Multimodal Agent MemorySubmit a paper

3DLLM-Mem

3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model

Wenbo Hu, Yining Hong, Yanjun Wang, Leison Gao, Zibu Wei, Xingcheng Yao, et al. · 2025-12-17

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: Image, depth, text
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. working memory and episodic memory

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    Current multi-view observations form 3D patch-token working memory, while prior observations and interactions persist as spatial-temporal episodic features.Full-paper method / memory representation and retrieval · PDF p. 2
    Working-memory tokens query episodic memory through attention and fuse task-relevant spatial and temporal features.Full-paper method / memory representation and retrieval · PDF p. 6

How memory changes (T2 operations)

  • Reorganization · Memory-level migration (short- to long-term)

    When the agent moves to a new environment, the preceding working memory is transferred into episodic memory. For 3DLLM-Mem, this specifically supports migration between memory tiers of working memory and episodic memory.

  • Revision · Rule-based state correction

    If a previously represented environment is revisited after modification, the corresponding memory entry is updated to the latest observed state. For 3DLLM-Mem, this specifically supports fixed-rule correction of stored state of working memory and episodic memory.

Related systems