Multimodal Agent MemorySubmit a paper

MTU3D

Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation

Ziyu Zhu, Xilin Wang, Yixuan Li, Zhuofan Zhang, Xiaojian Ma, Yixin Chen, et al. · 2025-07-30

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text, Image
Representation
Internalized latent & parametric

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. local object/frontier queries

    1. Represented asExperience-conditioned parametric memoryInternalized latent & parametric
    2. Stored inModular adapter bank
    3. Retrieved bySimilarity search, Metadata / spatiotemporal retrieval, Latent / parametric activation
    4. Source evidenceSource not retained
    Single-frame local queries are merged with historical queries and dynamically stored in a spatial memory bank. object queries retrieved from the spatial memory bank are fed with frontier queries for grounding and exploration.Full paper: Memory Read · PDF p. 4 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    During operation, MTU3D applies fixed-rule insertion or replacement to local object/frontier queries; the memory state changes while the governing update policy remains fixed.

  • Consolidation · Merging & deduplication

    Local queries are merged with historical global queries to update the spatial memory bank during exploration. For MTU3D, this specifically supports merging or deduplication of overlapping entries of local object/frontier queries.

Related systems