Multimodal Agent MemorySubmit a paper

MSGNav

MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation

Xun Huang, Shijia Zhao, Yunxiang Wang, Xin Lu, Wanfa Zhang, Rongsheng Qu, et al. · 2026-03-17

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text, Image
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. multi-modal 3D scene graph

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inGraph / relational store
    3. Retrieved bySimilarity search, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Incremental M3DSG updates with preserved image edges, adaptive vocabulary updates, and historical action repository. key subgraph selection retrieves task-relevant graph context; decision memory is prompted for closed-loop reasoning.Full paper: 3.3. MSGNav Embodied Navigation System · PDF p. 5 · Figure 1 / method overview

  2. object nodes

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inGraph / relational store
    3. Retrieved bySimilarity search, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceRetained but not accessible to the agent
    Incremental M3DSG updates with preserved image edges, adaptive vocabulary updates, and historical action repository. key subgraph selection retrieves task-relevant graph context; decision memory is prompted for closed-loop reasoning.Full paper: 3.3. MSGNav Embodied Navigation System · PDF p. 5 · Figure 1 / method overview

  3. image-based relation edges

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved bySimilarity search, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Incremental M3DSG updates with preserved image edges, adaptive vocabulary updates, and historical action repository. key subgraph selection retrieves task-relevant graph context; decision memory is prompted for closed-loop reasoning.Full paper: 3.3. MSGNav Embodied Navigation System · PDF p. 5 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    During operation, MSGNav applies fixed-rule insertion or replacement to multi-modal 3D scene graph; the memory state changes while the governing update policy remains fixed.

  • Consolidation · Merging & deduplication

    During operation, MSGNav applies merging or deduplication of overlapping entries to multi-modal 3D scene graph; the memory state changes while the governing update policy remains fixed.

Related systems