Multimodal Agent MemorySubmit a paper

CMMR-VLN

CMMR-VLN: Vision-and-Language Navigation via Continual Multimodal Memory Retrieval

Haozhou Li, Xiangyu Dong, Huiyan Jiang, Yaoming Zhou, Xiaoguang Ma · 2026-03-09

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. panoramic navigation evidence

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search
    4. Source evidenceSource directly available
    Experience memory retains panoramic images indexed with salient landmarks.Full paper: A. Multimodal Experience Memory (MEM) · PDF p. 3 · Figure 1

  2. structured route experience

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Similarity search
    4. Source evidenceSource linked & recoverable
    Successful routes and salient failure points are stored for retrieval in later navigation.Full paper: A. Multimodal Experience Memory (MEM) · PDF p. 3 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    Reflection selects complete successful paths and the key initial mistake from failures. For CMMR-VLN, this specifically supports abstraction of episodes into reusable experience of panoramic navigation evidence.

  • Update · Rule-based update

    A fixed reflection strategy admits selected route experiences after each navigation episode. For CMMR-VLN, this specifically supports fixed-rule insertion or replacement of panoramic navigation evidence.

Related systems