Multimodal Agent MemorySubmit a paper

UniWM

Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight

Yifei Dong, Fengyi Wu, Guangyu Chen, Lingdong Kong, Xu Zhu, Qiyu Hu, et al. · 2026-01-01

Read the paper

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text, Image
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Intra-step memory bank for short-term perceptual grounding

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inHierarchical multi-tier store
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    The autoregressive backbone writes visual-planning context into hierarchical memory.; Intra-step memory stores local perceptual cues during rollout generation.; Cross-step memory carries trajectory context across extended horizons.. memory-augmented planning reads intra-/cross-step banks to align imagined future states.Full paper: Memory · PDF p. 2 · Figure 1 / method overview

  2. Cross-step memory bank for longer trajectory context

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inHierarchical multi-tier store
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    The autoregressive backbone writes visual-planning context into hierarchical memory.; Intra-step memory stores local perceptual cues during rollout generation.; Cross-step memory carries trajectory context across extended horizons.. memory-augmented planning reads intra-/cross-step banks to align imagined future states.Full paper: Memory · PDF p. 2 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    During operation, UniWM applies fixed-rule insertion or replacement to Intra-step memory bank for short-term perceptual grounding; the memory state changes while the governing update policy remains fixed.

Related systems