Multimodal Agent MemorySubmit a paper

MemLearner

MemLearner: Learning to Query Context memory for Video World Models

Jiwen Yu, Jianxiong Gao, Jianhong Bai, Yiran Qin, Kaiyi Huang, Quande Liu, et al. · 2026-06-30

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. scene and object continuity needed for revisits under occlusion and dynamic objects

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inFlat sequential records
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    Context tokens (C) and query tokens (Q) inside a diffusion transformer; predicted video latent tokens (P). query tokens adaptively extract useful information from context tokens; P tokens condition on Q tokens during generation.Full paper: Diffusion Model · PDF p. 2 · Figure 1 / method overview

  2. scene and object continuity needed for revisits under occlusion and dynamic objects

    1. Represented asExperience-conditioned parametric memoryInternalized latent & parametric
    2. Stored inModular adapter bank
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    Context tokens (C) and query tokens (Q) inside a diffusion transformer; predicted video latent tokens (P). query tokens adaptively extract useful information from context tokens; P tokens condition on Q tokens during generation.Full paper: Diffusion Model · PDF p. 2 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    During operation, MemLearner applies fixed-rule insertion or replacement to scene and object continuity needed for revisits under occlusion and dynamic objects; the memory state changes while the governing update policy remains fixed.

Related systems