Multimodal Agent MemorySubmit a paper

PRISM

Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks

Rutav Shah, Rajat Kumar Jenamani, Xiaohan Zhang, Lingfeng Sun, Roberto Martín-Martín, Yuke Zhu, et al. · 2026-06-15

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: Image
Representation
Internalized latent & parametric

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. hierarchically compressed visuomotor history

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inHierarchical multi-tier store
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    Gated attention filters hierarchically compressed history tokens for action prediction.Full paper: Short-Term Memory · PDF pp. 4–8

How memory changes (T2 operations)

  • Update · Learned update module

    During operation, PRISM applies a trained but frozen update module to hierarchically compressed visuomotor history; the memory state changes while the governing update policy remains fixed.

Related systems