Multimodal Agent MemorySubmit a paper

MemoryVLA

MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation

Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, et al. · 2025-08-26

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: Image, text
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. persistent cognitive memory tokens

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Latent / parametric activation
    4. Source evidenceSource not retained
    The cognitive memory bank retains compact latent decision state that is retrieved for the action expert.Full paper: Memory · PDF p. 4 · Figure 1

  2. temporally indexed compressed perceptual tokens

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Latent / parametric activation
    4. Source evidenceSource not retained
    Visual encoder outputs are compressed into perceptual tokens, stored in a temporally indexed bank, and queried to influence the action policy.Full paper: Memory · PDF p. 4 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Merging & deduplication

    Redundancy-aware consolidation merges similar entries. For MemoryVLA, this specifically supports merging or deduplication of overlapping entries of persistent cognitive memory tokens.

Related systems