Multimodal Agent MemorySubmit a paper

AtlasVA

AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents

Pan Wang, Yihao Hu, Xiujin Liu, Jingchu Yang, Hang Wang, Zhihao Wen · 2026-05-18

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Image
Also: text, Embodied & 3D
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. spatial heatmaps

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Visual skill-memory layers; self-evolving atlas statistics; potential-based shaping rewards. VLM agents reuse visual skill memory and atlases as dense potential-based shaping signals for reinforcement learning.Full paper: Memory and Skill Evolution in RL Agents · PDF p. 3 · Figure 1 / method overview

  2. visual exemplars

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceRetained but not accessible to the agent
    Visual skill-memory layers; self-evolving atlas statistics; potential-based shaping rewards. VLM agents reuse visual skill memory and atlases as dense potential-based shaping signals for reinforcement learning.Full paper: Memory and Skill Evolution in RL Agents · PDF p. 3 · Figure 1 / method overview

  3. symbolic text skills

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Visual skill-memory layers; self-evolving atlas statistics; potential-based shaping rewards. VLM agents reuse visual skill memory and atlases as dense potential-based shaping signals for reinforcement learning.Full paper: Memory and Skill Evolution in RL Agents · PDF p. 3 · Figure 1 / method overview

Related systems