Multimodal Agent MemorySubmit a paper

ABot-AgentOS

ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

Jiayi Tian, Shiao Liu, Yuting Xu, Jia Lu, Zihao Guan, Honglin Han, et al. · 2026-07-17

Read the paperPDF

Management
T3 Self-Evolving Memory
Feedback during deployment changes the memory-management policy itself.
Primary modality
Embodied & 3D
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. source-grounded multimodal traces

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval, Planner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Dialogue, visual observations, and task traces remain source-grounded within lifelong memory.Full-paper method / memory representation and retrieval · PDF pp. 12–13 · Figure 1

  2. Universal Multi-modal Graph Memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Visual, spatial, temporal, dialogue, and task evidence becomes typed nodes and edges.Full-paper method / memory representation and retrieval · PDF pp. 12–13 · Figure 1

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    Diagnosed failures are converted into gated runtime evo-assets for later tasks. For ABot-AgentOS, this specifically supports LLM-mediated memory editing of source-grounded multimodal traces.

  • Revision · Evidence-weighted belief revision

    Multi-stage verification diagnoses memory failures before promotion. For ABot-AgentOS, this specifically supports evidence-weighted revision of stored belief or confidence of source-grounded multimodal traces.

How the policy evolves (T3)

  • Write policy

    For ABot-AgentOS, feedback produced during deployed inference persistently changes a reusable artifact governing later interactions: deployment-feedback evolution of the reusable write policy.

  • Retrieval policy

    For ABot-AgentOS, feedback produced during deployed inference persistently changes a reusable artifact governing later interactions: deployment-feedback evolution of the reusable retrieval policy or procedure.

Related systems