Multimodal Agent MemorySubmit a paper

ReflectWorld-MM

ReflectWorld-MM: An Entity-Oriented Multi-Media Memory System for Open-Ended Video Streams

Xiaokang Ma, Yifan Sun, Zhihong Jin, Jie Gu, Yudong Luo, Shenyi Shao, et al. · 2026-07-06

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, audio, text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. entity-resolved multimedia observations

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    A bounded front end creates entity-resolved observations.Full paper: Memory · PDF p. 2 · Figure 1

  2. episodic, semantic and procedural entity memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store, Hierarchical multi-tier store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Long-term memory couples multi-scale episodes with evolving entity semantics and procedures.Full paper: Memory · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Episodic / hierarchical summarization

    A fixed hierarchy consolidates observations. For ReflectWorld-MM, this specifically supports episodic or hierarchical summarization of entity-resolved multimedia observations.

  • Consolidation · Experience abstraction

    Repeated experience becomes semantic and procedural memory. For ReflectWorld-MM, this specifically supports abstraction of episodes into reusable experience of entity-resolved multimedia observations.

  • Revision · LLM-based conflict resolution

    Later observations revise conflicting entity knowledge. For ReflectWorld-MM, this specifically supports LLM-based resolution of conflicting records of entity-resolved multimedia observations.

  • Update · Prompted LLM/VLM record update

    A fixed GPT-based manager chooses Add, Update, Delete, or no-change actions after accumulating observations. For ReflectWorld-MM, this specifically supports LLM-mediated memory editing of entity-resolved multimedia observations.

  • Revision · Evidence-weighted belief revision

    Repeated confirmation increases an entity memory's importance through a fixed evidence-reweighting rule. For ReflectWorld-MM, this specifically supports evidence-weighted revision of stored belief or confidence of entity-resolved multimedia observations.

Related systems