Multimodal Agent MemorySubmit a paper

RRM

RRM: Experience-Driven Reflective Retrieval Memory for Long-Horizon Multimodal Reasoning

Jingxiang Fan, Junbao Zhuo, Bochao Zou · 2026-07-30

Read the paperPDF

Management
T3 Self-Evolving Memory
Feedback during deployment changes the memory-management policy itself.
Primary modality
Video & streaming
Also: Image, audio, text
Representation
Cross-represented explicit

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. entity-centric factual video graph

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    The graph preserves current-video factual evidence.Full-paper method / memory representation and retrieval · PDF pp. 4–5 · Figure 1

  2. successful and failed retrieval strategies

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Reflective memory stores procedural guidance distilled from task trajectories.Full-paper method / memory representation and retrieval · PDF pp. 4–5 · Figure 1

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    During operation, RRM applies LLM-mediated memory editing to entity-centric factual video graph; the memory state changes while the governing update policy remains fixed.

  • Consolidation · Experience abstraction

    During operation, RRM applies abstraction of episodes into reusable experience to entity-centric factual video graph; the memory state changes while the governing update policy remains fixed.

How the policy evolves (T3)

  • Retrieval policy

    Reuse feedback and outcomes change guidance for later searches. The persistent later-use effect is deployment-feedback evolution of the reusable retrieval policy or procedure.

Related systems