Multimodal Agent MemorySubmit a paper

PMMC

PMMC: Prospective Multimodal Memory Compilation for Long-Term LVLM Agents

Jingyu Sun, Yan Lin, Yuyang Xue, Yifan Wang, Zhengtao Yao, Rui Qian, et al. · 2026-08-02

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Image
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. raw-image interaction store

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Compiled programs preserve routes to raw images.Full-paper method / memory representation and retrieval · PDF pp. 3–4 · Figure 1

  2. verified question-conditioned programs

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Questioner, Planner and Doubter compile verified question-program pairs.Full-paper method / memory representation and retrieval · PDF pp. 3–4 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    Accumulated interactions become future-question programs. For PMMC, this specifically supports abstraction of episodes into reusable experience of raw-image interaction store.

  • Revision · Evidence-weighted belief revision

    The Doubter verifies evidence paths before admission. For PMMC, this specifically supports evidence-weighted revision of stored belief or confidence of raw-image interaction store.

Related systems