Multimodal Agent MemorySubmit a paper

CMA

Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing

Feng Wang, Canmiao Fu, Zhipeng Huang, Chen Li, Jing Lyu, Ge Li · 2026-07-09

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Image
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. episodic visual memory

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search
    4. Source evidenceSource directly available
    Visual inputs are externalized into selectively reactivated episodes.Full paper: Episodic Visual Memory (EVM), allowing visual content · PDF p. 2 · Figure 1

  2. structured visual abstractions

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Perceptual abstractions route retrieval.Full paper: Episodic Visual Memory (EVM), allowing visual content · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Update · Rule-based update

    A fixed deployed pipeline writes and organizes new episodes. For CMA, this specifically supports fixed-rule insertion or replacement of episodic visual memory.

Related systems