Multimodal Agent MemorySubmit a paper

FlexMem

Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism

Tao Chen, Kun Zhang, Qiong Wu, Xiao Chen, Chao Chang, Xiaoshuai Sun, et al. · 2026-03-31

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text
Representation
Internalized latent & parametric

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. compressed visual KV memory

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inHierarchical multi-tier store
    3. Retrieved byLatent / parametric activation
    4. Source evidenceRetained but not accessible to the agent
    Dual-pathway writing compresses visual KV caches and exposes multiple reading strategies.Full paper: 3. Method · PDF p. 3 · Figure 1

How memory changes (T2 operations)

  • Update · Learned update module

    A fixed visual-memory writer updates compressed KV state as the video grows. For FlexMem, this specifically supports a trained but frozen update module of compressed visual KV memory.

  • Reorganization · Memory-level migration (short- to long-term)

    Information is routed between short and long visual-memory pathways. For FlexMem, this specifically supports migration between memory tiers of compressed visual KV memory.

  • Consolidation · Merging & deduplication

    During operation, FlexMem applies merging or deduplication of overlapping entries to compressed visual KV memory; the memory state changes while the governing update policy remains fixed.

  • Forgetting · Discrete eviction

    During operation, FlexMem applies explicit deletion or eviction of stored items to compressed visual KV memory; the memory state changes while the governing update policy remains fixed.

Related systems