Multimodal Agent MemorySubmit a paper

Flash-VStream

Flash-VStream: Efficient Real-Time Understanding for Long Video Streams

Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Xiaojie Jin · 2025-06-30

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, text
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Context Synopsis Memory

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved bySimilarity search, Latent / parametric activation
    4. Source evidenceSource not retained
    A fixed-capacity context memory stores low-resolution visual-feature cluster centroids and temporal positions.Full paper: 3.1. Model Architecture · PDF p. 3

  2. Detail Augmentation Memory

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved bySimilarity search, Latent / parametric activation
    4. Source evidenceSource not retained
    High-resolution visual features nearest to high-density context clusters are retained for detailed spatial augmentation and jointly activated in the language model.Full paper: A. Implementation Details · PDF p. 12

How memory changes (T2 operations)

  • Update · Rule-based update

    A fixed K-means update incorporates each incoming low-resolution frame feature while maintaining a bounded centroid set. For Flash-VStream, this specifically supports fixed-rule insertion or replacement of Context Synopsis Memory.

  • Consolidation · Merging & deduplication

    Semantically similar frames are consolidated into cluster centroids to remove temporal redundancy. For Flash-VStream, this specifically supports merging or deduplication of overlapping entries of Context Synopsis Memory.

Related systems