Multimodal Agent MemorySubmit a paper

FOLIO

FOLIO: Focused Semantic Memory for Streaming Video Understanding

Haoyang Fan, Dhruv Parikh, Anvitha Ramachandran, Sameh Gobriel, Nilesh Jain, Rajgopal Kannan, et al. · 2026-07-14

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, text
Representation
Hybrid
source-linked semantic memory

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. short-term visual buffer and evidence cache

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    Detailed visual records are retained for focused entities.Full paper: Prior event memory · PDF p. 2

  2. entity-centered long-term semantic memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Similarity search
    4. Source evidenceSource linked & recoverable
    Long-term semantic memory organizes entities and temporal changes and links back to visual evidence.Full paper: Prior event memory · PDF p. 2

How memory changes (T2 operations)

  • Consolidation · Episodic / hierarchical summarization

    Surrounding context is stored compactly while focused entities retain detail. For FOLIO, this specifically supports episodic or hierarchical summarization of short-term visual buffer and evidence cache.

  • Reorganization · Relinking & reindexing

    The dynamic focus state updates entity organization and links to the visual cache. For FOLIO, this specifically supports relinking or reindexing of stored items of short-term visual buffer and evidence cache.

Related systems