Multimodal Agent MemorySubmit a paper

PyraVid

PyraVid: Hierarchical Multimodal Memory for Long-Horizon Video Reasoning

Sikuan Yan, Sicheng Dong, Haotong Wang, Ercong Nie, Yilun Liu, Jinhe Bi, et al. · 2026-05-16

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, Audio & speech, text
Representation
Hybrid
Coarse textual abstractions and graph links retain a route to the visual evidence stored in fact nodes.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. fact nodes with associated visual evidence

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store, Graph / relational store
    3. Retrieved bySimilarity search, Structure traversal
    4. Source evidenceSource directly available
    Each fact node stores temporal information, associated visual evidence, grounded text, and links to related nodes.Full paper: Hierarchical Memory · PDF p. 4

  2. fact, clip, and global textual-structured memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inHierarchical multi-tier store, Graph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Fact, clip, and evolving global levels are linked across granularity and by logical relations; inference expands and prunes evidence until sufficient.Full paper: Hierarchical Memory · PDF p. 4

How memory changes (T2 operations)

  • Consolidation · Episodic / hierarchical summarization

    Clip nodes summarize local events and the global node incrementally integrates clip-level context. For PyraVid, this specifically supports episodic or hierarchical summarization of fact nodes with associated visual evidence.

  • Reorganization · Relinking & reindexing

    New facts are connected to clip/global parents and sparsely linked to logically related facts. For PyraVid, this specifically supports relinking or reindexing of stored items of fact nodes with associated visual evidence.

  • Update · Prompted LLM/VLM record update

    Character-level memory is incrementally rewritten and profiled as new face and voice evidence arrives. For PyraVid, this specifically supports LLM-mediated memory editing of fact nodes with associated visual evidence.

Related systems