Multimodal Agent MemorySubmit a paper

EGAgent

Agentic Very Long Video Understanding

Aniket Rege, Arka Sadhu, Yuliang Li, Kejie Li, Ramya Korlakai Vinayak, Yuning Chai, et al. · 2026-01-26

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. entity-linked scene evidence

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Metadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    The agent retains searchable visual and audio evidence from continuous video.Full paper: Agentic Framework EGAgent · PDF p. 6 · Figure 1

  2. entity scene graph

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    People, places, objects, and their temporal relationships are organized into an entity scene graph.Full paper: EGAgent Pipeline · PDF p. 4 · Figure 1

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    During operation, EGAgent applies LLM-mediated memory editing to entity-linked scene evidence; the memory state changes while the governing update policy remains fixed.

  • Consolidation · Episodic / hierarchical summarization

    During operation, EGAgent applies episodic or hierarchical summarization to entity-linked scene evidence; the memory state changes while the governing update policy remains fixed.

Related systems