Multimodal Agent MemorySubmit a paper

Light-Omni

Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory

Chang Nie, Jiaju Wei, Junlan Feng, Chaoyou Fu, Caifeng Shan · 2026-07-06

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, audio, text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. finite global multimodal script

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource linked & recoverable
    Hierarchical merging preserves recent detail while summarizing old events.Full paper: 1 An asynchronous process that continuously updates memory without · PDF p. 2 · Figure 1

  2. parametric action and retrieval state

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inFlat sequential records
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    A latent state conditioned on the global script drives actions and retrieval embeddings.Full paper: 1 An asynchronous process that continuously updates memory without · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Episodic / hierarchical summarization

    Older events are progressively summarized. For Light-Omni, this specifically supports episodic or hierarchical summarization of finite global multimodal script.

  • Consolidation · Merging & deduplication

    A fixed merge procedure combines episodic content. For Light-Omni, this specifically supports merging or deduplication of overlapping entries of finite global multimodal script.

  • Update · Rule-based update

    During operation, Light-Omni applies fixed-rule insertion or replacement to finite global multimodal script; the memory state changes while the governing update policy remains fixed.

Related systems