Multimodal Agent MemorySubmit a paper

VideoLLaMB

VideoLLaMB: Long Video Understanding with Recurrent Memory Bridges

Yuxuan Wang, Cihang Xie, Yang Liu, Zilong Zheng · 2024-09-02

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text
Representation
Internalized latent & parametric

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. recurrent bridge memory tokens

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inFlat sequential records
    3. Retrieved byLatent / parametric activation
    4. Source evidenceRetained but not accessible to the agent
    Recurrent memory tokens bridge SceneTilling segments and carry latent context forward.Full paper: 2.3. Memory Cache with Retrieval · PDF p. 3 · Figure 1

How memory changes (T2 operations)

  • Update · Learned update module

    A learned recurrent bridge updates persistent memory tokens between semantic scenes. For VideoLLaMB, this specifically supports a trained but frozen update module of recurrent bridge memory tokens.

Related systems