Multimodal Agent MemorySubmit a paper

ReWind

ReWind: Understanding Long Videos with Instructed Learnable Memory

Anxhelo Diko, Tinghuai Wang, Wassim Swaileh, Shiyan Sun, Ioannis Patras · 2025-03-27

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, text
Representation
Hybrid
Learned memory states and selected modality-native frame features are combined as LLM input.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. long-term memory bank

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inFlat sequential records
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    Learnable read queries attend to stored history; write queries distill instruction-conditioned frame representations into compact tokens appended in temporal order.Full-paper method / memory representation and retrieval · PDF p. 4 · Figures 2 and 3

  2. original visual feature buffer

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved bySimilarity search
    4. Source evidenceRetained but not accessible to the agent
    Per-frame encoder features are preserved in a separate buffer, and Dynamic Frame Selection retrieves instruction-relevant high-resolution features for the second stage.Full-paper method / memory representation and retrieval · PDF p. 4 · Figure 2

How memory changes (T2 operations)

  • Update · Learned update module

    A trained cross-attention write interface uses learned queries to compress each new frame representation and persist it in the memory bank. For ReWind, this specifically supports a trained but frozen update module of long-term memory bank.

Related systems