Multimodal Agent MemorySubmit a paper

AViLA

AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction

Gengyuan Zhang, Tanveer Hannan, Hermine Kleiner, Beste Aydemir, Xinyu Xie, Jian Lan, et al. · 2025-06-23

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. comprehensive streaming observation memory

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    The memory retains historical streaming observations even when queries and evidence arrive asynchronously.Full paper: Memory · PDF p. 6 · Figure 1

  2. time-aware evidence index

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Metadata / spatiotemporal retrieval
    4. Source evidenceSource linked & recoverable
    Evidence identification links asynchronous queries to relevant historical observations and response triggers.Full paper: Memory · PDF p. 6 · Figure 1

How memory changes (T2 operations)

  • Update · Rule-based update

    New stream observations are continuously appended under a fixed retention procedure. For AViLA, this specifically supports fixed-rule insertion or replacement of comprehensive streaming observation memory.

  • Reorganization · Relinking & reindexing

    Temporal evidence links are updated as asynchronous support arrives. For AViLA, this specifically supports relinking or reindexing of stored items of comprehensive streaming observation memory.

Related systems