Multimodal Agent MemorySubmit a paper

VisualClaw

VisualClaw: A Real-Time, Personalized Agent for the Physical World

Haoqin Tu, Jianwen Chen, Zijun Wang, Siwei Han, Juncheng Wu, Hardy Chen, et al. · 2026-06-15

Read the paperPDF

Management
T3 Self-Evolving Memory
Feedback during deployment changes the memory-management policy itself.
Primary modality
Video & streaming
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. filtered streaming visual evidence

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    A cascaded gate retains informative streaming frames while reducing uploads.Full paper: Method · PDF pp. 4, 6, 19 · Figure 1

  2. hot/cold multimodal skill bank

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inHierarchical multi-tier store
    3. Retrieved bySimilarity search
    4. Source evidenceSource not retained
    A compressed skill bank is retrieved by hot/cold top-k injection and updated from failures.Full paper: Method · PDF pp. 4, 6, 19 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    Failures and retrieved memories are distilled into reusable skills. For VisualClaw, this specifically supports abstraction of episodes into reusable experience of filtered streaming visual evidence.

  • Update · Prompted LLM/VLM record update

    An evolver writes persistent skill-bank updates for later questions. For VisualClaw, this specifically supports LLM-mediated memory editing of filtered streaming visual evidence.

How the policy evolves (T3)

  • Retrieval policy

    Failure-driven skill updates persist and change future hot/cold skill activation and injection for later questions. The persistent later-use effect is deployment-feedback evolution of the reusable retrieval policy or procedure.

Related systems