Multimodal Agent MemorySubmit a paper

BehaviorVLA

From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model

Bing Hu, Zaijing Li, Rui Shao, Junda Chen, April Hua Liu, Wei-Shi Zheng, et al. · 2026-05-31

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Embodied & 3D
Also: Image, text
Representation
Internalized latent & parametric

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Behavior Memory Bank of scene-invariant behavioral prototypes distilled from demonstrations

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Latent / parametric activation
    4. Source evidenceSource not retained
    A bank of behavior prototypes is built by temporal mean pooling of behavior tokens (z_proto = 1/T sum_t h_t^z) over offline demonstrations, where behavior tokens come from spatial multimodal fusion of the vision and action streams (Sec. 3.2.1). At deployment a query q = MLP(Phi(O_0, L)) formed from the initial observation and the instruction selects the top-K prototypes by dot-product similarity with softmax weighting, and the aggregated prototype latent is injected as a behavioral prior that conditions action generation. The stored items are compressed latent vectors, not text or retained frames, and the demonstrations behind them are not recoverable from the bank.Full paper (arXiv HTML v2): Sec. 3.2.2 Behavior prototype construction and retrieval; Eq. 6; Fig. 2 · HTML Sec. 3.2.2 · Equation 6; Figure 2 caption

Related systems