Multimodal Agent MemorySubmit a paper

Ella

Ella: Embodied Social Agents with Lifelong Memory

Hongxin Zhang, Zheyuan Zhang, Zeyuan Wang, Zunzhe Zhang, Lixing Fang, Qinhong Zhou, et al. · 2025-06-30

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. spatiotemporal multimodal episodic memory

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved bySimilarity search, Metadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    Multimodal experiences from visual observations and social interactions are retained across days.Full paper: Update · PDF p. 5 · Figure 1

  2. name-centric semantic memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved bySimilarity search, Structure traversal
    4. Source evidenceSource linked & recoverable
    Acquired knowledge is organized by entities and updated for later social planning.Full paper: 4.1 Name-centric Semantic Memory · PDF p. 4 · Figure 1

How memory changes (T2 operations)

  • Update · Rule-based update

    New social and environmental observations update long-term memory. For Ella, this specifically supports fixed-rule insertion or replacement of spatiotemporal multimodal episodic memory.

  • Reorganization · Relinking & reindexing

    Entity-centric semantic relations evolve as agents acquire knowledge. For Ella, this specifically supports relinking or reindexing of stored items of spatiotemporal multimodal episodic memory.

Related systems