Multimodal Agent MemorySubmit a paper

EgoMem

EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models

Yiqun Yao, Naitong Yu, Xiang Li, Xin Jiang, Xuezhi Fang, Wenjia Ma, et al. · 2026-01-30

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Audio & speech
Also: text, Image, video
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. per-user facts

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search
    4. Source evidenceSource not retained
    Extracted user-fact entries; per-user face and voice embeddings. online face/voice identification routes to the per-user fact store; relevant context retrieved during dialog.Full paper: 5.1. Retrieval Evaluation · PDF p. 7 · Figure 1 / method overview

  2. preferences

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search
    4. Source evidenceSource not retained
    Extracted user-fact entries; per-user face and voice embeddings. online face/voice identification routes to the per-user fact store; relevant context retrieved during dialog.Full paper: 5.1. Retrieval Evaluation · PDF p. 7 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    During operation, EgoMem applies LLM-mediated memory editing to per-user facts; the memory state changes while the governing update policy remains fixed.

  • Revision · LLM-based conflict resolution

    During operation, EgoMem applies LLM-based resolution of conflicting records to per-user facts; the memory state changes while the governing update policy remains fixed.

Related systems