Multimodal Agent MemorySubmit a paper

M3-Agent

Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory

Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, et al. · 2025-10-09

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Audio & speech, Image, text
Representation
Hybrid
Entity nodes bind modality-specific face and voice features to explicit episodic and semantic knowledge.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. face and voice features associated with entities

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inGraph / relational store, Vector-indexed collection
    3. Retrieved bySimilarity search, Planner / tool-assisted retrieval
    4. Source evidenceRetained but not accessible to the agent
    Entity nodes associate face and voice information with textual knowledge in a multimodal graph.Full-paper method / memory representation and retrieval · PDF p. 9

  2. entity-centric episodic and semantic knowledge

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Online memorization generates concrete events and abstract knowledge, incrementally links them to entities, and control performs iterative retrieval over the graph.Full-paper method / memory representation and retrieval · PDF p. 3 · Figure 1

  3. stored image and audio observations linked to persistent entities

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    The system incrementally stores image and audio observations as entity-linked nodes that can be recovered during iterative graph retrieval.Full-paper method / memory representation and retrieval · PDF p. 9 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    Semantic memories abstract durable attributes, relationships, and world knowledge from observed clips. For M3-Agent, this specifically supports abstraction of episodes into reusable experience of face and voice features associated with entities.

  • Reorganization · Relinking & reindexing

    New face, voice, event, and knowledge entries are incrementally connected around persistent entity identities. For M3-Agent, this specifically supports relinking or reindexing of stored items of face and voice features associated with entities.

  • Update · Rule-based update

    Fixed similarity thresholds and match-or-create rules determine whether incoming memories update an existing node or create a new one. For M3-Agent, this specifically supports fixed-rule insertion or replacement of face and voice features associated with entities.

  • Revision · Evidence-weighted belief revision

    Repeated matches and reactivation update memory weights, which are subsequently used in voting and retrieval. For M3-Agent, this specifically supports evidence-weighted revision of stored belief or confidence of face and voice features associated with entities.

Related systems