Multimodal Agent MemorySubmit a paper

Echo-MC

Experience Transfer for Multimodal LLM Agents in Minecraft Game

Chenghao Li, Jun Liu, Songbo Zhang, Huadong Jian, Hao Ni, Lik-Hang Lee, et al. · 2026-04-07

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text, Image
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. transferable experience units along five knowledge dimensions: structure, attribute, process, function, interaction

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved bySimilarity search
    4. Source evidenceSource not retained
    Experience entries decomposed across the five dimensions. In-Context Analogy Learning (ICAL) retrieves relevant experiences and adapts them to unseen tasks via contextual examples.Full paper: Memory · PDF p. 5 · Figure 1 / method overview

  2. transferable experience units along five knowledge dimensions: structure, attribute, process, function, interaction

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inFlat sequential records
    3. Retrieved bySimilarity search
    4. Source evidenceSource not retained
    Experience entries decomposed across the five dimensions. In-Context Analogy Learning (ICAL) retrieves relevant experiences and adapts them to unseen tasks via contextual examples.Full paper: Memory · PDF p. 5 · Figure 1 / method overview

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    Memory grows as new interactions distill into the five-dimension format. For Echo-MC, this specifically supports abstraction of episodes into reusable experience of transferable experience units along five knowledge dimensions: structure, attribute, process, function, interaction.

  • Update · Rule-based update

    During operation, Echo-MC applies fixed-rule insertion or replacement to transferable experience units along five knowledge dimensions: structure, attribute, process, function, interaction; the memory state changes while the governing update policy remains fixed.

Related systems