Multimodal Agent MemorySubmit a paper

MUSE

MUSE: Agentic 3D Scene Authoring via Memory-Grounded Incremental Requirement Satisfaction

Ruijie Xu, Xinnan Zhu, Jiayu Ying, Daoguo Dong, Yuzhou Ji, Xin Tan · 2026-06-12

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text
Representation
Cross-represented explicit

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. working, scene, and skill memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inHierarchical multi-tier store
    3. Retrieved byPlanner / tool-assisted retrieval, Structure traversal
    4. Source evidenceSource linked & recoverable
    Structured requirements, scene state, and reusable skills track incremental construction and editing.Full paper: One-Shot Pipeline · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Update · Rule-based update

    The Inspector verifies each local scene operation and updates working, scene, and skill memory. For MUSE, this specifically supports fixed-rule insertion or replacement of working, scene, and skill memory.

  • Revision · Rule-based state correction

    Failed requirements trigger localized correction while preserving non-target scene content. For MUSE, this specifically supports fixed-rule correction of stored state of working, scene, and skill memory.

  • Consolidation · Experience abstraction

    Verified local operations are retained as reusable skill memory. For MUSE, this specifically supports abstraction of episodes into reusable experience of working, scene, and skill memory.

Related systems