Multimodal Agent MemorySubmit a paper

Kubrick

Kubrick: Multimodal Agent Collaborations for Synthetic Video Generation

Liu He, Yizhi Song, Hejun Huang, Pinxin Liu, Yunlong Tang, Daniel Aliaga, et al. · 2025-05-05

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text, Embodied & 3D
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Persistent 3D world state (geometry, assets, camera)

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource not retained
    3D scene representation (Blender) shared across shots. Each shot is rendered from the shared 3D world; continuity is inherent.Full-paper method / memory representation and retrieval · PDF p. 4 · Figure 1 / method overview

  2. Persistent 3D world state (geometry, assets, camera)

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inFlat sequential records
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource not retained
    3D scene representation (Blender) shared across shots. Each shot is rendered from the shared 3D world; continuity is inherent.Full-paper method / memory representation and retrieval · PDF p. 4 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    Agents modify the persistent 3D world state across shots (fixed editing rules). For Kubrick, this specifically supports LLM-mediated memory editing of Persistent 3D world state (geometry, assets, camera).

Related systems