Multimodal Agent MemorySubmit a paper

Open-World 3D Scene Graph RAG

Open-World 3D Scene Graph Generation for Retrieval-Augmented Reasoning

Fei Yu, Quan Deng, Shengeng Tang, Yuehua Li, Lechao Cheng · 2025-11-08

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text, Image
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. open-vocabulary objects

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inGraph / relational store, Vector-indexed collection
    3. Retrieved bySimilarity search, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceRetained but not accessible to the agent
    Dynamic 3D scene graph nodes and edges encoded into a vector database for retrieval-augmented reasoning. retrieval-augmented reasoning retrieves relevant stored scene-graph context for text/image-conditioned queries.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1 / method overview

  2. semantic relations

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store, Vector-indexed collection
    3. Retrieved bySimilarity search, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Dynamic 3D scene graph nodes and edges encoded into a vector database for retrieval-augmented reasoning. retrieval-augmented reasoning retrieves relevant stored scene-graph context for text/image-conditioned queries.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    During operation, Open-World 3D Scene Graph RAG applies fixed-rule insertion or replacement to open-vocabulary objects; the memory state changes while the governing update policy remains fixed.

  • Reorganization · Relinking & reindexing

    Dynamic graph construction updates semantic and spatial relationships as observations evolve. For Open-World 3D Scene Graph RAG, this specifically supports relinking or reindexing of stored items of open-vocabulary objects.

Related systems