Multimodal Agent MemorySubmit a paper

GraphEQA

GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answering

Saumya Saxena, Blake Buchanan, Chris Paxton, Peiqi Liu, Bingqing Chen, Narunas Vaskevicius, et al. · 2025-09-24

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: Image, depth, text
Representation
Hybrid
The VLM jointly receives the hierarchical scene graph and task-relevant source images for grounded planning.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. top-K task-relevant prior images

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved bySimilarity search, Planner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Buffered observations are ranked with SigLIP; only the top-K images are retained and jointly supplied with the current view to the VLM planner.Full-paper method / memory representation and retrieval · PDF p. 2

  2. online hierarchical 3D scene graph

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    The online 3DSG stores rooms, regions, objects, and frontier links; the planner traverses its hierarchy while grounding decisions in retained images.Full paper: Method · PDF p. 3 · Figure 3

How memory changes (T2 operations)

  • Update · Rule-based update

    The metric-semantic graph is incrementally updated online as the robot maps new space and adds object and frontier nodes. For GraphEQA, this specifically supports fixed-rule insertion or replacement of top-K task-relevant prior images.

  • Forgetting · Discrete eviction

    At each step, only the top-K relevant prior images remain in the buffer and the rest are discarded. For GraphEQA, this specifically supports explicit deletion or eviction of stored items of top-K task-relevant prior images.

Related systems