Multimodal Agent MemorySubmit a paper

GraphPad

GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering

Muhammad Qasim Ali, Saeejith Nair, Alexander Wong, Yuchen Cui, Yuhao Chen · 2025-06-01

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. frame-indexed navigation log

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    A navigation log indexes frame-by-frame visual content for later access.Full paper: 2.3. Task-Memory Alignment · PDF p. 2 · Figure 1

  2. mutable scene graph and scratchpad

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    API calls update a task-tailored 3D scene graph and task-specific notes at inference time.Full paper: 2.3. Task-Memory Alignment · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    The agent uses API calls to add, revise, and tailor scene-memory content. For GraphPad, this specifically supports LLM-mediated memory editing of frame-indexed navigation log.

  • Reorganization · Relinking & reindexing

    Inference-time graph updates add task-relevant objects and relations. For GraphPad, this specifically supports relinking or reindexing of stored items of frame-indexed navigation log.

Related systems