Multimodal Agent MemorySubmit a paper

SpatialGPT

SpatialGPT: Zero-Shot Vision-and-Language Navigation via Spatial CoT over Structured Spatial Memory

Zhiqiang Jiang, Xin Wang · 2026-01-01

Read the paper

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: text
Representation
Cross-represented explicit

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. spatial working memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inHierarchical multi-tier store
    3. Retrieved bySimilarity search, Metadata / spatiotemporal retrieval
    4. Source evidenceSource not retained
    Updates SKG with spatial knowledge, selected directions and alternatives. backtracking retrieves stored alternative paths or infers frontiers.Method: Spatial Knowledge Update · Proceedings pp. 423–435 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    During operation, SpatialGPT applies fixed-rule insertion or replacement to spatial working memory; the memory state changes while the governing update policy remains fixed.

Related systems