Multimodal Agent MemorySubmit a paper

NeSy-Spatial

Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning

Shi-Yu Tian, Zhuo-Xia Wang, Xuan-Yi Zhu, Zhi Zhou, Xinwei Yang, Kun-Yang Yu, et al. · 2026-08-08

Read the paperPDF

Management
T3 Self-Evolving Memory
Feedback during deployment changes the memory-management policy itself.
Primary modality
Image
Also: text, depth, 3d geometry
Representation
Cross-represented explicit

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. persistent tool-use and geometry skill library

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    The agent stores atomic neuro-symbolic instructions and Tool-Use and Geometry skills distilled from image-based spatial-reasoning trajectories; later problems retrieve and compose these structured skills with visual, depth, and reconstruction tools.Full paper: §§3.1–3.4 · HTML full text · Figures 2–3

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    Verified spatial-reasoning trajectories are distilled into reusable atomic instructions and higher-level tool-use and geometry skills.

  • Consolidation · Merging & deduplication

    The library-management stage analyzes, fuses, and prunes overlapping skill content before committing the next library state.

  • Forgetting · Discrete eviction

    Pruning removes skills that are redundant or no longer supported during library evolution.

How the policy evolves (T3)

  • Retrieval policy

    In the test-then-update protocol, each episode is predicted before its label is revealed; post-episode labels and verification traces update the persistent skill library, and the resulting library state is retrieved and composed on later spatial-reasoning tasks.

Related systems