Multimodal Agent MemorySubmit a paper

NS-Mem

Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory

Rongjie Jiang, Jianwei Wang, Gengda Zhao, Chengyang Luo, Kai Wang, Wenjie Zhang · 2026-03-16

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Image
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. episodic, semantic and logic-rule layers

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store, Hierarchical multi-tier store
    3. Retrieved byStructure traversal
    4. Source evidenceSource linked & recoverable
    Explicit structures and logic rules support deterministic queries.Full paper: Memory-Augmented Agents in Single Modality. Memory man- · PDF p. 2 · Figure 1

  2. paired neural memory

    1. Represented asPersistent multimodal latent stateInternalized latent & parametric
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search
    4. Source evidenceSource not retained
    Neural representations support similarity retrieval.Full paper: Update · PDF p. 3 · Figure 1

How memory changes (T2 operations)

  • Consolidation · Experience abstraction

    SK-Gen abstracts experience into structured knowledge. For NS-Mem, this specifically supports abstraction of episodes into reusable experience of episodic, semantic and logic-rule layers.

  • Update · Rule-based update

    SK-Gen incrementally updates neural and symbolic layers. For NS-Mem, this specifically supports fixed-rule insertion or replacement of episodic, semantic and logic-rule layers.

  • Consolidation · Merging & deduplication

    Recurring experience is consolidated. For NS-Mem, this specifically supports merging or deduplication of overlapping entries of episodic, semantic and logic-rule layers.

  • Revision · Evidence-weighted belief revision

    Similarity-thresholded reinforcement increases or decreases symbolic edge weights, removing a relation when its accumulated weight reaches zero. For NS-Mem, this specifically supports evidence-weighted revision of stored belief or confidence of episodic, semantic and logic-rule layers.

  • Reorganization · Relinking & reindexing

    Exponential-moving-average updates revise goal and step index vectors as new experience is integrated. For NS-Mem, this specifically supports relinking or reindexing of stored items of episodic, semantic and logic-rule layers.

Related systems