Multimodal Agent MemorySubmit a paper

MARDoc

MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA

Kaifeng Chen, Hongtao Liu, Qiyao Peng, Jian Yang, Yongqiang Liu, Xiaochen Zhang, et al. · 2026-06-04

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Image
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. multimodal document evidence

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource directly available
    The Explorer retrieves multi-granularity evidence from the original multimodal document.Full paper: Method · PDF p. 3 · Figure 1

  2. structured evidence and reasoning memory

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    The Refiner distills traces into a dynamically updated structure preserving evidence and logical dependencies.Full paper: Agent Memory. Memory mechanisms have be- · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    The Refiner repeatedly edits structured evidence and reasoning memory. For MARDoc, this specifically supports LLM-mediated memory editing of multimodal document evidence.

  • Consolidation · Episodic / hierarchical summarization

    Long interaction traces are distilled instead of accumulated verbatim. For MARDoc, this specifically supports episodic or hierarchical summarization of multimodal document evidence.

  • Revision · Evidence-weighted belief revision

    Reflector feedback checks evidence sufficiency and directs targeted memory refinement. For MARDoc, this specifically supports evidence-weighted revision of stored belief or confidence of multimodal document evidence.

Related systems