Multimodal Agent MemorySubmit a paper

OCR-Memory

OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory

Jinze Li, Yang Zhang, Xin Yang, Jiayi Qu, Jinfeng Xu, Shuo Yang, et al. · 2026-04-29

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
GUI & computer use
Also: Image, text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. annotated trajectory images

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Trajectories are rendered as anchored images.Full paper: Retrieval-Based Memory Systems. Existing · PDF p. 2 · Figure 1

  2. verbatim trajectory logs

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    Locate-and-transcribe fetches verbatim text keyed by an anchor.Full paper: Retrieval-Based Memory Systems. Existing · PDF p. 2 · Figure 1

How memory changes (T2 operations)

  • Forgetting · Progressive decay

    Age-aware resolution downscales old trajectory images and restores selected ones. For OCR-Memory, this specifically supports progressive decay of stored importance or retention of annotated trajectory images.

  • Update · Rule-based update

    During operation, OCR-Memory applies fixed-rule insertion or replacement to annotated trajectory images; the memory state changes while the governing update policy remains fixed.

Related systems