Multimodal Agent MemorySubmit a paper

Start from your modality

Memory problems look different for a robot, a video assistant, or a phone agent. Pick the setting you work in to see how its systems remember and how they are evaluated.

Management tiers by modality

Management tier by modality, shown as shares so modalities of different sizes compare fairly. Switch to Stacked for counts.

Systems by modality and management tier
  • Image 100%
  • Video & streaming 100%
  • Audio & speech 100%
  • GUI & computer use 100%
  • Embodied & 3D 100%
Systems by modality and management tier
CategoryT1 Static MemoryT2 Dynamic MemoryT3 Self-Evolving Memory
Image18397
Video & streaming27543
Audio & speech170
GUI & computer use13253
Embodied & 3D17794

Where memory is stored

Storage and indexing structures by modality. A system with several stores counts once in each.

Systems by storage structure and modality
  • Flat sequential records 106
  • Vector-indexed collection 94
  • Graph / relational store 102
  • Hierarchical multi-tier store 107
  • Modular adapter bank 27
Systems by storage structure and modality
CategoryImageVideo & streamingAudio & speechGUI & computer useEmbodied & 3D
Flat sequential records163641337
Vector-indexed collection332021326
Graph / relational store201631548
Hierarchical multi-tier store243532025
Modular adapter bank570213

Embodied and video lead; audio is just starting

Cumulative systems by primary modality and year of the latest paper version.

Jan 2027 · 297 systems
Image 64
Video & streaming 84
Audio & speech 8
GUI & computer use 41
Embodied & 3D 100
2023202420252026
Cumulative systems by primary modality, 2023–2026
0204060801002023202420252026● Embodied & 3D 100● Video & streaming 84● Image 64● GUI & computer use 41● Audio & speech 8
Cumulative systems by primary modality, 2023–2026
Series2023202420252026
Image241264
Video & streaming1143984
Audio & speech0118
GUI & computer use162041
Embodied & 3D91940100