Multimodal Agent MemorySubmit a paper

Benchmarks

82 benchmarks and evaluation resources from the survey, grouped by the modality they test.

78 of 82 test recall over long histories, but only 7 test whether memory handles conflicting information.

Directly evaluated, by shared capability
  • Cross-modal evidenceCE44
  • Long-range evidenceLE78
  • State changesSC38
  • Conflicting memoriesCF7
  • Unanswerable queriesUQ10
  • Memory-grounded actionMA33

Image19

  • ATM-Bench

    According to Me: Long-Term Personalized Referential Memory QA

    Real dataQuestion answering · Retrieval

    Paper
  • CamRoll

    Personal AI Agent for Camera Roll VQA

    Real dataQuestion answering · Retrieval

    Paper
  • DISBench

    DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories

    Real dataRetrieval · Localization

    Paper
  • LCMP

    TAMEing Long Contexts in Personalization: Towards Training-Free and State-Aware MLLM Personalized Assistant

    SyntheticQuestion answering · Generation

    Paper
  • LoCoMo

    Evaluating Very Long-Term Conversational Memory of LLM Agents

    Hybrid (real + synthetic/simulated)Question answering · Generation

    Paper
  • M2CA-Bench

    Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing

    SyntheticRetrieval · Generation

    Paper
  • M3Exam

    M3Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions

    Hybrid (real + synthetic/simulated)Question answering · Retrieval

    Paper
  • Mem-Gallery

    Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents

    SyntheticQuestion answering · Retrieval

    Paper
  • MemEye

    MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

    Hybrid (real + synthetic/simulated)Question answering

    Paper
  • MemLeak

    MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory

    Hybrid (real + synthetic/simulated)Question answering

    Paper
  • MemLens

    MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models

    SyntheticQuestion answering · Retrieval

    Paper
  • Memory-QA

    Memory-QA: Answering Recall Questions Based on Multimodal Memories

    Real dataQuestion answering · Retrieval

    Paper
  • MMA-Bench

    MMA: Multimodal Memory Agent

    SyntheticQuestion answering · Referring

    Paper
  • MMDU

    MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs

    Hybrid (real + synthetic/simulated)Question answering · Generation

    Paper
  • MMRC

    MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation

    Real dataQuestion answering · Generation

    Paper
  • Persona-MME

    PersonaVLM: Long-Term Personalized Multimodal LLMs

    Hybrid (real + synthetic/simulated)Question answering · Generation

    Paper
  • RefMem-Bench

    Connecting the Dots: Benchmarking Reflective Memory in Long-Horizon Dialogue

    SyntheticQuestion answering · Generation

    Paper
  • SMMBench

    SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory

    Hybrid (real + synthetic/simulated)Question answering · Action

    Paper
  • VisualMem

    Personal Visual Memory from Explicit and Implicit Evidence

    Hybrid (real + synthetic/simulated)Question answering · Retrieval

    Paper

Video & streaming23

  • EgoExoMem

    EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos

    Real dataQuestion answering

    Paper
  • EgoLifeQA

    EgoLife: Towards Egocentric Life Assistant

    Real dataQuestion answering · Retrieval

    Paper
  • EgoMem (VideoLucy)

    VideoLucy: Deep Memory Backtracking for Long Video Understanding

    Real dataQuestion answering · Generation

    Paper
  • EgoMemReason

    EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding

    Real dataQuestion answering · Retrieval

    Paper
  • EgoMonth

    EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory

    Real dataQuestion answering · Retrieval

    Paper
  • ELViM

    video-SALMONN S: Memory-Enhanced Streaming Audio-Visual LLM

    Real dataQuestion answering · Learning

    Paper
  • HippoVlog

    HippoMM: Hippocampal-inspired Multimodal Memory for Long Audiovisual Event Understanding

    Real dataQuestion answering · Retrieval

    Paper
  • LVOmniBench

    LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs

    Real dataQuestion answering

    Paper
  • M3-Bench

    Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory

    Real dataQuestion answering

    Paper
  • M3Eval

    M$^3$Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

    Hybrid (real + synthetic/simulated)Question answering

    Paper
  • MA-EgoQA

    MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents

    Real dataQuestion answering · Retrieval

    Paper
  • MM-Lifelong

    Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baseline

    Hybrid (real + synthetic/simulated)Question answering · Localization

    Paper
  • RIVER Bench

    RIVER: A Real-Time Interaction Benchmark for Video LLMs

    Hybrid (real + synthetic/simulated)Question answering · Action

    Paper
  • S-EMBER

    S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval

    Real dataQuestion answering · Retrieval

    Paper
  • SLVMBench

    SLVMBench: Skill Learning from Video Memory

    Real dataQuestion answering · Learning

    Paper
  • StreamArena

    StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

    Real dataQuestion answering · Action

    Paper
  • StreamBench

    Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

    Hybrid (real + synthetic/simulated)Question answering · Generation

    Paper
  • StreamingBench

    StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

    Real dataQuestion answering

    Paper
  • StreamMemBench

    StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance

    Real dataQuestion answering · Action · Learning

    Paper
  • SuperMemory-VQA

    SuperMemory-VQA: An Egocentric Visual Question-Answering Benchmark for Long-Horizon Memory

    Real dataQuestion answering · Retrieval

    Paper
  • TeleEgo

    TeleEgo: Benchmarking Egocentric AI Assistants in the Wild

    Real dataQuestion answering

    Paper
  • VideoWebArena

    VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks

    Hybrid (real + synthetic/simulated)Question answering · ActionCross-domain

    Paper
  • VStream-QA

    Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams

    Real dataQuestion answering

    Paper

Audio & speech10

  • Agentic ASR evaluation

    Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation

    Hybrid (real + synthetic/simulated)Generation · Revision

    Paper
  • AudioMarathon

    AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs

    Hybrid (real + synthetic/simulated)Question answering · Reasoning

    Paper
  • CarMem (text-only)

    CarMem: Enhancing Long-Term Memory in LLM Voice Assistants through Category-Bounding

    SyntheticRetrieval · Learning

    Paper
  • ChatMem-Bench

    VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction

    SyntheticQuestion answering · Generation

    Paper
  • ChronosAudio

    ChronosAudio: A Comprehensive Long-Audio Benchmark for Evaluating Audio-Large Language Models

    Hybrid (real + synthetic/simulated)Question answering · Classification

    Paper
  • EgoMem evaluation

    EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models

    Hybrid (real + synthetic/simulated)Retrieval · Generation · Learning

    Paper
  • H2HMem

    H2HMem: A Multimodal Memory Benchmark for Agents in Human-Human Interactions

    Hybrid (real + synthetic/simulated)Question answering · Learning

    Paper
  • LongAudioQA

    Don't Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understanding

    Real dataQuestion answering · Retrieval

    Paper
  • PlanRAG-Audio

    PlanRAG-Audio: Planning and Retrieval Augmented Generation for Long-form Audio Understanding

    Real dataQuestion answering · Retrieval

    Paper
  • Stateful streaming ASR

    Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents

    Real dataGeneration · State tracking

    Paper

GUI & computer use7

  • DataScope

    What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States

    Hybrid (real + synthetic/simulated)Action

    Paper
  • DMV-Bench

    DMV-Bench: Diagnosing Long-Horizon Multimodal Agents' Visual Memory with Incidental Cue Injection

    SimulatedRetrieval · Action

    Paper
  • EMemBench

    EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents

    SimulatedQuestion answering · Action

    Paper
  • KnowU-Bench

    KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation

    Hybrid (real + synthetic/simulated)Action · Planning

    Paper
  • MementoGUI-Bench

    MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents

    Real dataAction · Learning

    Paper
  • MemGUI-Bench

    MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments

    Real dataAction

    Paper
  • Memory-World

    STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments

    SimulatedAction

    Paper

Embodied & 3D23

  • 3DMem-Bench

    3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model

    SimulatedQuestion answering · Planning · Action

    Paper
  • APRS

    Seek to Segment: Active Perception for Panoramic Referring Segmentation

    Hybrid (real + synthetic/simulated)Localization · Action · Segmentation

    Paper
  • DunphyBench

    Long-Horizon Embodied Decision-Making via Multimodal Memory Compression

    SimulatedPlanning · Action

    Paper
  • EmbodiedWorldBench

    ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

    SimulatedAction · Learning

    Paper
  • eMEM-Bench v1

    eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents

    SimulatedQuestion answering · Retrieval

    Paper
  • FindingDory

    FindingDory: A Benchmark to Evaluate Memory in Embodied Agents

    SimulatedQuestion answering · Localization · Action

    Paper
  • HIOcc

    GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

    Real dataMapping

    Paper
  • LIBERO-Mem

    Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective

    SimulatedAction

    Paper
  • LMEE-Bench

    Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Exploration

    SimulatedQuestion answering · Action

    Paper
  • LongSpace-Bench

    LongSpace: Exploring Long-Horizon Spatial Memory from Perception to Recall in Video

    Real dataQuestion answering

    Paper
  • MEMORA-Bench

    MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning

    Real dataQuestion answering · Planning

    Paper
  • MIKASA-Robo (RGB+joints)

    Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning

    SimulatedAction

    Paper
  • ReMemBench

    Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks

    SimulatedAction

    Paper
  • RMBench

    RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design

    SimulatedAction

    Paper
  • RoboGraph benchmark

    Compiling and Benchmarking Task-State Horizons for Embodied Agents

    SimulatedPlanning · Action

    Paper
  • RoboMemArena

    RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark

    Hybrid (real + synthetic/simulated)Action · Learning

    Paper
  • RoboMME

    RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

    Hybrid (real + synthetic/simulated)Action

    Paper
  • RoboMME-Interference

    RoboMME-Interference: Benchmarking Robot Memory Under Interference

    SimulatedAction

    Paper
  • SEER-Bench

    Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Exploration

    SimulatedQuestion answering · Planning · Action

    Paper
  • SIS-Bench

    Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

    Real dataQuestion answering · Planning

    Paper
  • SpaMEM

    SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments

    SimulatedQuestion answering · Action

    Paper
  • WorldLines (semantic states)

    WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents

    SimulatedQuestion answering · Planning

    Paper
  • WorldMemArena

    WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction

    Hybrid (real + synthetic/simulated)Question answering · Action · LearningCross-domain

    Paper
Compare capabilitiesThe survey’s full coverage table: six shared and three modality-specific capabilities per benchmark.

● Direct◐ Partial Not evaluated Click a column to sort.

Benchmark
ATM-BenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF directly evaluatedUQ directly evaluatedMA not evaluatedFG directly evaluatedET directly evaluatedVC directly evaluated
CamRollCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG directly evaluatedET directly evaluatedVC partially evaluated
DISBenchCE partially evaluatedLE directly evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET directly evaluatedVC partially evaluated
LCMPCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET directly evaluatedVC directly evaluated
LoCoMoCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ directly evaluatedMA not evaluatedFG not evaluatedET not evaluatedVC not evaluated
M2CA-BenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET directly evaluatedVC partially evaluated
M3ExamCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG directly evaluatedET directly evaluatedVC partially evaluated
Mem-GalleryCE directly evaluatedLE directly evaluatedSC directly evaluatedCF directly evaluatedUQ directly evaluatedMA not evaluatedFG partially evaluatedET partially evaluatedVC directly evaluated
MemEyeCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG directly evaluatedET partially evaluatedVC directly evaluated
MemLeakCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET not evaluatedVC not evaluated
MemLensCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ directly evaluatedMA not evaluatedFG directly evaluatedET partially evaluatedVC directly evaluated
Memory-QACE partially evaluatedLE directly evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG directly evaluatedET not evaluatedVC partially evaluated
MMA-BenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF directly evaluatedUQ directly evaluatedMA not evaluatedFG partially evaluatedET not evaluatedVC directly evaluated
MMDUCE directly evaluatedLE partially evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET partially evaluatedVC not evaluated
MMRCCE directly evaluatedLE partially evaluatedSC directly evaluatedCF not evaluatedUQ directly evaluatedMA not evaluatedFG partially evaluatedET partially evaluatedVC directly evaluated
Persona-MMECE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET directly evaluatedVC directly evaluated
RefMem-BenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG partially evaluatedET partially evaluatedVC partially evaluated
SMMBenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF directly evaluatedUQ not evaluatedMA directly evaluatedFG partially evaluatedET directly evaluatedVC partially evaluated
VisualMemCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedFG directly evaluatedET not evaluatedVC partially evaluated
EgoExoMemCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
EgoLifeQACE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
EgoMem (VideoLucy)CE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
EgoMemReasonCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
EgoMonthCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
ELViMCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
HippoVlogCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
LVOmniBenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL partially evaluatedTR directly evaluatedME directly evaluated
M3-BenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
M3EvalCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL partially evaluatedTR directly evaluatedME directly evaluated
MA-EgoQACE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
MM-LifelongCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
RIVER BenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
S-EMBERCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
SLVMBenchCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
StreamArenaCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
StreamBenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR partially evaluatedME directly evaluated
StreamingBenchCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR partially evaluatedME directly evaluated
StreamMemBenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
SuperMemory-VQACE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
TeleEgoCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR directly evaluatedME directly evaluated
VideoWebArenaCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedTL partially evaluatedTR directly evaluatedME directly evaluated
VStream-QACE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedTL directly evaluatedTR partially evaluatedME directly evaluated
Agentic ASR evaluationCE directly evaluatedLE partially evaluatedSC directly evaluatedCF directly evaluatedUQ not evaluatedMA not evaluatedSP not evaluatedAC partially evaluatedRP directly evaluated
AudioMarathonCE partially evaluatedLE directly evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSP partially evaluatedAC directly evaluatedRP not evaluated
CarMem (text-only)CE not evaluatedLE directly evaluatedSC directly evaluatedCF directly evaluatedUQ not evaluatedMA not evaluatedSP not evaluatedAC not evaluatedRP partially evaluated
ChatMem-BenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF partially evaluatedUQ directly evaluatedMA not evaluatedSP directly evaluatedAC directly evaluatedRP partially evaluated
ChronosAudioCE partially evaluatedLE directly evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSP partially evaluatedAC directly evaluatedRP not evaluated
EgoMem evaluationCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA partially evaluatedSP directly evaluatedAC partially evaluatedRP partially evaluated
H2HMemCE directly evaluatedLE directly evaluatedSC directly evaluatedCF directly evaluatedUQ directly evaluatedMA not evaluatedSP partially evaluatedAC not evaluatedRP partially evaluated
LongAudioQACE directly evaluatedLE directly evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSP directly evaluatedAC directly evaluatedRP not evaluated
PlanRAG-AudioCE directly evaluatedLE directly evaluatedSC not evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSP directly evaluatedAC directly evaluatedRP not evaluated
Stateful streaming ASRCE not evaluatedLE partially evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSP partially evaluatedAC partially evaluatedRP not evaluated
DataScopeCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedGS directly evaluatedAO directly evaluatedXR not evaluated
DMV-BenchCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedGS directly evaluatedAO partially evaluatedXR directly evaluated
EMemBenchCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ directly evaluatedMA directly evaluatedGS partially evaluatedAO directly evaluatedXR not evaluated
KnowU-BenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedGS directly evaluatedAO partially evaluatedXR directly evaluated
MementoGUI-BenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedGS directly evaluatedAO directly evaluatedXR directly evaluated
MemGUI-BenchCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedGS directly evaluatedAO directly evaluatedXR directly evaluated
Memory-WorldCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedGS directly evaluatedAO directly evaluatedXR partially evaluated
3DMem-BenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS partially evaluated
APRSCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS partially evaluatedTS partially evaluated
DunphyBenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS partially evaluatedTS directly evaluated
EmbodiedWorldBenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
eMEM-Bench v1CE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSM directly evaluatedOS partially evaluatedTS partially evaluated
FindingDoryCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS partially evaluated
HIOccCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSM directly evaluatedOS partially evaluatedTS not evaluated
LIBERO-MemCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
LMEE-BenchCE directly evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
LongSpace-BenchCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA not evaluatedSM directly evaluatedOS directly evaluatedTS partially evaluated
MEMORA-BenchCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ directly evaluatedMA directly evaluatedSM partially evaluatedOS partially evaluatedTS partially evaluated
MIKASA-Robo (RGB+joints)CE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
ReMemBenchCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM partially evaluatedOS directly evaluatedTS directly evaluated
RMBenchCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM partially evaluatedOS directly evaluatedTS directly evaluated
RoboGraph benchmarkCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
RoboMemArenaCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
RoboMMECE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
RoboMME-InterferenceCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM partially evaluatedOS directly evaluatedTS directly evaluated
SEER-BenchCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
SIS-BenchCE partially evaluatedLE directly evaluatedSC partially evaluatedCF not evaluatedUQ not evaluatedMA partially evaluatedSM directly evaluatedOS partially evaluatedTS partially evaluated
SpaMEMCE partially evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS partially evaluated
WorldLines (semantic states)CE not evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
WorldMemArenaCE directly evaluatedLE directly evaluatedSC directly evaluatedCF not evaluatedUQ not evaluatedMA directly evaluatedSM directly evaluatedOS directly evaluatedTS directly evaluated
CE
Cross-modal evidence
LE
Long-range evidence
SC
State changes
CF
Conflicting memories
UQ
Unanswerable queries
MA
Memory-grounded action
M1
Modality-specific 1
M2
Modality-specific 2
M3
Modality-specific 3