MMRC
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, et al. · 2025-03-08
- Modality
- Image
- Data
- Real data
- Task · ground truth
- QA/Gen. · Ans.
- Scope
- Benchmark with memory tracks
What it tests
- CECross-modal evidenceDirectly evaluated
- LELong-range evidencePartially evaluated
- SCState changesDirectly evaluated
- CFConflicting memoriesNot evaluated
- UQUnanswerable queriesDirectly evaluated
- MAMemory-grounded actionNot evaluated
- FGFine-grained visual detailPartially evaluated
- ETEntity trackingPartially evaluated
- VCVisual state changeDirectly evaluated
Survey notes
History-conditioned probes explicitly test memory recall, image management and information updates, alongside cross-turn reasoning. The former dagger is not justified.