LoCoMo
Evaluating Very Long-Term Conversational Memory of LLM Agents
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, Yuwei Fang · 2024-02-27
- Modality
- Image
- Data
- Hybrid (real + synthetic/simulated)
- Task · ground truth
- QA/Gen. · Ans.+Ev.
- Scope
- Memory-specific benchmark
What it tests
- CECross-modal evidencePartially evaluated
- LELong-range evidenceDirectly evaluated
- SCState changesPartially evaluated
- CFConflicting memoriesNot evaluated
- UQUnanswerable queriesDirectly evaluated
- MAMemory-grounded actionNot evaluated
- FGFine-grained visual detailNot evaluated
- ETEntity trackingNot evaluated
- VCVisual state changeNot evaluated
Survey notes
Image-sharing conversations support long-term QA, event summarization and multimodal dialogue generation, including retrieval-based memory comparisons.