VideoWebArena
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks
Lawrence Jang, Yinheng Li, Dan Zhao, Charles Ding, Justin Lin, Paul Pu Liang, et al. · 2025-02-15
- Modality
- Video & streaming
- Data
- Hybrid (real + synthetic/simulated)
- Task · ground truth
- QA/Act. · Ans.+State
- Scope
- memory dependent task
What it tests
- CECross-modal evidencePartially evaluated
- LELong-range evidenceDirectly evaluated
- SCState changesPartially evaluated
- CFConflicting memoriesNot evaluated
- UQUnanswerable queriesNot evaluated
- MAMemory-grounded actionDirectly evaluated
- TLTemporal localizationPartially evaluated
- TRTemporal relationsDirectly evaluated
- MEMulti-event integrationDirectly evaluated
Survey notes
Previously observed videos supply facts and skills for subsequent web tasks.