Multimodal Agent MemorySubmit a paper

CuriosAI CASTLE

CuriosAI Submission to the CASTLE Challenge at EgoVis 2026

Yuto Kanda, Hayato Tanoue, Takayuki Hori · 2026-05-27

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Video & streaming
Also: text
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. candidate temporal windows

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byMetadata / spatiotemporal retrieval, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Captions; transcripts; timeline records; graph cells; evidence windows. SVA hierarchically narrows to a primary window then verifies sub-windows with a VLM; TMKG locates a primary cell through graph search.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1 / method overview

  2. verified sub-window evidence

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byMetadata / spatiotemporal retrieval, Structure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource not retained
    Captions; transcripts; timeline records; graph cells; evidence windows. SVA hierarchically narrows to a primary window then verifies sub-windows with a VLM; TMKG locates a primary cell through graph search.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1 / method overview

Related systems