Multimodal Agent MemorySubmit a paper

ToolMerge

Decomposing Queries into Tool Calls for Long-Video Keyframe Retrieval

Michal Shlapentokh-Rothman, Prachi Garg, Yu-Xiong Wang, Derek Hoiem · 2026-07-06

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Video & streaming
Also: Image, text
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. ranked keyframes

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource directly available
    An LLM decomposes the query into visual tool calls and Boolean-merges their keyframe rankings.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1

Related systems