Multimodal Agent MemorySubmit a paper

Co-Director

Co-Director: Agentic Generative Video Storytelling

Yale Song, Yiwen Song, Nick Losier, Nathan Hodson, Ye Jin, Rhyard Zhu, et al. · 2026-04-27

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Approved-keyframe sequence

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Keyframe images; Approval / flag annotations. Approved keyframes are reused; flagged keyframes are regenerated and the shots concatenated.Full-paper method / memory representation and retrieval · PDF p. 5 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    Flagged keyframes are regenerated while approved ones are kept (mutation by a fixed review rule). For Co-Director, this specifically supports LLM-mediated memory editing of Approved-keyframe sequence.

  • Revision · Evidence-weighted belief revision

    During operation, Co-Director applies evidence-weighted revision of stored belief or confidence to Approved-keyframe sequence; the memory state changes while the governing update policy remains fixed.

Related systems