Multimodal Agent MemorySubmit a paper

StoryAgent

StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration

Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun Chang, et al. · 2024-11-11

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Video & streaming
Also: text
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Subject identity embeddings

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    LoRA-BE 16-token identity embeddings learned from reference videos; Reference subject entries. Identity embeddings and reference subjects are reused across shots.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1 / method overview

  2. Reference subject bank

    1. Represented asExperience-conditioned parametric memoryInternalized latent & parametric
    2. Stored inModular adapter bank, Vector-indexed collection
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    LoRA-BE 16-token identity embeddings learned from reference videos; Reference subject entries. Identity embeddings and reference subjects are reused across shots.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1 / method overview

Related systems