Multimodal Agent MemorySubmit a paper

DreamFactory

DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework

Zhifei Xie, Daniel Tang, Dingwei Tan, Jacques Klein, Tegawend F. Bissyand, Saad Ezzini · 2024-08-21

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: text, Image
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Long- and short-term visual memory of generated scenes/frames

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Planner / tool-assisted retrieval
    4. Source evidenceSource directly available
    Image vector-DB entries; Base-frame references. New scenes are conditioned on retrieved reference frames; Monitor agent checks the new scene against stored frames.Full paper: Algorithm 2 Calculate CSSC Score · PDF p. 8 · Figure 1 / method overview

How memory changes (T2 operations)

  • Update · Rule-based update

    Short-term memory turns over and the Monitor agent re-conditions scenes; a fixed monitoring rule. For DreamFactory, this specifically supports fixed-rule insertion or replacement of Long- and short-term visual memory of generated scenes/frames.

Related systems