Multimodal Agent MemorySubmit a paper

YODO (You Only Demonstrate Once)

You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration

Bowen Wen, Wenzhao Lian, Kostas Bekris, Stefan Schaal · 2022-01-30

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Embodied & 3D
Also: Image, depth
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. fixed category-level 6-DoF demonstration trajectory and task-relevant local-frame sequence

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource not retained
    A single recorded demonstration video is parsed into an ordered object-pose trajectory in the receptacle coordinate frame, lifted into a category-level representation, and reprojected for a novel object. During online execution, Algorithm 1 iterates through the resulting fixed virtual target trajectory and uses each stored pose as the next manipulation goal.Full paper: Sec. IV-B Model-free 6 DoF Object Motion Tracking; Sec. IV-C Category-Level Behavior Cloning · PDF pp. 4-6 · Figure 1; Algorithm 1
    A demonstration-derived attention heatmap and local anchor sequence preserve task-relevant geometry along the manipulation horizon; online execution transfers these fixed references to the novel object while the controller follows the stored trajectory.Full paper: Sec. IV-C Local Attention · PDF p. 6 · Figure 2; Eq. 4

Related systems