Multimodal Agent MemorySubmit a paper

DRAE

DRAE: Dynamic Retrieval-Augmented Expert Networks for Lifelong Learning and Task Adaptation in Robotics

Yayu Long, Kewei Chen, Long Jin, Mingsheng Shang · 2025-01-01

Read the paper

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Embodied & 3D
Also: text
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Retrieved external documents or knowledge embeddings

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inGraph / relational store, Vector-indexed collection
    3. Retrieved bySimilarity search, Structure traversal, Latent / parametric activation
    4. Source evidenceRetained but not accessible to the agent
    P-RAG keeps knowledge in an external corpus rather than overwriting model weights.; DPMM expands the non-parametric knowledge base when new task distributions appear.; MoE experts are routed dynamically; the paper frames DPMM as memory expansion that avoids corrupting old skills.. graph query/traversal over the memory conditioned on the current goal.Full paper: 2. A non-parametric Bayesian approach using · PDF p. 2 · Figure 1 / method overview

  2. Expert-routing state and task-specialized expert activations

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store, Vector-indexed collection
    3. Retrieved bySimilarity search, Structure traversal, Latent / parametric activation
    4. Source evidenceSource not retained
    P-RAG keeps knowledge in an external corpus rather than overwriting model weights.; DPMM expands the non-parametric knowledge base when new task distributions appear.; MoE experts are routed dynamically; the paper frames DPMM as memory expansion that avoids corrupting old skills.. graph query/traversal over the memory conditioned on the current goal.Full paper: 2. A non-parametric Bayesian approach using · PDF p. 2 · Figure 1 / method overview

  3. Hierarchical RL memory state

    1. Represented asExperience-conditioned parametric memoryInternalized latent & parametric
    2. Stored inModular adapter bank, Graph / relational store
    3. Retrieved bySimilarity search, Structure traversal, Latent / parametric activation
    4. Source evidenceSource not retained
    P-RAG keeps knowledge in an external corpus rather than overwriting model weights.; DPMM expands the non-parametric knowledge base when new task distributions appear.; MoE experts are routed dynamically; the paper frames DPMM as memory expansion that avoids corrupting old skills.. graph query/traversal over the memory conditioned on the current goal.Full paper: 2. A non-parametric Bayesian approach using · PDF p. 2 · Figure 1 / method overview

Related systems