Multimodal Agent MemorySubmit a paper

PlanRAG-Audio

PlanRAG-Audio: Planning and Retrieval Augmented Generation for Long-form Audio Understanding

Masao Someki, Chien-yu Huang, Siddhant Arora, Samuele Cornell, Markus Müller, Nathan Susanj, et al. · 2026-05-19

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
Audio & speech
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. time-aligned audio spans

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inHierarchical multi-tier store
    3. Retrieved byMetadata / spatiotemporal retrieval, Planner / tool-assisted retrieval
    4. Source evidenceSource directly available
    The database preserves query-selectable raw audio spans.Full paper: Proposed Framework · PDF p. 3 · Figure 1

  2. speech, speaker, emotion and event streams

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byMetadata / spatiotemporal retrieval, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Planning selects time-aligned heterogeneous acoustic records.Full-paper method / memory representation and retrieval · PDF p. 5 · Figure 1

Related systems