Multimodal Agent MemorySubmit a paper

MemCtrl

MemCtrl: Using MLLMs as Active Memory Controllers on Embodied Agents

Vishnu Sashank Dorbala, Dinesh Manocha · 2026-01-28

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Embodied & 3D
Also: Image, text
Representation
Source- and modality-specific

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. retained embodied observations

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    A memory head gates which observations or reflections to retain, update, or discard.Full paper: Memory-Augmented MLLM Agents · PDF p. 4 · Figure 1

How memory changes (T2 operations)

  • Update · Learned update module

    The deployed fixed head controls writes. For MemCtrl, this specifically supports a trained but frozen update module of retained embodied observations.

Related systems