Multimodal Agent MemorySubmit a paper

IMPACT-CYCLE

IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory

Weitong Kong, Di Wen, Kunyu Peng, David Schneider, Zeyun Zhong, Alexander Jaus, et al. · 2026-04-22

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Video & streaming
Also: Image, text
Representation
Hybrid
Distinct operative representation paths are deliberately linked or jointly consumed by downstream reasoning, generation, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. retained keyframe set {I_t} selected from the long video by motion-driven temporal segmentation

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    K in {3,...,5} keyframes per video are selected from raw video V by motion-driven temporal segmentation and retained as the visual evidence base. In every verification round r the role-specialized agents are re-supplied with the keyframe images together with the current graph state G^(r-1) (Eq. 4), so the retained pixel-level evidence is queried repeatedly across rounds rather than consumed once.Full paper (arXiv HTML v1): Section III-B, Semantic Memory Construction · Eq. 4 · Eq. 1-4

  2. shared semantic memory: frame-level scene-graph slices G_t (entities N_t, relations E_t, attributes A_t), typed claim set C, and claim dependency graph D

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Metadata / spatiotemporal retrieval
    4. Source evidenceSource linked & recoverable
    The memory M = (G, C, D, Pi) stores a scene graph as a union of frame-level slices G_t holding entities, relations and attributes (Eq. 1-2), decomposed into atomic typed claims of four kinds (existence, label, attribute, subject-predicate-object relation, Eq. 3), plus a directed dependency graph D whose edge c -> c' is added when a revision to c structurally constrains c' via shared entity identity, shared relation endpoints, or overlapping temporal extent. Access is by structural traversal: after an edit to claim set Q only the dependency closure Gamma(Q) is re-verified (Eq. 13). Claims are grounded to keyframe index t, so the structured record links back to the retained keyframe evidence.Full paper (arXiv HTML v1): Section III-A, System Overview; Section III-B, Claim Decomposition; Section III-F, Dependency-Closure Re-verification · Eq. 1-3, Eq. 13 · Eq. 1-3, Eq. 13

  3. provenance log Pi of timestamped, attributed agent decisions and human edits

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource linked & recoverable
    Every agent decision and human edit is written to Pi as a timestamped, attributed entry when a revision is committed to G. The paper states Pi does not enter the belief computation directly but supports auditability, rollback, and supervisor inspection, so it is a persistent record consumed by later memory maintenance rather than by the belief update itself.Full paper (arXiv HTML v1): Section III-A, System Overview; Section III-E, Constrained Correction

How memory changes (T2 operations)

  • Revision · Evidence-weighted belief revision

    During deployment the three verification agents emit directional evidence that is aggregated with fixed role- and claim-type-dependent weights into S+, S-, S0 (Eq. 7-9); the stored per-claim belief is then revised as p^(r)(c) = (eps + S+(c)) / (2 eps + S+(c) + S-(c)) (Eq. 10). The belief attached to each persistent claim changes across verification rounds while the aggregation weights lambda, calibrated once on held-out development data, remain fixed.

  • Revision · Rule-based state correction

    A candidate correction o-hat is written into the scene graph in place only when its score Q(o-hat | c) exceeds a fixed revision threshold theta and exceeds the current graph value; otherwise the claim is escalated. Accepted edits (agent or human) mutate the persistent claim set and scene graph, and the dependency closure Gamma(Q) of the edited claims is then re-verified, so the stored memory state is corrected at inference time under a fixed acceptance rule.

Related systems