Multimodal Agent MemorySubmit a paper

OdysseyAgent

GUI Odyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices

Quanfeng Lu, Wenqi Shao, Zitao Liu, Lingxiao Du, Fanqing Meng, Boxuan Li, et al. · 2025-08-01

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
GUI & computer use
Also: text
Representation
Hybrid
Compressed historical screenshot tokens and the textual previous-action record are jointly concatenated into the agent input for next-action prediction.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. history screenshot memory: tokens of the previous delta=4 screenshots, compressed by a learned history resampler and concatenated with the current screen tokens, instruction and previous actions

    1. Represented asModality-specific encodings & geometric stateSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource linked & recoverable
    OdysseyAgent stores the previous screenshots of the episode and compresses their visual tokens with a single-layer cross-attention 'history resampler' whose learnable embeddings act as queries and whose keys/values are the historical screenshot tokens; the resulting 256 compressed history tokens (vs 1,024 for naive multi-image input) are concatenated with the current screenshot tokens, the instruction and the previous actions before the LLM decodes the next action. Default history length is delta=4. The raw historical screenshots exist in the trajectory and are re-encoded, so the perceptual source is recoverable but is not itself stored as memory content.Method / OdysseyAgent architecture (history resampler) · Architecture figure of OdysseyAgent; Table 4 (ablation on historical information)

  2. previous-action history string carried alongside the compressed screenshot history

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inFlat sequential records
    3. Retrieved byLatent / parametric activation
    4. Source evidenceSource not retained
    The agent input explicitly includes the previous actions of the episode together with the compressed historical screenshot tokens; the ablation removes previous actions and historical screenshots as distinct history sources, showing both contribute to cross-app navigation accuracy.Method / OdysseyAgent architecture; ablation · Table 4 (previous actions and historical screenshots ablated separately)

Related systems