Multimodal Agent MemorySubmit a paper

GUI-explorer

GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agent

Bin Xie, Rui Shao, Gongwei Chen, Kaiwen Zhou, Yinchuan Li, Jie Liu, et al. · 2025-05-22

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
GUI & computer use
Also: text
Representation
Hybrid
Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. exploration screenshots and trajectories

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inFlat sequential records
    3. Retrieved byMetadata / spatiotemporal retrieval
    4. Source evidenceSource directly available
    Autonomous exploration collects screenshots and function-aware interaction trajectories.Full-paper method / memory representation and retrieval · PDF p. 3 · Figure 1

  2. transition-aware UI knowledge

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Observation-action-outcome triples are mined into reusable screen-operation logic.Full-paper method / memory representation and retrieval · PDF p. 2 · Figure 1

Related systems