Multimodal Agent MemorySubmit a paper

GraphPilot

GraphPilot: GUI Task Automation with One-Step LLM Reasoning Powered by Knowledge Graph

Mingxian Yu, Siqi Luo, Xu Chen · 2026-01-24

Read the paperPDF

Management
T1 Static Memory
Memory is built once and reused without change during deployment.
Primary modality
GUI & computer use
Also: text
Representation
Cross-represented explicit

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. app-specific UI knowledge graph

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    Offline exploration constructs a graph of pages, element functions, and transition rules used for online planning and validation.Full paper: Knowledge and Memory Representations · PDF p. 2 · Figure 1

Related systems