Multimodal Agent MemorySubmit a paper

UI-KOBE

UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents

Yuxiang Chai, Han Xiao, Xinyu Fu, Jinpeng Chen, Rui Liu, Hongsheng Li · 2026-05-28

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
GUI & computer use
Also: text
Representation
Cross-represented explicit

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. app knowledge graph

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inGraph / relational store
    3. Retrieved byStructure traversal
    4. Source evidenceSource linked & recoverable
    Autonomous exploration creates UI-state nodes and executable transition edges used as runtime guidance.Full paper: Runtime Memory and Task Progress · PDF p. 6 · Figure 1

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    During operation, UI-KOBE applies LLM-mediated memory editing to app knowledge graph; the memory state changes while the governing update policy remains fixed.

Related systems