Mirage-1
Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills
Yuquan Xie, Zaijing Li, Rui Shao, Gongwei Chen, Kaiwen Zhou, Yinchuan Li, et al. · 2025-06-12
- Management
- T2 Dynamic Memory
- Memory changes during deployment through a stable management policy.
- Representation
- Hybrid
- Distinct operative representation paths are explicitly linked for downstream retrieval, reasoning, or control.
How experience becomes memory
Each path is one memory component, traced from what is stored to how it is found again.
multimodal execution-skill trajectories
- Represented asRetained perceptual source evidenceSource- and modality-specific
- Stored inHierarchical multi-tier store
- Retrieved bySimilarity search, Planner / tool-assisted retrieval
- Source evidenceSource directly available
Execution skills retain multimodal historical trajectories and step goals.Full paper: Update HMS · PDF p. 3 · Figure 1
core and meta skill hierarchy
- Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
- Stored inHierarchical multi-tier store
- Retrieved byStructure traversal, Planner / tool-assisted retrieval
- Source evidenceSource linked & recoverable
Trajectories are progressively abstracted into core skills and meta-skills for planning.Full paper: Update HMS · PDF p. 3 · Figure 1
How memory changes (T2 operations)
Consolidation · Experience abstraction
Historical trajectories are progressively abstracted into reusable skill levels. For Mirage-1, this specifically supports abstraction of episodes into reusable experience of multimodal execution-skill trajectories.
Consolidation · Episodic / hierarchical summarization
Execution, core, and meta skills form a consolidated hierarchy. For Mirage-1, this specifically supports episodic or hierarchical summarization of multimodal execution-skill trajectories.