REVIEW 28 cited by
GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devices
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Autonomous Graphical User Interface (GUI) navigation agents can enhance user experience in communication, entertainment, and productivity by streamlining workflows and reducing manual intervention. However, prior GUI agents often trained with datasets comprising tasks that can be completed within a single app, leading to poor performance in cross-app navigation. To address this problem, we present GUIOdyssey, a comprehensive dataset for cross-app mobile GUI navigation. GUIOdyssey comprises 8,334 episodes with an average of 15.3 steps per episode, covering 6 mobile devices, 212 distinct apps, and 1,357 app combinations. Each step is enriched with detailed semantic reasoning annotations, which aid the model in building cognitive processes and enhancing its reasoning abilities for complex cross-app tasks. Building on GUIOdyssey, we develop OdysseyAgent, an exploratory multimodal agent for long-step cross-app navigation equipped with a history resampler module that efficiently attends to historical screenshot tokens, balancing performance and inference speed. Extensive experiments conducted in both in-domain and out-of-domain scenarios validate the effectiveness of our approach. Moreover, we demonstrate that historial information involving actions, screenshots and context in our dataset can significantly enhances OdysseyAgent's performance on complex cross-app tasks.
Forward citations
Cited by 28 Pith papers
-
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
TSHA is a new 80,000-pair benchmark for indoor safety hazard assessment; current vision-language models score roughly 45-85, and fine-tuning on TSHA raised Qwen2.5-VL-3B by 18.3 points on TSHA's test set.
-
GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL
Action-weighted SFT plus KL-regularized GRPO on a curated 81K reasoning dataset lifts open-source GUI agents by 11–22 points on online task-completion benchmarks.
-
MobiAgent: A Systematic Framework for Customizable Mobile Agents
A full-stack mobile agent framework reports state-of-the-art task completion on its own DAG-based benchmark and 2-3x speedups from replaying recorded trajectories.
-
UItron: Foundational GUI Agent with Advanced Perception and Planning
UItron, trained with multi-stage SFT and curriculum RL on a new 1M-step Chinese app dataset, beats prior GUI agents on perception, grounding, planning, and especially Chinese mobile app scenarios.
-
PG-Agent: An Agent Powered by Page Graph
An MLLM GUI agent that stores past episodes as a page graph and retrieves action guidelines from it improves step success on three benchmarks.
-
Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents
A new 7,138-scenario benchmark shows mainstream smartphone AI agents frequently fail to notice or warn about private information.
-
SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience
A self-evolving computer-use agent trained with full-trajectory state judging and curriculum task generation goes from 11.3% to 34.5% average success on five OSWorld apps, and a specialist-to-generalist variant beats ...
-
MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents
MMBench-GUI provides a multi-platform, four-level benchmark and an efficiency-aware metric, and its experiments indicate visual grounding is the main bottleneck for current GUI agents.
-
OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?
A new 416-task desktop benchmark with a five-level automation taxonomy and a demand-based generalization hierarchy shows current computer-using agents succeed on roughly one in nine tasks, with near-zero success on hi...
-
Chain-of-Memory: Enhancing GUI Agents for Cross-Application Navigation
Explicitly storing short-term and long-term text memories, instead of raw screenshots, improves GUI agent accuracy on cross-app tasks, and a new annotated dataset helps 7B models approach 72B-level memory use.
-
GUI-Robust: A Comprehensive Dataset for Testing GUI Agent Robustness in Real-World Anomalies
A new benchmark with 5,318 GUI tasks, including 200 abnormal ones, shows that state-of-the-art GUI agents degrade sharply when real-world anomalies appear.
-
What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities
A self-generating graph benchmark produces 36k GUI agent tasks with controllable complexity and ten capability scores, and fine-tuning on its trajectories gives small gains on AndroidControl and OmniAct.
-
GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents
An attention-based action head with multi-patch supervision outperforms coordinate-generation baselines on GUI grounding, and a verifier further improves accuracy.
-
ZeroGUI: Automating Online GUI Learning at Zero Human Cost
ZeroGUI uses VLM-generated tasks and VLM-estimated rewards with two-stage GRPO to improve GUI agent success rates on OSWorld and AndroidLab without human annotations.
-
TransBench: Breaking Barriers for Transferable Graphical User Interface Agents in Dynamic Digital Environments
TransBench is a new benchmark of 1,459 screenshots and 22,000 grounding instructions for measuring how well GUI agents transfer across app versions, platforms, and applications.
-
LearnAct: Few-Shot Mobile GUI Agent with a Unified Demonstration Benchmark
Few-shot human demonstrations, parsed into structured descriptions and retrieved by instruction similarity, substantially raise mobile GUI agent accuracy and success rates across multiple models.
-
MobileA3gent: Training Mobile GUI Agents Using Decentralized Self-Sourced Data from Diverse Users
A hierarchical auto-annotation pipeline plus episode-aware federated aggregation lets mobile GUI agents be trained on automatically labeled user trajectories at about 1% of human annotation cost.
-
OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation
A large new benchmark and an offline judge model for open-ended interleaved image-text generation, with IntJudge matching human agreement better than GPT-4o.
-
ShowUI: One Vision-Language-Action Model for GUI Visual Agent
ShowUI is a lightweight 2B vision-language-action model that uses UI-guided token pruning and a curated 256K dataset to reach 75.1% zero-shot screenshot grounding accuracy.
-
Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents
A real-device-centric foundation GUI agent with hybrid GUI+CLI batched actions, AutoResearch data flywheel, online RL, and a proactive harness reaches SOTA mobile and competitive desktop/web scores.
-
Software Engineering for and with GUI Agent
A survey of 336 GUI-agent papers finds rapid growth alongside weak engineering support for recovery, human oversight, maintainability, and privacy, and calls for lifecycle-centered testing and governance.
-
Qwen-Audio-VAE Technical Report
A 12.5 Hz continuous audio VAE reconstructs speech, music, and sound well while encoding 64×30s clips in 541 ms after latency-aware encoder pruning.
-
Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Scheduling System
AI phone agents succeed on single-step commands but fail on multi-step compositions; a new benchmark measures this gap and a scheduler that decomposes tasks recovers most of it.
-
GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior
GUI-Reflection trains an 8B multimodal GUI agent to recognize mistakes, undo incorrect actions, and retry, improving AndroidWorld success rate from 14.58% (filtered BC baseline) to 34.72% with reflection data and onli...
-
Guiding VLM Agents with Process Rewards at Inference Time for GUI Navigation
GuidNav improves GUI navigation by reranking top-k VLM action candidates with a learned process reward model, yielding about 3.4% higher static action accuracy and up to around 33% relative dynamic task success gains.
-
Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
A GUI-controlling visual agent that uses edge-density-based cropping and a self-training loop matches or beats models trained on 10x more data.
-
SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control
A multi-agent RL workflow that interleaves single-agent updates, applied to mobile GUI control, achieves SOTA zero-shot performance and a +14.8 MATH500 gain.
-
Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms
A survey that proposes the Generalist Virtual Agent concept and taxonomies for agent environments, tasks, perceptions, actions, models, and evaluation, concluding that real-world-like environments favor human-like int...
Discussion (0). Continue with ORCID to comment.