Pith. sign in

REVIEW 12 cited by

AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.17490 v2 pith:2QFRJVZ2 submitted 2024-07-03 cs.HC cs.AIcs.MM

AMEX: Android Multi-annotation Expo Dataset for Mobile GUI Agents

Yuxiang Chai , Siyuan Huang , Yazhe Niu , Han Xiao , Liang Liu , Dingyu Zhang , Shuai Ren , Hongsheng Li This is my paper
classification cs.HC cs.AIcs.MM
keywords amexmobileagentsdatasetandroiddatasetselementexisting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

AI agents have drawn increasing attention mostly on their ability to perceive environments, understand tasks, and autonomously achieve goals. To advance research on AI agents in mobile scenarios, we introduce the Android Multi-annotation EXpo (AMEX), a comprehensive, large-scale dataset designed for generalist mobile GUI-control agents which are capable of completing tasks by directly interacting with the graphical user interface (GUI) on mobile devices. AMEX comprises over 104K high-resolution screenshots from popular mobile applications, which are annotated at multiple levels. Unlike existing GUI-related datasets, e.g., Rico, AitW, etc., AMEX includes three levels of annotations: GUI interactive element grounding, GUI screen and element functionality descriptions, and complex natural language instructions with stepwise GUI-action chains. We develop this dataset from a more instructive and detailed perspective, complementing the general settings of existing datasets. Additionally, we finetune a baseline model SPHINX Agent and illustrate the effectiveness of AMEX.The project is available at https://yxchai.com/AMEX/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents

    cs.AI 2025-12 accept novelty 8.0

    MobiBench is the first modular multi-path offline benchmark for mobile GUI agents, achieving 94.72% agreement with human evaluators while allowing component-level analysis.

  2. Grounding Computer Use Agents on Human Demonstrations

    cs.LG 2025-11 conditional novelty 7.0

    GroundCUA, a 3.56M-element human-annotated desktop grounding dataset, and GroundNext models achieve strong UI grounding with less than one-tenth the SFT data of prior work.

  3. GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

    cs.CV 2025-11 conditional novelty 7.0

    Supervising an MLLM's intrinsic self-attention with patch-level GUI labels, aggregated via a learnable anchor token and hidden-state-selected query tokens, reaches state-of-the-art 3B-scale GUI grounding accuracy with...

  4. GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents

    cs.CV 2025-04 unverdicted novelty 7.0

    GUI-R1 uses reinforcement fine-tuning with GRPO on a small curated dataset to create a generalist vision-language action model that outperforms prior GUI agent methods across mobile, desktop, and web benchmarks using ...

  5. MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents

    cs.AI 2025-12 conditional novelty 6.0

    MobiBench reaches near-human offline evaluation fidelity for mobile GUI agents by accepting any valid action at each step, and enables modular attribution of performance to agent components.

  6. RISK: A Framework for GUI Agents in E-commerce Risk Management

    cs.AI 2025-09 unverdicted novelty 6.0

    RISK introduces a dataset, benchmark, and R1-style RL fine-tuning for GUI agents that achieve 6.8-8.8% offline gains and 70.5% online task success in e-commerce risk management using 7.2% of baseline parameters.

  7. UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning

    cs.AI 2025-03 accept novelty 6.0

    UI-R1 shows rule-based RL with GRPO on 136 GUI tasks improves a 3B MLLM's action prediction accuracy by 6-22% over its base model and matches larger SFT-trained models.

  8. Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling

    cs.CV 2024-12 unverdicted novelty 6.0

    InternVL 2.5 is the first open-source MLLM to surpass 70% on the MMMU benchmark via model, data, and test-time scaling, with a 3.7-point gain from chain-of-thought reasoning.

  9. Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction

    cs.CL 2024-12 conditional novelty 6.0

    Aguvis presents a pure vision-based framework for autonomous GUI agents using structured reasoning via inner monologue, a new multimodal dataset, and two-stage training to reach SOTA on offline and online benchmarks.

  10. OS-ATLAS: A Foundation Action Model for Generalist GUI Agents

    cs.CL 2024-10 unverdicted novelty 6.0

    OS-Atlas, trained on the largest open-source cross-platform GUI grounding corpus of 13 million elements, outperforms prior open-source models on six benchmarks across mobile, desktop, and web platforms.

  11. StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design

    cs.CV 2026-07 conditional novelty 5.0

    A co-designed 0.9B UI vision-language model tops small-model benchmarks on screen Q&A and Chinese OCR and runs on a Snapdragon 8 Gen5 at 98 tokens/second.

  12. InquireMobile: Teaching VLM-based Mobile Agent to Request Human Assistance via Reinforcement Fine-Tuning

    cs.AI 2025-08 unverdicted novelty 5.0

    InquireMobile applies two-stage reinforcement fine-tuning and pre-action reasoning to VLM mobile agents, raising inquiry success rate by 46.8% on the introduced InquireBench benchmark.