Pith. sign in

REVIEW 5 cited by

ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.08199 v4 pith:JOLKPA55 submitted 2022-09-16 cs.CL cs.CVcs.HC

Yu-Chung Hsiao , Fedir Zubach , Gilles Baechler , Srinivas Sunkara , Victor Carbune , Jason Lin , Maria Wang , Yun Zhu
show 1 more author
Jindong Chen
This is my paper · ORCID
classification cs.CLcs.CVcs.HC
keywords datasetscreenscreenqaapplicationsmobilepairsquestion-answerscreenshots
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce ScreenQA, a novel benchmarking dataset designed to advance screen content understanding through question answering. The existing screen datasets are focused either on low-level structural and component understanding, or on a much higher-level composite task such as navigation and task completion for autonomous agents. ScreenQA attempts to bridge this gap. By annotating 86k question-answer pairs over the RICO dataset, we aim to benchmark the screen reading comprehension capacity, thereby laying the foundation for vision-based automation over screenshots. Our annotations encompass full answers, short answer phrases, and corresponding UI contents with bounding boxes, enabling four subtasks to address various application scenarios. We evaluate the dataset's efficacy using both open-weight and proprietary models in zero-shot, fine-tuned, and transfer learning settings. We further demonstrate positive transfer to web applications, highlighting its potential beyond mobile applications.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contexts

    cs.CV 2025-08 conditional novelty 6.0 of 10

    KRETA, a 2,577-item Korean text-rich VQA benchmark, shows vision-language models recognize Korean text well but lag in multi-step reasoning, especially in open-source models.

  2. MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents

    cs.CV 2025-07 conditional novelty 6.0 of 10

    MMBench-GUI provides a multi-platform, four-level benchmark and an efficiency-aware metric, and its experiments indicate visual grounding is the main bottleneck for current GUI agents.

  3. ZeroGUI: Automating Online GUI Learning at Zero Human Cost

    cs.AI 2025-05 conditional novelty 6.0 of 10

    ZeroGUI uses VLM-generated tasks and VLM-estimated rewards with two-stage GRPO to improve GUI agent success rates on OSWorld and AndroidLab without human annotations.

  4. StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A co-designed 0.9B UI vision-language model tops small-model benchmarks on screen Q&A and Chinese OCR and runs on a Snapdragon 8 Gen5 at 98 tokens/second.

  5. Atomic-to-Compositional Generalization for Mobile Agents with A New Benchmark and Scheduling System

    cs.CL 2025-06 conditional novelty 5.0 of 10

    AI phone agents succeed on single-step commands but fail on multi-step compositions; a new benchmark measures this gap and a scheduler that decomposes tasks recovers most of it.

Pith tools