Pith. sign in

REVIEW 7 cited by

Improved GUI Grounding via Iterative Narrowing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.13591 v7 pith:GSKR7OZD submitted 2024-11-18 cs.CV cs.AIcs.CL

Improved GUI Grounding via Iterative Narrowing

Anthony Nguyen This is my paper
classification cs.CV cs.AIcs.CL
keywords groundingperformancegeneraliterativemodelsnarrowingvariousacross
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Graphical User Interface (GUI) grounding plays a crucial role in enhancing the capabilities of Vision-Language Model (VLM) agents. While general VLMs, such as GPT-4V, demonstrate strong performance across various tasks, their proficiency in GUI grounding remains suboptimal. Recent studies have focused on fine-tuning these models specifically for zero-shot GUI grounding, yielding significant improvements over baseline performance. We introduce a visual prompting framework that employs an iterative narrowing mechanism to further improve the performance of both general and fine-tuned models in GUI grounding. For evaluation, we tested our method on a comprehensive benchmark comprising various UI platforms and provided the code to reproduce our results.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs

    cs.CV 2026-05 conditional novelty 7.0

    GUI grounding in VLMs is bottlenecked by prefill-stage candidate selection that decoding cannot fix, so Re-Prefill uses attention to extract and re-inject target tokens for up to 4.3% gains on ScreenSpot-Pro.

  2. One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding

    cs.CV 2026-06 unverdicted novelty 6.0

    InnerZoom bridges cross-layer evidence in one forward pass to achieve SOTA GUI grounding accuracy on six benchmarks while cutting latency up to 31.8% versus two-pass baselines.

  3. Dive into the Scene: Breaking the Perceptual Bottleneck in Vision-Language Decision Making via Focus Plan Generation

    cs.CV 2026-06 unverdicted novelty 6.0

    SceneDiver introduces a coarse-to-fine focus plan generation approach for VLMs that constructs holistic scene graphs then iteratively decomposes tasks, plus a distillation adapter for VLAs, to reduce visual hallucinat...

  4. UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding

    cs.CV 2026-04 unverdicted novelty 6.0

    UI-Zoomer uses uncertainty quantification to trigger and size adaptive zoom-ins only on uncertain GUI grounding predictions, yielding up to 13.4% gains on benchmarks with no training.

  5. Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI Grounding

    cs.LG 2026-04 unverdicted novelty 5.0

    A co-evolving proposer-critic RL framework improves GUI grounding accuracy by letting the model critique its own proposals rendered on screenshots.

  6. How Auxiliary Reasoning Unleashes GUI Grounding in VLMs

    cs.CV 2025-09 conditional novelty 5.0

    Overlaying labeled grids and axes on screenshots substantially improves zero-shot GUI grounding in most VLMs, with the best variant zooming into grid cells.

  7. Large Language Model-Brained GUI Agents: A Survey

    cs.AI 2024-11 unverdicted novelty 4.0

    A survey consolidating frameworks, data practices, large action models, benchmarks, applications, and research gaps in LLM-brained GUI agents.