Pith. sign in

REVIEW 4 major objections 4 minor 21 references

GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GUI-Lens replaces one-shot click prediction with a sequence of model-selected zoomed views and reports grounding accuracy gains of up to 24.9 percentage points across four GUI benchmarks.

desk verdict Solid framework with real gains, but the key ablation has a broken control—send to review but insist on fixing the w/o-Cropping comparison. read the letter →

arxiv 2608.03270 v1 pith:62MWV6RB submitted 2026-08-04 cs.CV cs.AI

Zichuan Fu , Shirong Wang , Wenlin Zhang , Guojing Li , Yimin Deng , Jingtong Gao , Junjia Qi , Hanyu Yan
show 5 more authors
Yefeng Zheng Xiaopeng Li Wanyu Wang Xian Wu Xiangyu Zhao
This is my paper
classification cs.CVcs.AI
keywords GUIgroundingcoarse-to-finecroppingvision-languagemodelscoordinatereferencesvisualverificationinference-timerefinementagentsOSWorld
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GUI grounding—mapping an instruction such as 'click the Align right icon' to a precise screen coordinate—fails on dense, high-resolution interfaces because a vision-language model can recognize a target without pinpointing it. The paper argues that this failure can be addressed at inference time: instead of predicting the click directly from the full screenshot, GUI-Lens lets the same general-purpose VLM choose successively focused crops, guided by OCR text and detected UI components exposed as coordinate references, and it checks each proposed crop and click against the instruction before committing. Evaluated on four grounding benchmarks and three VLM backends, the framework reports consistent gains of up to 24.9 percentage points, with GPT-5.5 reaching the best reported accuracy on ScreenSpot-Pro and the approach transferring to interactive computer-use tasks in OSWorld. If the claim holds, precise GUI interaction improves by changing how a model looks at a screen rather than only by training better models.

What carries the argument

The central mechanism is coarse-to-fine cropping: at each round the VLM selects both the region and the enlargement scale of the next view, instead of zooming around an earlier click estimate. Coordinate references (OCR text plus detected component boxes, serialized as id, label, bounding box, and source type) and a same-backend verifier that marks and accepts or rejects each proposal support the loop. The affine map $\Phi_r$ converts every local proposal back to original-screen coordinates, so the framework returns a single click point in the environment's coordinate system. The ablation identifies the cropping loop as the primary source of accuracy gain.

What would settle it

On a held-out sample of ScreenSpot-Pro, compute the verifier's accept/reject decisions against ground-truth point-in-box labels and report rejection rate and agreement; if accepted clicks are not clearly more often correct than rejected ones, or if the pipeline with verification decisions inverted keeps accuracy essentially unchanged, then self-verification is not the mechanism carrying the reported gains.

Watch

Extended reading notes

Core claim

GUI-Lens formalizes GUI grounding as sequential visual localization over focused views rather than one-shot prediction. Given a screenshot $I$ and instruction $g$, the VLM $M$ returns an action $a_r\in\{\mathrm{crop},\mathrm{click}\}$ and a spatial proposal $z_r$ at each round $r$; an accepted crop is padded, mapped to the original screen as $B_{r+1}=\mathrm{Pad}(\Phi_r(\hat C_r))\cap\Omega_I$, and becomes the next observation, while an accepted click is mapped back as $\hat p=\Phi_r(\hat p_r)$. Coordinate priming builds a reference set $R(I)=\mathcal{O}(I)\cup\mathcal{D}(I)$ from OCR and detected UI components so visible content is tied to screen locations, and visual verification marks eac

Load-bearing premise

The load-bearing premise is that the same model that proposes a click can be trusted to verify that click: the paper does not report how often the verifier accepts a wrong proposal, so if the verifier mostly says yes regardless, the observed gains could be inflated.

Editorial extensions

If this is right

  • On ScreenSpot-Pro, GUI-Lens improves every tested backend, so the gain comes from the grounding procedure rather than from swapping in a stronger model.
  • With GPT-5.5, GUI-Lens reports 87.9% average accuracy on ScreenSpot-Pro, ahead of the compared single-shot models and prior grounding systems, across both text and icon targets.
  • The method transfers to interactive use: Claude Opus 4.7 plus GUI-Lens scores 86.8 overall on OSWorld and outperforms the same-backbone Pointer Agent in all three domains.
  • Under a reduced budget of at most two crops and no verification, GUI-Lens still adds 10.34 points over single-shot prediction at 1.27x latency, and most samples finish within four crop rounds.
  • Ablations show that removing cropping causes the largest drop for every backend; coordinate references help most for the weakest backend, and verification adds a smaller but consistent gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the method only re-prompts an off-the-shelf VLM, the same loop should transfer to future general-purpose VLMs without retraining; one testable prediction is that the gains scale with the model's ability to follow multi-step visual instructions.
  • The coarse-to-fine observation loop is not inherently GUI-specific and could be evaluated on other dense-localization settings, such as document layouts, maps, or microscopy images, where small targets sit among visually similar neighbors; this extension is not claimed by the paper.
  • The efficiency analysis suggests a practical deployment rule—spend more crops when the target is an icon or in a dense region and stop early otherwise—since most samples terminate within a few rounds and verification is the main added cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces GUI-Lens, a GUI-grounding framework that replaces direct click prediction with a sequence of VLM-selected coarse-to-fine crops. OCR and a UI detector provide coordinate references, and a verification step (using the same VLM) checks proposed crops and clicks. Experiments cover ScreenSpot-Pro, ScreenSpot-v2, MMBench-GUI-L2, UI-Vision, and OSWorld, with GPT-5.5, Claude Opus 4.7, and MiniMax-M3 backends. The authors report improvements of up to 24.9 percentage points, state-of-the-art results with GPT-5.5, and attribute the main gain to cropping via a leave-one-component-out ablation on a 300-example subset.

Significance. If validated, GUI-Lens is a simple, backend-agnostic recipe for improving GUI grounding without fine-tuning: model-selected crops, coordinate references, and verification are all implementable over standard VLM APIs. The multi-benchmark, multi-backend evaluation and the accuracy–efficiency analysis (especially the Efficient variant, which improves over single-shot by 10.34 points without verification) are valuable and go beyond a single leaderboard. The released code is a strength. However, the central attribution of the gain to cropping currently rests on an ablation control that behaves anomalously, and the verification step is not independently characterized; these issues must be resolved before the main claim is fully supported.

major comments (4)
  1. [§3.6, Table 4; App. B] The w/o-Cropping ablation is not a clean component-removal. For Claude Opus 4.7, removing cropping reduces accuracy to 41.0 on the 300-sample subset, whereas the same backend's matched single-shot baseline is 57.4 on full ScreenSpot-Pro (Table 1). Since w/o-Cropping is described as a single full-screen prediction plus coordinate references and verification (a superset of the single-shot baseline), one would expect it to be at least as accurate as 57.4, not 16 points lower. The paper does not report the single-shot baseline on the same 300-example subset, so the 41.3-point 'cropping gain' is not a valid attribution. Please report the single-shot baseline on the same subset, disclose the exact prompt/action protocol for the w/o-Cropping variant, and rule out artifacts such as verification-driven restarts or residual crop actions.
  2. [§2.4, Eq. (5); Table 4] The verification step uses the same VLM that proposes the crop/click, but the paper reports no verifier accuracy, rejection rates, or correlation between verification decisions and ground-truth correctness. Without such statistics, accepted clicks could be self-confirmed errors, inflating the reported numbers. The Efficient variant without verification mitigates this concern for the cropping claim, but the main configuration's absolute numbers remain uninterpretable. Please report verifier precision/recall against ground-truth boxes, rejection counts by round, accuracy of accepted versus rejected proposals, and, if possible, results with an independent verifier.
  3. [§3.6, Table 4; App. D] All comparisons in Table 4 come from a single evaluation run per configuration, as Appendix D states. On 300 examples, the 1.0–2.0 point differences (e.g., w/o Coordinate Priming for GPT-5.5 and Claude) are within plausible noise, yet no confidence intervals, repeated runs, or significance tests are reported for the ablations. The main tables claim McNemar significance (p < 0.05) but do not give test statistics or clarify whether the tests apply to per-cell comparisons. Please provide confidence intervals or significance tests for the ablation table and test details for the main claims.
  4. [App. B, App. D] The text repeatedly states that 'Appendices B and D provide the complete grounding configuration, ablation definitions, prompts, and execution protocol,' but the actual prompt templates are not printed in the manuscript; only protocol descriptions appear. Since the method is prompt-based and reproducibility depends on the exact prompts, the prompts should be included in the appendix or in a reviewer-accessible supplement, not only in the code repository.
minor comments (4)
  1. [Table 4 vs. Table 1] The Full GUI-Lens accuracy for Claude Opus 4.7 is exactly 82.3 both on the 300-sample subset (Table 4) and on the full ScreenSpot-Pro average (Table 1). This coincidence is surprising; please explain or report the subset's single-shot baseline and full configuration scores for all backbones to make the subset's representativeness clear.
  2. [§3.4, Figure 4] The accuracy–latency curves are computed on a 300-example subset, while the crop-round distribution is from a separate 1,581-example full-scale run. Clarify why the full-scale run is not used for both panels, and whether the subset is the same one used in Table 4.
  3. [§3.3, Table 3] The OSWorld comparison is informative, but the paper should state more explicitly how GUI-Lens is integrated into the agent (e.g., which module calls the grounding pipeline, how clicks are executed, and whether the same crop budget and verification settings are used in the interactive setting).
  4. [Various] Minor wording issues: 'HCompany' should be 'H Company' in the affiliation/reference formatting; Table 2's column layout for MMBench-GUI-L2 is dense and hard to read; some cited works (e.g., 'Sager et al. 2026') appear in the introduction but not in the reference list in the visible text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GUI-Lens is an inference-time pipeline evaluated against external benchmarks, with no fitted parameters, no prediction reused as input by construction, and no load-bearing self-citation chain.

full rationale

The paper's derivation chain is a test-time procedure: coordinate references are obtained from external OCR and UI detectors (Eq. 2), the VLM proposes crops/clicks (Eq. 4), a verification call accepts or rejects proposals (Eq. 5), and accepted local coordinates are mapped back to screen coordinates via an affine transform (Eqs. 6-8). No fitted parameters are learned from the benchmarks, and no quantity is defined in terms of the target it is supposed to predict. The same VLM is used as both proposer and verifier, but this is not circular by construction: the verification call is a separate stage-specific model invocation, and the Efficient ablation without verification still improves over single-shot (Section 3.4, Table 4), so the reported gains do not reduce to self-verification. The Table 4 'w/o Cropping' result for Claude Opus 4.7 (41.0 on the 300-sample subset versus the 57.4 full single-shot baseline in Table 1) is a potential experimental-control concern, but it is an evaluation-validity issue rather than a derivation-level circularity. The paper's comparisons are against external benchmarks and matched single-shot baselines with the VLM held fixed, and it does not rely on self-citations or imported uniqueness theorems to justify its central claim. Therefore, the circularity score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

No new theoretical entities are introduced; the method uses existing detectors, OCR, and a VLM in a new inference loop. Free parameters are mostly engineering choices.

free parameters (7)
  • Max crop rounds = 8 (GPT-5.5, Claude Opus 4.7); 5 (MiniMax-M3)
    Hyperparameter limiting the number of zoom steps; not fitted but chosen without a described validation set.
  • Coordinate reference budget = 60 per crop, 80 for final
    Number of OCR/detector references serialized into the prompt.
  • Upscale factors = 5x intermediate, 8x final
    Enlargement applied to selected crops.
  • Contextual padding = 8% per side
    Padding added to crops before intersection with screen.
  • Verification retry limit = 6 (GPT-5.5, Claude); 3 (MiniMax-M3)
    Number of times a rejected proposal can restart cropping.
  • Detector confidence threshold = 0.1
    GPA-GUI-Detector confidence threshold, very low, may generate many spurious references.
  • NMS IoU threshold = 0.3
    Non-maximum suppression for detector boxes.
assumptions (5)
  • domain assumption OCR and UI detector outputs are accurate enough to serve as spatial references without misdirecting the VLM.
    The method relies on these references (Eq. 2) but reports no accuracy of the detector/OCR on the test screenshots.
  • domain assumption The VLM can produce valid JSON crop/click proposals and verification decisions through the staged prompts.
    Evidenced by reported results, but prompts are not fully shown in the text.
  • standard math The affine coordinate mapping (Eq. 8) correctly resizes crops; this is standard math.
    Coordinates are transformed linearly.
  • domain assumption Point-in-box evaluation is a fair measure of grounding accuracy.
    Standard benchmark protocol, but it ignores click precision within the box.
  • domain assumption The matched single-shot baselines use equally strong prompts as GUI-Lens's final grounding step.
    The prompt for the single-shot baseline is not provided, so fairness cannot be verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs." pith.science (2026). https://pith.science/paper/62MWV6RB

@misc{pith2026260803270,
  author       = {Pith},
  title        = {Pith review of: GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/62MWV6RB}},
  note         = {Machine review of arXiv:2608.03270}
}
read the original abstract

GUI grounding maps natural-language instructions to click locations and is essential for reliable GUI agents. The task remains difficult on high-resolution, densely populated interfaces because a vision-language model (VLM) may recognize a requested control without locating it precisely enough for interaction. Most existing methods provide various forms of localization assistance, but still rely on a direct click prediction, allowing visual ambiguity or an inaccurate initial estimate to propagate to the final result. In this paper, we introduce GUI-Lens, a coarse-to-fine grounding framework that allows a general-purpose VLM to determine the target through active visual observations. Specifically, GUI-Lens extracts OCR text and detected UI components from the screenshot and presents their positions as coordinate references. Using the instruction, the current view, and these references, the VLM selects the region and scale of the next view, which is cropped and enlarged to provide finer visual details. This process continues over successively focused views until the target is determined. Proposed crops and clicks are checked against the instruction throughout the process, and the final local position is mapped back to the original screen coordinates. Experiments on four GUI grounding benchmarks and three general-purpose VLM backends show that GUI-Lens improves overall grounding accuracy by up to 24.9 percentage points and achieves state-of-the-art performance with GPT-5.5.

Figures

Figures reproduced from arXiv: 2608.03270 by the authors.

Figure 1
Figure 1. Comparison of GUI grounding methods on a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison between coarse-to-fine cropping and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of GUI-Lens. Coordinate Priming converts OCR text and detected UI components into coordinate references. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Accuracy–efficiency analysis on ScreenSpot-Pro with GPT-5.5. (a) Accuracy and latency under four inference config [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: A grounding trajectory of GUI-Lens. Verification rejects crop proposals that omit the target, after which the framework [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 7 canonical work pages

  1. [1]

    Claude3.7SonnetSystemCard

    Anthropic.2025. Claude3.7SonnetSystemCard. https://as sets.anthropic.com/m/785e231869ea8b3b/original/claude- 3-7-sonnet-system-card.pdf. Anthropic. 2026a. Introducing Claude Opus 4.7. https: //www.anthropic.com/news/claude-opus-4-7. Accessed: 2026-06-17. Anthropic. 2026b. Introducing Claude Sonnet 4.6. https: //www.anthropic.com/news/claude-sonnet-4-6. Pu...

    work page 2025

  2. [4]

    InInternational Conference on Learning Representations

    Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents. InInternational Conference on Learning Representations. Gu,Z.;Zeng,Z.;Xu,Z.;Zhou,X.;Shen,S.;Liu,Y.;Zhou,B.; Meng,C.;Xia,T.;Chen,W.;etal.2025. UI-VenusTechnical Report: Building High-performance UI Agents with RFT. arXiv preprint arXiv:2508.10833. H Company. 2025a. Holo1....

    arXiv 2025

  3. [5]

    VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation

    Holo3-35B-A3B Model Card. https: //huggingface.co/Hcompany/Holo3-35B-A3B. Han, Q.; Tu, H.; Wang, Z.; Dai, H.; Zhou, Y.; Lau, N.; Car- denas, A. A.; Xu, Y.; Xu, R.; Xiong, C.; Zheng, Z.; Yao, H.; Zhou,Y.;andXie,C.2026. VLAA-GUI:KnowingWhento Stop, Recover, and Search, A Modular Framework for GUI Automation.arXiv preprint arXiv:2604.21375. HCIII Team, Lenovo

    work page Pith review arXiv 2026

  4. [6]

    https://github.com/wad ang/muscle-mem-agent

    HIPPO Agent: A Memory- Augmented Computer-Use Agent. https://github.com/wad ang/muscle-mem-agent. Hong,W.;Wang,W.;Lv,Q.;Xu,J.;Yu,W.;Ji,J.;Wang,Y.; Wang,Z.;Dong,Y.;Ding,M.;andTang,J.2024. CogAgent: AVisualLanguageModelforGUIAgents.InProceedingsof the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14281–14290. Jiang, Z.; Xie, S.; Li, W.; Zu...

    work page 2024

  5. [7]

    KimiTeam.2026.KimiK2.6:AdvancingOpen-SourceCod- ing

    Zoom In, Click Out:UnlockingandEvaluatingthePotentialofZoomingfor GUI Grounding.arXiv preprint arXiv:2512.05941. KimiTeam.2026.KimiK2.6:AdvancingOpen-SourceCod- ing. https://www.kimi.com/blog/kimi-k2-6. Kingsware and Vocaela AI

    arXiv 2026

  6. [8]

    ScreenSpot-Pro: GUI Grounding for ProfessionalHigh-ResolutionComputerUse.arXivpreprint arXiv:2504.07981. Lin, K. Q.; Li, L.; Gao, D.; Yang, Z.; Wu, S.; Bai, Z.; Lei, S.W.;Wang,L.;andShou,M.Z.2025.ShowUI:OneVision- Language-ActionModelforGUIVisualAgent.InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 19498–19508. Liu,Y.;Shen,H.;Yu,L...

    arXiv 2025

  7. [10]

    RecurrentModelsofVisualAttention.InAdvancesinNeural Information Processing Systems, volume

    Mnih,V.;Heess,N.;Graves,A.;andKavukcuoglu,K.2014. RecurrentModelsofVisualAttention.InAdvancesinNeural Information Processing Systems, volume

    work page 2014

  8. [12]

    GPT-4o System Card.arXiv preprint arXiv:2410.21276. OpenAI

    arXiv

Show all 21 references
  1. [13]

    AdaZoom-GUI:AdaptiveZoom-basedGUIGroundingwith Instruction Refinement.arXiv preprint arXiv:2603.17441

    Pei,S.;Tang,L.;Duan,T.;Chen,L.;Li,S.;Huang,K.;Jing, Y.;Yan,Y.;Zhang,B.;Jiang,C.;Zhang,B.;andLu,J.2026. AdaZoom-GUI:AdaptiveZoom-basedGUIGroundingwith Instruction Refinement.arXiv preprint arXiv:2603.17441. Qin, Y.; et al

    2026

  2. [14]

    Sager, P.; Meyer, B.; Yan, P.; von Wartburg-Kottler, R.; Etaiwi, L.; Enayati, A.; Nobel, G.; Abdulkadir, A.; Grewe, B

    UI-TARS: Pioneering Automated GUI Interaction with Native Agents.arXiv preprint arXiv:2501.12326. Sager, P.; Meyer, B.; Yan, P.; von Wartburg-Kottler, R.; Etaiwi, L.; Enayati, A.; Nobel, G.; Abdulkadir, A.; Grewe, B. F.; and Stadelmann, T

    arXiv

  3. [15]

    Salesforce/GPA-GUI-Detector

    GPA-GUI-Detector: General- Purpose Agent for GUI Element Detection.HuggingFace Model Hub. Salesforce/GPA-GUI-Detector. Shi,B.;Bai,X.;andYao,C.2017. AnEnd-to-EndTrainable NeuralNetworkforImage-BasedSequenceRecognitionand Its Application to Scene Text Recognition.IEEE Transac- t...

    2017

  4. [16]

    Venus Team; Gao, C.; Gu, Z.; Liu, Y.; Qiu, X.; Shen, S.; Wen,Y.;Xia,T.;Xu,Z.;Zeng,Z.;etal.2026

    UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI Grounding.arXiv preprint arXiv:2604.14113. Venus Team; Gao, C.; Gu, Z.; Liu, Y.; Qiu, X.; Shen, S.; Wen,Y.;Xia,T.;Xu,Z.;Zeng,Z.;etal.2026. UI-Venus-1.5 Technical Report.arXiv preprint arXiv:2602.09082. Wang,P.;Bai,S.;Tan,S...

    2026 arXiv

  5. [17]

    Wu,Q.;Cheng,K.;Yang,R.;Zhang,C.;Yang,J.;Jiang,H.; Mu, J.; Peng, B.; Qiao, B.; Tan, R.; Qin, S.; Liden, L.; Lin, Q.; Zhang, H.; Zhang, T.; Zhang, J.; Zhang, D.; and Gao, J.2025a

    MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents.arXiv preprint arXiv:2507.19478. Wu,Q.;Cheng,K.;Yang,R.;Zhang,C.;Yang,J.;Jiang,H.; Mu, J.; Peng, B.; Qiao, B.; Tan, R.; Qin, S.; Liden, L.; Lin, Q.; Zhang, H.; Zhang, T.; Zhang, J.; Zhang, D.; and Gao...

    arXiv

  6. [18]

    InNeurIPS

    OSWorld: BenchmarkingMultimodalAgentsforOpen-EndedTasksin Real Computer Environments. InNeurIPS. Xu, H.; Chen, Q.; Wang, L.; and Liu, L. 2025a. Attention- Driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models Without Fine-Tuning. In Walsh, T.; Shah, J.; ...

    2025 arXiv

  7. [19]

    Zhang,C.;Yang,Z.;Liu,J.;Li,Y.;Han,Y.;Chen,X.;Huang, Z.;Fu,B.;andYu,G.2025a

    GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents.arXiv preprint arXiv:2509.15532. Zhang,C.;Yang,Z.;Liu,J.;Li,Y.;Han,Y.;Chen,X.;Huang, Z.;Fu,B.;andYu,G.2025a. AppAgent:MultimodalAgents asSmartphoneUsers. InProceedingsofthe2025CHICon- ference on Human ...

  8. [20]

    A Baseline Systems TheexperimentscompareGUI-Lenswiththreecomplemen- tary classes of prior systems

    GUI Agents with Foundation Models: AComprehensiveSurvey.arXivpreprintarXiv:2411.04890. A Baseline Systems TheexperimentscompareGUI-Lenswiththreecomplemen- tary classes of prior systems. General-purpose VLMs test whether the proposed grounding procedure improves mod- els that a...

    2026 arXiv

  9. [21]

    Easy- OCR (Baek et al

    extracts visual components with a confidence threshold of 0.1 and an NMS IoU threshold of 0.3. Easy- OCR (Baek et al. 2019; Shi, Bai, and Yao

    2019

  10. [27]

    Nayak, S.; Jian, X.; Lin, K

    Curran Asso- ciates, Inc. Nayak, S.; Jian, X.; Lin, K. Q.; Rodriguez, J. A.; Kalsi, M.; Chapados,N.;Özsu,M.T.;Agrawal,A.;Vazquez,D.;Pal,C.; Taslakian,P.;Gella,S.;andRajeswar,S.2025. UI-Vision:A Desktop-centric GUI Benchmark for Visual Perception and Interaction. InProceedings ...

    2025

  11. [2024]

    OmniParser for Pure Vision Based GUI Agent.arXiv preprint arXiv:2408.00203. MiniMax

    arXiv

  12. [2025]

    arXiv preprint arXiv:2510.02250

    Scaling Agents for Computer Use. arXiv preprint arXiv:2510.02250. Gou, B.; Wang, R.; Zheng, B.; Xie, Y.; Chang, C.; Shu, Y.; Sun, H.; and Su, Y

  13. [2026]

    Char- acter Region Awareness for Text Detection

    Baek,Y.;Lee,B.;Han,D.;Yun,S.;andLee,H.2019. Char- acter Region Awareness for Text Detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, 9365–9374. Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng,L.;Ding,W.;Gao,C.;Ge,C...

    2019 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.