ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding

ShengJing Yang; Tzer-Jen Wei; ZongHan Hsieh

REVIEW 3 major objections 5 minor 12 references

ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 3B vision-language model trained on a single consumer GPU with just 24K samples matches or beats several 7B GUI-grounding systems.

desk verdict ZonUI-3B is a useful lightweight GUI-grounding recipe with plausible numbers, but the missing train/eval overlap check makes every headline accuracy conditional. read the letter →

arxiv 2506.23491 v3 pith:SAHKLTD2 submitted 2025-06-30 cs.CV cs.AI

ZongHan Hsieh , Tzer-Jen Wei , ShengJing Yang This is my paper · ORCID

classification cs.CVcs.AI

keywords GUIgroundingvision-languagemodellow-rankadaptationtwo-stagefine-tuningcross-platformdatasetdataredundancyScreenSpotbenchmarkshigh-resolutiondesktopinterfaces

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a 3-billion-parameter vision-language model, fine-tuned with low-rank adaptation on a single consumer GPU, can match or beat models more than twice its size at GUI grounding, the task of pointing to the correct interface element from a natural-language instruction. The model, ZonUI-3B, reaches 84.9% on ScreenSpot, 86.4% on ScreenSpot-v2, and 28.7% on ScreenSpot-Pro, surpassing all prior models under 4B parameters and tying several 7B systems. The authors attribute this to a deliberately compact 24K-example training corpus assembled from mobile, web, and desktop sources, a balanced sampling scheme, and a two-stage schedule that first learns general GUI structure and then specializes to high-resolution screens. If correct, the result means competitive GUI grounding does not require massive datasets or multi-GPU training; a careful data and training recipe on modest hardware can close most of the gap to large models.

What carries the argument

The load-bearing mechanism is the pairing of a redundancy-reduced, cross-platform dataset with a two-stage LoRA fine-tuning schedule. LoRA (low-rank adaptation) inserts small trainable matrices into the frozen transformer layers of the Qwen2.5-VL-3B backbone, making full fine-tuning feasible on one RTX 4090. Stage 1 trains on balanced samples from mobile, web, and desktop sources to build platform-general grounding ability; Stage 2 fine-tunes on a high- and multi-resolution subset of the UGround web-hybrid corpus to adapt to dense, high-resolution layouts. The authors argue that this staged separation, rather than architectural novelty or scale, carries the performance gains, and ablations attribute +3.3% desktop and +1.8% web accuracy to the two-stage schedule.

What would settle it

Collect the four training corpora and the three benchmark sets, then compute image-similarity (e.g., perceptual hashing or embedding distance) between every benchmark screenshot and every training screenshot; if a meaningful number of benchmark elements or full screens match training images, the reported gains reflect memorized screens rather than the training recipe.

Watch

Extended reading notes

Core claim

The paper's central claim is that data redundancy, not model scale, is the dominant constraint in lightweight GUI grounding. Randomly cutting the training corpus from roughly 120K examples to 16.1K examples leaves ScreenSpot accuracy nearly unchanged (82.8% versus 82.9%), and merging two diverse sources into 24.1K examples outperforms either corpus alone. Adding a two-stage fine-tuning schedule—general cross-platform training followed by high-resolution specialization—then lifts accuracy to 84.9% on ScreenSpot, 86.4% on ScreenSpot-v2, and 28.7% on ScreenSpot-Pro, setting a new bar among sub-4B models and matching several 7B baselines.

Load-bearing premise

The reported accuracy numbers assume that the training screenshots from ShowUI-Web, UGround, AMEX, and ShowUI-Desktop do not appear in ScreenSpot, ScreenSpot-v2, or ScreenSpot-Pro, but the paper reports no overlap or deduplication check.

Editorial extensions

If this is right

A 3B model trained on 24K examples beats the strongest prior 2B model (UI-TARS-2B) by +2.6% on ScreenSpot and +1.7% on ScreenSpot-v2, while matching several 7B systems.
Cutting training data to about one-seventh of the available corpus (16.1K of 119.4K examples) costs only 0.1% ScreenSpot accuracy, so existing GUI datasets contain large amounts of redundant signal.
Two-stage fine-tuning contributes +3.3% on desktop and +1.8% on web over single-stage training, so high-resolution specialization can be added after general GUI learning without architectural changes.
The full pipeline runs on a single consumer GPU with LoRA and mixed-precision training, lowering the hardware barrier for building deployable GUI agents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the absence of benchmark overlap were confirmed, the results would suggest that GUI grounding is more data-diversity-limited than capacity-limited, implying that even smaller or older backbones might reach similar accuracy with the same recipe.
A direct testable extension is to swap Stage 2's data source for professional-software screenshots (e.g., CAD or design tools) and check whether the ScreenSpot-Pro gain concentrates in the corresponding categories.
The random-sampling result implies that simple subsampling of GUI datasets may compete with explicit deduplication pipelines; measuring per-sample novelty (embedding distance to the rest of the corpus) against downstream accuracy would test this.
Since the paper reports no overlap analysis, rerunning the exact recipe on a deduplicated corpus is the cleanest check on whether the reported gap to 7B models reflects the training strategy or memorized benchmark images.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

ZonUI-3B is a clean, honest systems paper. It shows a 3B LoRA-tuned Qwen2.5-VL, trained on a 24K curated subset on a single RTX 4090, hitting 84.9/86.4/28.7 on ScreenSpot/ScreenSpot-v2/ScreenSpot-Pro. The results are plausible and the recipe is useful, but the paper never verifies that its training data (ShowUI-Web, UGround-WebHybrid, AMEX, ShowUI-Desktop) is disjoint from the evaluation sets. That's the one assumption all three headline numbers rest on.

What's actually new: not the components—LoRA, Qwen2.5-VL, public datasets, and two-stage fine-tuning are all known. The new content is the specific combination and the compact 24K data subset, plus the ablations showing that 24K random samples beat 119K, and that balanced sampling and the two-stage schedule each add a few points. Those ablations are the most useful part; they give the paper a clear practical takeaway.

Soft spots, in order of severity: 1. No overlap analysis or deduplication between the training corpora in Sec 2.1 and the three ScreenSpot benchmarks. The risk is concrete: Stage 2 fine-tunes on high-res UGround web-hybrid screenshots, and ScreenSpot-Pro tests professional high-res desktop interfaces. If any eval screenshot (or near-duplicate) is in the 24K training set, the reported gains over UI-TARS-2B and others are inflated. This needs to be addressed before the numbers can be trusted. 2. Everything is a single run. No error bars, and the ablation deltas are small (+0.9% balanced sampling, +0.1% for 119K vs 16K). Some of those differences are within noise. 3. Table 2 has a garbled cell: the ZonUI-3B row shows '27.954.8', where two numbers got concatenated. Easy to fix, but it suggests the tables weren't carefully proofread.

The paper also links a GitHub repo; I didn't check whether code and data are actually released. If they are, that raises confidence.

This paper is for people building lightweight GUI agents or trying to reproduce small-model results on a single GPU. It's a solid empirical data point, not a field re-organization. It deserves a serious referee: the core claim is important enough, the methodology is reproducible in principle, and the gaps are concrete and addressable.

Recommendation: send it to peer review, but have the reviewers ask the authors to (a) run an overlap analysis or explicitly deduplicate their training set against the eval benchmarks, and (b) report variance across at least three runs for the main results. If the overlap is clean and the numbers hold, this is a useful contribution. If not, it's a lesson about dataset contamination.

Referee Report

3 major / 5 minor

Summary. The paper presents ZonUI-3B, a 3B-parameter vision-language model for GUI grounding obtained by LoRA fine-tuning Qwen2.5-VL-3B on a 24K-example corpus assembled from ShowUI, UGround, AMEX, and ShowUI-Desktop data. The method uses a two-stage fine-tuning schedule: cross-platform balanced training followed by high-resolution specialization. The authors report 84.9% on ScreenSpot, 86.4% on ScreenSpot-v2, and 28.7% on ScreenSpot-Pro, surpassing prior sub-4B models and matching several 7B baselines. Ablations claim that balanced sampling, multi-source diversity, and the two-stage schedule are the main contributors, all trained on a single RTX 4090.

Significance. If the reported numbers are trustworthy, this is a practically valuable result: it shows that a compact, single-GPU-trainable model can be competitive with much larger GUI-grounding systems. The paper's strengths are its use of three standard benchmarks, a public model release, and a simple, reproducible training recipe. However, the empirical support is currently under-powered: all results are single runs with no error bars, the ScreenSpot-Pro table is malformed, and the paper does not report any train/eval overlap analysis. These issues must be resolved before the central claim can be accepted.

major comments (3)

[§2.1 / §4.1] The paper reports no overlap analysis or deduplication between the training corpus (ShowUI-Web, UGround-WebHybrid, AMEX, ShowUI-Desktop) and the evaluation benchmarks (ScreenSpot, ScreenSpot-v2, ScreenSpot-Pro). Because the central claim is that 24K examples suffice to surpass prior sub-4B models, and because Stage 2 explicitly trains on high-resolution UGround web-hybrid screenshots, the reader cannot rule out test-set contamination from near-duplicate screenshots. Please provide a quantitative overlap check or at least a clear dataset-construction statement showing that evaluation screenshots, and near-duplicates of them, are excluded from training.
[Table 2] The ZonUI-3B row in the ScreenSpot-Pro table contains concatenated numeric cells "27.954.8", "50.019.6", and "39.211.7", so the per-category breakdown is unreadable and the reported 28.7% overall cannot be independently verified from the table. Please reformat the table so that every cell contains a single value and clearly distinguish per-category averages from the overall average.
[§4.3, Tables 3–5] All ablation results are single runs with no error bars, confidence intervals, or multiple seeds. The differences that motivate the core recipe are small (e.g., +0.9% for balanced sampling in Table 3, +0.3% for adding UGround data in Table 5, and the 82.8 vs 82.9 volume comparison in Table 4), so without variance estimates these conclusions are not statistically supported. Please provide at least three seeds or bootstrap confidence intervals for the main ablation comparisons.

minor comments (5)

[Abstract] The phrase "combine cross-platform, multi-resolution dataset" should be "combines a cross-platform, multi-resolution dataset"; similar grammar issues appear in other places.
[§4.1] The text says "24,GB VRAM" with a stray comma; it should read "24GB VRAM".
[§2.2] The paper says the reduced dataset is "one-seventh of the full dataset (16.1K examples)", but 120K / 7 is approximately 17.1K; please reconcile these numbers.
[Tables 3–5] The rows labeled "2-Stage Training (Ours)" use the same data size as "UGround + ShowUI-Web" but it is not stated what exactly differs between the two-stage recipe and the single-stage baseline in terms of data subset selection and training schedule; please clarify.
[§3.3/§4.1] The paper does not report the number of training epochs for Stage 1 and Stage 2; please include this information for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports empirical benchmark results from a training recipe, with no equation-level derivation or self-citation chain that reduces to its inputs.

full rationale

ZonUI-3B is an empirical fine-tuning paper. The central claims are benchmark accuracies (84.9% on ScreenSpot, 86.4% on ScreenSpot-v2, 28.7% on ScreenSpot-Pro) obtained by LoRA fine-tuning Qwen2.5-VL-3B on a 24K-example corpus. There is no formal derivation, no fitted equation, and no prediction that is equivalent to an input by construction. The training data sources (ShowUI-Web, UGround-WebHybrid, AMEX, ShowUI-Desktop) are external open datasets, and the evaluation benchmarks (ScreenSpot, ScreenSpot-v2, ScreenSpot-Pro) are also external. The paper does not define any metric in terms of its own training choices, nor does it invoke a uniqueness theorem or self-citation to force a result. The ablations (Tables 3-5) use ScreenSpot to select training configurations, which is a form of test-set tuning and could inflate the reported number, but it is not a circularity in the sense of the target score: the final accuracy is not logically entailed by the training data or by the paper's own definitions. The absence of an overlap/deduplication analysis between training and evaluation sets is a data-leakage risk, not a circularity risk, and it is not established by the text that any evaluation screenshot appears in the training set. Therefore no circular step can be exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests entirely on empirical choices rather than derivation: hand-picked hyperparameters (LoRA rank/alpha, learning rates, 1:1:1 sampling ratio, subset sizes) and three domain assumptions about benchmark annotations, data representativeness, and train/test disjointness. No new entities are introduced.

free parameters (5)

LoRA rank and alpha = rank=8, alpha=16
Chosen in Section 3.3 and 4.1 without a reported sweep; the grounding results depend on this capacity choice.
Stage 1 learning rate = 2e-4
Set in Section 4.1; central accuracy depends on this optimization choice.
Stage 2 learning rate = 5e-5
Set in Section 4.1 for the resolution specialization stage.
Balanced sampling ratio = 1:1:1
Chosen in Section 3.2 and Table 3 to mitigate platform imbalance; the +0.9% gain rests on this hand-picked ratio.
Training subset sizes = 16.1K and 24.1K
Random samples drawn in Section 2.2 with no seed; the redundancy-reduction claim depends on these specific subset sizes.

assumptions (4)

domain assumption Benchmark success is correctly measured by whether predicted coordinates fall inside the annotated bounding box.
Section 4.1 defines success this way; errors in benchmark annotations would shift all numbers.
domain assumption The selected public training corpora are representative of the GUI distributions in the evaluation benchmarks.
Section 2.1 assumes ShowUI, UGround, and AMEX cover the platforms and resolutions needed for ScreenSpot, ScreenSpot-v2, and ScreenSpot-Pro.
domain assumption Training and evaluation data do not overlap.
No deduplication or overlap analysis is reported between Section 2.1 training sources and Section 4.1 benchmarks; all benchmark numbers depend on this.
domain assumption LoRA with rank 8 preserves the backbone's coordinate grounding ability.
Section 3.1 adopts LoRA without justification that low-rank updates are sufficient for exact coordinate grounding.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding." pith.science (2026). https://pith.science/paper/SAHKLTD2

@misc{pith2026250623491,
  author       = {Pith},
  title        = {Pith review of: ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SAHKLTD2}},
  note         = {Machine review of arXiv:2506.23491}
}

read the original abstract

In this paper, we present ZonUI-3B, a lightweight Vision-Language Model (VLM) that can be fully trained on a single consumer-grade GPU (RTX 4090) while delivering performance comparable to significantly larger models on GUI grounding tasks. The model incorporates several key innovations: (i) combine cross-platform, multi-resolution dataset of 24K examples from diverse sources including mobile, desktop, and web GUI screenshots to effectively address data scarcity in high-resolution desktop environments; (ii) a two-stage fine-tuning strategy, where initial cross-platform training establishes robust GUI understanding, followed by specialized fine-tuning on high-resolution data to significantly enhance model adaptability; and (iii) data curation and redundancy reduction strategies, demonstrating that randomly sampling a smaller subset with reduced redundancy achieves performance comparable to larger datasets, emphasizing data diversity over sheer volume. Empirical evaluation on standard GUI grounding benchmarks, including ScreenSpot, ScreenSpot-v2, and the challenging ScreenSpot-Pro, highlights ZonUI-3B's exceptional accuracy, achieving 84.9% on ScreenSpot and 86.4% on ScreenSpot-v2, surpassing prior models under 4B parameters. Ablation studies validate the critical role of balanced sampling and two-stage fine-tuning in enhancing robustness, particularly in high-resolution desktop scenarios. The ZonUI-3B is available at: https://github.com/Han1018/ZonUI-3B

Figures

Figures reproduced from arXiv: 2506.23491 by the authors.

**Figure 2.** Per-category accuracy on ScreenSpot, ScreenSpot-v2, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Overview of two-stage fine-tuning pipeline. Stage 1 builds platform-general GUI grounding ability across mobile, web, and [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 12 linked inside Pith

[1]

Qwen2.5-vl technical report

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen 6 Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, Junyang Lin, et al. Qwen2.5-vl technical re...

arXiv 2025
[2]

Amex: Android multi-annotation expo dataset for mobile gui agents

Yuxiang Chai, Siyuan Huang, Yazhe Niu, Han Xiao, Liang Liu, Dingyu Zhang, Shuai Ren, and Hongsheng Li. Amex: Android multi-annotation expo dataset for mobile gui agents. arXiv preprint arXiv:2407.17490, 2024. 1, 2, 3, 5, 6

arXiv 2024
[3]

Seeclick: Har- nessing gui grounding for advanced visual gui agents

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yan- tao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Har- nessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401.10935, 2024. Version 2, Feb 23, 2024. 1, 4, 5

arXiv 2024
[4]

Navigating the digital world as humans do: Universal visual grounding for gui agents

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world as humans do: Universal visual grounding for gui agents. arXiv preprint arXiv:2410.05243 , 2024. Ac- cepted to ICLR 2025 (Oral). 1, 2, 3, 4, 5, 6

arXiv 2024
[5]

Cogagent: A visual language model for gui agents

Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogagent: A visual language model for gui agents. arXiv preprint arXiv:2312.08914, 2023. CVPR 2024 (High- light). 4

arXiv 2023
[6]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3, 4, 6

arXiv 2021
[7]

Screenspot-pro: Gui grounding for professional high- resolution computer use

Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high- resolution computer use. arXiv preprint arXiv:2504.07981,

arXiv
[8]

Showui: One vision-language-action model for gui visual agent

Kevin Qinghong Lin, Linjie Li, Difei Gao, et al. Showui: One vision-language-action model for gui visual agent. arXiv preprint arXiv:2411.17465, 2024. 1, 2, 3, 4, 5, 6

arXiv 2024

Show all 12 references

[9]

Ui-tars: Pioneering automated gui inter- action with native agents

Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shi- jue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei Zheng,...

arXiv
[10]

Os-atlas: A founda- tion action model for generalist gui agents

Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, and Yu Qiao. Os-atlas: A founda- tion action model for generalist gui agents. arXiv preprint arXiv:2410.23218, 2024. 1, 2, 4, 5, 6

arXiv
[11]

Aguvis: Unified pure vision agents for autonomous gui interaction

Yiheng Xu, Zekun Wang, Junli Wang, Dunjie Lu, Tian- bao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. Aguvis: Unified pure vision agents for autonomous gui interaction. arXiv preprint arXiv:2412.04454, 2024. Ac- cepted to ICML 2025. 1, 2, 4, 5, 6

2024 arXiv
[12]

Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze L...

2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Qwen2.5-vl technical report

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen 6 Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, Junyang Lin, et al. Qwen2.5-vl technical re...

arXiv 2025

[2] [2]

Amex: Android multi-annotation expo dataset for mobile gui agents

Yuxiang Chai, Siyuan Huang, Yazhe Niu, Han Xiao, Liang Liu, Dingyu Zhang, Shuai Ren, and Hongsheng Li. Amex: Android multi-annotation expo dataset for mobile gui agents. arXiv preprint arXiv:2407.17490, 2024. 1, 2, 3, 5, 6

arXiv 2024

[3] [3]

Seeclick: Har- nessing gui grounding for advanced visual gui agents

Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yan- tao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Har- nessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401.10935, 2024. Version 2, Feb 23, 2024. 1, 4, 5

arXiv 2024

[4] [4]

Navigating the digital world as humans do: Universal visual grounding for gui agents

Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world as humans do: Universal visual grounding for gui agents. arXiv preprint arXiv:2410.05243 , 2024. Ac- cepted to ICLR 2025 (Oral). 1, 2, 3, 4, 5, 6

arXiv 2024

[5] [5]

Cogagent: A visual language model for gui agents

Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogagent: A visual language model for gui agents. arXiv preprint arXiv:2312.08914, 2023. CVPR 2024 (High- light). 4

arXiv 2023

[6] [6]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3, 4, 6

arXiv 2021

[7] [7]

Screenspot-pro: Gui grounding for professional high- resolution computer use

Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high- resolution computer use. arXiv preprint arXiv:2504.07981,

arXiv

[8] [8]

Showui: One vision-language-action model for gui visual agent

Kevin Qinghong Lin, Linjie Li, Difei Gao, et al. Showui: One vision-language-action model for gui visual agent. arXiv preprint arXiv:2411.17465, 2024. 1, 2, 3, 4, 5, 6

arXiv 2024