Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ScreenExplorer trains a 3B vision-language model to explore GUI environments more diversely than much larger static models, using a curiosity reward from a learned world model and experience stream distillation.

desk verdict A real-world RL training pipeline for GUI agents with a serious evaluation flaw: the diversity metric is the training objective. read the letter →

arxiv 2505.19095 v1 pith:B77TG5T2 submitted 2025-05-25 cs.AI

Runliang Niu , Jinglong Ji , Yi Chang , Qi Wang This is my paper · ORCID
classification cs.AI
keywords GUIagentsVision-languagemodelReinforcementlearningExplorationWorldCuriosity-drivenrewardExperiencestreamdistillationGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ScreenExplorer claims that a small vision-language model can learn to explore a real, open-ended desktop environment more diversely than much larger frozen models if it is trained online with reinforcement learning in that environment. The paper builds a GUI environment, rewards the agent for both valid interactions and state novelty, and pairs GRPO training with world-model curiosity and distillation of past exploration experience. A 3B-parameter model improved its average diversity score from 0.21 to 0.53, surpassing static deployments of gpt-4o, Qwen2.5-VL-72B, OpenAI Computer Use, and a 3B base. The value of the claim lies in a scalable path toward self-improving agents that gradually need less human-labeled data.

What carries the argument

A LLaMA-style world model with a VAE image tokenizer and text embeddings predict the next GUI state from the current state and chosen action. The world model's prediction discrepancy forms the core curiosity reward, combined with additional rewards for formatted actions, instantaneous and subsequent state changes, environment description, and intent interpretation. This reward design is paired with GRPO for policy optimization, then followed by experience stream distillation with manual or automated filtering to fine-tune the base model for the next generation.

What would settle it

A direct test is to evaluate the same models on diverse downstream tasks (e.g., file navigation, web search, software web-based tasks) and compare task success rates with the reported diversity metrics. If the cosine-similarity-based diversity scores do not correlate positively with meaningful task completion or human preference, the equivalence between these metrics and true exploration would be broken.

Watch

Extended reading notes

Core claim

The paper's central claim is that a 3B-parameter VLM, trained in a real open GUI desktop environment with GRPO, a world-model curiosity reward, and experience stream distillation, develops better environmental adaptation and sustained exploration than static deployment models. Training rewards diverse, novel state changes and meaningful interaction, and the results show a notable jump in diversity scores versus the base Qwen2.5-VL-3B and versus larger static models such as gpt-4o, Qwen2.5-VL-72B, and GUI-specific agents like OpenAI Computer Use. The paper also finds that the world model's predicted-state discrepancy is the most critical reward component, accelerating effective exploration during the cold-start phase and smoothing gradient differentiation in GRPO.

Load-bearing premise

The claim that cosine similarity between VAE image embeddings and OCR text embeddings measures meaningful exploration diversity is load-bearing: it is used both inside the reward function and inside the evaluation metrics, and the paper does not validate it against human judgment or an external benchmark.

Editorial extensions

If this is right

  • A 3B model can match or exceed exploration diversity of much larger models in a real desktop environment when trained online with a curiosity-driven reward and RL.
  • World-model curiosity is a key ingredient for overcoming the cold-start exploration phase in open GUI environments.
  • Self-collected exploration experience, without additional labels, can be distilled to improve exploration capabilities and sustain improvement.
  • The training and distillation cycle offers a general recipe for reducing reliance on carefully curated human-annotated data in GUI agent development.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The underlying mechanism might transfer to other open-ended environments beyond GUI, such as 3D worlds or web browsing, where a world model can predict next-state embeddings and quantify novelty from prediction error.
  • A testable extension is to compare the cosine-similarity-based diversity scores with human judgments or task-completion statistics, to see whether the reported diversity improvements align with truly meaningful exploration.
  • The paper's argument implies a major shift: for exploration-focused open environments, a small trainable model can outperform large frozen ones, which casts doubt on the assumption that raw scale alone is the main bottleneck in such settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ScreenExplorer, a 3B-parameter vision-language model trained with GRPO in a real Linux GUI environment. The training reward combines a format reward, visual and textual state-change rewards, a world-model prediction-error curiosity reward, and intent-state alignment terms (Eq. (1)). After RL, the authors also distill the collected experience streams via SFT. The central claim is that this framework improves diverse exploration in open GUI environments, supported by Table 1, which reports trajectory-level and group-level diversity computed from cosine similarity of VAE image embeddings and OCR text embeddings (Eqs. (5)-(6)), plus qualitative case studies and ablations.

Significance. If the central claim were established, the contribution would be notable: it would show that a small 3B VLM, trained with a world-model curiosity reward and experience-stream distillation in a live GUI environment, can explore more diversely than much larger frozen models. The paper also ships source code, builds a real GUI RL environment, and includes ablation studies on reward components, which are concrete strengths. However, the quantitative evidence for the central claim is currently compromised because the evaluation metrics in Section 4.1 use the same cosine-similarity definitions that appear in the training reward in Eq. (1). The reported exploration gains may therefore reflect optimization of the training objective rather than meaningful exploration, so the significance of the claimed result is not yet established.

major comments (4)
  1. [Section 3.2, Eq. (1) and Section 4.1, Eqs. (5)-(6)] The evaluation metrics d_vis_seq, d_text_seq, D_vis_grp, and D_text_grp are average pairwise cosine dissimilarities of the same VAE image embeddings and OCR text embeddings that appear in the reward terms r_vis_seq and r_text_seq. Because GRPO directly maximizes Eq. (1), the improvement in Table 1 from average diversity 0.21/0.17 to 0.53/0.54 is partially by construction and does not by itself demonstrate meaningful exploration. Please report independent evaluation metrics, such as OS-level state coverage, number of distinct applications or windows visited, human diversity ratings, or held-out exploration tasks, and provide error bars over multiple seeds.
  2. [Table 1 and Section 4.4] The main comparison in Table 1 is against frozen, statically deployed models; the only RL-trained control appears in the ablation study of Section 4.4, but that control is not included in Table 1. Without a baseline trained under the same RL loop with, for example, only the format reward, the reported gains cannot be attributed to the exploration rewards rather than to RL in general or to reward hacking of the similarity metric.
  3. [Section 5] The paper admits that agents get stuck on perpetually novel stimuli such as news and video pages, which is exactly the behavior incentivized by reward terms that maximize pairwise dissimilarity of screen embeddings. This admission means the high d_seq and D_grp scores in Table 1 may reflect trivial novelty rather than useful exploration breadth. The paper should quantify the distribution of visited applications, domains, and action types, and separate useful exploration from the noisy-TV trap.
  4. [Section 4.2, Table 1] The evaluation is based on only 20 episodes of 10 steps each (200 frames) per model, and no standard deviations, confidence intervals, or significance tests are reported. Given the high variance of GUI exploration trajectories, single-run point estimates are not sufficient to support the claim that trained models outperform baselines across the board.
minor comments (6)
  1. [Section 4.3] The text refers to 'Equation 3.2' when discussing format reward penalties; the correct reference is Eq. (1).
  2. [Figure 2 caption] The caption says 'All 9 Rewards' but the list in Section 3.2 contains six bullet categories; please clarify how the nine individual reward terms map to the six categories.
  3. [Table 1] The caption mentions 'box indicates the lowest scores,' but no boxes are visible in the rendered table; please check the formatting or remove the reference.
  4. [Table 1] The 'Avg. Diversity' column is not defined; please state explicitly that it is the mean of the four preceding diversity columns.
  5. [Section 3.4, Eq. (3)] The GRPO advantage formula uses group normalization but the notation omits the group index in the mean and standard deviation; please clarify the grouping notation.
  6. [Section 4.3 and Figure 4] The claim that a high and oscillating world-model loss 'indicates sustained curiosity' is not self-evident; a persistently high reconstruction loss could also mean the world model is failing to learn, which should be discussed.

Circularity Check

2 steps flagged · score 8.0 of 10

Evaluation diversity metric is the training reward; the reported exploration gains are by construction.

  1. self definitional [Section 4.1 Evaluation Metrics (Eqs. 5-6) vs. Section 3.2 Reward Function (Eq. 1)]
    "We use the cosine similarity sim(·,·) to measure the similarity between environmental states, as defined in subsection 3.2."

    The reward in Eq. (1) includes rvis_seq and rtext_seq, defined as E_{i<t,j>t}[1−sim(o'_i,o'_j)] and E_{i<t,j>t}[1−sim(e'_i,e'_j)], plus instantaneous and world-model dissimilarities in the same VAE image and OCR text embedding spaces. The evaluation metrics in Eqs. (5)-(6), dvis_seq and Dvis_grp, are the same average pairwise dissimilarities over the same embeddings. Since GRPO directly maximizes these dissimilarity terms, the reported improvement from 0.21/0.17 to 0.53/0.54 is the expected consequence of optimizing the evaluation metric itself, not an independent measure of exploration quality. No external validation, such as OS-level state coverage, human judgment, or held-out task diversity, is reported.

  2. fitted input called prediction [Appendix E Experience Stream Distillation and Section 4.2/Table 1]
    "We first filter the trajectories generated during RL training, identifying and retaining diverse exploration steps that successfully complete specific tasks. ... 3. Advantages greater than 0."

    The automated filtering criterion keeps transitions with positive GRPO advantage, and advantages are computed from r in Eq. (1), which contains the same cosine-similarity exploration rewards used as evaluation metrics in Eqs. (5)-(6). Distillation therefore performs SFT on trajectories selected for high values of the very quantity used to score the resulting model. The further diversity gains reported for ScreenExplorer-3B-Distill in Table 1 are thus a selection artifact of the reward-defined filter as much as evidence of an independently improved exploration capability.

full rationale

The central claim—that ScreenExplorer achieves better 'diverse exploration' than static models—is measured with Eqs. (5)-(6), which are average pairwise cosine dissimilarities of VAE image and OCR text embeddings. The training reward in Eq. (1) directly maximizes the same family of dissimilarities (rvis_seq, rtext_seq, rvis_inst, rtext_inst, rvis_world, rtext_world). Consequently, the headline improvement from 0.21/0.17 to 0.53/0.54 is the expected result of optimizing the evaluation metric itself, not an independent confirmation of exploration quality. The distillation loop compounds this: experience streams are filtered by positive advantage computed from the same reward, then the distilled model is evaluated with the same cosine-similarity metrics. The paper's own Limitations section concedes that only screenshots and OCR text were used to measure state similarity, and no OS-level state coverage, human judgment, or held-out metric is provided. The self-citation to ScreenAgent is not load-bearing, and the world-model prediction error is a real additional signal, but it is scored in the same embedding spaces and does not break the reward-evaluation loop. Score 8 reflects that the central diversity result is largely forced by the definition of the metric as the training objective.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numeric free parameters are fitted to data in the reward or evaluation. However, the choice of pretrained image/text encoders and the reward components themselves are selected by hand; these are assumptions rather than fitted numbers. The paper does not introduce new physical or theoretical entities; the world model is a learned neural network, not a postulated entity.

assumptions (4)
  • domain assumption Cosine similarity between learned embeddings measures GUI state diversity and novelty.
    Used in every reward term (Eq. 1) and in the evaluation metrics (Eqs. 5-6). If this similarity does not reflect meaningful exploration, both the training signal and the reported results are invalid. The paper provides no external validation of this measure.
  • domain assumption A single configured Linux desktop with internet access represents the open GUI world.
    All training and evaluation happen in one desktop environment (Appendix A). The claim of open-ended exploration and generalization to novel environments is not tested on any other environment.
  • ad hoc to paper The world model's prediction error is a useful curiosity signal that helps overcome cold start.
    The paper asserts that high world model loss indicates continued curiosity, but the loss can also reflect a poorly fitting model or noisy TV content. The assumption is central to the proposed curiosity reward.
  • domain assumption Pretrained VAE and text embedding models (Cosmos-Tokenizer, BGE-M3) faithfully represent screenshots and OCR text.
    All similarities and reconstructions operate in these pretrained latent spaces; no analysis checks whether these embeddings preserve GUI-relevant differences.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World." pith.science (2026). https://pith.science/paper/B77TG5T2

@misc{pith2026250519095,
  author       = {Pith},
  title        = {Pith review of: ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B77TG5T2}},
  note         = {Machine review of arXiv:2505.19095}
}
read the original abstract

The rapid progress of large language models (LLMs) has sparked growing interest in building Artificial General Intelligence (AGI) within Graphical User Interface (GUI) environments. However, existing GUI agents based on LLMs or vision-language models (VLMs) often fail to generalize to novel environments and rely heavily on manually curated, diverse datasets. To overcome these limitations, we introduce ScreenExplorer, a VLM trained via Group Relative Policy Optimization(GRPO) in real, dynamic, and open-ended GUI environments. Innovatively, we introduced a world-model-based curiosity reward function to help the agent overcome the cold-start phase of exploration. Additionally, distilling experience streams further enhances the model's exploration capabilities. Our training framework enhances model exploration in open GUI environments, with trained models showing better environmental adaptation and sustained exploration compared to static deployment models. Our findings offer a scalable pathway toward AGI systems with self-improving capabilities in complex interactive settings.

Figures

Figures reproduced from arXiv: 2505.19095 by the authors.

Figure 1
Figure 1. ScreenExplorer-3B-E1’s RL training leads to better GUI exploration diversity versus static models. During training, we design rewards to encourage actions that lead to successful interaction in novel environments. To further incentivize exploration, we introduce a World Model(WM) that learns the environment’s transition dy￾namics. We quantify the novelty of visited states by mea￾suring the discrepancy between the pr… view at source ↗
Figure 2
Figure 2. Framework overview: (a)We run M parallel environments for T steps per episode. At each step, the VLM takes state s and outputs an intent i and action a, the environment returns the post-action state s ′ , and the world model predicts the next state sˆ. All transitions are stored in a rollout buffer, where a reward function computes an exploration reward for each action. The VLM is then updated via GRPO, while the wo… view at source ↗
Figure 3
Figure 3. Examples of Trajectories from ScreenExplorer-3B-E1. Through RL training, the model developed increasingly effective interactions with the environment, enabling exploration of deeper pages. 0 100 Episode 0 2 4 6 8 Overall Reward r rmin-rmax 0 100 Episode 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 Format Reward r format 0 100 Episode 0.0 0.2 0.4 0.6 0.8 1.0 1.2 Visual Change Reward r vis inst r vis seq 0 100 Episode 0.0 0.2 0.4 … view at source ↗
Figures from the paper (27 more)
Figure 4
Figure 4. Figure 4: Indicators of ScreenExplorer-3B-E1 in training. During RL training, the VLM actor’s increasing rewards show improved environment interaction and state space exploration. The world model loss demonstrates sustained curiosity that drives further exploration. We trained t…
Figure 5
Figure 5. Figure 5: Rewards and metrics of exploration diversity in world model ablation study. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Advantage Range. By introducing the world model reward increases the variance of GRPO [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: World Model Architecture For action a, we use the LLaMA tokenizer to encode the action’s function-call-style string into word tokens. For all text on the screen extracted by the OCR module, we use a pre-trained text embedding model to project all text into one dense ve…
Figure 8
Figure 8. Figure 8: Training Pipeline Integrating RL and Experience Stream Distillation. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: The Qwen2.5-VL-3B model demonstrates spatial awareness capabilities. When the intent specifies "top-left corner of the screen", the model outputs coordinates (100, 100), which accurately corresponds to the screen’s top-left position. 19 [PITH_FULL_IMAGE:figures/full_f…
Figure 10
Figure 10. Figure 10: The Qwen2.5-VL-3B model followed the instructions in the prompt, attempting to utilize double click to interact with desktop icons. However, the specific coordinates were directly copied from the prompt text, indicating that, the language processing capabilities domin…
Figure 11
Figure 11. Figure 11: The original Qwen2.5-VL-3B model demonstrates the capability to correctly launch applications in a limited number of cases, with the intent descriptions aligning with the target actions. RL training can facilitate the emergence of such effective interactions. Intent: …
Figure 12
Figure 12. Figure 12: The original Qwen2.5-VL-72B model, due to its larger model scale, demonstrates enhanced capability in interpreting action intentions, recognizing UI elements within images and generates more precise coordinate outputs. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Impact of ablation studies on exploration metrics across different exploration reward [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: In a case study from the ablation group w/o Instant, while the agent successfully learned to open the browser, it demonstrated limited capability for deeper exploration. We conducted two additional interesting ablation studies: In the ablation setting w/o Visual, we r…
Figure 15
Figure 15. Figure 15: Intent descriptions generated by ScreenExplorer-3B-E1 and ablation w/o Intent-State Alignment group. the ScreenExplorer-3B-E1 model demonstrates a stronger propensity to reference existing on-screen text when generating intents, resulting in more specific and contextu…
Figure 16
Figure 16. Figure 16: The original Qwen2.5-VL-3B model: The initial Qwen2.5-VL-3B model generated near￾random coordinates, making it difficult to accurately click on icons and effectively interact with the environment, resulting in extremely low exploration rewards and diversity metrics. 2…
Figure 17
Figure 17. Figure 17: Episode-50 of ScreenExplorer-3B-E1: The model demonstrated the capability to output commands for launching desktop applications, resulting in significant screen state changes, though it had not yet developed exploration behaviors in specific apps [PITH_FULL_IMAGE:fig…
Figure 18
Figure 18. Figure 18: Episode-80 of ScreenExplorer-3B-E1: The model demonstrated the ability to navigate from Firefox’s homepage to explore a specific webpage [PITH_FULL_IMAGE:figures/full_fig_p023_18.png]
Figure 19
Figure 19. Figure 19: Episode-100 of ScreenExplorer-3B-E1: The model demonstrated the capability to navigate through news feeds and access articles for reading [PITH_FULL_IMAGE:figures/full_fig_p023_19.png]
Figure 20
Figure 20. Figure 20: Episode-100 of ScreenExplorer-3B-E1: The model was exploring the LibreOffice software interface. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_20.png]
Figure 21
Figure 21. Figure 21: At episode 70, the agent detected a DRM warning in the browser and observed that "stories [PITH_FULL_IMAGE:figures/full_fig_p024_21.png]
Figure 22
Figure 22. Figure 22: At episode 80, the agent scrolled down on the news page to acquire additional information, [PITH_FULL_IMAGE:figures/full_fig_p024_22.png]
Figure 23
Figure 23. Figure 23: At episode 100, the agent demonstrated cross-modal translation capabilities, recognizing [PITH_FULL_IMAGE:figures/full_fig_p024_23.png]
Figure 24
Figure 24. Figure 24: At episode 110, the intent description from [PITH_FULL_IMAGE:figures/full_fig_p024_24.png]
Figure 25
Figure 25. Figure 25: Comparison of rewards and metrics during RL training for free exploration tasks using [PITH_FULL_IMAGE:figures/full_fig_p025_25.png]
Figure 26
Figure 26. Figure 26: Episode-50 of Qwen2.5-VL-7B: Compared to the 3B model, the 7B model demonstrates superior proficiency in utilizing the ScrollDown command for web page navigation. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_26.png]
Figure 27
Figure 27. Figure 27: Episode-50 of Qwen2.5-VL-7B: The 7B model successfully completed a web browsing and software download sequence [PITH_FULL_IMAGE:figures/full_fig_p026_27.png]
Figure 28
Figure 28. Figure 28: Episode-90 of Qwen2.5-VL-7B: The 7B model discovered that browsing through images yields higher exploration rewards [PITH_FULL_IMAGE:figures/full_fig_p026_28.png]
Figure 29
Figure 29. Figure 29: Episode-100 of Qwen2.5-VL-7B: The 7B model successfully completed a product purchase flow, from product selection to adding items to the shopping cart. Intent: “To continue exploring the webpage and find relevant information about the incident and the corresponding vi…
Figure 30
Figure 30. Figure 30: At episode 160, the 7B model demonstrated superior image reasoning capabilities and [PITH_FULL_IMAGE:figures/full_fig_p026_30.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative Visual Code Mobile World Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    gWorld converts mobile GUI world modeling into renderable HTML generation, and its fine-tuned 8B and 32B VLMs outperform frontier open-weight models up to 50x larger on next-state accuracy.

Reference graph

Works this paper leans on

51 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    2023

  2. [2]

    Large language model-brained gui agents: A survey, 2025

    Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Zhang. Large language model-brained gui agents: A survey, 2025

    work page 2025

  3. [3]

    Karlsson, Bo An, Shuicheng Yan, and Zongqing Lu

    Weihao Tan, Wentao Zhang, Xinrun Xu, Haochong Xia, Ziluo Ding, Boyu Li, Bohan Zhou, Junpeng Yue, Jiechuan Jiang, Yewen Li, Ruyi An, Molei Qin, Chuqiao Zong, Longtao Zheng, Yujie Wu, Xiaoqiang Chai, Yifei Bi, Tianbao Xie, Pengjie Gu, Xiyun Li, Ceyao Zhang, Long Tian, Chaojie Wang, Xinrun Wang, Börje F. Karlsson, Bo An, Shuicheng Yan, and Zongqing Lu. Cradl...

    arXiv 2024

  4. [4]

    UFO: A UI-Focused Agent for Windows OS Interaction.arXiv preprint arXiv:2402.07939, 2024

    Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, Minghua Ma, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Zhang. UFO: A UI-Focused Agent for Windows OS Interaction.arXiv preprint arXiv:2402.07939, 2024

    arXiv 2024

  5. [5]

    UFO2: The Desktop AgentOS.arXiv preprint arXiv:2504.14603, 2025

    Chaoyun Zhang, He Huang, Chiming Ni, Jian Mu, Si Qin, Shilin He, Lu Wang, Fangkai Yang, Pu Zhao, Chao Du, Liqun Li, Yu Kang, Zhao Jiang, Suzhen Zheng, Rujia Wang, Jiaxu Qian, Minghua Ma, Jian-Guang Lou, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang. UFO2: The Desktop AgentOS.arXiv preprint arXiv:2504.14603, 2025

    arXiv 2025

  6. [6]

    Os-copilot: Towards generalist computer agents with self-improvement, 2024

    Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. Os-copilot: Towards generalist computer agents with self-improvement, 2024

    work page 2024

  7. [7]

    Agent S: An Open Agentic Framework that Uses Computers Like a Human

    Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent S: An Open Agentic Framework that Uses Computers Like a Human. InInternational Conference on Learning Representations (ICLR), 2025

    work page 2025

  8. [8]

    Agent s2: A compositional generalist-specialist framework for computer use agents, 2025

    Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang, Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist-specialist framework for computer use agents, 2025

    2025

Show all 51 references
  1. [9]

    Cogagent: A visual language model for gui agents, 2024

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogagent: A visual language model for gui agents, 2024

    2024

  2. [10]

    Rico: A mobile app dataset for building data-driven design applications

    Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. Rico: A mobile app dataset for building data-driven design applications. InProceedings of the 30th Annual ACM Symposium on User Interface Software and Techn...

    2017

  3. [11]

    Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, and Jeffrey P. Bigham. Screen recognition: Creating accessibility metadata for mobile applications from pixels, 2021. 10

    2021

  4. [12]

    Mind2web: Towards a generalist agent for the web, 2023

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web, 2023

    2023

  5. [13]

    Mobilevlm: A vision-language model for better intra- and inter-ui understanding, 2024

    Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Jianfeng Liu, Ang Li, Jian Luan, Bin Wang, and Shuo Shang. Mobilevlm: A vision-language model for better intra- and inter-ui understanding, 2024

    2024

  6. [14]

    Screenagent: A vision language model-driven computer control agent

    Runliang Niu, Jindong Li, Shiqi Wang, Yali Fu, Xiyu Hu, Xueyuan Leng, He Kong, Yi Chang, and Qi Wang. Screenagent: A vision language model-driven computer control agent. In Kate Larson, editor,Proceedings of the Thirty-Third International Joint Conference on Artificial Intel- ...

    2024

  7. [15]

    Guicourse: From general vision language models to versatile gui agents, 2024

    Wentong Chen, Junbo Cui, Jinyi Hu, Yujia Qin, Junjie Fang, Yue Zhao, Chongyi Wang, Jun Liu, Guirong Chen, Yupeng Huo, Yuan Yao, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Guicourse: From general vision language models to versatile gui agents, 2024

    2024

  8. [16]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025

  9. [17]

    Exploration by random network distillation, 2018

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation, 2018

    2018

  10. [18]

    Noveld: A simple yet effective exploration criterion

    Tianjun Zhang, Huazhe Xu, Xiaolong Wang, Yi Wu, Kurt Keutzer, Joseph E Gonzalez, and Yuandong Tian. Noveld: A simple yet effective exploration criterion. In M. Ranzato, A. Beygelz- imer, Y . Dauphin, P.S. Liang, and J. Wortman Vaughan, editors,Advances in Neural Information Pr...

    2021

  11. [19]

    Diversity is all you need: Learning skills without a reward function, 2018

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function, 2018

    2018

  12. [20]

    Dynamics- aware unsupervised discovery of skills, 2020

    Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman. Dynamics- aware unsupervised discovery of skills, 2020. 11

    2020

  13. [21]

    V oyager: An open-ended embodied agent with large language models, 2023

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models, 2023

    2023

  14. [22]

    Tenenbaum, Tim Rocktäschel, and Edward Grefenstette

    Andres Campero, Roberta Raileanu, Heinrich Küttler, Joshua B. Tenenbaum, Tim Rocktäschel, and Edward Grefenstette. Learning with amigo: Adversarially motivated intrinsic goals, 2021

    2021

  15. [23]

    Open-world reinforcement learning over long short-term imagination, 2025

    Jiajian Li, Qi Wang, Yunbo Wang, Xin Jin, Yang Li, Wenjun Zeng, and Xiaokang Yang. Open-world reinforcement learning over long short-term imagination, 2025

    2025

  16. [24]

    Huajian Xin, Z. Z. Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, Wenjun Gao, Qihao Zhu, Dejian Yang, Zhibin Gou, Z. F. Wu, Fuli Luo, and Chong Ruan. Deepseek-prover-v1.5: Harnessing proof assistant feedback for reinforcem...

    2024

  17. [25]

    Let’s verify step by step, 2023

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step, 2023

    2023

  18. [26]

    Mutual reasoning makes smaller llms stronger problem-solvers, 2024

    Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. Mutual reasoning makes smaller llms stronger problem-solvers, 2024

    2024

  19. [27]

    Lan Pan, Hanbo Xie, and Robert C. Wilson. Large language models think too fast to explore effectively, 2025

    2025

  20. [28]

    Improving rl exploration for llm reasoning through retrospective replay, 2025

    Shihan Dou, Muling Wu, Jingwen Xu, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. Improving rl exploration for llm reasoning through retrospective replay, 2025

    2025

  21. [29]

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning, 2025

    Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding self-evolution in ll...

    2025

  22. [30]

    Showui: One vision-language-action model for gui visual agent, 2024

    Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou. Showui: One vision-language-action model for gui visual agent, 2024

    2024

  23. [31]

    Foundations and recent trends in multimodal mobile agents: A survey.arXiv preprint arXiv:2411.02006, 2024

    Biao Wu, Yanda Li, Meng Fang, Zirui Song, Zhiwei Zhang, Yunchao Wei, and Ling Chen. Foundations and recent trends in multimodal mobile agents: A survey.arXiv preprint arXiv:2411.02006, 2024

    2024 arXiv

  24. [32]

    Mobile-agent: Autonomous multi-modal mobile device agent with visual perception, 2024

    Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent: Autonomous multi-modal mobile device agent with visual perception, 2024

    2024

  25. [33]

    Karlsson, Bo An, and Zongqing Lu

    Weihao Tan, Ziluo Ding, Wentao Zhang, Boyu Li, Bohan Zhou, Junpeng Yue, Haochong Xia, Jiechuan Jiang, Longtao Zheng, Xinrun Xu, Yifei Bi, Pengjie Gu, Xinrun Wang, Börje F. Karlsson, Bo An, and Zongqing Lu. Towards general computer control: A multimodal agent for red dead redem...

    2024

  26. [34]

    Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning, 2024

    Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning, 2024

    2024

  27. [35]

    Au- tonomous evaluation and refinement of digital agents, 2024

    Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Au- tonomous evaluation and refinement of digital agents, 2024

    2024

  28. [36]

    Distrl: An asynchronous distributed reinforcement learning framework for on-device control agents, 2025

    Taiyi Wang, Zhihao Wu, Jianheng Liu, Jianye Hao, Jun Wang, and Kun Shao. Distrl: An asynchronous distributed reinforcement learning framework for on-device control agents, 2025

    2025

  29. [37]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. 12

    2024

  30. [38]

    Lee, and Sanjeev Arora

    Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei, Jason D. Lee, and Sanjeev Arora. What makes a reward model a good teacher? an optimization perspective, 2025

    2025

  31. [39]

    Mavor-Parker, Kimberly A

    Augustine N. Mavor-Parker, Kimberly A. Young, Caswell Barry, and Lewis D. Griffin. How to stay curious while avoiding noisy tvs using aleatoric uncertainty estimation, 2024

    2024

  32. [40]

    Space") Key(

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating System...

    2023

  33. [42]

    The intent text description should clearly indicate a specific action (e.g., click, type, scroll) and specify a target (e.g., an icon, button, field, or screen location)

  34. [43]

    The executed action should accomplish the goal described in the intent

  35. [44]

    In addition to manual filtering, we implemented an automated filtering process without human intervention, based on the following criteria:

    The language in the intent should flow smoothly and contain no word repetitions. In addition to manual filtering, we implemented an automated filtering process without human intervention, based on the following criteria:

  36. [45]

    Begin from the 30-th episode

  37. [46]

    The output format is correct, wherer f ormat= 1

  38. [47]

    Advantages greater than0

  39. [48]

    The automated filtering employs the following prompts: You are evaluating whether an intent string clearly specifies a computer operation instruction

    The intent text should clearly indicate a specific action, evaluated bygpt-4o-mini-2024-07- 18. The automated filtering employs the following prompts: You are evaluating whether an intent string clearly specifies a computer operation instruction. A good intent should:

    2024

  40. [49]

    Clearly indicate a specific action (e.g., click, type, scroll)

  41. [50]

    Specify a target (e.g., an icon, button, field, or screen location)

  42. [51]

    Be unambiguous about what the user wants to accomplish

  43. [52]

    {{intent}}

    The language flows smoothly and there are no words repeated. Intent to evaluate: "{{intent}}" If the intent meets the criteria above, rewrite it as a clear task objective. If the intent does not meet the criteria, mark it as not good and use an empty string as the task. Keep u...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.