Pith. sign in

REVIEW 3 major objections 5 minor 45 references

ProgRM: Build Better GUI Agents with Progress Rewards

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that dense, stepwise progress rewards, produced by an LCS-based self-annotation pipeline, train GUI agents that outperform leading proprietary LLMs and outcome-reward-trained actors on the WikiHow benchmark.

desk verdict The LCS progress-labeling idea is real and honestly analyzed, but the scalable variant's gain over ORM is within noise; the strong result comes from environment-reward labels. read the letter →

arxiv 2505.18121 v1 pith:DW2QGAHF submitted 2025-05-23 cs.AI cs.CLcs.LG

Danyang Zhang , Situo Zhang , Ziyue Yang , Zichen Zhu , Zihan Zhao , Ruisheng Cao , Lu Chen , Kai Yu This is my paper
classification cs.AIcs.CLcs.LG
keywords GUIagentsprogressrewardmodelreinforcementlearningoutcomelongestcommonsubsequenceself-annotationWikiHowbenchmarkdenserewards
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that GUI agents learn better when reinforcement learning rewards are dense and stepwise, reflecting how much of a task has been completed, rather than a single final success or failure signal. To obtain such rewards without human annotation, it introduces an LCS-based self-annotation scheme that extracts common action subsequences, or recipes, from successful trajectories and labels key steps by matching to those recipes. On the WikiHow Android benchmark, actors trained with this Progress Reward Model reach 62.00% success rate, beating an outcome-reward-trained actor (58.67%) and strong proprietary LLMs such as Claude-3.7-Sonnet (56.00%).

What carries the argument

The central object is the Progress Reward Model (ProgRM): a pretrained LLM with an MLP and sigmoid head that maps the task instruction, action history, and current screen to a progress estimate $p_t \in [0,1]$, and the reward is the progress delta $\Delta p = p_t - p_{t-k}$. The labels that train it come from an LCS-based self-annotation pipeline that extracts recipes from successful trajectories, identifies key steps by matching to the best recipe, and assigns progress as the matched position in the recipe. The RL training adapts REINFORCE++ with token-level credit assignment so that progress rewards flow through the agent's action tokens.

What would settle it

Run the same RL pipeline on a second GUI benchmark that has no environment milestone rewards and compare the LCS-based ProgRM actor against an ORM-trained actor; if the LCS-based actor does not beat the ORM actor, the core claim is falsified. Alternatively, on a held-out set of WikiHow tasks, measure the correlation between LCS-assigned key steps and human-judged progress steps; a low correlation would show the labels do not capture true progress.

Watch

Extended reading notes

Core claim

The paper's central claim is that the change in predicted task-completion progress between successive states is a superior RL reward signal for GUI agents than the sparse outcome reward used by ORMs. Its ProgRM models progress with an LLM plus a sigmoid head, and the reward is the progress delta over a short history. The LCS-based self-labeler first groups successful trajectories by soft-LCS similarity, extracts a shared recipe per group, and then labels a trajectory's steps by their position in the best-matching recipe. The authors report that the resulting actor outperforms ORM-trained and proprietary-model baselines on WikiHow, and that the automatically labeled variant, while effective (59.33% success), leaves a substantial gap in key-step accuracy (0.126 error) compared to environment-reward-based labels (0.036), which they attribute to the difficulty of automatic key-step discovery.

Load-bearing premise

The whole pipeline rests on the assumption that successful trajectories of a task contain a shared core step sequence, and that matching steps to that sequence marks genuine progress; if that assumption is false, the automatically assigned progress labels misalign with true task progress and the rewards mislead training.

Editorial extensions

If this is right

  • If the central claim holds, dense progress rewards can replace outcome rewards for GUI-agent RL, giving credit to partially successful trajectories and improving both success rate and training stability.
  • The LCS-based self-annotation algorithm provides a way to generate process-level supervision without human experts or Monte-Carlo search, lowering the cost of RL training for GUI agents.
  • The measured gap between LCS-based and environment-based labels implies that better key-step discovery could push auto-labeled actors closer to the 62.00% level, and the authors show that longer training (30K steps) raises success to 67.33%.
  • Because ProgRM is a trajectory-level reward model, it can be combined with any trajectory-level RL algorithm, not only the REINFORCE++ variant used in the experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper leaves implicit: applying the LCS recipe extraction to web or desktop GUI benchmarks that lack milestone rewards would show whether the labeler's assumption of shared task recipes holds beyond the WikiHow app.
  • The large key-step error gap (0.126 vs 0.036) suggests that a better grouping or matching criterion—perhaps incorporating screen state or semantic action embeddings—could close most of the gap between the automatic and environment-based variants.
  • The finding that history length $k=1$ outperforms $k=3$ suggests that very local credit assignment matters, which could be relevant to other agentic RL settings where progress is measurable per step.
  • If tasks are highly heterogeneous and successful trajectories share few steps, the recipe library may become too sparse, and the method may need per-task few-shot recipes or a fallback to outcome rewards to stay reliable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ProgRM, a progress reward model that predicts a scalar completion fraction for each step of a GUI-agent trajectory, and uses the per-step progress gain as a dense reward during online reinforcement learning. To avoid manual progress annotations, the authors introduce an LCS-based self-annotation pipeline that extracts "recipes" from successful trajectories, matches unseen trajectories to these recipes, and assigns progress labels according to matched positions. The method is evaluated on the WikiHow Android benchmark. In Table 1, the actor trained with ProgRM using environment milestone rewards reaches 62.00% success rate, surpassing Claude-3.7-Sonnet (56.00%) and an ORM-trained actor (58.67%); the LCS-based variant reaches 59.33%. The authors also analyze RM accuracy, key-step progress estimation error, latency, and failure modes.

Significance. If the result were robust, dense progress rewards would be a meaningful step beyond sparse outcome rewards for GUI-agent RL, and the LCS self-annotation pipeline would be an attractive way to avoid manual progress labeling. The paper has genuine strengths: the WikiHow environment's milestone rewards provide an independent yardstick for validating the LCS labels; the authors directly compare LCS-based and environment-based labels (Tables 2 and 3); and the reported inference latency of the trained RMs is appropriate for online training. The failure-mode analysis in Figure 3 is also useful. However, the central empirical claim is currently supported by a single 150-task test split with no variance estimates, and the headline number comes from the environment-oracle variant rather than the scalable LCS variant. The paper's own Section H and Table 3 concede a substantial gap between LCS-based and environment-based label quality, which makes the current evidence for the scalable contribution thinner than the abstract suggests.

major comments (3)
  1. [§3.2, Table 1] The main claim that ProgRM-trained actors "outperform" ORM-trained actors is not statistically supported for the scalable LCS variant. PROGRMLCS achieves 59.33% versus ORM's 58.67% on a single test set of 150 tasks, with no seeds, no confidence intervals, and no significance test. A binomial standard error at p≈0.59 with N=150 is about 4 percentage points, so the 0.66-point gap is well within sampling noise. Moreover, the per-category comparison is mixed: PROGRMLCS is below ORM on Cross-Page tasks (69.49 vs 72.88). The paper needs multiple independent seeds and either confidence intervals or a paired significance test across tasks before this comparison can be evaluated.
  2. [§2.2, Table 1, Section H] The headline result, 62.00% success rate, comes from PROGRMEnv, whose progress labels are derived directly from environment milestone rewards. As the paper itself says in §2.2, this signal is only available in environments that provide milestone-style intermediate rewards, so it is not the scalable contribution claimed in the abstract. The scalable contribution, PROGRMLCS, is not shown to improve over ORM beyond sampling noise. The paper should either present the oracle variant explicitly as a proof-of-concept upper bound, or provide substantially stronger evidence for the LCS variant. As written, the abstract's claim that the proposed self-annotation pipeline leads to better actors is not backed by the reported experiments.
  3. [§3.3, Table 3, Section H] Table 3 shows that PROGRMLCS has a key-step progress estimation error of 0.126, which is more than three times the 0.036 error of PROGRMEnv, and Section H admits that the LCS-based labels still have a "remarkable gap" relative to environment-reward labels. This is not merely a secondary quality metric: the method's central premise is that the LCS pipeline produces progress labels that can serve as reliable training signals. The current evidence suggests the LCS labels are substantially misaligned with true task progress, and the paper does not analyze how this label noise affects the downstream RL training. I would like to see an analysis of label noise versus actor performance, or an improved labeling algorithm that closes part of this gap.
minor comments (5)
  1. [§3.1] The word "devided" should be "divided" in the description of the WikiHow test-set categories.
  2. [Appendix B] The text states that when no successful trajectories are available, "a successful trajectory is manually annotated by the authors." This is worth acknowledging in the main text, because it slightly qualifies the claim that the pipeline is fully self-annotating.
  3. [Figures 5 and 6] The trajectories shown in Figure 5 and Figure 6(a) appear to be identical, and Figure 6(b) repeats another example; please clarify whether these are independent cases or whether the figures are duplicated unintentionally.
  4. [Section H] There is a typo in the last paragraph: "ethe nvironment" should be "the environment."
  5. [Table 4] The k=3 ablation row reports a single success-rate value with no variance estimate; given the small effect sizes elsewhere, a confidence interval would help the reader judge whether this degradation is real.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: progress labels are validated against external environment milestones, and the paper's own Limitations section shows the LCS labels are not equivalent to the environment-based labels.

full rationale

The paper's derivation chain is: LCS recipes are extracted from successful trajectories (Section 2.2); key steps are matched to recipes (Eq. 4); progress labels are assigned from recipe positions under the stated uniform-gain assumption; the ProgRM is trained to predict those labels with BCE loss (Eq. 5); and the predicted progress gains (Eq. 2) are used as dense RL rewards. Each of these steps defines a training target, but the paper does not present the training target itself as the evidence of success. The central empirical claim is actor success rate on the WikiHow benchmark (Table 1), which is an external environment signal not used to construct the PROGRMLCS labels. The paper additionally measures key-step progress error against environment milestone rewards (Section 3.3, Table 3), and the LCS-variant error (0.126) is substantially worse than the environment-label variant (0.036); the Limitations section (Section H) explicitly acknowledges this gap. This admission demonstrates that the LCS progress labels are not identical to the external yardstick by construction, which is the opposite of a self-definitional collapse. The only author-group self-citation that is load-bearing is the Mobile-Env/WikiHow benchmark [38], but that is a published, externally falsifiable benchmark with annotated tasks and milestone rewards, so it counts as independent support rather than circularity. No uniqueness theorem or ansatz is imported from the authors' prior work. The small PROGRMLCS-versus-ORM margin (59.33 vs 58.67 on 150 tasks, no significance test) is a statistical-evidence concern, not a circularity concern, and does not change this verdict.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The method depends on hand-set thresholds (theta_L, epsilon, k), a linear progress model over recipe positions, and the assumption that LCS-matched steps are the true key steps. WikiHow milestone rewards provide the only external check on these choices.

free parameters (4)
  • Trajectory grouping similarity threshold theta_L = 0.6
    Used to group successful trajectories before LCS recipe extraction; set by hand in Appendix A and affects which recipes are constructed and thus which steps are labeled as key steps.
  • Soft match penalty for NOTHING actions epsilon = 0.4
    Set in Appendix A for soft LCS matching; controls how empty actions count toward trajectory similarity and recipe construction.
  • Progress reward history length k = 1
    Hyperparameter in Eq. 2; the authors ablate k=3 and find worse results, so this choice materially changes the reward signal.
  • Failed-step sampling ratio = 62.58% of failed trajectory steps retained
    Appendix C samples failed steps to balance the PROGRM training set, changing the label distribution the model is trained on.
assumptions (5)
  • domain assumption Successful trajectories for the same task share common behavior patterns, and the LCS of those trajectories identifies key progress steps.
    Section 2.2, recipe library construction and key step discovery; this is the load-bearing premise behind the LCS self-annotation pipeline.
  • domain assumption Progress increases uniformly along the recipe, so a key step at recipe position kappa has progress kappa divided by the recipe length.
    Section 2.2, progress label assignment; this imposes a linear progress model on recipe steps.
  • domain assumption Non-key steps inherit the progress of the nearest preceding key step, meaning they contribute zero progress gain.
    Section 2.2, progress label assignment; this may under-credit genuinely useful exploratory steps.
  • domain assumption WikiHow milestone rewards are a reliable ground-truth proxy for task progress.
    Used in Section 3.3 to validate LCS key steps and to train PROGRMEnv.
  • standard math The LCS dynamic programming recurrence (Eq. 7) and its soft variant (Eq. 8) correctly compute the intended subsequence similarity.
    Appendix A; this is standard dynamic programming, but the soft-match weighting is a design choice.
invented entities (1)
  • Per-step task progress scalar Prog(s_t; g) independent evidence
    purpose: Provides dense intermediate reward via progress gain between states in Eq. 2.
    The paper validates progress estimates against WikiHow environment milestone rewards in Table 3, giving a falsifiable check within the benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ProgRM: Build Better GUI Agents with Progress Rewards." pith.science (2026). https://pith.science/paper/DW2QGAHF

@misc{pith2026250518121,
  author       = {Pith},
  title        = {Pith review of: ProgRM: Build Better GUI Agents with Progress Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DW2QGAHF}},
  note         = {Machine review of arXiv:2505.18121}
}
read the original abstract

LLM-based (Large Language Model) GUI (Graphical User Interface) agents can potentially reshape our daily lives significantly. However, current LLM-based GUI agents suffer from the scarcity of high-quality training data owing to the difficulties of trajectory collection and reward annotation. Existing works have been exploring LLMs to collect trajectories for imitation learning or to offer reward signals for online RL training. However, the Outcome Reward Model (ORM) used in existing works cannot provide finegrained feedback and can over-penalize the valuable steps in finally failed trajectories. To this end, we propose Progress Reward Model (ProgRM) to provide dense informative intermediate rewards by predicting a task completion progress for each step in online training. To handle the challenge of progress reward label annotation, we further design an efficient LCS-based (Longest Common Subsequence) self-annotation algorithm to discover the key steps in trajectories and assign progress labels accordingly. ProgRM is evaluated with extensive experiments and analyses. Actors trained with ProgRM outperform leading proprietary LLMs and ORM-trained actors, illustrating the effectiveness of ProgRM. The codes for experiments will be made publicly available upon acceptance.

Figures

Figures reproduced from arXiv: 2505.18121 by the authors.

Figure 1
Figure 1. Comparison of policy optimization methods. (a) Imitation Learning optimizes the agent’s [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Progress labeling algorithm. The proposed labeling algorithm consists of three stages: (a) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Actor failure mode analysis [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Statistics of the collected reward model (RM) training data for WikiHow. Figure 4(a) dis [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: A failed trajectory with partial progress showing with the progress scores predicted by [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Temporal variation of progress measurement over successful episodes. Each line in the [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 12 canonical work pages

  1. [1]

    Digi-q: Learning vlm q-value functions for training device-control agents

    Hao Bai, Yifei Zhou, Li Erran Li, Sergey Levine, and Aviral Kumar. Digi-q: Learning vlm q-value functions for training device-control agents. In The Thirteenth International Conference on Learning Representations

    work page

  2. [2]

    Digirl: Training in-the-wild device-control agents with autonomous re- inforcement learning

    Hao Bai, Yifei Zhou, Jiayi Pan, Mert Cemri, Alane Suhr, Sergey Levine, and Avi- ral Kumar. Digirl: Training in-the-wild device-control agents with autonomous re- inforcement learning. In Amir Globersons, Lester Mackey, Danielle Belgrave, An- gela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processing Syst...

    work page 2024

  3. [3]

    Learning about progress from experts

    Jake Bruce, Ankit Anand, Bogdan Mazoure, and Rob Fergus. Learning about progress from experts. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/ forum?id=sKc6fgce1zs

    work page 2023

  4. [4]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168

    arXiv

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv doi:10.48550/arxiv.2501.12948 2025

  6. [6]

    Dungeons and data: A large-scale nethack dataset

    Eric Hambro, Roberta Raileanu, Danielle Rothermel, Vegard Mella, Tim Rocktäschel, Heinrich Küttler, and Naila Murray. Dungeons and data: A large-scale nethack dataset. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors,Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processi...

    work page 2022

  7. [7]

    Reinforce++: A simple and efficient approach for aligning large language models

    Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262, 2025

    arXiv 2025

  8. [8]

    Exploring Expert Failures Improves LLM Agent Tuning

    Li-Cheng Lan, Andrew Bai, Minhao Cheng, Ruochen Wang, Cho-Jui Hsieh, and Tianyi Zhou. Exploring expert failures improves llm agent tuning. arXiv preprint arXiv:2504.13145, 2025

    work page Pith review arXiv 2025

Show all 45 references
  1. [9]

    Making language models better reasoners with step-aware verifier

    Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making language models better reasoners with step-aware verifier. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Associati...

    2023 doi

  2. [10]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-1...

    2024

  3. [11]

    UI-R1: enhancing action prediction of GUI agents by reinforcement learning

    Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Guanjing Xiong, and Hongsheng Li. UI-R1: enhancing action prediction of GUI agents by reinforcement learning. CoRR, abs/2503.21620, 2025. doi: 10.48550/ARXIV .2503.21620. URL https: //doi.org/10.48550/arXiv.2503.21620

    2025 doi arXiv

  4. [12]

    Shikhar Murty, Dzmitry Bahdanau, and Christopher D. Manning. Nnetscape navigator: Complex demonstrations for web agents without a demonstrator. CoRR, abs/2410.02907, 2024. doi: 10.48550/ARXIV .2410.02907. URLhttps://doi.org/10.48550/arXiv.2410.02907

    2024 doi arXiv

  5. [13]

    Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, and Shuyan Zhou

    Tianyue Ou, Frank F. Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, and Shuyan Zhou. Synatra: Turning indirect knowledge into direct demonstrations for digital agents at scale. In Amir Globersons, Lester Mackey, Danielle Bel...

    2024

  6. [14]

    Explorer: Scaling exploration-driven web trajectory synthesis for multimodal web agents

    Vardaan Pahuja, Yadong Lu, Corby Rosset, Boyu Gou, Arindam Mitra, Spencer Whitehead, Yu Su, and Ahmed Awadallah. Explorer: Scaling exploration-driven web trajectory synthesis for multimodal web agents. CoRR, abs/2502.11357, 2025. doi: 10.48550/ARXIV .2502.11357. URL https://do...

    2025 doi arXiv

  7. [15]

    Au- tonomous evaluation and refinement of digital agents

    Jiayi Pan, Yichi Zhang, Nicholas Tomlin, Yifei Zhou, Sergey Levine, and Alane Suhr. Au- tonomous evaluation and refinement of digital agents. CoRR, abs/2404.06474, 2024. doi: 10.48550/ARXIV .2404.06474. URLhttps://doi.org/10.48550/arXiv.2404.06474

    2024 doi arXiv

  8. [16]

    Webrl: Training LLM web agents via self-evolving online curriculum reinforcement learning

    Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong. Webrl: Training LLM web agents via self-evolving online curriculum reinforcement learning. CoRR, abs/2411.02...

    2024 doi arXiv

  9. [17]

    UI-TARS: pioneering automated GUI interaction with native agents.CoRR, abs/2501.12326, 2025

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuanye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei Zheng, C...

    2025 doi arXiv

  10. [18]

    Yuxiao Qu, Matthew Y . R. Yang, Amrith Setlur, Lewis Tunstall, Edward Emanuel Beeching, Ruslan Salakhutdinov, and Aviral Kumar. Optimizing test-time compute via meta reinforcement fine-tuning. CoRR, abs/2503.07572, 2025. doi: 10.48550/ARXIV .2503.07572. URL https: //doi.org/10...

    2025 doi arXiv

  11. [19]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors,Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Interna- tiona...

    2019

  12. [20]

    Step: Stacked llm policies for web actions

    Paloma Sodhi, SRK Branavan, Yoav Artzi, and Ryan McDonald. Step: Stacked llm policies for web actions. In First Conference on Language Modeling

  13. [21]

    R1-searcher: Incentivizing the search capability in llms via reinforcement learning

    Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. arXiv preprint arXiv:2503.05592, 2025

    2025 arXiv

  14. [22]

    Hongjin Su, Ruoxi Sun, Jinsung Yoon, Pengcheng Yin, Tao Yu, and Sercan Ö. Arik. Learn-by- interact: A data-centric framework for self-adaptive agents in realistic environments. CoRR, abs/2501.10893, 2025. doi: 10.48550/ARXIV .2501.10893. URL https://doi.org/10. 48550/arXiv.2501.10893

    2025 doi arXiv

  15. [23]

    Os-genesis: Automating GUI agent trajectory construction via reverse task synthesis

    Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, and Zhiyong Wu. Os-genesis: Automating GUI agent trajectory construction via reverse task synthesis. CoRR...

    2024 doi arXiv

  16. [24]

    Mobile-agent: Autonomous multi-modal mobile device agent with visual perception

    Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. Mobile-agent: Autonomous multi-modal mobile device agent with visual perception. CoRR, abs/2401.16158, 2024. doi: 10.48550/ARXIV .2401.16158. URL https://doi.org/ 10.48550/arXiv.2401.16158

    2024 doi arXiv

  17. [25]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeti...

    2024 doi

  18. [26]

    Distrl: An asynchronous distributed reinforcement learning framework for on-device control agents.CoRR, abs/2410.14803, 2024

    Taiyi Wang, Zhihao Wu, Jianheng Liu, Jianye Hao, Jun Wang, and Kun Shao. Distrl: An asynchronous distributed reinforcement learning framework for on-device control agents.CoRR, abs/2410.14803, 2024. doi: 10.48550/ARXIV .2410.14803. URL https://doi.org/10. 48550/arXiv.2410.14803

    2024 doi arXiv

  19. [27]

    Reinforcing language agents via policy optimization with action decomposition

    Muning Wen, Ziyu Wan, Weinan Zhang, Jun Wang, and Ying Wen. Reinforcing language agents via policy optimization with action decomposition. arXiv preprint arXiv:2405.15821, 2024

    2024 arXiv

  20. [28]

    Gui-r1: A generalist r1-style vision-language action model for gui agents

    Xiaobo Xia and Run Luo. Gui-r1: A generalist r1-style vision-language action model for gui agents. arXiv preprint arXiv:2504.10458, 2025

    2025 arXiv

  21. [29]

    Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768, 2025

    2025 arXiv

  22. [30]

    Aguvis: Unified pure vision agents for autonomous GUI interaction

    Yiheng Xu, Zekun Wang, Junli Wang, Dunjie Lu, Tianbao Xie, Amrita Saha, Doyen Sahoo, Tao Yu, and Caiming Xiong. Aguvis: Unified pure vision agents for autonomous GUI interaction. CoRR, abs/2412.04454, 2024. doi: 10.48550/ARXIV .2412.04454. URL https://doi.org/ 10.48550/arXiv.2...

    2024 doi arXiv

  23. [31]

    Agenttrek: Agent trajectory synthesis via guiding replay with web tutorials

    Yiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang, Zekun Wang, Yuchen Mao, Caiming Xiong, and Tao Yu. Agenttrek: Agent trajectory synthesis via guiding replay with web tutorials. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 2...

    2025

  24. [32]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 doi arXiv

  25. [33]

    Outcome-supervised verifiers for planning in mathematical reasoning

    Fei Yu, Anningzhe Gao, and Benyou Wang. Outcome-supervised verifiers for planning in mathematical reasoning. CoRR, abs/2311.09724, 2023. doi: 10.48550/ARXIV .2311.09724. URL https://doi.org/10.48550/arXiv.2311.09724

    2023 doi arXiv

  26. [34]

    Free process rewards without process labels

    Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kai Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels. CoRR, abs/2412.01981,

    arXiv

  27. [35]

    UFO: A ui-focused agent for windows OS interaction

    Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, Minghua Ma, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, and Qi Zhang. UFO: A ui-focused agent for windows OS interaction. CoRR, abs/2402.07939, 2024. doi: 10.48550/ARXIV .2402.07939. URL https://doi.org/...

    2024 doi arXiv

  28. [36]

    Appagent: Multimodal agents as smartphone users

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. Appagent: Multimodal agents as smartphone users. In Naomi Yamashita, Vanessa Evers, Koji Yatani, Sharon Xianghua Ding, Bongshin Lee, Marshini Chetty, and Phoebe O. Toups Duga...

    2025

  29. [37]

    Large language models are semi-parametric reinforcement learning agents

    Danyang Zhang, Lu Chen, Situo Zhang, Hongshen Xu, Zihan Zhao, and Kai Yu. Large language models are semi-parametric reinforcement learning agents. In Alice Oh, Tris- tan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Informat...

    2023

  30. [38]

    Mobile-env: Building qualified evaluation benchmarks for llm-gui interaction

    Danyang Zhang, Zhennan Shen, Rui Xie, Situo Zhang, Tianbao Xie, Zihan Zhao, Siyuan Chen, Lu Chen, Hongshen Xu, Ruisheng Cao, et al. Mobile-env: Building qualified evaluation benchmarks for llm-gui interaction. arXiv preprint arXiv:2305.08144, 2023

    2023 arXiv

  31. [39]

    The lessons of developing process reward models in mathematical reasoning

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025

    2025 arXiv

  32. [40]

    Gpt-4v(ision) is a generalist web agent, if grounded

    Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. Gpt-4v(ision) is a generalist web agent, if grounded. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openreview. net/forum?id...

    2024

  33. [41]

    VEM: environment-free exploration for training GUI agent with value environment model

    Jiani Zheng, Lu Wang, Fangkai Yang, Chaoyun Zhang, Lingrui Mei, Wenjie Yin, Qingwei Lin, Dongmei Zhang, Saravan Rajmohan, and Qi Zhang. VEM: environment-free exploration for training GUI agent with value environment model. CoRR, abs/2502.18906, 2025. doi: 10.48550/ARXIV .2502....

    2025 doi arXiv

  34. [42]

    Proposer-agent-evaluator(pae): Autonomous skill discovery for foundation model internet agents

    Yifei Zhou, Qianlan Yang, Kaixiang Lin, Min Bai, Xiong Zhou, Yu-Xiong Wang, Sergey Levine, and Li Erran Li. Proposer-agent-evaluator(pae): Autonomous skill discovery for foundation model internet agents. CoRR, abs/2412.13194, 2024. doi: 10.48550/ARXIV .2412.13194. URL https://...

    2024 doi arXiv

  35. [43]

    prototype

    Zichen Zhu, Hao Tang, Yansi Li, Dingye Liu, Hongshen Xu, Kunyao Lan, Danyang Zhang, Yixuan Jiang, Hao Zhou, Chenrun Wang, et al. Moba: Multifaceted memory-enhanced adaptive planning for efficient mobile task automation. In Proceedings of the 2025 Conference of the Nations of t...

    2025

  36. [2019]

    URL https://doi.org/10.18653/v1/D19-1410

    doi: 10.18653/V1/D19-1410. URL https://doi.org/10.18653/v1/D19-1410

    doi

  37. [2024]

    URL https://doi.org/10.48550/arXiv.2412

    doi: 10.48550/ARXIV .2412.01981. URL https://doi.org/10.48550/arXiv.2412. 01981

    doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.