Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Online RL for multi-turn computer-use agents works when trajectory advantages, task selection, and async infrastructure are redesigned for context-managed steps.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 15:59 UTC pith:R7B5XDZ6

load-bearing objection Solid systems-and-methods paper: STEPO’s length-bias fix is real, the 63.2% is competitive, and the main soft spot is policy-specific filtering plus missing variance—not a broken objective. the 3 major comments →

arxiv 2607.09773 v1 pith:R7B5XDZ6 submitted 2026-07-07 cs.AI cs.CLcs.LG

EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents

Mianqiu Huang , Taofeng Xue , Chong Peng , Jinrui Ding , Sicheng Fan , Jiale Hong , Yufei Gao , Xiaocheng Zhang
show 7 more authors
Linsen Guo Xin Yang Dengchang Zhao Yuchen Xie Peng Pei Xunliang Xie Xipeng Qiu
This is my paper
classification cs.AI cs.CLcs.LG
keywords computer-use agentsonline reinforcement learningmulti-turn GUI agentsstep-level policy optimizationadaptive curriculumasynchronous RLOSWorldcontext management
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that computer-use agents cannot fully learn long-horizon desktop tasks from static offline traces, because each action changes the screen, the future action space, and recovery options. It extends prior offline self-evolution into online reinforcement learning inside executable sandboxes with verifiable terminal rewards. The authors claim that single-turn language RL recipes fail here: context management turns each trajectory into many step-level samples, rewards are sparse, trajectory lengths vary, and environments are slow. Their fix is a joint stack—Step-Level Policy Optimization that redistributes trajectory advantages evenly across steps, policy-aware filtering of verifiable tasks, a three-channel adaptive curriculum, and staleness-controlled asynchronous training with mini-group batching. With that stack, a 32B agent reaches 63.2% success on OSWorld-Verified, beating comparable open-weight models and approaching much larger systems. A sympathetic reader cares because the paper treats online computer-use RL as a coupled algorithm–data–system problem rather than a prompt-level objective transplant.

Core claim

The paper establishes that multi-turn computer-use agents can be trained effectively with online RL if trajectory-level advantages are conserved after context-managed step decomposition, training data is filtered and calibrated to the current policy’s pass rates, curricula balance learnable tasks with hard positive replay and controlled infeasible exposure, and experience generation is decoupled from updates under staleness control. Under those conditions, online interaction becomes a scalable training signal and yields 63.2% Pass@1 on OSWorld-Verified for a 32B model.

What carries the argument

Step-Level Policy Optimization (STEPO): compute a group-normalized trajectory advantage Ai, then assign Ai/|Ti| to each context-managed step so total advantage mass and zero-sum group balance are preserved after decomposition into turn-level samples.

Load-bearing premise

The whole result leans on sandbox validators and policy-dependent pass-rate filters giving a clean, non-hackable signal that intermediate-difficulty tasks stay informative as the agent improves.

What would settle it

Train the same backbone and budget with STEPO, DTAC, and async mini-group batching, but replace the filtered intermediate-pass-rate task pool with unfiltered or misvalidated tasks; if OSWorld-Verified gains disappear or PRM-style hacking reappears while terminal success stays flat, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Naive GRPO after multi-turn trajectory decomposition should be replaced by advantage redistribution that keeps group balance.
  • Online computer-use data must be recalibrated per policy; a high-SNR subset for one model need not help another.
  • Mini-group batching and staleness windows become first-class requirements once trajectories have variable step counts and slow sandbox feedback.
  • Process reward models need alignment checks against executable final success, or they can raise intermediate scores without improving completion.
  • Cross-domain office-only online RL can still lift Daily and Professional categories by strengthening shared atomic skills.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If validators remain the bottleneck, scaling environments without better milestone or state-difference credit assignment will mostly add reward noise rather than capability.
  • Matching training-time and inference-time context management is itself a policy decision; agents that learn adaptive folding or summarization may need STEPO-style objectives redefined over those managed contexts.
  • GUI–CLI hybrid agents will inherit the same length-bias and group-structure issues whenever sparse terminal rewards are decomposed across mixed action streams.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. EvoCUA-1.5 proposes an online RL framework for multi-turn computer-use agents that interact with executable desktop sandboxes under context management and sparse terminal rewards. The main technical pieces are STEPO (trajectory-level GRPO advantages redistributed as Ai/|Ti| after step decomposition; Eqs. 5–13), policy-aware filtering/pass-rate calibration of verifiable synthesized tasks, Dynamic Tri-Adaptive Curriculum (VAS + AdaPR + ICS), and a staleness-controlled asynchronous rollout–buffer–training stack with mini-group batching. On OSWorld-Verified the 32B model reports 63.2% Pass@1 (100 steps), with supporting ablations for STEPO (Fig. 6), mini-group batching (Table 3), filtered vs full data (Table 4), DTAC (Table 6), and some cross-platform/cross-domain transfer (Tables 2, 7).

Significance. If the gains hold under stronger controls, this is a useful systems-and-algorithm contribution for multi-turn GUI agents: it correctly identifies that context-managed trajectories break naive GRPO length balance, and it packages a practical online stack (objective + curriculum + async infrastructure) rather than a single-turn RL transplant. Strengths include a clear bias analysis for STEPO (Eqs. 5–12 and Appendix B), executable binary validators as the primary reward, and component ablations that move beyond pure leaderboard reporting. The work is of interest to the computer-use and agent RL communities even if the headline 63.2% is partly data-selection driven.

major comments (3)
  1. [§6.2, Table 1] Table 1 reports a single 63.2% Pass@1 for EvoCUA-1.5-32B with no multi-seed variance, confidence intervals, or repeated evaluation protocol. Given stochastic multi-turn rollouts and policy-dependent task selection, this is insufficient to support the claim of a new 32B-scale SOTA or of stable improvement over EvoCUA-32B (56.7%/57.8%). At minimum, report evaluation variance and clarify whether the same max-step budget, backbone iteration, and decoding settings are held fixed against the closest open baselines (e.g., CUA-GYM-35B, Kimi-K2.5).
  2. [§6.3; Fig. 6; Tables 3, 4, 6] The central claim attributes the OSWorld result to the full stack (STEPO + filtering + DTAC + async mini-group training), but ablations are partial and mostly not on the final 32B EvoCUA-1.5 run: STEPO is shown on OpenCUA-32B (Fig. 6), mini-group batching on Qwen3-VL-8B (Table 3), DTAC on Qwen3-VL-32B (Table 6). There is no leave-one-component-out or matched-budget ablation that isolates each piece on the model that produces 63.2%. Without that, the headline number cannot be cleanly attributed to the proposed methods versus policy-specific data curation.
  3. [§4.1–4.2; Tables 4–5, 8] Tables 4–5 show that filtering helps and that a subset effective for EvoCUA-8B fails to transfer (or slightly hurts) EvoCUA-32B, which the paper itself treats as policy-dependent data quality. Combined with fixed curriculum bounds in Table 8 (Plow=0.125, Phigh=0.375, ρinf=0.025) and no sensitivity analysis, this leaves open the skeptic’s concern that intermediate-pass-rate filtering may overfit the current policy rather than yield a general high-SNR curriculum. Please add sensitivity on these thresholds and, if possible, a held-out or cross-policy evaluation of the filtered pool.
minor comments (4)
  1. [Figure 1; Table 1] Figure 1 and Table 1 mix models with different max-step budgets (50 vs 100); make the comparison protocol more explicit in the caption and text so readers do not over-read cross-budget rankings.
  2. [§3.2, Eqs. (10)–(13)] Notation for advantages switches among Ai, ÂSTEPO_i,t, and Âi,t in Eqs. (10)–(13); unify symbols and state whether KL regularization is used in the reported runs.
  3. [§6.3, Figure 7] The PRM reward-hacking observation (Fig. 7) is useful but qualitative; a short quantitative summary (e.g., PRM score vs terminal success correlation by difficulty) would strengthen §6.3.
  4. [§5; Appendix D] Appendix D lists hyperparameters but not group size G, number of tasks, total rollout steps, or wall-clock/async throughput; these are needed for reproducibility of the infrastructure claims in §5.

Circularity Check

1 steps flagged

No significant circularity: empirical multi-turn RL methods with external binary validators and held-out OSWorld-style evaluation; only mild prior-work dependence for task synthesis.

specific steps
  1. self citation load bearing [Section 4 / Abstract; EvoCUA citation [Xue et al., 2026]]
    "For candidate generation, EvoCUA-1.5 follows the verifiable task synthesis pipeline introduced in EvoCUA [Xue et al., 2026]: tasks are generated from environment priors and atomic computer-use abilities, and are paired with sandbox configurations and executable validators."

    The online training pool is built on the authors’ prior EvoCUA synthesis pipeline rather than an independent external task source. This is ordinary sequential research scaffolding, not a uniqueness theorem or a fit renamed as prediction: rewards and final OSWorld evaluation remain external. Flagged only as mild self-dependence, not as a reduction of the 63.2% claim to its inputs.

full rationale

EvoCUA-1.5 is an empirical systems/methods paper, not a first-principles derivation that redefines its target. Terminal rewards are external executable validators R(sT;g)=I[Vg(sT)=True], not quantities fitted to the reported OSWorld metric. STEPO redistributes already-computed group-normalized trajectory advantages as Ai/|Ti| so that sum_t Âi,t = Ai and group balance is preserved after context-managed step decomposition; this is a length-normalization design choice, not a prediction forced by fitting the evaluation score. DTAC (VAS/AdaPR/ICS) and policy-aware pass-rate filtering select training tasks from a synthesized pool using current-policy statistics, but success is measured on external OSWorld-Verified / WindowsAgentArena / MacOSArena benchmarks rather than on the filtered training pass rates themselves. The only mild dependence is inheritance of EvoCUA’s verifiable task-synthesis pipeline (self-citation of prior work by overlapping authors), which supplies candidate tasks and validators but does not force the 63.2% Pass@1 result or the STEPO objective by construction. Ablations and cross-platform tables further treat components as empirical interventions, not uniqueness theorems. Score 1 reflects that minor prior-work scaffolding without load-bearing circular reduction of the central claim.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

The central empirical claim rests on standard RL math plus several operational choices that define the training distribution and advantage assignment. Free parameters are mostly curriculum/buffer knobs in Table 8 and GRPO/PPO-style clipping/KL settings. Domain assumptions include binary executable validators as ground truth, sliding-window context as the training view, and Bernoulli pass-rate variance as a proxy for learnability. No new physical entities are postulated; STEPO/DTAC are algorithmic constructs.

free parameters (6)
  • EMA smoothing coefficient β = 0.7
    Controls how fast task pass-rate estimates track the policy; set to 0.7 in Table 8 and shapes VAS weights.
  • AdaPR pass-rate interval [Plow, Phigh] = 0.125–0.375
    Defines which hard tasks get positive replay; hand-set to 0.125–0.375 and directly affects curriculum composition.
  • ICS infeasible sampling ratio ρinf = 0.025
    Fixed fraction of infeasible tasks per batch; set to 0.025 and changes failure-handling training signal.
  • Staleness threshold = 3
    Maximum policy-version lag retained in the async buffer; set to 3 and trades throughput vs on-policyness.
  • Maximum AdaPR replay count = 3
    Caps reuse of rare successful hard groups; set to 3 to limit over-amplification.
  • Policy-clipping coefficient εc / KL regularization strength
    Standard PPO/GRPO stability knobs in the STEPO objective; values not fully enumerated but required for the reported optimization behavior.
axioms (5)
  • domain assumption Group-normalized trajectory advantages (GRPO-style) with approximately zero group mean are a valid contrastive learning signal for binary terminal rewards.
    Section 3 builds STEPO on Ai = (Ri − mean(R))/(std(R)+ε) and conserves Σ Ai after step decomposition.
  • domain assumption Training samples must be built after inference-time context management (sliding window), so each executable turn is a separate step-level sample.
    Section 2.1 and Figure 2 make this the reason naive trajectory masking is invalid.
  • domain assumption Executable sandbox validators provide sufficiently reliable binary success labels for online RL.
    Reward definition R = I[Vg(sT)=True] in Section 2 and filtering pipeline in Section 4.1 depend on validator trustworthiness.
  • ad hoc to paper Bernoulli variance/entropy of pass rate is a good proxy for task informativeness under grouped rollouts.
    Section 4.2.1 sets w = 4 P_EMA (1−P_EMA) from Var(r)=P(1−P); useful heuristic, not a theorem of optimal curriculum.
  • ad hoc to paper Uniform redistribution of trajectory advantage across steps (Ai/|Ti|) is preferable to length-weighted reuse for computer-use trajectories.
    STEPO definition in Section 3.2; motivated by bias analysis and a mild length preference, supported by one ablation figure rather than a general optimality proof.
invented entities (2)
  • Step-Level Policy Optimization (STEPO) no independent evidence
    purpose: Assign trajectory-level GRPO advantages to context-managed step samples without length-weighted group imbalance.
    Core algorithmic construct of the paper; independent evidence is empirical ablation (Fig. 6), not an external physical prediction.
  • Dynamic Tri-Adaptive Curriculum (DTAC) no independent evidence
    purpose: Compose each batch from variance-adaptive tasks, hard positive replay, and controlled infeasible tasks.
    Curriculum mechanism introduced to stabilize sparse-reward online RL; support is ablation Table 6 within this training setup.

pith-pipeline@v1.1.0-grok45 · 22687 in / 4023 out tokens · 50944 ms · 2026-07-14T15:59:55.381178+00:00 · methodology

0 comments
read the original abstract

Computer-use agents must solve long-horizon tasks through repeated interaction with partially observable, multimodal desktop environments. Although imitation learning and offline trajectory refinement provide strong priors, static traces cannot cover the causal feedback loop of real computer use: each action changes the screen state, future action space, and recovery options. EvoCUA-1.5 extends self-evolving computer-use agents from offline experience learning to online reinforcement learning, where policies interact with executable sandbox environments and improve from verifiable task outcomes. Online RL in this setting requires more than directly reusing single-turn language-RL recipes. Multi-turn interaction introduces context-managed observations, sparse terminal rewards, variable-length trajectories, and slow environment feedback. EvoCUA-1.5 addresses these challenges with Step-Level Policy Optimization (STEPO), which preserves trajectory-level advantage balance after decomposition into step-level samples; policy-aware filtering and pass-rate calibration over verifiable synthesized tasks; Dynamic Tri-Adaptive Curriculum (DTAC), which combines learnable tasks, difficult positive replay, and controlled infeasible-task exposure; and a fully asynchronous RL infrastructure with staleness control and mini-group batching. Experiments show that these components improve training stability and downstream performance. EvoCUA-1.5 achieves 63.2\% success on OSWorld-Verified, outperforming comparable 32B/35B-scale open-weight baselines and even approaching models with significantly larger parameter counts. Overall, EvoCUA-1.5 provides a practical framework for scaling online RL in multi-turn computer-use agents.

Figures

Figures reproduced from arXiv: 2607.09773 by Chong Peng, Dengchang Zhao, Jiale Hong, Jinrui Ding, Linsen Guo, Mianqiu Huang, Peng Pei, Sicheng Fan, Taofeng Xue, Xiaocheng Zhang, Xin Yang, Xipeng Qiu, Xunliang Xie, Yuchen Xie, Yufei Gao.

Figure 1
Figure 1. Figure 1: Performance comparison on OSWorld-Verified. EvoCUA-1.5 achieves 63.2% success rate, outperforming [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Context management in multi-turn computer-use agent training. The full trajectory history is transformed [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Atomic-ability coverage in the synthesized task pool. EvoCUA-1.5 builds on EvoCUA’s verifiable task [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of Dynamic Tri-Adaptive Curriculum (DTAC). Each training batch combines tasks selected by [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Asynchronous online RL infrastructure. Rollout workers continuously generate computer-use trajectories, [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Training dynamics of naive multi-turn GRPO and STEPO on OpenCUA-32B. STEPO improves reward more [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Process reward models can provide misleading optimization signals when their preferences are not aligned [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

    cs.AI 2026-07 conditional novelty 5.5

    A real-device-centric foundation GUI agent with hybrid GUI+CLI batched actions, AutoResearch data flywheel, online RL, and a proactive harness reaches SOTA mobile and competitive desktop/web scores.

Reference graph

Works this paper leans on

17 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025a

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv

  2. [2]

    Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Henry Wu, et al

    GitHub repository. Xinyuan Wang, Bowen Wang, Dunjie Lu, Junlin Yang, Tianbao Xie, Junli Wang, Jiaqi Deng, Xiaole Guo, Yiheng Xu, Chen Henry Wu, et al. Opencua: Open foundations for computer-use agents.arXiv preprint arXiv:2508.09123, 2025a. Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Lia...

    arXiv

  3. [3]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al

    URL https: //arxiv.org/abs/2601.15876. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    arXiv

  4. [4]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Pith/arXiv arXiv

  5. [5]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    Pith/arXiv arXiv

  6. [6]

    Haolong Yan, Jia Wang, Xin Huang, Yeqing Shen, Ziyang Meng, Zhimin Fan, Kaijun Tan, Jin Gao, Lieyu Shi, Mi Yang, et al

    Accessed: 2025-10-01. Haolong Yan, Jia Wang, Xin Huang, Yeqing Shen, Ziyang Meng, Zhimin Fan, Kaijun Tan, Jin Gao, Lieyu Shi, Mi Yang, et al. Step-gui technical report.arXiv preprint arXiv:2512.15431,

    arXiv 2025

  7. [7]

    Qwen Team

    Accessed: 2025-10-31. Qwen Team. Qwen3.7-plus: Multimodal agent intelligence.https://qwen.ai/blog?id=qwen3.7-plus,

    2025

  8. [8]

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shi- jue Huang, et al

    URLhttps://github.com/OpenGVLab/ScaleCUA. Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shi- jue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents.arXiv preprint arXiv:2501.12326,

    Pith/arXiv arXiv

  9. [9]

    Mobile-agent-v3: Foundamental agents for gui automation.arXiv preprint arXiv:2508.15144,

    Jiabo Ye, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Zhaoqing Zhu, Ziwei Zheng, Feiyu Gao, Junjie Cao, Zhengxi Lu, et al. Mobile-agent-v3: Foundamental agents for gui automation.arXiv preprint arXiv:2508.15144,

    Pith/arXiv arXiv

  10. [10]

    Kimi Team

    URLhttps://arxiv.org/abs/2605.25624. Kimi Team. Kimi k2.5: Visual agentic intelligence, 2026a. URLhttps://arxiv.org/abs/2602.02276. Kimi Team. Kimi k2.6: Advancing open-source coding.https://www.kimi.com/blog/kimi-k2-6, 2026b. MiniMax Team. Minimax m3: Frontier coding, 1m context, and native multimodality. https://www.minimaxi.com /blog/minimax-m3,

    Pith/arXiv arXiv

  11. [11]

    URLhttps://arxiv.org/abs/2409.08264. Xuehui Wang, Zhenyu Wu, JingJing Xie, Zichen Ding, Bowen Yang, Zehao Li, Zhaoyang Liu, Qingyun Li, Xuan Dong, Zhe Chen, Weiyun Wang, Xiangyu Zhao, Jixuan Chen, Haodong Duan, Tianbao Xie, Chenyu Yang, Shiqian Su, Yue Yu, Yuan Huang, Yiqian Liu, Xiao Zhang, Yanting Zhang, Xiangyu Yue, Weijie Su, Xizhou Zhu, Wei Shen, Jif...

    Pith/arXiv arXiv

  12. [12]

    Digi-q: Learning q-value functions for training device-control agents.arXiv preprint arXiv:2502.15760, 2025b

    Hao Bai, Yifei Zhou, Li Erran Li, Sergey Levine, and Aviral Kumar. Digi-q: Learning q-value functions for training device-control agents.arXiv preprint arXiv:2502.15760, 2025b. Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents.arXiv pr...

    Pith/arXiv arXiv

  13. [13]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. InInternational Conference on Learning Representations, volume 2024, pages 15585–15606,

    2024

  14. [14]

    Visualwebarena: Evaluating multimodal agents on realistic visual web tasks

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. InICLR 2024 Workshop on Large Language Model (LLM) Agents. Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, ...

    Pith/arXiv arXiv 2024

  15. [15]

    Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning

    18 EvoCUA-1.5: Online RL for Computer-Use Agents Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, et al. Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning. InICML 2025 Workshop on Computer Use Agents. Yuhang Liu, Zeyu Liu, Shuanghe Zhu, Pengxiang Li, Congkai Xi...

    2025

  16. [16]

    Ui-s1: Advancing gui automation via semi-online reinforcement learning.arXiv preprint arXiv:2509.11543,

    Zhengxi Lu, Jiabo Ye, Fei Tang, Yongliang Shen, Haiyang Xu, Ziwei Zheng, Weiming Lu, Ming Yan, Fei Huang, Jun Xiao, et al. Ui-s1: Advancing gui automation via semi-online reinforcement learning.arXiv preprint arXiv:2509.11543,

    arXiv

  17. [17]

    Efficient multi-turn rl for gui agents via decoupled training and adaptive data curation.arXiv preprint arXiv:2509.23866,

    Pengxiang Li, Zechen Hu, Zirui Shang, Jingrong Wu, Yang Liu, Hui Liu, Zhi Gao, Chenrui Shi, Bofei Zhang, Zihao Zhang, et al. Efficient multi-turn rl for gui agents via decoupled training and adaptive data curation.arXiv preprint arXiv:2509.23866,

    arXiv