Pith. sign in

REVIEW 3 major objections 5 minor 63 references

Training computer-use agents improves more from deep, co-evolving worlds than from more environments.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 18:37 UTC pith:BQQQYEFX

load-bearing objection Solid systems paper: depth and targeted worlds beat bulk gyms, with a real RL substrate—but the co-evolution lift is intentionally non-causal. the 3 major comments →

arxiv 2607.28074 v1 pith:BQQQYEFX submitted 2026-07-30 cs.AI cs.LG

Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale

Yash Pandya , Sahil Gupta , Sarthak Harne , Archana Yadav , Kavyansh Chourasia , Hussein Mozannar , Vibhav Vineet , Sara Abdali
show 5 more authors
Corby Rosset Yash Lara Ahmed Awadallah Ece Kamar Akshay Nambi
This is my paper
classification cs.AI cs.LG
keywords computer-use agentssynthetic environmentsgrounded verificationco-evolutionenvironment depthreinforcement learningweb agentstask curriculum
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Computer-use agents only learn from actions that change real application state, yet the login-gated, stateful apps that matter cannot be trained on directly. Synthetic stand-ins therefore have to carry the load. This paper argues that once such environments can be generated in bulk, the bottleneck is no longer how many exist but what is inside each one: behavioural depth relative to target workflows, focus on the exact interactions the agent fails, and a loop that repairs the environment, tasks and verifier from the same graded rollouts used to train the model. Echoverse compiles specifications into stateful apps graded against their own databases, then co-evolves those worlds with the policy. A 9B model trained on twelve such worlds rises from 36.5% to 67.1% across fourteen splits, shallow clones can hurt live-site transfer while deep ones help, and the same worlds support reinforcement learning that lifts held-out score from 58.8% to 68.0%.

Core claim

Gains for computer-use agents come less from adding more synthetic environments than from three interior properties of each world: how completely it supports the workflows it is meant to teach, whether it targets the specific interaction the agent fails, and whether environment, tasks and verifier improve alongside the model. On the same domains, shallow worlds can push live accuracy below the base model while deep ones raise it; repairing one world more than doubles the model trained on it; and the same grounded worlds serve unmodified as RL environments.

What carries the argument

The co-evolution loop: every graded rollout is read twice—once as repairs to the environment, its tasks and its database-grounded verifier (world first, without weakening goals), and once as training signal for the model—so a static benchmark saturates while the loop compounds.

Load-bearing premise

That measured gains from repairing a world reflect genuine agent skill rather than mainly reshaping which tasks exist and how they are scored, since environment-only or verifier-only ablations are not separable in this design.

What would settle it

Train matched models on deep versus shallow versions of the same domains and on pre- versus post-repair versions of one world, then score them on fixed live-web tasks and on a frozen task set graded by an unchanged external verifier; if shallow or unrepaired worlds match or beat deep repaired ones on those fixed measures, the interior-quality claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Below a depth threshold, adding environments can inject noise and hurt live transfer rather than help.
  • Drilling one failing control across many renderings transfers to held-out widget families and to the open web.
  • The same database-owned worlds that supply clean supervised data also meet reset, throughput and reward needs for RL without modification.
  • A mid-size student can close most of the gap to its much larger teacher when trained only on deep, targeted, checkable synthetic trajectories.
  • Public progress should emphasize factories that find failures and repair worlds, not only larger inventories of synthetic sites.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If depth and co-evolution dominate count, environment-generation pipelines should ship machine-checkable workflow claims and repair traces, not only page counts.
  • Live-web benchmarks that judge only from screenshots will remain weak training rewards even as synthetic grounded verifiers improve, widening the train-eval substrate split.
  • Capability worlds that isolate single controls may become a standard complement to full-domain clones whenever agents stall on one widget class across many sites.
  • Whole-loop lifts without separable ablations will keep making it hard to credit ‘better environments’ versus ‘easier or differently filtered tasks’ unless frozen external graders become standard.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Echoverse argues that once synthetic computer-use environments are plentiful, returns come from three interior properties—behavioural depth (completeness w.r.t. a target workflow set), capability targeting (mass-varying failed controls), and co-evolution of environment, tasks, and verifier with the model—rather than from environment count. A factory compiles seeds into FastAPI/React/SQLite apps with database-grounded graders; a loop reads each graded rollout as both world repair and training signal. On twelve worlds, a 9B model rises from 36.5% to 67.1% across fourteen splits; shallow clones hurt live transfer while deep ones help; capability worlds transfer to held-out widgets and the open web; repairing ECHOSTAY lifts a model from 16.2% to 38.5%; and RL with a grounded trajectory reward plus dense per-step judge raises held-out score from 58.8% to 68.0%. Four worlds are released as a benchmark.

Significance. If the results hold, the paper usefully shifts the field from scaling environment count to interior quality and closed-loop repair, with an operational depth definition discharged by machine-checkable claims, database-grounded verification that is harder to game than screenshot judges, and unmodified worlds that meet RL reset/throughput/reward needs. The main scorecard, capability ablations with held-out widget families, dual scaling axes, live-web transfer, and the RL run are coherent contributions. Releasing runnable apps, seed data, and grounded graders is a concrete community asset. The work is complementary to bulk environment generators and to live-web benchmarks that cannot supply login-gated write workflows or exact reset.

major comments (3)
  1. [Sec. 6.7] Sec. 6.7 presents the ECHOSTAY v1→v2 model lift (16.2%→38.5%) as evidence for co-evolution, one of the three central levers. The paper states that environment-only or verifier-only ablations are “not well defined” because repairs bring new tasks into existence and re-mint references from the repaired DB. The measured jump therefore confounds cleaner supervision on a fixed distribution with task-set expansion and grader re-grounding. Without a frozen held-out task panel scored under a fixed verifier across world versions (or an explicit decomposition of solve-rate vs. corpus-composition effects), the causal claim that repairing world fidelity sharpens agent capability on a stable measure remains under-identified. Either add that panel or reframe co-evolution strictly as a joint loop effect and stop treating 16.2→38.5 as isolated evidence for depth/repair quality.
  2. [Sec. 6.2, Figure 5] Sec. 6.2 is the primary support for the depth thesis on live transfer, but uses only two WebVoyager domains (Allrecipes, Hugging Face) with a few dozen tasks each, and shallow vs. deep corpora that are “comparable” but not trajectory-matched. The paper itself reads this as directional evidence that shallow can be worse than no training, not as an estimate of depth’s value. That is appropriately cautious in the text, yet the abstract and thesis still lean on 80→75 vs. 80→85 / 48→65 as a main result. Strengthen with more domains, matched trajectory budgets, or demote the quantitative claim to a qualitative negative result with explicit N limits in the abstract.
  3. [Sec. 4.3, 4.8, Table 4] Phase-2 feasibility is white-box (Playwright plus source and DB access) and deliberately excludes the student policy, which is good, but the supervised corpus is still teacher-filtered (GPT-5.4 trajectories that pass the grounded verifier). Evaluation splits are therefore shaped by what the factory and teacher can complete. Table 4’s comparison of πSFT to GPT-5.4 is informative as distillation progress, but the paper should state more clearly which gaps are student-capacity vs. residual world/task hardness the teacher also fails, and whether any evaluation tasks were ever filtered by teacher success (Sec. 4.3 says no; Sec. 4.8 says failed teacher trajectories contribute no demos—confirm this holds for the released benchmark tasks).
minor comments (5)
  1. [Table 4, Sec. 5.3] Table 4 averages are unweighted means over fourteen splits of very different difficulty and size; a weighted or per-category breakdown (communication / regulated / capability) would aid interpretation.
  2. [Sec. 6.6, Figure 8] Figure 8’s trajectory-scaling axis holds per-world mixture fixed—good—but absolute counts at each subsample point are hard to read off the prose; add exact N labels on the x-axis or a small table.
  3. [Sec. 7.4–7.5] RL uses a 50-turn train cap vs. 100-turn eval budget (Sec. 7.4); note whether any held-out gain is partly longer-horizon tolerance rather than better policy.
  4. [Appendix C, Sec. 3.2] Appendix C verifier prompts are a strength; consider reporting inter-judge agreement or a small human audit of write-diff decisions to quantify residual LLM-comparison error.
  5. [Sec. 1–2] Typos/style: “WebV oyager” spacing appears repeatedly; “aworld” → “a world” early in Sec. 1; ensure consistent πbase / πSFT notation in figures.

Circularity Check

0 steps flagged

Empirical systems paper with external live-web and held-out checks; no load-bearing result reduces to its inputs by construction.

full rationale

Echoverse reports training and ablation results, not a first-principles derivation. Success on synthetic splits is graded by database-grounded verifiers (SQL diffs / reference answers minted from D0), which are independent of the policy’s self-report and of screenshot judges; held-out widget families, WebVoyager/Online-Mind2Web transfer, and a disjoint RL validation set further break any train–score tautology. The co-evolution ECHOSTAY lift (16.2%→38.5%) is causally under-identified—the paper itself says environment-only ablations are ‘not well defined’ because repairs expand the poseable task set—but that is confounding of the measured effect, not a prediction forced by definition or by fitting a parameter then re-reading it. Self-citations to Fara-7B/1.5 are explicitly scoped as prior pipeline context and are not used as uniqueness theorems or load-bearing premises for the three-lever thesis. No step matches self-definitional reduction, fitted-input-as-prediction, or uniqueness-by-self-citation. Score 1 only for the mild design entanglement in Sec. 6.7, not for circular derivation.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 4 invented entities

Empirical systems paper; load-bearing commitments are methodological assumptions about what makes a training world valid and how success is measured, plus engineering choices in distillation and RL—not free physical constants. Invented entities are software constructs (worlds, factory roles, composite reward), with partial independent handles via the public release.

free parameters (4)
  • Claim/corpus pass threshold (95%) = 95%
    Phase-1 and Phase-2 advance when ≥95% of claims or tasks pass; this bar is a design choice that shapes which worlds and tasks exist.
  • RL mid-band task filter (solve 1–3 of 4) = pass@4 ∈ {1,2,3}
    Curriculum filter discards always-fail and always-pass tasks for the starting policy; band choice affects gradient availability and reported RL gains.
  • Trajectory vs step reward weights and wrong-termination penalty = λ=1, penalty=0.1
    Composite reward uses unit weights, λ=1 on step reward, and a 0.1 wrong-termination penalty; these are hand-set and affect the 58.8→68.0 lift.
  • SFT/RL hyperparameters (LR, KL, group size, temp, step caps) = see Table 9
    Standard training knobs (e.g. LR 1e-7, KL 0.001, group 8, temp 0.7, 50-turn RL cap vs 100-turn eval) chosen by authors; not derived.
axioms (5)
  • domain assumption Application database state is the true environment state; success is a property of DB outcomes (plus narrow LLM semantic equivalence), not of pixels or agent self-report.
    Sec. 3.1–3.2; entire grounded-grading and RL reward stack rests on this.
  • ad hoc to paper Depth equals completeness w.r.t. a pre-specified target workflow set, discharged by machine-checkable claims and interface solvability—not feature count or visual fidelity.
    Sec. 3.3 operational definition; drives factory design and the deep-vs-shallow experiment.
  • domain assumption Factory agents (builders/verifiers/triagers/fixers) plus human oversight can attribute failures to world vs model well enough that repairing first yields trustworthy supervision.
    Sec. 4.6–4.7; without reliable attribution, co-evolution could scrub hard tasks or bake in bugs.
  • domain assumption Verifier-filtered distillation from a frontier teacher (GPT-5.4) plus optional GRPO-style RL is a valid path to improve a 9B browser policy.
    Sec. 4.8 and 7; standard in the subfield but inherits teacher ceiling and judge biases in the dense term.
  • domain assumption Semantic equivalence of answers and SQL diffs can be judged by an LLM under fixed prompts without trajectory context.
    Sec. 3.2 and Appendix C; narrows the judge but still inserts model judgment into ‘grounded’ scores.
invented entities (4)
  • Echoverse world (E, T, V) bundle independent evidence
    purpose: Unit that couples environment, task corpus, and grounded verifier as the trainable/repairable object.
    Core abstraction of the paper; software construct rather than physical entity.
  • Capability worlds (datepicker and nested-filter mass generators) independent evidence
    purpose: Isolate failing UI controls across many themes/widgets to teach transferable skills.
    Novel training substrate type within the suite; held-out families provide a falsifiable transfer test.
  • Co-evolution loop (one rollout, two signals) no independent evidence
    purpose: Use each graded failure both as world repair and as model training signal under repair-first discipline.
    Methodological invention; evidence is empirical lifts, not an external physical prediction.
  • Composite grounded+dense RL reward no independent evidence
    purpose: Supply terminal DB truth plus per-step shaping for long browser trajectories.
    Engineering construct; validation scored on grounded term only, which is a partial external handle.

pith-pipeline@v1.2.0-daily-grok45 · 26572 in / 4113 out tokens · 73177 ms · 2026-07-31T18:37:44.777876+00:00 · methodology

0 comments
read the original abstract

Computer-use agents learn from what their actions change, so training one needs applications it can act on, break and reset. The applications that matter most are login-gated and stateful, so synthetic environments stand in for them. Recent pipelines generate such environments in bulk, which moves the bottleneck from how many exist to what is inside each one. The returns, we find, come from three properties: how much behavioural depth an environment carries, whether it targets the interaction an agent actually fails, and whether it improves alongside the model. We present Echoverse, which compiles specifications into stateful applications whose tasks are graded against the application's own database, and a co-evolution loop that reads every graded rollout twice: as repairs to the environment, its tasks and its verifier, and as training signal for the model. Trained on twelve such environments, a 9B model improves from $36.5\%$ to $67.1\%$ across fourteen evaluation splits, within fourteen points of the much larger frontier model that taught it. We examine each property in turn. On the same domains, shallow environments push live-site accuracy below the base model ($80.0 \to 75.0$) while deep ones raise it ($80.0 \to 85.0$ and $48.0 \to 65.0$); drilling one interface control across many renderings transfers to held-out widget families and to the open web; and repairing a single environment lifts the model trained on it from $16.2\%$ to $38.5\%$. The same worlds serve as reinforcement-learning environments, where a reward combining the grounded verifier with a dense per-step judge raises held-out score from $58.8\%$ to $68.0\%$. We release four environments as a benchmark, with their applications, seed data and grounded graders. Code: https://aka.ms/echoverse

Figures

Figures reproduced from arXiv: 2607.28074 by Ahmed Awadallah, Akshay Nambi, Archana Yadav, Corby Rosset, Ece Kamar, Hussein Mozannar, Kavyansh Chourasia, Sahil Gupta, Sara Abdali, Sarthak Harne, Vibhav Vineet, Yash Lara, Yash Pandya.

Figure 1
Figure 1. Figure 1: The learning loop. Every graded run is read twice. Failures that survive triage become model training data; defects in the environment, its tasks or its verifier become repairs. The same graded run that measures the model also sharpens the world. et al., 2026], and this work builds on that progress. Once worlds are plentiful, the bottleneck moves inside them, into whether each one holds together under real… view at source ↗
Figure 2
Figure 2. Figure 2: The environment factory. Phase 1 expands a handful of seeds into an application, then repairs the database, backend and frontend until it passes machine-checkable claims. Phase 2 regrounds tasks on live data, runs a panel of verifier, triage and layer-specific fixer agents, and re-scores against database ground truth until the corpus clears its verification bar. Many of those fixes land in the environment … view at source ↗
Figure 3
Figure 3. Figure 3: The suite as an agent meets it. The first ten panels are the full-domain worlds tabulated above, each a behaviourally faithful clone of a class of application rather than of any particular product. The last two are held-out renderings from the capability worlds of Sec. 5.2: a calendar heatmap in a legal-services frontend and a compound booking-search panel in an airline frontend, neither of which appears i… view at source ↗
Figure 4
Figure 4. Figure 4: Thematic spread of the two capability worlds. The same control is re-themed so that the skill cannot be learned as a single layout: nested filters over six verticals, date pickers over ten contexts. Percentages are shares of generated frontends within each world. 5.2 Two capability worlds Not every weakness is a missing domain. Some are a single control the agent cannot reliably operate. Consider an agent … view at source ↗
Figure 5
Figure 5. Figure 5: Deep versus shallow worlds on two live WebVoyager domains, with the same domain coverage and differing task depth. Deep lifts both; shallow drops below base on Allrecipes and stalls on Hugging Face. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Closing the gap to the frontier, per evaluation split. The solid bar is the climb from πbase to πSFT; the faded remainder is the distance still to GPT-5.4. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Targeted training, targeted gains. Training either skill lifts both controls, including held-out widgets and compositions neither was trained on, and training both is the best all-rounder on every split. 6.5 Transfer to the live web To test whether the skill survives the open web, we evaluate πSFT, unchanged, on WebVoyager and Online-Mind2Web, benchmarks it never trained on. These barely overlap with what … view at source ↗
Figure 8
Figure 8. Figure 8: Two scaling axes, scored without a hosted browser. Left: more trajectories over a fixed set of worlds, subsampled from the full corpus at its own per-world proportions, with the horizontal axis spaced by actual trajectory count. The synthetic average keeps rising, but live-web transfer saturates. Right: more environments, where breadth keeps both the synthetic average and WebVoyager climbing. More trajecto… view at source ↗
Figure 9
Figure 9. Figure 9: Co-evolution lifts the model on ECHOSTAY. As the world went from v1 to v2, the model trained on its corpus more than doubled, from 16.2% to 38.5%. This is a separate measurement from the world’s own solve rate. As the world sharpens, the model climbs with it ( [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Reinforcement learning on an Echoverse world. From the supervised policy we roll out a group of trajectories in one environment; each is a sequence of act-and-execute steps that changes the database. A grader, the same grounded verifier that filtered the supervised data, sits outside the environment and scores each rollout’s final database state into a reward. The group of rewards updates the policy, and … view at source ↗
Figure 11
Figure 11. Figure 11: Reinforcement learning on five worlds, a little over two epochs. Left: the held-out judged score, 25 tasks per world graded by the database-grounded verifier. Right: the training reward, which unlike the judged score includes the dense per-step term. 8 Conclusion A world is more than a benchmark to score against: it is something the loop keeps improving, because the same graded run that measures the model… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 15 linked inside Pith

  1. [1]

    He, Hongliang and Yao, Wenlin and Ma, Kaixin and Yu, Wenhao and Dai, Yong and Zhang, Hongming and Lan, Zhenzhong and Yu, Dong , journal =

  2. [3]

    Deng, Xiang and Gu, Yu and Zheng, Boyuan and Chen, Shijie and Stevens, Samuel and Wang, Boshi and Sun, Huan and Su, Yu , booktitle =

  3. [4]

    Zhou, Shuyan and Xu, Frank F. and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham , booktitle =

  4. [5]

    Koh, Jing Yu and Lo, Robert and Jang, Lawrence and Duvvur, Vikram and Lim, Ming Chong and Huang, Po-Yu and Neubig, Graham and Zhou, Shuyan and Salakhutdinov, Ruslan and Fried, Daniel , booktitle =

  5. [6]

    Xie, Tianbao and Zhang, Danyang and Chen, Jixuan and Li, Xiaochuan and Zhao, Siheng and Cao, Ruisheng and Hua, Toh Jing and Cheng, Zhoujun and Shin, Dongchan and Lei, Fangyu and Liu, Yitao and Xu, Yiheng and Zhou, Shuyan and Savarese, Silvio and Xiong, Caiming and Zhong, Victor and Yu, Tao , booktitle =

  6. [7]

    and Del Verme, Manuel and Marty, Tom and Boisvert, L

    Drouin, Alexandre and Gasse, Maxime and Caccia, Massimo and Laradji, Issam H. and Del Verme, Manuel and Marty, Tom and Boisvert, L. International Conference on Machine Learning (ICML) , year =

  7. [8]

    Rawles, Christopher and Clinckemaillie, Sarah and Chang, Yifan and Waltz, Jonathan and Lau, Gabrielle and Fair, Marybeth and Li, Alice and Bishop, William and Li, Wei and Campbell-Ajala, Folawiyo and Toyama, Daniel and Berry, Robert and Hakimi, Divya and Xu, Yuan and Riva, Oriana and Lillicrap, Timothy , booktitle =

  8. [11]

    Zhou, Shuyan , year =

  9. [12]

    Zhang, Ziyun and Wang, Zezhou and Zhang, Xiaoyi and Guo, Zongyu and Li, Jiahao and Li, Bin and Lu, Yan , journal =

  10. [13]

    Aggarwal, Pranjal and Neubig, Graham and Welleck, Sean , journal =

  11. [14]

    Wang, Bowen and Lu, Dunjie and Wang, Junli and Bai, Tianyi and Liu, Shixuan and Zhang, Zhipeng and Wang, Haiquan and Hu, Hao and Xie, Tianbao and Bai, Shuai and Liu, Dayiheng and Shen, Que and Lin, Junyang and Yu, Tao , journal =

  12. [15]

    Yang, Yuhao and Yang, Zhen and Dou, Zi-Yi and Nguyen, Anh and You, Keen and Attia, Omar and Szot, Andrew and Feng, Michael and Ramrakhya, Ram and Toshev, Alexander and Huang, Chao and Yang, Yinfei and Gan, Zhe , journal =

  13. [16]

    Awadallah, Ahmed and Gupta, Sahil and Lara, Yash and Lu, Yadong and Mozannar, Hussein and Nambi, Akshay and Nussbaum, Zach and Pandya, Yash and Rajeswaran, Aravind and Rosset, Corby and Taymanov, Alexey and do Valle, Luiz and Vineet, Vibhav and Whitehead, Spencer and Zhao, Andrew , journal =

  14. [17]

    Awadallah, Ahmed and Lara, Yash and Magazine, Raghav and Mozannar, Hussein and Nambi, Akshay and Pandya, Yash and Rajeswaran, Aravind and Rosset, Corby and Taymanov, Alexey and Vineet, Vibhav and Whitehead, Spencer and Zhao, Andrew , journal =

  15. [19]

    2026 , note =

    A Practical Recipe for Training Computer-Use Agents with Reinforcement Learning , author =. 2026 , note =

    2026

  16. [20]

    Understanding

    Liu, Zichen and Chen, Changyu and Li, Wenjun and Qi, Penghui and Pang, Tianyu and Du, Chao and Lee, Wee Sun and Lin, Min , journal =. Understanding

  17. [21]

    Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Song, Junxiao and Bi, Xiao and Zhang, Haowei and Zhang, Mingchuan and Li, Y. K. and Wu, Y. and Guo, Daya , journal =

  18. [23]

    , booktitle =

    Zelikman, Eric and Wu, Yuhuai and Mu, Jesse and Goodman, Noah D. , booktitle =

  19. [25]

    and Zhang, Hao and Gonzalez, Joseph E

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , booktitle =. Judging

  20. [26]

    , journal =

    Wang, Rui and Lehman, Joel and Clune, Jeff and Stanley, Kenneth O. , journal =. Paired Open-Ended Trailblazer (

  21. [27]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Emergent Complexity and Zero-Shot Transfer via Unsupervised Environment Design , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  22. [28]

    International Conference on Machine Learning (ICML) , year =

    Evolving Curricula with Regret-Based Environment Design , author =. International Conference on Machine Learning (ICML) , year =

  23. [30]

    Science , volume =

    A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go Through Self-Play , author =. Science , volume =

  24. [31]

    and Zhang, Hao and Stoica, Ion , booktitle =

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph E. and Zhang, Hao and Stoica, Ion , booktitle =. Efficient Memory Management for Large Language Model Serving with

  25. [32]

    Cox, Murray , year =. Inside

  26. [33]

    2024 , howpublished =

    Playwright: Fast and Reliable End-to-End Testing for Modern Web Apps , author =. 2024 , howpublished =

    2024

  27. [34]

    2025 , howpublished =

    Browserbase: Headless Browser Infrastructure for. 2025 , howpublished =

    2025

  28. [35]

    Gym-Anything : Turn any software into an agent environment

    Pranjal Aggarwal, Graham Neubig, and Sean Welleck. Gym-Anything : Turn any software into an agent environment. arXiv preprint arXiv:2604.06126, 2026

    Pith/arXiv arXiv 2026

  29. [36]

    Model context protocol

    Anthropic . Model context protocol. https://modelcontextprotocol.io, 2024

    2024

  30. [37]

    Fara-7B : An efficient agentic model for computer use

    Ahmed Awadallah, Yash Lara, Raghav Magazine, Hussein Mozannar, Akshay Nambi, Yash Pandya, Aravind Rajeswaran, Corby Rosset, Alexey Taymanov, Vibhav Vineet, Spencer Whitehead, and Andrew Zhao. Fara-7B : An efficient agentic model for computer use. arXiv preprint arXiv:2511.19663, 2025

    arXiv 2025

  31. [38]

    Fara-1.5 : Scalable learning environments for computer use agents

    Ahmed Awadallah, Sahil Gupta, Yash Lara, Yadong Lu, Hussein Mozannar, Akshay Nambi, Zach Nussbaum, Yash Pandya, Aravind Rajeswaran, Corby Rosset, Alexey Taymanov, Luiz do Valle, Vibhav Vineet, Spencer Whitehead, and Andrew Zhao. Fara-1.5 : Scalable learning environments for computer use agents. arXiv preprint arXiv:2606.20785, 2026

    Pith/arXiv arXiv 2026

  32. [39]

    Browserbase: Headless browser infrastructure for AI agents

    Browserbase . Browserbase: Headless browser infrastructure for AI agents. https://www.browserbase.com, 2025

    2025

  33. [40]

    Inside A irbnb: Adding data to the debate

    Murray Cox. Inside A irbnb: Adding data to the debate. http://insideairbnb.com, 2024

    2024

  34. [41]

    Xu, Siva Reddy, Quentin Cappart, Graham Neubig, Ruslan Salakhutdinov, Nicolas Chapados, and Alexandre Lacoste

    Thibault Le Sellier de Chezelles, Maxime Gasse, Alexandre Drouin, Massimo Caccia, L \'e o Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han L \`u , Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Quentin Cappart, Graham Neubig, Ruslan Salakhutdinov, Nicolas Chapados, and Alexandre Lacoste. The BrowserGym ...

    Pith/arXiv arXiv 2025

  35. [42]

    DeepSeek-R1 : Incentivizing reasoning capability in LLM s via reinforcement learning

    DeepSeek-AI . DeepSeek-R1 : Incentivizing reasoning capability in LLM s via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  36. [43]

    Mind2Web : Towards a generalist agent for the web

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2Web : Towards a generalist agent for the web. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  37. [44]

    Emergent complexity and zero-shot transfer via unsupervised environment design

    Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, and Sergey Levine. Emergent complexity and zero-shot transfer via unsupervised environment design. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  38. [45]

    Laradji, Manuel Del Verme, Tom Marty, L \'e o Boisvert, Megh Thakkar, Quentin Cappart, David Vazquez, Nicolas Chapados, and Alexandre Lacoste

    Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, L \'e o Boisvert, Megh Thakkar, Quentin Cappart, David Vazquez, Nicolas Chapados, and Alexandre Lacoste. WorkArena : How capable are web agents at solving common knowledge work tasks? In International Conference on Machine Learning (ICML), 2024

    2024

  39. [46]

    WebVoyager : Building an end-to-end web agent with large multimodal models

    Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebVoyager : Building an end-to-end web agent with large multimodal models. arXiv preprint arXiv:2401.13919, 2024

    Pith/arXiv arXiv 2024

  40. [47]

    VisualWebArena : Evaluating multimodal agents on realistic visual web tasks

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. VisualWebArena : Evaluating multimodal agents on realistic visual web tasks. In Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  41. [48]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention . In ACM Symposium on Operating Systems Principles (SOSP), 2023

    2023

  42. [49]

    A practical recipe for training computer-use agents with reinforcement learning

    Daisy Lin and XJ Wang. A practical recipe for training computer-use agents with reinforcement learning. https://www.amazon.science/blog/a-practical-recipe-for-training-computer-use-agents-with-rl, 2026. Amazon Science Blog, Amazon AGI Lab

    2026

  43. [50]

    Understanding R1 -zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding R1 -zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    Pith/arXiv arXiv 2025

  44. [51]

    Playwright: Fast and reliable end-to-end testing for modern web apps

    Microsoft . Playwright: Fast and reliable end-to-end testing for modern web apps. https://playwright.dev, 2024

    2024

  45. [52]

    Evolving curricula with regret-based environment design

    Jack Parker-Holder, Minqi Jiang, Michael Dennis, Mikayel Samvelyan, Jakob Foerster, Edward Grefenstette, and Tim Rockt \"a schel. Evolving curricula with regret-based environment design. In International Conference on Machine Learning (ICML), 2022

    2022

  46. [53]

    Automatic curriculum learning for deep RL : A short survey

    R \'e my Portelas, C \'e dric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer. Automatic curriculum learning for deep RL : A short survey. arXiv preprint arXiv:2003.04664, 2020

    Pith/arXiv arXiv 2003

  47. [54]

    Qwen3.5 : Towards native multimodal agents

    Qwen Team . Qwen3.5 : Towards native multimodal agents. https://qwen.ai/blog?id=qwen3.5, February 2026

    2026

  48. [55]

    AndroidWorld : A dynamic benchmarking environment for autonomous agents

    Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, Daniel Toyama, Robert Berry, Divya Hakimi, Yuan Xu, Oriana Riva, and Timothy Lillicrap. AndroidWorld : A dynamic benchmarking environment for autonomous agents. In International Conference on Learn...

    2025

  49. [56]

    The art of building verifiers for computer use agents

    Corby Rosset, Pratyusha Sharma, Andrew Zhao, Miguel Gonzalez-Fernandez, and Ahmed Awadallah. The art of building verifiers for computer use agents. arXiv preprint arXiv:2604.06240, 2026

    Pith/arXiv arXiv 2026

  50. [57]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath : Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  51. [58]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362 0 (6419): 0 1140--1144, 2018

    2018

  52. [59]

    CUA-Gym : Scaling verifiable training environments and tasks for computer-use agents

    Bowen Wang, Dunjie Lu, Junli Wang, Tianyi Bai, Shixuan Liu, Zhipeng Zhang, Haiquan Wang, Hao Hu, Tianbao Xie, Shuai Bai, Dayiheng Liu, Que Shen, Junyang Lin, and Tao Yu. CUA-Gym : Scaling verifiable training environments and tasks for computer-use agents. arXiv preprint arXiv:2605.25624, 2026

    Pith/arXiv arXiv 2026

  53. [60]

    Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O. Stanley. Paired open-ended trailblazer ( POET ): Endlessly generating increasingly complex and diverse learning environments and their solutions. arXiv preprint arXiv:1901.01753, 2019

    Pith/arXiv arXiv 1901

  54. [61]

    OSWorld : Benchmarking multimodal agents for open-ended tasks in real computer environments

    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld : Benchmarking multimodal agents for open-ended tasks in real computer environments. In Advances in Neural Information...

    2024

  55. [62]

    An illusion of progress? assessing the current state of web agents

    Tianci Xue, Weijian Qi, Tianneng Shi, Chan Hee Song, Boyu Gou, Dawn Song, Huan Sun, and Yu Su. An illusion of progress? assessing the current state of web agents. arXiv preprint arXiv:2504.01382, 2025

    arXiv 2025

  56. [63]

    UltraCUA : A foundation model for computer use agents with hybrid action

    Yuhao Yang, Zhen Yang, Zi-Yi Dou, Anh Nguyen, Keen You, Omar Attia, Andrew Szot, Michael Feng, Ram Ramrakhya, Alexander Toshev, Chao Huang, Yinfei Yang, and Zhe Gan. UltraCUA : A foundation model for computer use agents with hybrid action. arXiv preprint arXiv:2510.17790, 2026

    Pith/arXiv arXiv 2026

  57. [64]

    -bench: A benchmark for tool-agent-user interaction in real-world domains

    Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. -bench: A benchmark for tool-agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045, 2024

    Pith/arXiv arXiv 2024

  58. [65]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825, 2023

    Pith/arXiv arXiv 2023

  59. [66]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. STaR : Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  60. [67]

    InfiniteWeb : Scalable web environment synthesis for GUI agent training

    Ziyun Zhang, Zezhou Wang, Xiaoyi Zhang, Zongyu Guo, Jiahao Li, Bin Li, and Yan Lu. InfiniteWeb : Scalable web environment synthesis for GUI agent training. arXiv preprint arXiv:2601.04126, 2026. Accepted to ACL 2026

    Pith/arXiv arXiv 2026

  61. [68]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM -as-a-judge with MT-Bench and chatbot arena. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2023

    2023

  62. [69]

    WebArena-Infinity : Generating browser environments with verifiable tasks at scale

    Shuyan Zhou. WebArena-Infinity : Generating browser environments with verifiable tasks at scale. https://webarena.dev/webarena-infinity/, March 2026

    2026

  63. [70]

    Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig

    Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena : A realistic web environment for building autonomous agents. In International Conference on Learning Representations (ICLR), 2024

    2024