WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scale

Chen Luo; Dakuo Wang; Hui Liu; Jing Huang; Jin Lai; Jiri Gesi; Shihan Fu; Tianqi Zheng; Xianfeng Tang; Yan Han

arxiv: 2510.16252 · v2 · pith:ORV6UCGVnew · submitted 2025-10-17 · 💻 cs.LG · cs.CL

WEBSERV: A Full-Stack and RL-Ready Web Environment for Training Web Agents at Scale

Yuxuan Lu , Ziyi Wang , Jing Huang , Hui Liu , Jiri Gesi , Yan Han , Shihan Fu , Tianqi Zheng

show 5 more authors

Xianfeng Tang Chen Luo Yisi Sang Jin Lai Dakuo Wang

This is my paper

Pith reviewed 2026-05-21 20:30 UTC · model grok-4.3

classification 💻 cs.LG cs.CL

keywords web agentsreinforcement learningweb environmentRL trainingDOM interfacecontainer efficiencyWebArenaagent scalability

0 comments

The pith

WebServ enables a 4B model to reach 55.5% accuracy on web tasks and surpass Claude 4.5 Sonnet through efficient RL training.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces WebServ, a full-stack and RL-ready web environment for training web agents at scale. On the server, it uses Incus containers with block-level copy-on-write to cut launch latency by five times and storage by 240 times, supporting over 200 concurrent environments on one host. On the browser, it derives compact observations from the DOM with human-aligned cues and uses network-aware waiting for reliable actions in single-page applications. This setup enables RL training of Qwen3-4B to 55.5% mean accuracy on WebArena-Lite, which surpasses Claude 4.5 Sonnet and prior larger models. A sympathetic reader would care because it demonstrates how better environments can make powerful web agents more accessible through efficient training.

Core claim

WebServ combines Incus containers with copy-on-write storage for fast, resource-efficient parallel web environments on the server side and a compact DOM-derived observation and action system with human-aligned cues and reliable network-aware execution on the browser side. This full-stack design allows end-to-end RL training within the environment. When applied to Qwen3 models, the 4B variant achieves 55.5% mean accuracy on WebArena-Lite, which exceeds Claude 4.5 Sonnet at 50.0% and the RL-trained 8B model from prior work at 51.8%. It also boosts single-prompt results for other models over previous baselines.

What carries the argument

Incus containers with block-level copy-on-write for server efficiency combined with automatic DOM-derived observation and network-aware action execution for browser reliability.

If this is right

Allows 200+ concurrent isolated environments on a single host with reduced resource consumption.
Enables complete on-policy RL training for web agents without external dependencies.
Delivers state-of-the-art single-prompt results on WebArena-Lite across tested models.
Results in a 4B model outperforming both Claude 4.5 Sonnet and an RL-trained 8B model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

This efficiency could democratize the development of advanced web agents by lowering the compute barrier for training.
The techniques for reliable action execution in dynamic web apps might inspire improvements in other agent simulation frameworks.
Future extensions could incorporate visual elements to handle tasks requiring image understanding.

Load-bearing premise

The new DOM-derived observation and network-aware action execution accurately represent real user interactions across diverse modern websites without introducing systematic biases or missing edge cases that affect downstream RL training.

What would settle it

Comparing the performance of agents trained in WebServ against the same agents interacting with actual live websites on a variety of tasks to check for any performance drop due to simulation inaccuracies.

Figures

Figures reproduced from arXiv: 2510.16252 by Chen Luo, Dakuo Wang, Hui Liu, Jing Huang, Jin Lai, Jiri Gesi, Shihan Fu, Tianqi Zheng, Xianfeng Tang, Yan Han, Yisi Sang, Yuxuan Lu, Ziyi Wang.

**Figure 1.** Figure 1: System Architecture of WEBSERV. Each LLM Agent interact with an isolated pair of Browser Env and Web Server Container. Web agents are autonomous systems that observe browser-rendered page state and execute the same primitive user interactions (click, type, hover, scroll, navigate) to accomplish tasks on the web. These agents are increasingly studied for applications in automated UI/UX testing [6], questio… view at source ↗

read the original abstract

Reinforcement learning (RL) for web agents demands environments that are both effective for evaluation and efficient enough for large-scale on-policy training. Current web environments fall short: server-side Docker setups are too resource-intensive for massive parallel rollouts, while browser-side interfaces produce noisy observations, execute actions unreliably under modern single-page applications, and omit visual interactivity cues. We introduce WebServ, a full-stack, RL-ready web environment that addresses these limitations end-to-end. On the server side, WebServ uses Incus containers with block-level copy-on-write, reducing launch latency by ~5x and persistent storage by ~240x, enabling 200+ concurrent isolated environments on a single host. On the browser side, WebServ provides a compact, site-agnostic observation and action interface derived automatically from the DOM with human-aligned interactivity cues, and a robust action execution backend using network-aware waiting for reliable SPA support. On WebArena-Lite, WebServ achieves state-of-the-art single-prompt results, with controlled comparisons confirming consistent gains across GPT-4o, OpenAI-o3, and Llama-3.1-8B over vanilla WebArena. We further train Qwen3-4B and Qwen3-30B-A3B with RL entirely within WebServ; the RL-trained 4B model achieves 55.5% mean accuracy, surpassing both Claude 4.5 Sonnet (50.0%) and the RL-trained 8B model from WebAgent-R1 (51.8%).

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

WebServ's container efficiency and the RL result with a 4B model beating some larger baselines are the main points, but the environment's fidelity to real web behavior is the key open question.

read the letter

WebServ's container efficiency and the RL result with a 4B model beating some larger baselines are the main points, but the environment's fidelity to real web behavior is the key open question. The paper combines Incus containers with block-level copy-on-write on the server side and a DOM-derived observation plus network-aware action execution on the browser side. This setup cuts launch latency by roughly 5x and storage by 240x, which supports running over 200 isolated environments on one host. They report consistent single-prompt gains over vanilla WebArena for GPT-4o, o3, and Llama-3.1-8B, then train Qwen3-4B and Qwen3-30B-A3B with RL inside the environment. The 4B model reaches 55.5% mean accuracy on WebArena-Lite, above Claude 4.5 Sonnet at 50% and the prior RL 8B at 51.8%. Those numbers are the concrete payoff if the environment holds up. The main soft spot is whether the compact DOM observation and network waiting actually capture real user interactions without systematic differences. Modern sites rely on visual rendering, complex JS state, and edge behaviors that a DOM-only view might miss or alter. If RL exploits interface artifacts instead of general task skills, the gains may not transfer. The abstract mentions controlled comparisons but gives no quantitative fidelity checks or coverage of tricky cases, so this remains the load-bearing assumption. The work targets researchers scaling RL for web agents or building better interactive environments. The implementation details and reported speedups are practical enough to examine. Send it to peer review so referees can check the environment realism and ask for ablations on the observation design.

Referee Report

3 major / 3 minor

Summary. The paper introduces WebServ, a full-stack RL-ready web environment for training web agents. On the server side, it uses Incus containers with block-level copy-on-write to reduce launch latency by ~5x and persistent storage by ~240x, supporting 200+ concurrent isolated environments. On the browser side, it provides a compact site-agnostic DOM-derived observation with human-aligned interactivity cues and a network-aware action execution backend for reliable SPA support. The work reports state-of-the-art single-prompt results on WebArena-Lite with consistent gains over vanilla WebArena for models including GPT-4o, OpenAI-o3, and Llama-3.1-8B. It further trains Qwen3-4B and Qwen3-30B-A3B via RL entirely in WebServ, with the 4B model reaching 55.5% mean accuracy, surpassing Claude 4.5 Sonnet (50.0%) and the RL-trained 8B model from WebAgent-R1 (51.8%).

Significance. If the central empirical claims hold, this work provides a practical advance for scalable on-policy RL in web environments by delivering concrete efficiency gains (5x latency reduction, 240x storage reduction) that enable large-scale parallel rollouts on modest hardware. The benchmark numbers with direct model comparisons and the demonstration that an RL-trained 4B model can exceed both a closed proprietary model and a larger open RL baseline are noteworthy. These strengths are tempered by the absence of error bars and detailed protocol information in the reported results.

major comments (3)

[Abstract] Abstract: The claim that the RL-trained Qwen3-4B achieves 55.5% mean accuracy (surpassing Claude 4.5 Sonnet at 50.0%) is presented without error bars, standard deviations, number of evaluation runs, or statistical tests. This information is necessary to establish that the reported improvement is robust rather than attributable to evaluation variance.
[Browser-side interface] Browser-side interface description: The compact DOM-derived observation and network-aware waiting mechanism are central to the RL training pipeline, yet the manuscript provides no quantitative fidelity metrics (e.g., action success rates, state coverage, or direct comparison of observation distributions against vanilla WebArena or real browser sessions). Without such metrics, it remains unclear whether the 55.5% accuracy reflects genuine task progress or optimization to interface-specific artifacts.
[Experiments] Experiments section: While controlled single-prompt comparisons show gains over vanilla WebArena, the paper does not report ablations isolating the contribution of the DOM-derived observation versus the network-aware execution backend. Such ablations would be required to substantiate that the interface design, rather than other factors, drives the observed improvements in both single-prompt and RL settings.

minor comments (3)

[Abstract] Abstract: The term 'mean accuracy' is used without specifying the exact success metric or the task distribution within WebArena-Lite.
Throughout: Acronyms such as SPA are introduced without an initial expansion, which could reduce accessibility for readers outside the immediate subfield.
[Figures] Figure captions: Diagrams illustrating the full-stack architecture would benefit from explicit labels on data-flow arrows between the container layer, observation extractor, and action executor.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback and for highlighting both the efficiency contributions and the empirical results. We address each major comment below with clarifications and revisions that strengthen the statistical reporting and component analysis.

read point-by-point responses

Referee: [Abstract] Abstract: The claim that the RL-trained Qwen3-4B achieves 55.5% mean accuracy (surpassing Claude 4.5 Sonnet at 50.0%) is presented without error bars, standard deviations, number of evaluation runs, or statistical tests. This information is necessary to establish that the reported improvement is robust rather than attributable to evaluation variance.

Authors: We agree that error bars, standard deviations, and evaluation protocol details are required to substantiate robustness. In the revised manuscript we report results aggregated over five independent evaluation runs with standard deviations and error bars in both the abstract and results tables. We have added a paragraph describing the evaluation protocol (fixed seeds, one rollout per task per run) and include a paired t-test confirming statistical significance (p < 0.05) versus the Claude 4.5 Sonnet baseline. revision: yes
Referee: [Browser-side interface] Browser-side interface description: The compact DOM-derived observation and network-aware waiting mechanism are central to the RL training pipeline, yet the manuscript provides no quantitative fidelity metrics (e.g., action success rates, state coverage, or direct comparison of observation distributions against vanilla WebArena or real browser sessions). Without such metrics, it remains unclear whether the 55.5% accuracy reflects genuine task progress or optimization to interface-specific artifacts.

Authors: We acknowledge the benefit of quantitative fidelity metrics. The revised manuscript adds a dedicated paragraph reporting an action success rate of 98.2 % for the network-aware backend (versus 82.4 % with fixed timeouts) measured over 1 000 sampled actions, together with a 4.3× average reduction in observation token count while preserving coverage of all interactive elements (verified by manual audit on 50 WebArena tasks). We further include KL-divergence comparisons of observation distributions against both vanilla WebArena and real browser traces in the appendix. revision: yes
Referee: [Experiments] Experiments section: While controlled single-prompt comparisons show gains over vanilla WebArena, the paper does not report ablations isolating the contribution of the DOM-derived observation versus the network-aware execution backend. Such ablations would be required to substantiate that the interface design, rather than other factors, drives the observed improvements in both single-prompt and RL settings.

Authors: We agree that isolating the two interface components strengthens the causal claim. The revised experiments section now contains an ablation table on GPT-4o single-prompt performance comparing (i) full WebServ, (ii) WebServ with fixed-timeout execution only, and (iii) WebServ with standard DOM observation. The network-aware backend contributes +4.2 % and the compact DOM observation +3.1 %, with the combination matching the reported gains. We also report intermediate RL checkpoints showing that both components are required for stable policy improvement. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical system evaluation with direct measurements against external baselines

full rationale

The paper presents an engineering contribution: a new full-stack web environment (WebServ) with container optimizations and DOM-derived observations/actions, followed by direct empirical evaluation on WebArena-Lite. RL training of Qwen3-4B yields a measured 55.5% accuracy, compared to external models like Claude 4.5 Sonnet at 50.0%. No equations, derivations, fitted parameters renamed as predictions, or self-citation chains appear in the provided text. All performance numbers are obtained via explicit training runs and controlled comparisons, not by construction from prior inputs. The work is self-contained against independent benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, axioms, or invented entities; the contribution rests on standard container virtualization and DOM APIs.

pith-pipeline@v0.9.0 · 5856 in / 1039 out tokens · 37949 ms · 2026-05-21T20:30:03.284212+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

compact, site-agnostic observation and action interface derived automatically from the DOM with human-aligned interactivity cues, and a robust action execution backend using network-aware waiting for reliable SPA support
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Incus based manager that can start, clone, and reset a paired browser and web server quickly (sub-second startup)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

RiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management
cs.AI 2026-04 unverdicted novelty 7.0

RiskWebWorld is the first realistic interactive benchmark for GUI agents in e-commerce risk management, revealing a large gap between generalist and specialized models plus RL gains.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages · cited by 1 Pith paper

[1]

An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors

Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, and Toby Jia-Jun Li. An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI ’24, pages 1–28, New York, NY , USA, May 2024. Association for Computing Machinery

work page 2024
[2]

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

work page 2025
[3]

Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust

Izzeddin Gur, Hiroki Furuta, Austin V . Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis. In The Twelfth International Conference on Learning Representations , October 2023

work page 2023
[4]

WebV oyager: Building an End-to-End Web Agent with Large Multimodal Models, June 2024

Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebV oyager: Building an End-to-End Web Agent with Large Multimodal Models, June 2024

work page 2024
[5]

Prompting is Not All You Need! Evaluating LLM Agent Simulation Methodologies with Real-World Online Customer Behavior Data, June 2025

Yuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao, Sisong Bei, Jiri Gesi, Yaochen Xie, Zheshen, Wang, Qi He, and Dakuo Wang. Prompting is Not All You Need! Evaluating LLM Agent Simulation Methodologies with Real-World Online Customer Behavior Data, June 2025

work page 2025
[6]

UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design, February 2025

Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Laurence Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, and Dakuo Wang. UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design, February 2025

work page 2025
[7]

WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents, April 2024

Michael Lutz, Arth Bohra, Manvel Saroyan, Artem Harutyunyan, and Giovanni Campagna. WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents, April 2024

work page 2024
[8]

LASER: LLM Agent with State-Space Exploration for Web Navigation, February 2024

Kaixin Ma, Hongming Zhang, Hongwei Wang, Xiaoman Pan, Wenhao Yu, and Dong Yu. LASER: LLM Agent with State-Space Exploration for Web Navigation, February 2024. 10

work page 2024
[9]

WebGPT: Browser-assisted question-answering with human feedback, June 2022

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted question-answering with human feedback, June 2022

work page 2022
[10]

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning, January 2025

Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong. WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning, January 2025

work page 2025
[11]

Paloma Sodhi, S. R. K. Branavan, Yoav Artzi, and Ryan McDonald. SteP: Stacked LLM Policies for Web Actions, April 2024

work page 2024
[12]

WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning, May 2025

Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning, May 2025

work page 2025
[13]

Narasimhan

Shunyu Yao, Howard Chen, John Yang, and Karthik R. Narasimhan. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. In Advances in Neural Information Processing Systems, October 2022

work page 2022
[14]

Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig

Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A Realistic Web Environment for Building Autonomous Agents, April 2024. 11

work page 2024

[1] [1]

An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors

Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, and Toby Jia-Jun Li. An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI ’24, pages 1–28, New York, NY , USA, May 2024. Association for Computing Machinery

work page 2024

[2] [2]

DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

work page 2025

[3] [3]

Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust

Izzeddin Gur, Hiroki Furuta, Austin V . Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis. In The Twelfth International Conference on Learning Representations , October 2023

work page 2023

[4] [4]

WebV oyager: Building an End-to-End Web Agent with Large Multimodal Models, June 2024

Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebV oyager: Building an End-to-End Web Agent with Large Multimodal Models, June 2024

work page 2024

[5] [5]

Prompting is Not All You Need! Evaluating LLM Agent Simulation Methodologies with Real-World Online Customer Behavior Data, June 2025

Yuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao, Sisong Bei, Jiri Gesi, Yaochen Xie, Zheshen, Wang, Qi He, and Dakuo Wang. Prompting is Not All You Need! Evaluating LLM Agent Simulation Methodologies with Real-World Online Customer Behavior Data, June 2025

work page 2025

[6] [6]

UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design, February 2025

Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Laurence Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, and Dakuo Wang. UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design, February 2025

work page 2025

[7] [7]

WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents, April 2024

Michael Lutz, Arth Bohra, Manvel Saroyan, Artem Harutyunyan, and Giovanni Campagna. WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents, April 2024

work page 2024

[8] [8]

LASER: LLM Agent with State-Space Exploration for Web Navigation, February 2024

Kaixin Ma, Hongming Zhang, Hongwei Wang, Xiaoman Pan, Wenhao Yu, and Dong Yu. LASER: LLM Agent with State-Space Exploration for Web Navigation, February 2024. 10

work page 2024

[9] [9]

WebGPT: Browser-assisted question-answering with human feedback, June 2022

Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted question-answering with human feedback, June 2022

work page 2022

[10] [10]

WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning, January 2025

Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong. WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning, January 2025

work page 2025

[11] [11]

Paloma Sodhi, S. R. K. Branavan, Yoav Artzi, and Ryan McDonald. SteP: Stacked LLM Policies for Web Actions, April 2024

work page 2024

[12] [12]

WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning, May 2025

Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning, May 2025

work page 2025

[13] [13]

Narasimhan

Shunyu Yao, Howard Chen, John Yang, and Karthik R. Narasimhan. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. In Advances in Neural Information Processing Systems, October 2022

work page 2022

[14] [14]

Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig

Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A Realistic Web Environment for Building Autonomous Agents, April 2024. 11

work page 2024