Pith. sign in

REVIEW 2 major objections 6 minor 2 cited by

WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B web agent trained on about 4,000 synthesized trajectories beats agents trained on 7,400 real ones and 20,000 tutorial ones on WebArena-Lite.

desk verdict A plausible new way to synthesize web trajectories offline, but the headline gain depends on unvalidated world-model and reward-model proxies. read the letter →

arxiv 2507.04370 v1 pith:4T3TBJYM submitted 2025-07-06 cs.AI

Yifei Gao , Junhong Ye , Jiaqi Wang , Jitao Sang This is my paper
classification cs.AI
keywords webagentsworldmodelMonteCarlotreesearchtrajectorysynthesisArenacurriculumlearningUIunderstandingsupervisedfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that high-quality web-agent training data can be manufactured offline, replacing expensive and noisy real-environment interaction. It proposes WebSynthesis, which trains a world model to predict the next web-page observation for a given action, then runs Monte Carlo tree search in that simulated environment to generate goal-directed navigation trajectories. A policy fine-tuned on about 4,000 such synthetic trajectories, preceded by a curriculum of UI-understanding tasks, reaches 20.15% Pass@3 (success within three tries) on WebArena-Lite, higher than OS-Genesis (18.66%, trained on 7.4k real trajectories) and AgentTrek (11.94%, trained on 20k tutorial trajectories). If this holds, the practical implication is that agent self-improvement no longer depends on collecting large amounts of real user interaction data.

What carries the argument

The load-bearing object is the learned world model $\omega_\theta(o_t \mid o_{t-1}, a_t)$, an LLM prompted to output the full next accessibility tree for any action, enabling reversible and cheap simulation. It is coupled with a process reward model $\gamma$ that assigns a 1–5 score to each predicted state given the user instruction, and an MCTS controller that uses the UCB score $U_C = v_C + \epsilon \sqrt{\ln n_P / n_C}$ to balance exploration and exploitation, backpropagating a weighted-average value up the tree. The product is a web action tree whose high-value root-to-node paths become 'valuable' training trajectories and whose failed sibling branches become 'rollback' trajectories of the form $S \to P \to C$, teaching the agent to recognize and recover from mistakes. The policy, the world model, and the reward model are all implemented as LLM prompts; the policy and world model are LoRA-tuned Qwen2.5-7B, and the reward model is GPT-4.

What would settle it

Run the extracted 'valuable trajectories' back in the real WebArena environment and measure the actual task-completion rate; if a large fraction (e.g., more than half) fail in the real environment, the world-model-and-reward selection is not tracking genuine success. A complementary check is to compare, step by step, the world model's predicted accessibility tree against the real page's accessibility tree after the same action and measure how often element IDs, URLs, and content match.

Watch

Extended reading notes

Core claim

WebSynthesis is a full data-generation-and-training pipeline. A policy agent proposes actions in an imagined environment; a learned world model $\omega_\theta$ predicts the resulting accessibility tree (the text representation of page elements); a GPT-4 process reward model scores progress toward the user instruction. An MCTS loop with UCB selection, expansion of at least three candidate actions per node, and URL-based caching builds a web action tree. From this tree, 'valuable trajectories' (paths whose nodes exceed a value threshold) and 'rollback trajectories' (corrective go_back paths from failed siblings) are extracted and used for supervised fine-tuning, after a curriculum that teaches dense captioning, element functionality, and state-transition prediction. The central empirical claim is that the resulting 7B agent scores 20.15% overall Pass@3 on WebArena-Lite, outperforming OS-Genesis-7B trained on 7.4k real trajectories and AgentTrek-7B trained on 20k tutorial-based trajectories, and exceeding GPT-4 with chain-of-thought prompting (13.58%). The paper also reports a monotonic scaling curve: performance rises from 12.5% to 100% of the synthetic dataset, and 75% of the data already matches GPT-4.

Load-bearing premise

The pipeline assumes that the world model's imagined next-page states and the GPT-4 reward model's progress scores correspond to what truly happens and truly succeeds on the real web; if either proxy is false, the selected 'valuable' trajectories are not actually successful and the policy learns incorrect navigation.

Editorial extensions

If this is right

  • Agent-training trajectories can be manufactured offline at a fraction of the API cost of real interaction, because MCTS reuses predicted states and caches pages by URL.
  • Roughly 4,000 synthetic trajectories suffice to match or exceed 7.4k real trajectories, implying that goal-directed search selects higher-information trajectories than open-ended real or tutorial-collected data.
  • A UI-understanding warm-up (dense captioning, element functionality, state-transition prediction) lifts trajectory-trained agents across OS-Genesis, AgentTrek, and WebSynthesis, indicating that UI comprehension and trajectory imitation are complementary training signals.
  • The reported scaling curve from 12.5% to 100% of the synthetic dataset is monotonic and reaches GPT-4-level performance at 75% of the data, so further scaling the offline generator is expected to keep improving policy performance.
  • Rollback trajectories extracted from failed branches teach explicit go_back recovery, adding robustness that valuable-success trajectories alone do not provide.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the world model generalizes beyond its WebArena training domains, the same search-and-synthesize recipe could transfer to other GUI environments (e.g., mobile or desktop interfaces) without building a new real-data collection pipeline.
  • A quick check on the framework's soundness would be to compare the world model's next-step predictions against real page transitions at the same URLs; a high mismatch would mean the search is operating on hallucinated pages.
  • The GPT-4 process reward model is a hidden cost and a potential bottleneck; replacing it with a small learned reward model would make the pipeline cheaper and clarify whether the world model or the reward model is driving the improvement.
  • A stronger variant of the claim would be to show the synthesized trajectories improve online reinforcement learning in a real environment; the paper's own conclusion identifies closed-loop integration and world-model fidelity as the open challenges.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes WebSynthesis, a pipeline that trains a Qwen2.5-7B web agent entirely on synthetic trajectories generated by a learned LLM-based world model and Monte Carlo Tree Search (MCTS). The method has two training stages: (i) curriculum fine-tuning on 'UI fundamental understanding' tasks (dense captioning, element functionality, state transition prediction) and (ii) behavior cloning on 'valuable' and 'rollback' trajectories extracted from the MCTS tree. The main empirical claim, stated in the abstract and supported by Table 2, is that fine-tuning on roughly 4k synthetic trajectories reaches 20.15% Pass@3 on WebArena-Lite, surpassing OS-Genesis-7B (18.66%, 7.4k real trajectories) and AgentTrek-7B (11.94%, 20k tutorial trajectories). The paper also reports a +33.4% improvement from the TextUI warm-up stage and several ablations.

Significance. If the central claim holds, WebSynthesis would be a meaningful advance in sample-efficient web-agent training, showing that self-supervised trajectory synthesis in a learned world model can match or beat training on real or tutorial-collected data with roughly half the data volume. The paper's strengths include evaluation on a standard external benchmark (WebArena-Lite), direct comparison with strong baselines fine-tuned on the same base model, a two-stage curriculum that is ablated carefully, and a clearly described MCTS formulation. However, the empirical validity of the headline comparison rests on the fidelity of two unvalidated proxies: the world model that generates next-page observations and the GPT-4 process reward model that scores partial trajectories. The paper itself acknowledges 'compounding errors in multi-step rollouts' and 'the fidelity of learned world models' as remaining limitations in the conclusion, but it provides no experiment measuring either proxy. Consequently, the current evidence does not yet establish that the synthesized trajectories actually solve the WebArena tasks in the real environment.

major comments (2)
  1. [Sec. 4.1] The load-bearing assumption of the pipeline is that the world model ω (Eq. 1) produces observations that match real WebArena transitions and that the process reward model γ assigns scores tracking true task success. Valuable trajectories are identified in Sec. 3.3 by thresholding the node value vC, which is computed from γ on top of ω-predicted states. Neither proxy is validated: there is no comparison of ω-predicted next pages against real transitions, no measurement of γ agreement with actual task success, and no replay of extracted trajectories in the real environment. If ω hallucinates plausible pages or γ rewards internally coherent but wrong branches, the 'valuable' trajectories may be nonsensical for real WebArena, and the improvement in Table 2 could be an artifact of learning patterns that transfer coincidentally. This gap is acknowledged in the conclusion ('compounding errors in multi-step rollouts', 'fidelity of learned world models') but no experiment addresses it. I recommend adding: (a) a world-model prediction accuracy metric (e.g., element-level or sentence-level match between predicted and actual next A11y trees on a held-out set); (b) a comparison of γ scores with real binary success on a sample of rollouts; and (c) a sanity check that a subset of extracted trajectories is executable in the real WebArena sandbox.
  2. [Sec. 4.1; Sec. 3.2] The world model training details are missing. Section 4.1 says only that 'we construct our dataset in online setting using the provided web environment for world model training,' with no information about the amount of transition triples, the training procedure, the number of epochs, or the final world-model performance. Without this, the reader cannot assess whether ω is a faithful simulator or a memorizing language model. Please report the data size, training recipe, and at least one quantitative evaluation of ω's predictive fidelity.
minor comments (6)
  1. [Table 1] The term 'State Transition Perdition' in Table 1 appears to be a typo; it should be 'State Transition Prediction'.
  2. [Table 7] The template label 'State T ransmission' appears to be a formatting/typographical error; the surrounding text uses 'State Transition Prediction'.
  3. [Sec. 4.1] The evaluation section says 165 test cases from WebArena-Lite are used, but it does not report the breakdown of tasks per site (the number of Maps tasks is not shown in Table 2). Please include the site-wise counts for transparency.
  4. [Fig. 4] The data-scaling curve in Figure 4 is described in terms of performance gains over data percentage, but the exact y-axis scale and the number of evaluation runs are not stated; please clarify the metric and add error bars or at least a note on variance.
  5. [Sec. 3.2] The equation in Eq. (1) writes 'argmax{a0,a1,···aT } Σ γθ(ot, at)' but the argument of the sum is not fully specified; as written, the objective depends on the policy and world model through ot, which should be stated explicitly for clarity.
  6. [Sec. 2.2] The related work section mentions WebEvolver, WMA, and WebDreamer, but does not compare their world-model validation strategies with the proposed approach; a short paragraph on how those works evaluate world-model fidelity would help position WebSynthesis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: WebSynthesis is validated on external WebArena, so the internal world-model/reward-model selection loop is not a by-construction reduction.

full rationale

The paper's central claim is an externally measured result: the policy trained on WebSynthesis trajectories is evaluated on WebArena-Lite via standard string/url matching, independent of both the learned world model and the GPT-4 process reward model. The world model and reward model only shape the synthetic training set through Eq. (1) and the MCTS selection in Sections 3.2-3.3; they do not define the evaluation metric. Selecting trajectories by internal node value v_C and then measuring real WebArena success is empirically testable and is tested in Table 2, so the headline comparison is not manufactured by construction. The paper's acknowledged limitations about world-model fidelity and compounding errors in multi-step rollouts are genuine validity risks, but they concern proxy accuracy, not circularity: an unvalidated proxy is different from a claim that reduces to its own inputs. No load-bearing self-citation or imported uniqueness theorem appears; the cited world-model work (WMA, WebDreamer) is external prior work, and the world model is trained on real WebArena transitions rather than assumed by citation. The term 'valuable trajectories' is an internal selection label, not a redefinition of the external success metric. Therefore no circular step is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The system-level claim rests on four unvalidated domain assumptions: world-model fidelity, reward-model fidelity, no task leakage between synthesis and evaluation, and accuracy of GPT-4o annotations. The paper reports no calibration checks for these proxies. There are no invented physical entities.

free parameters (4)
  • MCTS exploration constant epsilon (UCB) = not reported
    Controls exploration versus exploitation during tree search in Section 3.2; the value is not given yet it affects trajectory diversity and quality.
  • DFS target-node value threshold = not reported
    Section 3.3 keeps trajectories only when node value vC exceeds a predefined threshold; the threshold is not specified.
  • Minimum candidate actions per expansion = 3
    Section 3.2 samples at least three distinct actions per node; this hand-chosen breadth parameter affects search coverage and synthesis cost.
  • Curriculum dataset sizes = 2k / 6k / 7k / 4k
    Table 1 sets per-stage data scales by hand; the claimed sample efficiency (4k trajectories) depends on these choices.
assumptions (4)
  • domain assumption The LLM world model omega accurately simulates web page transitions for arbitrary actions.
    Section 3.2 defines ot ~ omega_theta(ot | o_{t-1}, a_t) and uses predicted states to score, select, and train trajectories; no accuracy evaluation is reported.
  • domain assumption The GPT-4 process reward model gamma scores partial trajectories proportionally to true task completion.
    Sections 3.2 and 3.3 use vC to select valuable trajectories and backpropagate values; no agreement with real task success is measured.
  • domain assumption Trajectories synthesized inside WebArena transfer to WebArena-Lite without task-template leakage.
    Stage 1 data are collected by random exploration in WebArena and evaluation uses WebArena-Lite; the paper does not state whether MCTS task prompts overlap with the 165 test tasks.
  • domain assumption GPT-4o-generated dense captions and functionality descriptions are accurate enough to teach UI understanding.
    Section 3.1 trains the agent on GPT-4o captions and functional descriptions without human verification or quality checks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis." pith.science (2026). https://pith.science/paper/4T3TBJYM

@misc{pith2026250704370,
  author       = {Pith},
  title        = {Pith review of: WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4T3TBJYM}},
  note         = {Machine review of arXiv:2507.04370}
}
read the original abstract

Recent advancements in large language models (LLMs) have significantly improved the capabilities of web agents. However, effectively navigating complex and dynamic web environments still requires more advanced trajectory-level planning and execution. Prior studies have addressed self-improving agents by collecting extensive GUI trajectories from real-environment interactions. Despite their effectiveness, these approaches encounter two critical challenges: (1) Uncontrollable environment states, where real or sandboxed web environments often yield unstable and non-deterministic feedback, complicating the reproduction and debugging of agent behaviors; and (2) High API costs, as generating even a single interaction trajectory can involve hundreds of queries, leading to considerable API usage and computational expenses. To address these limitations and enable scalable self-improvement for agents, we propose WebSynthesis, a novel framework for trajectory synthesis and training. WebSynthesis leverages a learned world model to simulate virtual web environments, allowing a policy agent to perform efficient and reversible tree-based planning. This approach supports the large-scale generation of diverse and high-quality trajectories, which are subsequently utilized to refine the agent's policy. Experimental results demonstrate that an agent trained using WebSynthesis on a small-scale synthetic dataset achieves performance comparable to or even surpassing that of models trained on large-scale real-world data.

Figures

Figures reproduced from arXiv: 2507.04370 by the authors.

Figure 1
Figure 1. An overview of the UI Fundamental Understanding datasets, which offer single-step, fine [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of world model-guided Monte Carlo Tree Search and trajectory collection. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. An overview of the UI Behavior Cloning dataset, which provides multi-step demonstrations [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance improvement with the syn￾thetic trajectory data scaling up By controlling the proportion of the WebSyn￾thesis dataset, we aimed to explore the impact of increasing the synthetic data scale on the per￾formance of policy agents. As shown in the Figure, our ev…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SEE: Structure-aware Exploring \& Exploiting for Long-horizon GUI Agent Trajectory Synthesis

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A graph-based pipeline automatically synthesizes long, multi-step mobile-app interaction trajectories, and fine-tuning GUI agents on them improves task success and cross-benchmark transfer.

  2. Quo Vadis, World Modeling?

    cs.CV 2026-08 conditional novelty 5.0 of 10

    An agent-centric reframing of world modeling, replacing physical state prediction with 'information transitions' organized into six proxy functions and three empowerment levels.

Reference graph

Works this paper leans on

48 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [1]

    Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods

    Yuji Cao, Huan Zhao, Yuheng Cheng, Ting Shu, Yue Chen, Guolong Liu, Gaoqi Liang, Junhua Zhao, Jinyue Yan, and Yun Li. Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods. IEEE Transactions on Neural Networks and Learning Systems, 36(6):9737–9757, 2025

    work page 2025

  2. [2]

    Web agents with world models: Learning and leveraging environment dynamics in web navigation

    Hyungjoo Chae, Namyoung Kim, Kai Tzu-iunn Ong, Minju Gwak, Gwanwoo Song, Jihoon Kim, Sunghwan Kim, Dongha Lee, and Jinyoung Yeo. Web agents with world models: Learning and leveraging environment dynamics in web navigation. arXiv preprint arXiv:2410.13232, 2024

    arXiv

  3. [3]

    Understanding world or predicting future? a comprehensive survey of world models, 2024

    Jingtao Ding, Yunke Zhang, Yu Shang, Yuheng Zhang, Zefang Zong, Jie Feng, Yuan Yuan, Hongyuan Su, Nian Li, Nicholas Sukiennik, Fengli Xu, and Yong Li. Understanding world or predicting future? a comprehensive survey of world models, 2024

    work page 2024

  4. [4]

    Plan-and-act: Improving planning of agents for long-horizon tasks

    Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anu- manchipalli, Kurt Keutzer, and Amir Gholami. Plan-and-act: Improving planning of agents for long-horizon tasks. arXiv preprint arXiv:2503.09572, 2025

    arXiv 2025

  5. [5]

    Webevolver: Enhancing web agent self-improvement with coevolving world model

    Tianqing Fang, Hongming Zhang, Zhisong Zhang, Kaixin Ma, Wenhao Yu, Haitao Mi, and Dong Yu. Webevolver: Enhancing web agent self-improvement with coevolving world model. arXiv preprint arXiv:2504.21024, 2025

    arXiv 2025

  6. [6]

    From llm reasoning to autonomous ai agents: A comprehensive review

    Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. From llm reasoning to autonomous ai agents: A comprehensive review. arXiv preprint arXiv:2504.19678, 2025

    arXiv 2025

  7. [7]

    Is your llm secretly a world model of the internet? model-based planning for web agents

    Yu Gu, Kai Zhang, Yuting Ning, Boyuan Zheng, Boyu Gou, Tianci Xue, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, et al. Is your llm secretly a world model of the internet? model-based planning for web agents. arXiv preprint arXiv:2411.06559, 2024

    arXiv 2024

  8. [8]

    Simulate before act: Model-based planning for web agents

    Yu Gu, Boyuan Zheng, Boyu Gou, Kai Zhang, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, and Yu Su. Simulate before act: Model-based planning for web agents

    work page

Show all 48 references
  1. [9]

    Webcot: Enhancing web agent reasoning by reconstructing chain-of-thought in reflection, branching, and rollback, 2025

    Minda Hu, Tianqing Fang, Jianshu Zhang, Junyu Ma, Zhisong Zhang, Jingyan Zhou, Hongming Zhang, Haitao Mi, Dong Yu, and Irwin King. Webcot: Enhancing web agent reasoning by reconstructing chain-of-thought in reflection, branching, and rollback, 2025

    2025

  2. [10]

    Al Sallab, Senthil Yogamani, and Patrick Pérez

    B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Yogamani, and Patrick Pérez. Deep reinforcement learning for autonomous driving: A survey, 2021

    2021

  3. [11]

    Visualwebarena: Evaluating multimodal agents on realistic visual web tasks

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. arXiv preprint arXiv:2401.13649, 2024. 11

    2024 arXiv

  4. [12]

    Tree search for language model agents, 2024

    Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents, 2024

    2024

  5. [13]

    Autowebglm: A large language model- based web navigating agent

    Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, et al. Autowebglm: A large language model- based web navigating agent. In Proceedings of the 30th ACM SIGKDD Conference on Knowl- edge Discovery and D...

    2024

  6. [14]

    Visualagentbench: Towards large multimodal models as visual foundation agents, 2024

    Xiao Liu, Tianjie Zhang, Yu Gu, Iat Long Iong, Yifan Xu, Xixuan Song, Shudan Zhang, Hanyu Lai, Xinyi Liu, Hanlin Zhao, Jiadai Sun, Xinyue Yang, Yu Yang, Zehan Qi, Shuntian Yao, Xueqiao Sun, Siyi Cheng, Qinkai Zheng, Hao Yu, Hanchen Zhang, Wenyi Hong, Ming Ding, Lihang Pan, Xia...

    2024

  7. [15]

    Infiguiagent: A multimodal generalist gui agent with native reasoning and reflection, 2025

    Yuhang Liu, Pengxiang Li, Zishu Wei, Congkai Xie, Xueyu Hu, Xinchen Xu, Shengyu Zhang, Xiaotian Han, Hongxia Yang, and Fei Wu. Infiguiagent: A multimodal generalist gui agent with native reasoning and reflection, 2025

    2025

  8. [16]

    Moerland, Joost Broekens, Aske Plaat, and Catholijn M

    Thomas M. Moerland, Joost Broekens, Aske Plaat, and Catholijn M. Jonker. Model-based reinforcement learning: A survey. Foundations and Trends® in Machine Learning, 16(1):1–118, 2023

    2023

  9. [17]

    Agent q: Advanced reasoning and learning for autonomous ai agents

    Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.07199, 2024

    2024 arXiv

  10. [18]

    Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning

    Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, et al. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning. arXiv preprint arXiv:2411.02337, 2024

    2024 arXiv

  11. [19]

    Ui-tars: Pioneering automated gui interaction with native agents

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326, 2025

    2025 arXiv

  12. [20]

    Mastering atari, go, chess and shogi by planning with a learned model

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Si- mon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, 2020

    2020

  13. [21]

    Zerosearch: Incentivize the search capability of llms without searching, 2025

    Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou. Zerosearch: Incentivize the search capability of llms without searching, 2025

    2025

  14. [22]

    Os-genesis: Automating gui agent trajectory construction via reverse task synthesis

    Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, et al. Os-genesis: Automating gui agent trajectory construction via reverse task synthesis. arXiv preprint arXiv:2412.19723, 2024

    2024 arXiv

  15. [23]

    Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning

    Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, et al. Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning. arXiv preprint arXiv:2505.16421, 2025

    2025

  16. [24]

    Agenttrek: Agent trajectory synthesis via guiding replay with web tutorials

    Yiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang, Zekun Wang, Yuchen Mao, Caiming Xiong, and Tao Yu. Agenttrek: Agent trajectory synthesis via guiding replay with web tutorials. arXiv preprint arXiv:2412.09605, 2024

    2024 arXiv

  17. [25]

    Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023

    2023

  18. [26]

    Agentoccam: A simple yet strong baseline for llm-based web agents, 2024

    Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala. Agentoccam: A simple yet strong baseline for llm-based web agents, 2024. URL https://arxiv. org/abs/2410.13825. 12

    2024 arXiv

  19. [27]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search, 2024

    2024

  20. [28]

    Large language model-brained gui agents: A survey

    Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, et al. Large language model-brained gui agents: A survey. arXiv preprint arXiv:2411.18279, 2024

    2024 arXiv

  21. [29]

    Rest-mcts*: Llm self-training via process reward guided tree search, 2024

    Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mcts*: Llm self-training via process reward guided tree search, 2024

    2024

  22. [30]

    Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration

    Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and V olker Tresp. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 23378–23386, 2025

    2025

  23. [31]

    Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning

    Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning. arXiv preprint arXiv:2504....

    2025 arXiv

  24. [32]

    Synapse: Trajectory-as-exemplar prompting with memory for computer control, 2024

    Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An. Synapse: Trajectory-as-exemplar prompting with memory for computer control, 2024

    2024

  25. [33]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023

    2023 arXiv

  26. [34]

    Proposer-agent-evaluator (pae): Autonomous skill discovery for foundation model internet agents

    Yifei Zhou, Qianlan Yang, Kaixiang Lin, Min Bai, Xiong Zhou, Yu-Xiong Wang, Sergey Levine, and Erran Li. Proposer-agent-evaluator (pae): Autonomous skill discovery for foundation model internet agents. arXiv preprint arXiv:2412.13194, 2024

    2024 arXiv

  27. [35]

    Postmill,

    Maciej ´Swiechowski, Konrad Godlewski, Bartosz Sawicki, and Jacek Ma´ndziuk. Monte carlo tree search: a review of recent modifications and applications. Artificial Intelligence Review, 56(3):2497–2562, July 2022. 13 A UI Fundamental Understanding Instruction Template You’re a ...

    2022

  28. [36]

    Let’s think step-by-step

    You should think step by step and then issue the next action. Start with a "Let’s think step-by-step." phrase

  29. [37]

    You should only issue an action that is valid given the current web page

  30. [38]

    You should only issue one action at a time

  31. [39]

    In summary, the next action I will perform is

    Generate the action in the correct format. Start with a "In summary, the next action I will perform is" phrase, followed by action inside “““. For example, "In summary, the next action I will perform is “‘click [1234]“‘"

  32. [40]

    Don’t generate anything after stop

    Issue stop action when you think you have achieved the objective. Don’t generate anything after stop. Table 8: Policy Agent Prompt Template. 15 You are an autonomous intelligent agent tasked with navigating a web browser. You will be given a web GUI-based task. Specifically, y...

  33. [41]

    Please think step by step based on the current page observation and the actions taken, and give the maximum possible next page observation

  34. [42]

    You should ensure the richness of the observations of the web page to be predicted and support the continuous operation process

  35. [43]

    In summary, the next web page observation is

    Please generate the content of the next page in the correct format. Start with the phrase "In summary, the next web page observation is" and then add supplements within “‘<your generated contents>“‘. For example, "In summary, the next web page observation is “‘Tab 0 (current):...

  36. [44]

    Do not generate any content after stopping.""", Table 9: World Model Prompt Template

    When you think you have achieved the full content prediction of the next page, issue a stop operation with [END]. Do not generate any content after stopping.""", Table 9: World Model Prompt Template. You are an expert in evaluating GUI agent task trajectories. Your task is to ...

  37. [45]

    User Instruction: Describes the user’s intended task

  38. [46]

    - The accessibility tree of the current web page: This is a simplified representation of the webpage, providing key information

    Action History: Includes two key parts: - Reasoning and Action for Each Step: A sequence of actions performed by the agent, including the reasoning thought and final executed action. - The accessibility tree of the current web page: This is a simplified representation of the w...

  39. [47]

    Trajectory Coherence: - Do the steps and corresponding actions follow a logical sequence toward the goal? - Are the actions clearly described and specific? - Are there redundant or unnecessary actions?

  40. [48]

    The sequence is logically clear with no noticeable redundancies

    Task Completion: - Does the trajectory successfully achieve the instructed task? - Are all necessary interactions completed? - Are error cases handled appropriately? Scoring Guidelines: Rate the trajectory on a scale of 1 to 5 based on the evaluation criteria: - 5: The task is...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.