Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

InSTA: Towards Internet-Scale Training For Agents

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a fully automatic, human-free pipeline—an LLM task proposer, an LLM agent, and an LLM judge—can generate enough web-navigation training data from 150k live sites that a 1.7B-parameter model reaches a 56.9% success…

desk verdict A serious data flywheel with a conditional headline: the small-model-beats-frontier claim needs independent human-verified evaluation before it is believable. read the letter →

arxiv 2502.06776 v2 pith:EDSDDNC5 submitted 2025-02-10 cs.LG cs.AI

Brandon Trabucco , Gunnar Sigurdsson , Robinson Piramuthu , Ruslan Salakhutdinov This is my paper · ORCID
classification cs.LGcs.AI
keywords webnavigationagentsLLM-as-judgesynthetictrainingdatainternet-scaleflywheeltaskgenerationsmalllanguagemodelszero-shottransferagentpipeline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Web-agent training is bottlenecked by human demonstrations, which cover a few hundred popular sites and go stale as the web changes. This paper claims that all three ingredients of a training pipeline—writing tasks, attempting them, and deciding which attempts succeeded—can be replaced by pretrained LLMs operating at scale. The resulting pipeline annotated 150k live sites with tasks, collected 20k filtered trajectories from a Qwen 3 235B policy, and fine-tuned a 1.7B-parameter model that reaches a 56.9% success rate on 3,000 held-out sites and tasks. That number beats the 235B data collection policy and a 400B-parameter Llama 4 Maverick, and reaches 94.7% of Gemini 2.5 Flash's performance, while the data collection itself cost about $521 in spot compute. If these numbers hold, the paper has shown that frontier-competitive web agents can be trained without human annotation, on a budget that small labs can afford.

What carries the argument

The load-bearing object is the closed-loop task proposer combined with an LLM judge for trajectory filtering. In the first phase the proposer, given only a website URL, generates an easy initial task or marks the site unsafe; an agent then explores the site, and the resulting trajectory is fed back to the proposer, which writes a harder, grounded task based on the actual page state. A separate judge consumes the task plus the agent's full trajectory of screenshots-as-markdown and actions, and emits a continuous success score in JSON; filtering to trajectories scored exactly 1.0 selects the training data. The judge's confidence score, defined as $2 \cdot |r_T - 1/2|$, is the piece that makes the flywheel trustworthy, because the paper reports accuracy rising from 82.6% overall to 93.1% on maximally confident judgments, and staying stable as site PageRank falls.

What would settle it

Have fresh human annotators independently label a new random sample of, say, 200 trajectories the judge scored as successful and 200 it scored as failed, drawn from low-PageRank sites; if human-judge agreement on this larger sample is far below the reported 82.6%, the filtered-SFT gains may reflect judge-pleasing rather than task competence. A sharper version is to run the trained 1.7B agent on a benchmark whose ground truth is deterministic and human-verified, where the judge never writes the tasks or checks the answers; the central claim would collapse if the 56.9% success rate does not transfer there.

Watch

Extended reading notes

Core claim

The central claim is that an internet-scale data flywheel for agents can run without human annotations: a language-model task proposer filters and labels 150k safe sites drawn from the top 1M by PageRank, a language-model agent (Qwen 3 235B) explores each site and produces trajectories with reasoning traces, and a language-model judge scores each trajectory for success, efficiency, and self-correction. Training a Qwen 3 1.7B model by supervised fine-tuning on the judge-filtered successful trajectories yields an agent with a 56.9% success rate on a held-out set of 3,000 websites and tasks, outperforming its own data collection policy (Qwen 3 235B) and Llama 4 Maverick, a model roughly 235 times larger, and reaching 94.7% of Gemini 2.5 Flash's success rate. The same checkpoint transfers zero-shot to WebVoyager without any WebVoyager data, and mixing 20% of the pipeline's data with human data improves generalization on WebLINX and Mind2Web test splits by 149.0% and 156.3% respectively. The paper interprets these results as evidence that LLMs are powerful data curation tools: the safety filter reaches 97% accuracy, the judge reaches 82.6% accuracy against human labels, and judge confidence is calibrated enough that high-confidence judgments reach 93.1% accuracy.

Load-bearing premise

The held-out test set—3,000 websites and tasks generated and scored by the same LLM family that produced the training data—is assumed to faithfully measure general web-agent ability, rather than measuring skill at satisfying the pipeline's own judge.

Editorial extensions

If this is right

  • Success on the held-out test improves monotonically with the number of training trajectories, and judge-filtered data improves faster than uniformly sampled data, so scaling the pipeline further should keep paying off.
  • A 1.7B agent trained only on pipeline data transfers zero-shot to WebVoyager, matching frontier LLMs on three of four judges, which implies the learned skill is not memorized site-specific behavior.
  • Adding 20% pipeline data to human demonstrations from WebLINX and Mind2Web improves test-set step accuracy by 149.0% and 156.3% respectively without lowering scores on the benchmarks' own test splits, so the data is complementary to human data rather than a substitute only.
  • Raising the reasoning-token budget from 50 to 500 improves success monotonically and does not appear saturated at 500, so coupling the pipeline with test-time reasoning is a promising axis.
  • The 150k-site data collection run cost about $521.55 in spot compute, so the recipe, if correct, removes cost as the main barrier to training frontier-competitive agents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The judge's calibrated confidence suggests a natural extension the paper does not run: use the judge score as a dense reward for reinforcement learning instead of a binary filter, which could push the same 1.7B agent well past its data collection policy.
  • Because the pipeline re-annotates the live web, re-running it on fresh Common Crawl snapshots would yield continuously updated agents; the paper frames the environment as 'dynamic' but evaluates only a single snapshot.
  • The headline comparison may flatter the small model if the task proposer and judge share blind spots with the training data; a human-written, human-scored held-out task suite would be the stricter test, and is within reach at modest cost.
  • The $521 cost figure only covers data collection; a full cost comparison would need training compute, judge API calls, and evaluation runs, so the economic claim should be read narrowly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents InSTA, an automated pipeline that generates web-navigation training data at scale: a task proposer LLM annotates roughly 150k websites with tasks, a policy LLM attempts them in a live browser, and a judge LLM scores trajectory success. The authors train Qwen 3 1.7B with SFT on judge-filtered trajectories and report a 56.9% success rate on a held-out set of 3,000 sites and tasks, claiming to surpass the Qwen 3 235B data-collection policy and Llama 4 Maverick, while reaching 94.7% of Gemini 2.5 Flash's performance. Additional experiments show zero-shot transfer to WebVoyager and improvements on WebLINX and Mind2Web when mixing InSTA data with human data.

Significance. The paper's main contribution is a concrete, scalable recipe for removing human annotation from web-agent training data collection, with code, models, and datasets released. The data-scaling trends, the cost analysis, and the zero-shot transfer results are valuable and go beyond typical fixed-benchmark training. If the headline evaluation is independently confirmed on a human-verified sample of the actual test set, the claim that a 1.7B model can compete with frontier LLMs on diverse live websites would be an important result for the community. The authors are transparent about limitations (e.g., one task per site, 30-action cap) and include a thoughtful discussion of safeguards, which strengthens the paper.

major comments (3)
  1. [§5.1, §6.1, Appendix H] The headline success rate of 56.9% is measured with the same Qwen 3 235B judge used to filter the SFT trajectories, and this judge is not among the models human-calibrated in Figure 5 (which covers Llama 3.1 70B, GPT-4o, and Gemini 1.5 Pro on only 100 trajectories). Appendix H states that the held-out test set is "produced by the task generation feedback loop," so the test tasks come from the same pipeline whose judge-filtered outputs shape the trained agent. The human verifiability study in Table 2 / Appendix E.2 is limited to initial-phase tasks and does not validate the refined feedback-loop tasks that constitute the test set. The independent secondary judges in Figure 7 mitigate this concern, but those judges are also unverified on the InSTA test set and may share biases in inferring success from trajectories. Concretely, the authors should (a) human-annotate a few hundred trajectories from the actual 3,000-site test set, (b) report per-judge accuracy on that sample, and (c) report the headline success rates using only judges whose calibration on the test set has been established.
  2. [§5.2 vs §6.1] There is an inconsistency about which policy generated the training trajectories. Section 5.2 says the large data-collection experiment used "a fine-tuned Qwen3 1.7B as the agent (refer to the next section)" with "Qwen3 235B zero-shot as the judge," whereas Section 6.1 and the caption of Figure 7 say the 20k trajectories used for SFT were "produced by a Qwen 3 235B data collection policy." Since the abstract claims to "outperform the data collection policy Qwen 3 235B," it matters which agent actually collected the data. Please clarify the exact provenance of the 20k trajectories and of the 150k dataset, and reconcile the two statements.
  3. [Figures 7, 8, 10] All success-rate comparisons, including the claims of outperforming Qwen 3 235B and Llama 4 Maverick, are reported as single point estimates without error bars, confidence intervals, or repeated seeds. Given that the best human-calibrated judge accuracy is 82.6% (Section 5.1), judge noise is substantial relative to the reported differences; the authors should report means and variance across at least a few seeds, or bootstrap resamples over the 3,000-site test set.
minor comments (6)
  1. [Appendix E.1] The unsafe_sites_list is not included in the manuscript ("refer to the code release"); to make Table 1 reproducible, please list the 50 unsafe sites in the appendix.
  2. [Abstract / Table 3] The abstract says 150k sites, but Table 3 lists 146,746 tasks after filtering; the paper should state whether the 150k figure is approximate.
  3. [Figure 7 caption] "Gpt 4.1 Nano Judge" should be "GPT-4.1 Nano" for consistency with the model naming elsewhere.
  4. [Equation (4)] The equation line "st at ∼ πLLM(at|st, ˜c)" mixes the policy and the state transition; use separate equations for the policy and the environment transition.
  5. [Appendix A] The phrase "we provide instruct the task proposer" should be corrected to "we instruct the task proposer."
  6. [Appendix H] Specify how the 3,000 test-set websites were sampled (e.g., uniformly from the 146,746, or stratified by PageRank), and confirm the exact overlap-removal procedure against the training websites.

Circularity Check

1 steps flagged · score 5.0 of 10

The headline 56.9% success rate is measured by the same Qwen3 235B judge used to filter the training data, so the primary metric is partly self-referential; independent secondary judges and WebVoyager transfer partially break the loop.

  1. self definitional [Section 6.1 (Performance Improves With Data Scale) and Figure 7 caption; Appendix H]
    "To proceed, we collect 20k trajectories using a Qwen 3 235B data collection policy, annotated with scores from a Qwen 3 235B judge. ... To filter the data, we select trajectories where Judge(Success) = 1. ... We report success rates on a test set of 3,000 held-out websites and tasks."

    The same Qwen3 235B judge both selects the training data and defines the headline evaluation metric. Section 6.1 keeps only trajectories with Judge(Success)=1, and the 56.9% success rate reported in the abstract and Section 7 is the fraction of held-out test trajectories that the same judge scores as successful (Figure 7, Qwen3 235B Judge panel). The trained model is therefore optimized to imitate behavior the judge already approves, and the primary 'success rate' measures agreement with that judge's preferences rather than an independently verified task-completion rate. The held-out website/task split prevents task leakage, but it does not break the judge loop.

full rationale

The paper's main derivation chain is: an LLM task proposer generates tasks, LLM agents produce trajectories, an LLM judge filters successful trajectories, a small model is fine-tuned on the filtered set, and the model is then evaluated on a held-out set of 3,000 websites and tasks. The partial circularity is that the filter and the primary evaluator are the same Qwen3 235B judge: training-data quality is defined by Judge(Success)=1, and the headline success rate is the fraction of held-out trajectories receiving Judge(Success)>0.5 from that same judge. This makes the primary metric a measure of fit to the judge's preferences, not an independent measurement of real-world task completion. The paper mitigates this in several ways: Figure 7 reports success rates under GPT-4.1 Nano, Llama 4 Maverick, and Gemini 2.5 Flash judges; Figure 8 shows zero-shot transfer to the externally defined WebVoyager benchmark; and Appendix H confirms WebVoyager websites are absent from the 20k training trajectories. These are genuine independent checks and substantially reduce the severity of the circularity. Two further limitations, though not circularity, weaken the ground truth of the primary metric and should be weighed in the verdict: Section 5.1 human-calibrates judges only for Llama 3.1 70B, GPT-4o, and Gemini 1.5 Pro (78.0-82.6% accuracy), not for the Qwen3 235B judge used for filtering and the headline curve; and the human verifiability study in Appendix E.2 covers only initial-phase tasks, not the refined feedback-loop tasks that constitute the held-out test set. The paper does not rely on load-bearing self-citation or imported uniqueness theorems; the Trabucco et al. (2024) citation for data-mixing hyperparameters is not load-bearing. Overall, the central result has independent content through the secondary judges and WebVoyager, but the primary 56.9% number is partially circular because the training filter and evaluation metric share the same judge.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities or mathematical objects. The load-bearing assumptions are empirical: that LLM judges and task proposers validated on small curated sets behave reliably at 150k-site scale, and that a self-generated test set measures real agent competence. Free parameters are mostly data-pipeline choices rather than fitted constants.

free parameters (4)
  • Judge filter threshold = Judge(Success) = 1.0
    Selects 10.5k of 20k trajectories for SFT; the central data quality mechanism depends on this binary cutoff.
  • Human-data mixing ratio = 80% human / 20% InSTA
    Used in Section 6.2 static benchmark experiments; chosen to mirror prior synthetic data work and directly affects the reported +149% and +156% generalization gains.
  • InSTA test set size = 3,000 websites and tasks
    The primary evaluation benchmark is a design choice from the same pipeline; reported success rates depend on its task difficulty and judge bias properties.
  • Task proposer feedback loops = 1
    Only one round of task refinement is used; the authors state that more loops or RL would generate harder tasks, so task difficulty is set by this choice.
assumptions (4)
  • domain assumption LLM judge scores generalize from 100 human-labeled trajectories to the full 150k-site distribution and to live websites.
    Section 5.1 reports 82.6% agreement with human labels on 100 trajectories and uses this to justify judge-based evaluation and filtering at scale.
  • domain assumption The safety filter's 97% accuracy on 100 curated websites generalizes to the top 1M Common Crawl sites.
    Section 4.2 and Table 1 report high precision/recall on a hand-picked 100-site set, which is the only safety validation for filtering 1M sites down to 150k.
  • domain assumption The held-out InSTA test set of 3,000 sites/tasks is a meaningful measure of general agent capability.
    Appendix H describes the test set as produced by the same task generation feedback loop, so task distribution and judge biases may be shared with training data.
  • domain assumption Generated tasks are achievable and verifiable, based on 300 human annotations across 100 sites.
    Section 4.2 and Appendix E.2 report an 89% verifiable rate from this small sample and use it to argue the pipeline produces reliable tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InSTA: Towards Internet-Scale Training For Agents." pith.science (2026). https://pith.science/paper/EDSDDNC5

@misc{pith2026250206776,
  author       = {Pith},
  title        = {Pith review of: InSTA: Towards Internet-Scale Training For Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EDSDDNC5}},
  note         = {Machine review of arXiv:2502.06776}
}
read the original abstract

The predominant approach for training web navigation agents is to gather human demonstrations for a set of popular websites and hand-written tasks, but it is becoming clear that human data is an inefficient resource. We develop a pipeline to facilitate internet-scale training for agents without laborious human annotations. In the first stage, an LLM annotates 150k sites with agentic tasks. In the next stage, LLM agents complete tasks and produce trajectories. In the final stage, an LLM filters trajectories by judging their success. Language models are powerful data curation tools, identifying harmful content with an accuracy of 97%, judging successful trajectories with an accuracy of 82.6%, and producing effective data. We train agents based on Qwen 3 1.7B that are competitive with frontier LLMs as web agents, while being smaller and faster. Our top agent reaches a success rate of 56.9%, outperforming the data collection policy Qwen 3 235B, a 235 times larger Llama 4 Maverick, and reaching 94.7% of the performance of Gemini 2.5 Flash. We are releasing code, models and data at: https://data-for-agents.github.io.

Figures

Figures reproduced from arXiv: 2502.06776 by the authors.

Figure 1
Figure 1. Overview of the InSTA pipeline. Our work unlocks a dynamic internet-scale environment that allows training small models to match frontier LLMs as agents, on a fraction of the budget. Starting from the top 1M sites on the internet, we annotate 150k sites with challenging agentic tasks, and release the entire pipeline, including code, models and an official huggingface dataset, on our website: https://data-for-agents.… view at source ↗
Figure 2
Figure 2. Annotating 150k live sites with agentic tasks. Starting from 1,000,000 websites, we employ a pretrained language model that marks sites as safe/unsafe for annotation, and assigns a realistic task that a hypothetical user might want to accomplish on each site. The task proposer aggressively filters out 85% of websites from the pipeline, resulting in 150k safe websites annotated with realistic tasks. astop = (“stop”,(… view at source ↗
Figure 3
Figure 3. Most frequent words in our tasks. This wordcloud shows the top 500 most frequent words in tasks from the training set of our official huggingface dataset. The size of each word corresponds to its frequency in the dataset. Our tasks span diverse categories and lexicon. Prompt Details. The task proposer operates in two phases. In an initial phase when just the url of a website is observed, the task proposer generates … view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Automatic evaluation for agents with language model judges. Building on the large and diverse set of tasks generated by the pipeline, we employ pretrained language models to attempt and evaluate web navigation tasks. We dispatch language model agents to perform tasks b…
Figure 5
Figure 5. Figure 5: Language models are robust evaluators. We measure the accuracy of language models for de￾tecting successful trajectories, and find that accuracy remains stable relative to PageRank values (left plot). As models become more confident, their accuracy im￾proves (right plo…
Figure 7
Figure 7. Figure 7: InSTA unlocks great potential in small models. We train agents based on Qwen 3 1.7B using trajectories produced by a Qwen 3 235B data collection policy, and optionally filtered by a Qwen 3 235B judge (see Judge Filtered vs. Uniformly Sampled). We report success rates o…
Figure 8
Figure 8. Figure 8: Our agents zero-shot transfer to WebVoyager. With no additional training or specialized data, our checkpoints for Qwen 3 1.7B in Section 6.1 zero-shot transfer to the WebVoyager benchmark. Trends found on our test set appear to hold for WebVoyager as well, and our top …
Figure 9
Figure 9. Figure 9: Our data transfers to static benchmarks. We train agents with all human data from the We￾bLINX and Mind2Web training sets, and resulting agents struggle to generalize to more diverse test data. Adding our data improves generalization by +149.0% for WebLINX, and +156.3%…
Figure 10
Figure 10. Figure 10: Agents improve with a larger reasoning budget. We ablate the number of tokens in the reasoning budget for the top-performing agent, and see a monotonic improvement in the success rate as the reasoning budget increases. Gemini 2.5 Flash has a 70% success rate with a bu…
Figure 11
Figure 11. Figure 11: System prompt for the exploration phase of task generation. We design the system prompt for task generation to detect and remove unsafe websites. This prompt ensures that tasks are passive, and do not modify content on a website. Refer to the next figures for the in-c…
Figure 12
Figure 12. Figure 12: to assign a category in 3 words or less to encourage simple categories. Categories have 16.9 tasks on average, and 953 categories have more than the mean, while 7741 have less than the mean. There is occasional overlap between categories, which can be observed in [PI…
Figure 13
Figure 13. Figure 13: Largest categories for task generation. We categorize 150k tasks generated by our pipeline in Section 4, and visualize the number of tasks in the largest 70 categories. Top categories include article search, news search, recipe search, and product lookup. The top 12 t…
Figure 14
Figure 14. Figure 14: Most solved categories for task generation. We explore the completion rates for the top categories of tasks generated by our pipeline. We restrict our focus to categories where at least 100 tasks are assigned, and plots the success rates for the top 70 categories. Res…
Figure 15
Figure 15. Figure 15: Least successful categories for internet-scale task generation. Similar to the previous figure, we explore the rates of task completion for the bottom 70 categories that have at least 100 tasks assigned to them. While the majority of the least successful categories ha…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

    cs.AI 2026-07 accept novelty 6.0 of 10

    VLM judges of CUA trajectories are systematically lenient; OSReward measures this with human gold, and OS-Shepherd open models close most of the reliability gap cheaply.

  2. Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Scaling the number of interaction steps, trained via a curriculum over rollout horizon, improves web-agent task success and outperforms scaling per-step reasoning under fixed token budgets.

Reference graph

Works this paper leans on

58 extracted references · 7 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page

  5. [5]

    Language models as agent models

    Jacob Andreas. Language models as agent models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp.\ 5769--5779, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi:10.18653/v1/2022.findings-emnlp.423. URL https://aclanthology.org/2...

    doi:10.18653/v1/2022.findings-emnlp.423 2022

  6. [6]

    Graph of thoughts: Solving elaborate problems with large language models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (16): 0 17682--17690, Mar. 202...

    doi:10.1609/aaai.v38i16.29720 2024

  7. [7]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    arXiv 2020

  8. [8]

    Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023. URL https://arxiv.org/abs/2303.12712

    arXiv 2023

Show all 58 references
  1. [9]

    Fireact: Toward language agent fine-tuning, 2023

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning, 2023. URL https://arxiv.org/abs/2310.05915

    2023 arXiv

  2. [10]

    Xu, Siva Reddy, Quentin Cappart, Graham Neubig, Ruslan Salakhutdinov, Nicolas Chapados, and Alexandre Lacoste

    Thibault Le Sellier De Chezelles, Maxime Gasse, Alexandre Drouin, Massimo Caccia, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han Lù, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Quentin Cappart, Graham Neubig, Ruslan...

    2024 arXiv

  3. [11]

    Mind2web: Towards a generalist agent for the web, 2023

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web, 2023. URL https://arxiv.org/abs/2306.06070

    2023 arXiv

  4. [12]

    Better synthetic data by retrieving and transforming existing datasets, 2024

    Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, and Graham Neubig. Better synthetic data by retrieving and transforming existing datasets, 2024. URL https://arxiv.org/abs/2404.14361

    2024 arXiv

  5. [13]

    The llama 3 herd of models, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, and et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  6. [14]

    W eb V oyager: Building an end-to-end web agent with large multimodal models

    Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. W eb V oyager: Building an end-to-end web agent with large multimodal models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting o...

    2024 doi

  7. [15]

    Cogagent: A visual language model for gui agents, 2023

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogagent: A visual language model for gui agents, 2023. URL https://arxiv.org/abs/2312.08914

    2023 arXiv

  8. [16]

    Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023. URL https://arxiv.org/abs/2312.06674

    2023 arXiv

  9. [17]

    Visualwebarena: Evaluating multimodal agents on realistic visual web tasks, 2024 a

    Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks, 2024 a . URL https://arxiv.org/abs/2401.13649

    2024 arXiv

  10. [18]

    Tree search for language model agents, 2024 b

    Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents, 2024 b . URL https://arxiv.org/abs/2407.01476

    2024

  11. [19]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023. URL https://arxiv.org/abs/2309.06180

    2023 arXiv

  12. [20]

    Rlaif vs

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback, 2024. URL https://arxiv....

    2024 arXiv

  13. [21]

    From generation to judgment: Opportunities and challenges of llm-as-a-judge

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. From generation to judgment: Opportunities and challenges of llm-as-a-judge. arXiv preprint arXiv: ...

    2024

  14. [22]

    Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding?, 2024

    Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, and Xiang Yue. Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding?, 2024. URL https://arxiv.org/abs/2404.05955

    2024 arXiv

  15. [23]

    Weblinx: Real-world website navigation with multi-turn dialogue, 2024

    Xing Han Lù, Zdeněk Kasner, and Siva Reddy. Weblinx: Real-world website navigation with multi-turn dialogue, 2024. URL https://arxiv.org/abs/2402.05930

    2024

  16. [24]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

    2023

  17. [25]

    Playwright

    Microsoft. Playwright. https://github.com/microsoft/playwright, 2024

    2024

  18. [26]

    Agentinstruct: Toward generative teaching with agentic flows, 2024

    Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei ge Chen, Olga Vrousgos, Corby Rosset, Fillipe Silva, Hamed Khanpour, Yash Lara, and Ahmed Awadallah. Agentinstruct: Toward generative teaching with agentic flows, 2024. ...

    2024 arXiv

  19. [27]

    Shikhar Murty, Hao Zhu, Dzmitry Bahdanau, and Christopher D. Manning. Nnetnav: Unsupervised learning of browser agents through environment interaction in the wild, 2025. URL https://arxiv.org/abs/2410.02907

    2025 arXiv

  20. [28]

    Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, and Shuyan Zhou

    Tianyue Ou, Frank F. Xu, Aman Madaan, Jiarui Liu, Robert Lo, Abishek Sridhar, Sudipta Sengupta, Dan Roth, Graham Neubig, and Shuyan Zhou. Synatra: Turning indirect knowledge into direct demonstrations for digital agents at scale, 2024. URL https://arxiv.org/abs/2409.15637

    2024 arXiv

  21. [29]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2024

  22. [30]

    Large language models can self-improve at web agent tasks, 2024

    Ajay Patel, Markus Hofmarcher, Claudiu Leoveanu-Condrei, Marius-Constantin Dinu, Chris Callison-Burch, and Sepp Hochreiter. Large language models can self-improve at web agent tasks, 2024. URL https://arxiv.org/abs/2405.20309

    2024 arXiv

  23. [31]

    REFINER : Reasoning feedback on intermediate representations

    Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. REFINER : Reasoning feedback on intermediate representations. In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter ...

    2024

  24. [32]

    Agent q: Advanced reasoning and learning for autonomous ai agents, 2024

    Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous ai agents, 2024. URL https://arxiv.org/abs/2408.07199

    2024 arXiv

  25. [33]

    Language models are unsupervised multitask learners, 2019

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners, 2019. URL https://api.semanticscholar.org/CorpusID:160025533

    2019

  26. [34]

    Android in the wild: A large-scale dataset for android device control, 2023

    Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. Android in the wild: A large-scale dataset for android device control, 2023. URL https://arxiv.org/abs/2307.10088

    2023 arXiv

  27. [35]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. In Thirty-seventh Conference on Neural Information Processing Systems...

    2023

  28. [36]

    Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold, 2024

    Amrith Setlur, Saurabh Garg, Xinyang Geng, Naman Garg, Virginia Smith, and Aviral Kumar. Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold, 2024. URL https://arxiv.org/abs/2406.14532

    2024 arXiv

  29. [37]

    Scribeagent: Towards specialized web agents using production-scale workflow data, 2024

    Junhong Shen, Atishay Jain, Zedian Xiao, Ishan Amlekar, Mouad Hadji, Aaron Podolny, and Ameet Talwalkar. Scribeagent: Towards specialized web agents using production-scale workflow data, 2024. URL https://arxiv.org/abs/2411.15004

    2024 arXiv

  30. [38]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024. URL https://arxiv.org/abs/2408.03314

    2024 arXiv

  31. [39]

    Fast best-of-n decoding via speculative rejection

    Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://open...

    2024

  32. [40]

    Preference fine-tuning of llms should leverage suboptimal, on-policy data, 2024

    Fahim Tajwar, Anikait Singh, Archit Sharma, Rafael Rafailov, Jeff Schneider, Tengyang Xie, Stefano Ermon, Chelsea Finn, and Aviral Kumar. Preference fine-tuning of llms should leverage suboptimal, on-policy data, 2024. URL https://arxiv.org/abs/2404.14367

    2024 arXiv

  33. [41]

    Common crawl, 2025

    The Common Crawl Foundation . Common crawl, 2025. URL https://commoncrawl.org/

    2025

  34. [42]

    Llama: Open and efficient foundation language models, 2023 a

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

    2023 arXiv

  35. [43]

    Llama 2: Open foundation and fine-tuned chat models, 2023 b

    Hugo Touvron, Louis Martin, Kevin Stone, and et al. Llama 2: Open foundation and fine-tuned chat models, 2023 b . URL https://arxiv.org/abs/2307.09288

    2023 arXiv

  36. [44]

    Effective data augmentation with diffusion models

    Brandon Trabucco, Kyle Doherty, Max A Gurinas, and Ruslan Salakhutdinov. Effective data augmentation with diffusion models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ZWzUA9zeAg

    2024

  37. [45]

    Llms still can't plan; can lrms? a preliminary evaluation of openai's o1 on planbench, 2024

    Karthik Valmeekam, Kaya Stechly, and Subbarao Kambhampati. Llms still can't plan; can lrms? a preliminary evaluation of openai's o1 on planbench, 2024. URL https://arxiv.org/abs/2409.13373

    2024 arXiv

  38. [46]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18 0 (6), March 2024. ISSN...

    2024 doi

  39. [47]

    Large multimodal agents: A survey, 2024

    Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, and Guanbin Li. Large multimodal agents: A survey, 2024. URL https://arxiv.org/abs/2402.15116

    2024 arXiv

  40. [48]

    An illusion of progress? assessing the current state of web agents

    Tianci Xue, Weijian Qi, Tianneng Shi, Chan Hee Song, Boyu Gou, Dawn Song, Huan Sun, and Yu Su. An illusion of progress? assessing the current state of web agents. 2025. URL https://arxiv.org/abs/2504.01382

    2025

  41. [49]

    Webshop: Towards scalable real-world web interaction with grounded language agents, 2023 a

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents, 2023 a . URL https://arxiv.org/abs/2207.01206

    2023 arXiv

  42. [50]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models, 2023 b . URL https://arxiv.org/abs/2305.10601

    2023 arXiv

  43. [51]

    differentiation

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. Textgrad: Automatic "differentiation" via text, 2024. URL https://arxiv.org/abs/2406.07496

    2024 arXiv

  44. [52]

    Agenttuning: Enabling generalized agent abilities for llms, 2023

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttuning: Enabling generalized agent abilities for llms, 2023. URL https://arxiv.org/abs/2310.12823

    2023 arXiv

  45. [53]

    Appagent: Multimodal agents as smartphone users, 2023

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. Appagent: Multimodal agents as smartphone users, 2023. URL https://arxiv.org/abs/2312.13771

    2023 arXiv

  46. [54]

    Generative verifiers: Reward modeling as next-token prediction, 2024

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction, 2024. URL https://arxiv.org/abs/2408.15240

    2024 arXiv

  47. [55]

    Evaluation of openai o1: Opportunities and challenges of agi, 2024

    Tianyang Zhong, Zhengliang Liu, Yi Pan, and et al. Evaluation of openai o1: Opportunities and challenges of agi, 2024. URL https://arxiv.org/abs/2409.18486

    2024

  48. [56]

    Language agent tree search unifies reasoning acting and planning in language models, 2024 a

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning acting and planning in language models, 2024 a . URL https://arxiv.org/abs/2310.04406

    2024 arXiv

  49. [57]

    Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig

    Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents, 2024 b . URL https://arxiv.org/abs/2307.13854

    2024 arXiv

  50. [58]

    Proposer-agent-evaluator(pae): Autonomous skill discovery for foundation model internet agents, 2024 c

    Yifei Zhou, Qianlan Yang, Kaixiang Lin, Min Bai, Xiong Zhou, Yu-Xiong Wang, Sergey Levine, and Erran Li. Proposer-agent-evaluator(pae): Autonomous skill discovery for foundation model internet agents, 2024 c . URL https://arxiv.org/abs/2412.13194

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.