Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Exploring Expert Failures Improves LLM Agent Tuning

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Failed expert trajectories, mined by re-simulating from intermediate states, train LLM agents to solve hard subtasks that rejection-sampling fine-tuning leaves unsolved.

desk verdict The idea is right, but Algorithm 1 doesn't train on any action from failed expert trajectories, so the headline results don't currently support the claimed mechanism. read the letter →

arxiv 2504.13145 v2 pith:6YL7BNQ6 submitted 2025-04-17 cs.AI

Li-Cheng Lan , Andrew Bai , Minhao Cheng , Cho-Jui Hsieh , Tianyi Zhou This is my paper · ORCID
classification cs.AI
keywords LLMagentsagentfine-tuningrejectionsamplingnegativetrajectoriesbeneficialactionminingstatesimulationWebShopScienceWorld
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rejection-sampling fine-tuning (RFT) improves LLM agents by imitating successful expert and self-generated trajectories, but because experts and students both succeed mostly on easy subtasks, hard subtasks stay out-of-distribution. The paper claims those hard subtasks are not actually unsolvable: failed expert trajectories often contain correct plans and key actions, with the failure coming only from late mistakes. The proposed method, EEF, identifies the good parts by restarting the current policy from sampled intermediate states of failed expert trajectories; when the policy succeeds from a state, the expert actions immediately before it are treated as beneficial and added to the training set, while other actions are discarded. On WebShop, EEF reaches a 62% win rate and a 0.81 reward with GPT-3.5 plus GPT-4 data, surpassing RFT and GPT-4, and it is the first reported method to exceed 81 on ScienceWorld. If correct, this turns rejected negative data into a reusable resource instead of waste.

What carries the argument

The load-bearing object is the expert-state simulation protocol used to label beneficial actions. Given a failed expert trajectory $\tau_e=[s_0,a_0,s_1,\ldots]$, EEF picks states at equal skip length $l=\lfloor|\tau_e|/(M+1)\rfloor$, rolls out the current policy from each selected state, and compares outcomes: if the policy succeeds from $s_{i-l}$ but fails from $s_i$, the actions $a_{i-l:i-1}$ are classified as harmful and $s_i$ is marked a recovery state; if the policy succeeds from $s_i$, the preceding expert actions are treated as beneficial prefixes. Important states are initial subtask states plus the first such recovery state in each trajectory, and from the stored positive-trajectory repository EEF selects at most one solution per important state, training the model only on actions after that state with the supervised fine-tuning loss. This simulation signal is what converts negative expert data into positive training segments and what the authors argue prevents contamination from suboptimal actions.

What would settle it

Take a set of failed expert trajectories, split each at sampled states, and scramble which expert action prefixes are paired with which successful rollouts; if scrambled EEF training retains the same win-rate gains on WebShop or ScienceWorld, then the success-from-state signal is not actually selecting expert-specific beneficial actions, and the claimed mechanism would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a failed expert trajectory should not be discarded as a unit: although the whole trajectory has zero reward, it can be partitioned into beneficial action segments and harmful ones, and supervised fine-tuning on only the beneficial segments lets a smaller LLM agent solve subtasks that rejection-sampling fine-tuning cannot. The supporting evidence is empirical: EEF-style training solves previously unsolved WebShop subtasks and lifts win rates from 35.6% for GPT-4 and 53.6% for RFT to 58.4% with GPT-4 data alone and 62.0% with additional GPT-3.5 data, with reward passing 0.81; on ScienceWorld, the score exceeds 81. The paper also claims that because harmful actions are excluded, EEF can learn from weaker, cheaper experts such as GPT-3.5 Turbo without being dragged down by their mistakes.

Load-bearing premise

The load-bearing assumption is that a successful rollout of the current policy from an expert intermediate state certifies the expert actions just before that state as beneficial for the original starting state; that inference would fail if the intermediate state is easy or the policy's own later choices are what produce success.

Editorial extensions

If this is right

  • Hard, persistently out-of-distribution subtasks become learnable, so expert failure trajectories are a resource rather than noise.
  • Training on a few selected beneficial actions instead of whole trajectories should reduce exposure to suboptimal behavior and may improve sample efficiency relative to whole-trajectory imitation.
  • The same setup can absorb demonstrations from cheaper, weaker experts, shifting the data budget toward low-cost exploration.
  • Agents learn navigation and recovery skills—using Next and Back in WebShop—that both RFT and GPT-4 attempt but fail to deploy successfully.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the same success-versus-failure simulation signal could be turned into an automatic curriculum, starting the policy from states it already solves and pushing toward states just beyond its reach; the paper does not develop this framing.
  • My inference: the equal-interval sampling is a coarse credit-assignment heuristic, so a binary-search variant that isolates the exact harmful action boundary should extract more signal per failed trajectory.
  • My inference: if the labeling rule holds for expert failures, it should also hold for failures of the student policy, which would make the method self-improving rather than dependent on a stronger expert.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Exploring Expert Failures (EEF), an extension of Rejection Sampling Fine-Tuning (RFT) for LLM agents. The method is described as identifying beneficial actions inside failed expert trajectories by simulating the current policy from selected expert intermediate states, then training on the expert actions that lead to successful simulations while excluding harmful ones. The authors claim that this enables solving previously unsolvable subtasks and report state-of-the-art results on WebShop and ScienceWorld, including a 62% win rate on WebShop-11k and rewards above 0.81 and 81 on the two benchmarks. The paper includes ablations, case studies, and an efficiency analysis comparing EEF with RFT and other baselines.

Significance. The motivating idea is practically valuable: failed expert trajectories are abundant and cheap, and mining them without inheriting their errors could mitigate the simplicity bias of RFT in sparse-reward agentic environments. The reported results, if reproducible, would be a meaningful advance, and the use of weaker, cheaper experts (GPT-3.5) is an attractive feature. The paper also provides concrete case studies and a navigation-skill analysis that support the intuition that EEF helps acquire non-trivial skills. However, the central mechanism as described in the text and Figure 1 is not implemented in Algorithm 1, the labeling rule that certifies actions as beneficial is unvalidated, and the empirical results lack error bars and a fixed checkpoint-selection protocol. These issues are load-bearing because they directly affect whether the experiments support the paper's main claim. The contribution is therefore promising but not yet established.

major comments (3)
  1. [Section 3.2, Algorithm 1, Figure 1] Algorithm 1, lines 7–10 and 14, does not train on the claimed 'beneficial actions' from failed expert trajectories. Line 9 samples policy rollouts from expert intermediate states s_{m×l}; line 10 adds only positive rollouts to D+. In the subsequent training step, get_traj(s,D+) returns a positive trajectory containing state s, so a rollout starting at s_l contains s_l and the policy's own subsequent actions, not the expert prefix a_{0:l-1}. The SFT loss at line 14 is therefore computed only on policy-generated action tokens from the simulated rollouts. The statement in Section 3.2 that 'the agent only needs to learn the expert actions [a_0,...,a_{l-1}]' and the green arrows in Figure 1 have no corresponding implementation. Similarly, calling τ_{s_l} a 'solution for s0' contradicts the formal definition of a solution as a positive trajectory containing s0, since τ_{s_l} does not contain s0. Because of this mismatch, the experiments cannot support the central claim: the reported gains could come entirely from extra exploration starting at expert states, rather than from learning beneficial actions inside failed expert trajectories. This gap must be resolved either by changing the algorithm so that the selected expert prefix is actually included in the training set, or by reframing the contribution and claims to match what is implemented.
  2. [Section 3.2, important state selection] The rule that a successful rollout from expert state s_l certifies the preceding expert actions a_{0:l-1} as beneficial for the original state s0 is an unproven assumption, and the symmetric failure-based rule for identifying recovery states is likewise unvalidated. Success from s_l could be an artifact of s_l being an easy state or of the policy's own later choices; failure from s_i after success from s_{i-l} does not necessarily attribute the failure to the intervening expert actions. No experiment tests this certificate directly, for example by comparing the selected action segments with random segments or by ablating the selection rule. Since this certificate is supposed to distinguish EEF from merely adding extra rollouts, it needs direct empirical support before the main claim can be accepted.
  3. [Section 4.1 and Table 3] All reported win rates and rewards in Table 3 are single numbers with no error bars, no number of seeds, and no measure of variance. The text says 'We select the best model from all iterations' but does not specify the number of checkpoints, the validation criterion, or how many iterations were considered. The headline differences (e.g., 58.4% vs. 53.6% on WebShop-11k) may be within run-to-run variance or may reflect best-checkpoint selection over many fine-tuned models. The paper should report mean and standard deviation over at least three seeds and fix a single checkpoint-selection rule. In addition, Section 4.1 states 'The default parameters are M = 5, I = 4', while Table 2 lists I = 3 for both EEF rows; this inconsistency must be resolved.
minor comments (5)
  1. [Title and Section 2] The title contains a spacing typo ('T uning') and Section 2 is headed 'Backgroud'; please proofread the manuscript.
  2. [Section 3.1 and Figure 1] The notation τ_{s_l} is used ambiguously: it sometimes denotes a rollout generated from s_l and sometimes a trajectory that includes the expert prefix before s_l. Please define a single consistent notation and use it throughout.
  3. [Algorithm 1, line 12] The function 'need recover states(Di)' is not formally defined. The text describes recovery states as expert states, but the argument Di is the set of policy rollouts; please specify how the function identifies states from Di and how it relates to the expert trajectory states.
  4. [Table 2] For EEF GPT-3&4, Table 2 lists Sim. # = 11, but the text says M=5 for expert states; please explain how the eleven simulations are budgeted across initial states, expert states, and GPT-3.5 trajectories.
  5. [Abstract and Table 3] The claim of being 'the first method to surpass a score of 0.81 in WebShop' should be justified against published state-of-the-art numbers, not only against the baselines in Table 3; also, the abstract's SciWorld claim should state explicitly which EEF variant supports it, since the GPT-3&4 row in Table 3 is left as '–' for ScienceWorld.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EEF's selection heuristic is training-loop feedback and its gains are measured on external test benchmarks, so no prediction reduces to its inputs by construction.

full rationale

The paper's load-bearing claim is that beneficial actions from failed expert trajectories can be identified by simulating the current policy from expert intermediate states and then used to improve fine-tuning. This is a data-selection heuristic, not a derivation that assumes what it predicts. The 'beneficial' label is assigned by an external environment reward on rollouts generated by πθ from expert states, and the reported 62% WebShop win rate and 81 SciWorld score are evaluated on test subtasks, so the central claim is not equivalent to its training inputs. The one self-citation (Lan et al. 2023) is used only to justify the standard environment-reset assumption enabling mid-trajectory simulation; this assumption is also demonstrated by the implemented experiments and is not load-bearing. The skeptical observation that Algorithm 1 (lines 7–14) adds only policy rollouts starting at expert states to D+, while Section 3.2 describes learning expert prefix actions [a0, ..., a_{l-1}], identifies a text–algorithm faithfulness issue rather than a circularity: no fitted parameter is renamed as a prediction and no equation reduces to its own input. I therefore find no circular step requiring a nonzero score.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central method adds no new entities or fitted constants that function like physical parameters. The free parameters are standard training hyperparameters plus the M and I choices that shape the rollout labeling. The main additional postulate is the labeling rule connecting a successful rollout from an intermediate state to the quality of preceding expert actions. The paper also assumes the environment can be reset to arbitrary states, which is true for WebShop and ScienceWorld but is an extra capability not present in every agent environment.

free parameters (3)
  • M (number of expert states simulated per expert trajectory) = 5 (default; ablation uses 1, 2, 5)
    Hand-chosen hyperparameter that controls the granularity l = floor(|tau_e|/(M+1)) of the interval sampling from failed expert trajectories; the beneficial-action labels depend directly on which states are sampled.
  • I (number of fine-tuning iterations) = 4 (default; results reported for iteration 3)
    Hand-chosen stopping point; the authors select the best model across iterations, which is a form of model selection that can bias reported numbers if done on the test set.
  • Training setup (learning rate, epochs, batch size) = 5e-5, 6, 64
    Standard SFT hyperparameters stated in Section 4.1; not fitted to the central claim but required for reproduction.
assumptions (3)
  • domain assumption The environment is an MDP and permits setting the environment state to arbitrary intermediate expert states.
    Stated directly in Section 2, and used by Algorithm 1 lines 7-9 for all expert-state simulations. Without resets, EEF cannot run.
  • ad hoc to paper A successful simulation from expert state sl makes the expert prefix leading to sl beneficial for the original state s0.
    The central labeling heuristic in Figure 1 and Section 3.2; never proven, and the paper's own algorithm listing does not cleanly implement it.
  • domain assumption The sparse final reward R(tau) is the only training signal and correctly identifies good behavior.
    Used to partition D+ and to define need-recover states; standard for these benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Expert Failures Improves LLM Agent Tuning." pith.science (2026). https://pith.science/paper/6YL7BNQ6

@misc{pith2026250413145,
  author       = {Pith},
  title        = {Pith review of: Exploring Expert Failures Improves LLM Agent Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6YL7BNQ6}},
  note         = {Machine review of arXiv:2504.13145}
}
read the original abstract

Large Language Models (LLMs) have shown tremendous potential as agents, excelling at tasks that require multiple rounds of reasoning and interactions. Rejection Sampling Fine-Tuning (RFT) has emerged as an effective method for finetuning LLMs as agents: it first imitates expert-generated successful trajectories and further improves agentic skills through iterative fine-tuning on successful, self-generated trajectories. However, since the expert (e.g., GPT-4) succeeds primarily on simpler subtasks and RFT inherently favors simpler scenarios, many complex subtasks remain unsolved and persistently out-of-distribution (OOD). Upon investigating these challenging subtasks, we discovered that previously failed expert trajectories can often provide valuable guidance, e.g., plans and key actions, that can significantly improve agent exploration efficiency and acquisition of critical skills. Motivated by these observations, we propose Exploring Expert Failures (EEF), which identifies beneficial actions from failed expert trajectories and integrates them into the training dataset. Potentially harmful actions are meticulously excluded to prevent contamination of the model learning process. By leveraging the beneficial actions in expert failures, EEF successfully solves some previously unsolvable subtasks and improves agent tuning performance. Remarkably, our approach achieved a 62\% win rate in WebShop, outperforming RFT (53. 6\%) and GPT-4 (35. 6\%), and to the best of our knowledge, setting a new state-of-the-art as the first method to surpass a score of 0.81 in WebShop and exceed 81 in SciWorld.

Figures

Figures reproduced from arXiv: 2504.13145 by the authors.

Figure 1
Figure 1. τe = [s0,s1, . . . ] is a failed expert (GPT-4) trajectory for a challenging sub￾task s0 with R(τe) = 0. Despite its overall failure, τe contains partial insights to train agent πθ . To identify beneficial actions in τe , our method (EEF) performs simula￾tions from expert states [s0,sl , . . . ] at inter￾vals of length l, resulting in [τs0 , τsl , . . . ], revealing that certain expert actions a0:l−1 and a2l:3l−1 en… view at source ↗
Figure 2
Figure 2. The percentage of success￾ful subtasks that involve the use of navigation skills: Next and Back. Navigation skills In this experiment, we evaluate the mitigation of simplicity bias by measuring the effectiveness of navigation skills on the Webshop 11k dataset. Specifically, we focus on two types of naviga￾tion actions: “Next page” (Next) and “Back to search” (Back), as shown in [PITH_FULL_IMAGE:figures/full_fig_p00… view at source ↗
Figure 3
Figure 3. Win rates of different meth￾ods training the same base model for one iteration under varying sim￾ulation budgets (exploration cost) by adjusting N or M. RFT × N: Initial state s0 simulated N times. EEF: Dif￾ferent expert state simulation num￾bers M = 1, 2, 5 on different expert datasets. Case studies To further substantiate the necessity of navigation skills for challenging subtasks, we ex￾amine specific subtasks wh… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    FailForge converts failed SWE-bench trajectories into training signal via distilled procedural skills, lifting Qwen3.5-4B pass@1 by 6.6 points over RFT.

  2. ProgRM: Build Better GUI Agents with Progress Rewards

    cs.AI 2025-05 conditional novelty 6.0 of 10

    ProgRM, a per-step progress reward model trained with LCS-based self-annotated labels, improves RL-trained GUI agent success rates on WikiHow relative to outcome reward models.

Reference graph

Works this paper leans on

25 extracted references · 2 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [4]

    Agent-flan: Designing data and methods of effective agent tuning for large language models

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-flan: Designing data and methods of effective agent tuning for large language models. arXiv preprint arXiv:2403.12881,

    arXiv

  3. [5]

    Atlas: Agent tuning via learning critical steps

    Zhixun Chen, Ming Li, Yuxuan Huang, Yali Du, Meng Fang, and Tianyi Zhou. Atlas: Agent tuning via learning critical steps. arXiv preprint arXiv:2503.02197,

    arXiv

  4. [7]

    Contextual markov decision pro- cesses

    Assaf Hallak, Dotan Di Castro, and Shie Mannor. Contextual markov decision pro- cesses. ArXiv, abs/1502.02259,

    arXiv

  5. [9]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International conference on machine learning , pp. 9118–9147. PMLR, 2022a. 11 Preprint. Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor M...

    arXiv

  6. [10]

    Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectories

    Li-Cheng Lan, Huan Zhang, and Cho-Jui Hsieh. Can agents run relay race with strangers? generalization of rl to out-of-distribution trajectories. ArXiv, abs/2304.13424,

    work page Pith review arXiv

  7. [11]

    Zhiwei Liu, Weiran Yao, Jianguo Zhang, Liangwei Yang, Zuxin Liu, Juntao Tan, Prafulla K Choubey, Tian Lan, Jason Wu, Huan Wang, et al

    URL https://api.semanticscholar.org/CorpusID:258331519. Zhiwei Liu, Weiran Yao, Jianguo Zhang, Liangwei Yang, Zuxin Liu, Juntao Tan, Prafulla K Choubey, Tian Lan, Jason Wu, Huan Wang, et al. Agentlite: A lightweight library for building and advancing task-oriented llm agent system. arXiv preprint arXiv:2402.15538,

    arXiv

  8. [12]

    Agentboard: An analytical evaluation board of multi-turn llm agents

    Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn llm agents. ArXiv, abs/2401.13178,

    arXiv

Show all 25 references
  1. [13]

    org/CorpusID:267199917

    URL https://api.semanticscholar. org/CorpusID:267199917. Qianli Ma, Haotian Zhou, Tingkai Liu, Jianbo Yuan, Pengfei Liu, Yang You, and Hongxia Yang. Let’s reward step by step: Step-level reward model as the navigators for reasoning. arXiv preprint arXiv:2310.10080,

    arXiv

  2. [14]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. ArXiv, abs/2305.18290,

    arXiv

  3. [15]

    Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Peter J

    Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzed...

    2024

  4. [16]

    Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, and Sujian Li

    URL https://api.semanticscholar.org/CorpusID:266163375. Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Agentbank: Towards generalized llm agents via fine-tuning on 50000+ interaction trajectories. arXiv preprint arXiv:...

    arXiv

  5. [17]

    Hugo Touvron, Louis Martin, Kevin R

    12 Preprint. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cant´on Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude ...

    arXiv

  6. [18]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar

    URL https://api.semanticscholar.org/CorpusID: 259950998. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023a. Han...

    arXiv

  7. [21]

    Watch every step! llm agent learning via iterative step-level process refinement

    Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Watch every step! llm agent learning via iterative step-level process refinement. arXiv preprint arXiv:2406.11176,

    arXiv

  8. [22]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825,

    arXiv

  9. [23]

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang

    URL https://api.semanticscholar.org/CorpusID:247762790. Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttuning: Enabling generalized agent abilities for llms. arXiv preprint arXiv:2310.12823,

    arXiv

  10. [24]

    Agentohana: Design unified data and training pipeline for effective agent learning

    Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Ming Zhu, Juntao Tan, Thai Hoang, Zuxin Liu, Liangwei Yang, et al. Agentohana: Design unified data and training pipeline for effective agent learning. arXiv preprint arXiv:2402.15506,

    arXiv

  11. [25]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854,

    arXiv

  12. [2006]

    From novice to expert: Llm agent policy optimization via step-wise reinforcement learning

    Zhirui Deng, Zhicheng Dou, Yutao Zhu, Ji-Rong Wen, Ruibin Xiong, Mang Wang, and Weipeng Chen. From novice to expert: Llm agent policy optimization via step-wise reinforcement learning. arXiv preprint arXiv:2411.03817,

    arXiv

  13. [2015]

    Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, and Roberta Raileanu

    URL https://api.semanticscholar.org/CorpusID: 14616648. Alex Havrilla, Sharath Raparthy, Christoforus Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, and Roberta Raileanu. Glore: When, where, and how to improve llm reasoning via global and local refinements. arX...

    arXiv

  14. [2022]

    Zihan Wang, Yunxuan Li, Yuexin Wu, Liangchen Luo, Le Hou, Hongkun Yu, and Jingbo Shang

    URL https://api.semanticscholar.org/CorpusID: 247451124. Zihan Wang, Yunxuan Li, Yuexin Wu, Liangchen Luo, Le Hou, Hongkun Yu, and Jingbo Shang. Multi-step problem solving through a verifier: An empirical analysis on model- induced process supervision. arXiv preprint arXiv:240...

    arXiv

  15. [2023]

    Fireact: Toward language agent fine-tuning

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:2310.05915,

    arXiv

  16. [2024]

    Renat Aksitov, Sobhan Miryoosefi, Zonglin Li, Daliang Li, Sheila Babayan, Kavya Koppa- rapu, Zachary Fisher, Ruiqi Guo, Sushant Prakash, Pranesh Srinivasan, et al

    URL https://github.com/meta-llama/llama3/blob/ main/MODEL CARD.md. Renat Aksitov, Sobhan Miryoosefi, Zonglin Li, Daliang Li, Sheila Babayan, Kavya Koppa- rapu, Zachary Fisher, Ruiqi Guo, Sushant Prakash, Pranesh Srinivasan, et al. Rest meets re- act: Self-improvement for multi...

    arXiv

  17. [2025]

    Math-shepherd: A label-free step-by-step verifier for llms in mathematical reasoning

    Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Y Wu, and Zhifang Sui. Math-shepherd: A label-free step-by-step verifier for llms in mathematical reasoning. arXiv preprint arXiv:2312.08935, 2023b. Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Ti...

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.