Pith. sign in

REVIEW 4 major objections 5 minor 9 cited by

AgentRefine: Enhancing Agent Generalization through Refinement Tuning

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training LLM agents on synthetic trajectories that include deliberate mistakes and subsequent refinements makes them generalize to new agent tasks better than prior agent-tuning methods.

desk verdict Plausible recipe, solid held-out gains, but the causal claim that refinement drives the gains is confounded by a regenerated comparison corpus. read the letter →

arxiv 2501.01702 v2 pith:L5HOVRFZ submitted 2025-01-03 cs.AI cs.CLcs.RO

Dayuan Fu , Keqing He , Yejie Wang , Wentao Hong , Zhuoma Gongque , Weihao Zeng , Wei Wang , Jingang Wang
show 2 more authors
Xunliang Cai Weiran Xu
This is my paper · ORCID
classification cs.AIcs.CLcs.RO
keywords agentgeneralizationrefinementtuningsyntheticdatainstructionLLMagentsself-refinementReActBoard
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that open-weights LLM agents fail to generalize across agent environments because standard agent-tuning data only teaches them to memorize observation-action associations; once the environment changes, they repeat errors and never recover. AgentRefine is proposed as a fix: generate a large, persona-driven variety of synthetic environments and tasks, have a strong LLM act out trajectories that contain deliberate mistakes followed by refinements, and fine-tune the agent on those trajectories while masking the loss on the erroneous turns. The resulting model, trained only on synthetic data, improves held-out success and progress rates on five decision-making benchmarks compared to prior agent-tuning methods and stays stable when the action vocabulary of a training environment is perturbed. The paper claims this shows a causal connection between self-refinement during training and generalization at inference.

What carries the argument

The core mechanism is the constructed refinement trajectory plus a modified training loss. Data is built in three stages: an LLM generates an environment script (locations, items, goal, available actions with regex validation) from a sampled persona; the same LLM plays both Dungeon Master and player to produce multi-turn ReAct-style trajectories in which the player sometimes acts wrongly and the DM returns feedback; and a verifier checks the script and trajectory, requiring at least two error-refine pairs before a successful finish. Training then uses Equation 1, which computes the log-likelihood of each turn's thought and action conditioned on prior turns but multiplies by an indicator $\mathbb{1}(A_j)$ that is 1 only when the action is correct, so erroneous tokens are never learned. This is what makes refinement tuning distinct from ordinary supervised fine-tuning on agent trajectories.

What would settle it

Take the AgentRefine synthetic corpus and flip or corrupt a fraction of the error labels, for example marking 20% of correct turns as errors and vice versa, while keeping trajectory diversity identical; if held-out success does not fall as corruption increases, then refinement labels are not what drives the gains, and the argument would instead point to trajectory diversity or formatting.

Watch

Extended reading notes

Core claim

AgentRefine's central claim is that refinement tuning, not just more or more diverse agent data, is what carries generalization. On trajectories synthesized by GPT-4o, each player turn is followed by a Dungeon Master observation, and a verifier keeps only trajectories containing at least two error-refinement cycles before a successful finish. Training on these trajectories with the erroneous turns' tokens masked out yields models (LLaMA-3-8B/70B, Mistral-7B) that outperform Agent-FLAN, AgentGym, and AgentGen on held-out AgentBoard tasks, for example leading Agent-FLAN by 13.3% success on SciWorld, and that also transfer to the HotpotQA reasoning task. Ablations show that removing or masking the refinement loss cuts held-out performance sharply, while including loss on erroneous tokens can cost roughly 75% of SciWorld success. The paper positions this as establishing a link between step-level self-refinement and agent generalization.

Load-bearing premise

The training signal assumes that GPT-4o's simulated environment feedback and error labels are faithful proxies for real environment feedback, so that a model learning to refine on synthetic feedback will also refine well in real environments; this fidelity is checked on only 50 judged trajectories in the paper.

Editorial extensions

If this is right

  • Trained on synthetic data with no overlap with the five test environments, AgentRefine improves held-out success and progress rates on SciWorld, BabyAI, PDDL, and Jericho, and on Alfworld when no training data came from it.
  • Because refinement trajectories are the key ingredient, other agent-tuning pipelines could be upgraded by inserting error-and-refine cycles into their data and masking erroneous turns, without changing the model architecture.
  • Models become resistant to small environment perturbations: while Agent-FLAN and AgentGym drop sharply in success when action wording is reordered, AgentRefine's average success rate stays flat or slightly increases.
  • The same recipe transfers to a reasoning task such as HotpotQA, suggesting refinement tuning teaches general recovery behavior rather than task-specific exploration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If refinement is the true driver, then data diversity alone is not enough; this suggests a testable recipe of deliberately injecting controlled failures into any synthetic or collected agent corpus.
  • The loss masking turns each error-refine cycle into something close to implicit preference data; one could extract pairs of erroneous versus corrected turns and train with direct preference optimization or reinforcement learning, which the paper does not do.
  • The fidelity of GPT-4o's simulated feedback is the critical transfer assumption; an extension would corrupt a fraction of error labels during synthesis and measure how held-out performance degrades, separating refinement learning from mere trajectory diversity.
  • Because gains appear on text-based simulation benchmarks, testing on web navigation or tool use would show whether the refinement habit transfers across environment types.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AgentRefine, a framework for instruction-tuning LLM agents on synthetic trajectories that deliberately include error-and-refinement turns, with a loss function that masks erroneous turns. The authors claim that this refinement tuning significantly improves generalization to held-out agent tasks compared to prior agent-tuning methods such as Agent-FLAN and AgentGym, and that it also improves robustness to environment perturbations and performance on a reasoning task (HotpotQA). The method is evaluated on five AgentBoard environments and on HotpotQA using LLaMA-3 and Mistral models.

Significance. The paper addresses an important problem: open-source LLM agents trained by instruction tuning on existing agent corpora tend to overfit to training environments and fail to generalize. If the central claim is correct, adding synthetic error-and-refinement trajectories is a simple and potentially widely applicable recipe for improving held-out agent performance, and the connection between self-refinement and generalization would be a useful conceptual contribution. The paper includes several analyses (diversity, perturbation robustness, Best-of-N, open-source synthesis) that go beyond a single benchmark comparison. However, the strength of the causal claims about refinement is not fully supported by the experiments as designed.

major comments (4)
  1. [§4.2, Table 2] The 'w/o refinement data' ablation is confounded. The text states that this condition uses a re-generated training set without error and refinement trajectories; because the data are regenerated, the condition differs from AgentRefine along many dimensions other than the presence of refinement, including environment coverage, trajectory length, task difficulty, and thought diversity. The paper itself credits such diversity for generalization (§4.4), so the observed performance drop cannot be uniquely attributed to refinement tuning. A matched-pair ablation that removes only the refinement turns from the same trajectories (or otherwise controls for trajectory-level attributes) is needed to support the paper's central causal claim. The 'w/o refinement loss' condition is more controlled, but it only tests the effect of loss masking, not the effect of the refinement content on learned behavior.
  2. [Appendix D (and §4.1, Table 1)] The reported results are obtained by selecting the checkpoint with the best average results on the evaluation benchmarks after training for 10 epochs. This is a form of test-set model selection and can inflate reported performance and mask variance. For a paper whose central claim is that AgentRefine 'significantly outperforms' prior methods, the selection procedure should be reported transparently, and the main results should either be averaged over checkpoints or selected using a validation set that is disjoint from the reported test tasks. Otherwise, the improvements over baselines may be partly due to cherry-picking a favorable epoch.
  3. [§4.3 and Appendix I, Table 11] The robustness claim is undercut by the 'w/o refinement data' condition. Table 11 shows that this condition achieves essentially the same average success rate (48.78 versus 48.48) and a comparable standard deviation (5.47 versus 5.78) under the same perturbations as AgentRefine. This contradicts the attribution of perturbation robustness to refinement tuning. The paper should either temper the robustness claim or provide an explanation of why the 'no refinement' model retains robustness; the data diversity explanation is a plausible alternative that is not ruled out by the current experiments.
  4. [§8, Figure 10] The reliability check of the synthetic data uses only 50 trajectories and covers only whether GPT-4's turn-level error labels agree with human annotations. It does not validate whether the refinement steps (the corrected actions and their surrounding thoughts) faithfully resemble behaviors that would recover from real environment feedback in the target benchmarks. Since the transfer of the learned refinement behavior to AgentBoard and HotpotQA rests on the fidelity of the synthetic feedback, a larger and more direct fidelity check is necessary. At minimum, the paper should acknowledge this limitation explicitly in the main text rather than only presenting the 94%/82% agreement numbers.
minor comments (5)
  1. [§3.3, Eq. (1)] The loss equation is not written correctly: the indicator 1(Aj) appears inside the logarithm, which would produce log(0) for erroneous turns. The intended operation is clearly to multiply the log-likelihood by the indicator (i.e., 1(Aj) * log(...)), and the equation should be corrected to avoid confusion about the masking scheme.
  2. [§3.2] The cross-reference 'In Appendix 5, we use deepseek-v2.5' is incorrect; Section 5 is in the main text, not an appendix. The reference should be fixed.
  3. [§4.4] The text says 'As shown in Figure 3' when referring to the environment similarity heatmap, but the heatmap is in Figure 7. The figure reference is wrong.
  4. [§4.1] The paper says it changes AgentBoard's prompts from Act-only to ReAct. This is a modification of the evaluation setup, and although it is applied consistently across all methods, the deviation from the standard AgentBoard protocol should be described more precisely (e.g., exact prompt format, which example prompts were adjusted) so that readers can replicate or compare with other published results.
  5. [Appendix K, Table 3] Perturbation 5 is described as removing all in-distribution training data and retraining the model, which is not a perturbation of the test environment but a change to the trained model. Reporting this as a perturbation alongside four genuine environment perturbations can be misleading; it should be presented as a separate analysis of training-data dependence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main generalization claim is tested on external benchmarks, and no load-bearing step reduces to its inputs by construction.

full rationale

The paper's central claim is that refinement tuning on synthetic error-and-refine trajectories improves held-out agent generalization. The training data are synthesized from personas with GPT-4o, and the evaluation is performed on external AgentBoard tasks (SciWorld, Alfworld, BabyAI, PDDL, Jericho) and HotpotQA, which are not used in training; no parameter is fitted to those test sets. The refinement loss in Equation 1 masks erroneous turns using verifier labels, and Section 8 validates those GPT-4 error labels against human judgments on 50 sampled trajectories, so the labels are not assumed by construction. The paper does not invoke a self-citation chain or an authors' uniqueness theorem to force its choice, and the cited prior work on self-refinement and persona diversity is used as inspiration, not as a load-bearing justification for the empirical result. The 'w/o refinement data' ablation compares against a separately regenerated corpus, which is a potential confound for the causal attribution to refinement content, but that is an experimental identification limitation rather than a definitional circularity: the comparison is not equivalent to the target claim by construction. Overall, the derivation is self-contained against external benchmarks and no circular step was found.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on domain assumptions about the fidelity and diversity of GPT-4o-generated synthetic environments and feedback, plus standard SFT machinery. No new physical or formal entities are introduced. The two free parameters are data-scale and trajectory-verification choices, not fitted to the evaluation benchmarks.

free parameters (2)
  • training data size = 32,000 for main results; scaling experiments use 4k, 8k, 16k, 32k, 64k
    The main results use 32,000 training trajectories, and performance scales with data size (Figure 5), so the reported gains depend on this chosen quantity.
  • minimum error-refine turns per trajectory = 2
    The trajectory verifier rejects trajectories with fewer than two error-refine turns (Algorithm 1, step 33), a design choice that shapes the training distribution.
assumptions (4)
  • domain assumption GPT-4o's simulated environment feedback and error labels are a faithful proxy for real environment feedback in the test environments.
    The entire training signal is synthesized by GPT-4o; if the simulated feedback is unrealistic, refinement tuning may not transfer. Invoked in Sections 3.1 and 3.2.
  • domain assumption The rule-based and GPT-4o verifier correctly identifies error and refinement turns.
    Loss masking and data filtering rely on verifier accuracy; only 50 trajectories were human-checked for GPT-4o judgment reliability (Section 8).
  • domain assumption Persona diversity from Persona-Hub generates sufficiently diverse agent environments for generalization.
    The method samples personas and asks GPT-4o to create environments; if the environments cluster narrowly, generalization gains would be limited. Section 3.1.
  • standard math Standard supervised fine-tuning with cross-entropy loss is an appropriate training objective for refinement behavior.
    Uses standard language-model loss with masking; no new mathematical framework is introduced.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AgentRefine: Enhancing Agent Generalization through Refinement Tuning." pith.science (2026). https://pith.science/paper/L5HOVRFZ

@misc{pith2026250101702,
  author       = {Pith},
  title        = {Pith review of: AgentRefine: Enhancing Agent Generalization through Refinement Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L5HOVRFZ}},
  note         = {Machine review of arXiv:2501.01702}
}
read the original abstract

Large Language Model (LLM) based agents have proved their ability to perform complex tasks like humans. However, there is still a large gap between open-sourced LLMs and commercial models like the GPT series. In this paper, we focus on improving the agent generalization capabilities of LLMs via instruction tuning. We first observe that the existing agent training corpus exhibits satisfactory results on held-in evaluation sets but fails to generalize to held-out sets. These agent-tuning works face severe formatting errors and are frequently stuck in the same mistake for a long while. We analyze that the poor generalization ability comes from overfitting to several manual agent environments and a lack of adaptation to new situations. They struggle with the wrong action steps and can not learn from the experience but just memorize existing observation-action relations. Inspired by the insight, we propose a novel AgentRefine framework for agent-tuning. The core idea is to enable the model to learn to correct its mistakes via observation in the trajectory. Specifically, we propose an agent synthesis framework to encompass a diverse array of environments and tasks and prompt a strong LLM to refine its error action according to the environment feedback. AgentRefine significantly outperforms state-of-the-art agent-tuning work in terms of generalization ability on diverse agent tasks. It also has better robustness facing perturbation and can generate diversified thought in inference. Our findings establish the correlation between agent generalization and self-refinement and provide a new paradigm for future research.

Figures

Figures reproduced from arXiv: 2501.01702 by the authors.

Figure 1
Figure 1. Overall progress score among 5 tasks. Agent-FLAN has been trained on Held-in task. Language agents (Mialon et al., 2023; Sumers et al., 2023), which harness the powerful capabilities of large language models (LLMs) to perceive environments, make decisions, and take actions, have emerged as an effective solution to com￾plex real-world problems. Plenty of agent projects such as Au￾toGPT (Sig), GPT-Engineer (gpt), and … view at source ↗
Figure 2
Figure 2. Example of parameter memorization in Agent-FLAN. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The success rate varia￾tion via perturbation error action, the entire process will be stuck in the same error mode for a while, regardless of the observation (§7). These ex￾perimental results indicate that traditional approaches merely memorize the correct trajectory information, fundamentally leading to a lack of generalization capability. Not memorize but self-refine. Inspired by recent work (Shinn et al., 2024; M… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: The script generation requires the LLM to generate a script with the environment, tasks, [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 4
Figure 4. Figure 4: The pipeline of AgentRefine data generation and refinement tuning. observing, and evaluating. In the thinking stage, we require the LLM to evaluate the player’s state and known information so far and analyze the observations the player can obtain based on the last acti…
Figure 5
Figure 5. Figure 5: The model’s performance as the AgentRefine train data scales up. In our proposed Refinement Tuning, we mask the loss of erroneous turn tokens to prevent the model from learning incorrect thought processes. To verify whether this process is necessary, we train a model l…
Figure 6
Figure 6. Figure 6: The t-SNE figure among Agent￾FLAN, AgentGym, and AgentRefine’s Thought. Thought Diversity [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: The similarity heatmap between different environments in 6 sources. and 10 from each test set) and removed the one-shot examples from the test sets. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Model Performance on reasoning task, Hotpot QA [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Comparison case study on Jericho and SciWorld between Agent-FLAN and AgentRefine. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: The comparison of GPT-4’s judgement and human’s judgement. The right column/line means human/GPT-4 con￾siders this turn doesn’t need to be refined. The wrong column/line means human/GPT￾4 considers this turn needs to be refined [PITH_FULL_IMAGE:figures/full_fig_p009_…
Figure 11
Figure 11. Figure 11: The success rate by in￾corporating ShareGPT Both Agent-FLAN and AgentTuning have found that incor￾porating general data can enhance the model’s generalization ability. This improvement arises from the improvement of instruction-following capability [PITH_FULL_IMAGE:f…
Figure 12
Figure 12. Figure 12: Model Performance on Different Tasks We use MT-bench (Zheng et al., 2023) to test mod￾els’ instruction-following ability and use gpt-4o￾2024-05-13 to judge the score. The score of AgentRefine is approximately 0.2 points higher than that of Agent-FLAN regardless of whe…
Figure 13
Figure 13. Figure 13: The statistics of Continuous Error Turns in AgentRefine [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories

    cs.AI 2026-07 accept novelty 6.0 of 10

    Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.

  2. RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents

    cs.LG 2025-07 conditional novelty 6.0 of 10

    RLVMR adds rule-based rewards for planning, exploration, reflection, and monitoring tags to outcome-based reinforcement learning, and reports state-of-the-art success rates and fewer redundant actions on ALFWorld and ...

  3. PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization

    cs.AI 2025-06 conditional novelty 6.0 of 10

    Pseudocode-structured plans paired with preference optimization improve LLM agent success rates and generalization across interactive benchmarks.

  4. Agent-Environment Alignment via Automated Interface Generation

    cs.AI 2025-05 conditional novelty 6.0 of 10

    ALIGN automatically generates and verifies an interface wrapper that enriches environment observations, improving LLM agent success rates by up to 45.67 percentage points on ALFWorld without modifying the agent or env...

  5. Beyond Action Imitation: Learning a Decision-Aware User Simulator for Online Advertising

    cs.IR 2026-07 conditional novelty 5.5 of 10

    DASH folds cross-domain user histories, distills teacher thinking traces, and RL-tunes a small LLM with action plus rubric rewards to jointly predict ad actions and decision traces on Tencent data.

  6. Source Component Shift Adaptation via Offline Decomposition and Online Mixing Approach

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    The claimed result is that source-component-shift adaptation splits cleanly into offline component learning via EM and online mixing-weight updates, cutting cumulative test loss by up to 67.4%.

  7. Leveraging OS-Level Primitives for Robotic Action Management

    cs.OS 2025-08 conditional novelty 4.0 of 10

    Applying OS-style exception handling, context caching, and replay to robotic action slices raises success rates 7x to 24x and cuts execution steps up to 74% for repetitive manipulation tasks, without retraining the VLA model.

  8. Large Language Models for Planning: A Comprehensive and Systematic Survey

    cs.AI 2025-05 conditional novelty 3.0 of 10

    A structured survey of LLM planning methods, benchmarks, and interpretability work, organized around a three-way taxonomy.

  9. Knowledge Augmented Complex Problem Solving with Large Language Models: A Survey

    cs.LG 2025-05 unverdicted novelty 2.0 of 10

    A survey that organizes LLM-based complex problem solving into three components, multi-step reasoning, domain knowledge, and result verification, and reviews methods and challenges across four application domains.

Reference graph

Works this paper leans on

32 extracted references · 9 canonical work pages · cited by 9 Pith papers

  1. [1]

    our mission is to provide the tools, so that you can focus on what matters

    Significant-gravitas/autogpt: Autogpt is the vision of accessible ai for everyone, to use and to build on. our mission is to provide the tools, so that you can focus on what matters. https:// github.com/Significant-Gravitas/AutoGPT. (Accessed on 09/29/2024). gpt-engineer-org/gpt-engineer: Platform to experiment with the ai software engineer. termi- nal ba...

    arXiv 2024

  2. [3]

    Babyai: A platform to study the sample efficiency of grounded language learning

    Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning. arXiv preprint arXiv:1810.08272,

    arXiv

  3. [5]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    arXiv

  4. [6]

    Preact: Predicting future in react enhances agent’s planning ability

    Dayuan Fu, Jianzhao Huang, Siyuan Lu, Guanting Dong, Yejie Wang, Keqing He, and Weiran Xu. Preact: Predicting future in react enhances agent’s planning ability. arXiv preprint arXiv:2402.11534,

    arXiv

  5. [8]

    Large language models cannot self-correct reasoning yet

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798,

    arXiv

  6. [9]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825,

    arXiv

  7. [10]

    Deepseek-v2: A strong, economical, and efficient mixture- of-experts language model

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture- of-experts language model. arXiv preprint arXiv:2405.04434,

    arXiv

  8. [11]

    Agentboard: An analytical evaluation board of multi-turn llm agents

    Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Ling- peng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn llm agents. arXiv preprint arXiv:2401.13178,

    arXiv

Show all 32 references
  1. [12]

    Self-refine: Iterative refinement with self-feedback

    11 Published as a conference paper at ICLR 2025 Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Informa...

    2025

  2. [13]

    Augmented language models: a survey

    Gr´egoire Mialon, Roberto Dess `ı, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozi `ere, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. Augmented language models: a survey. Trans. Mach. ...

    2023

  3. [14]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al

    URL https://arxiv.org/abs/2405.14205. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789,

    arXiv

  4. [15]

    Alfworld: Aligning text and embodied environments for interactive learning

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre C ˆot´e, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768,

    2010 arXiv

  5. [16]

    Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L

    Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive architectures for language agents. Trans. Mach. Learn. Res., 2024,

    2024

  6. [17]

    Mauro Vallati, Lukas Chrpa, Marek Grze ´s, Thomas Leo McCluskey, Mark Roberts, Scott Sanner, et al

    URL https: //api.semanticscholar.org/CorpusID:261556862. Mauro Vallati, Lukas Chrpa, Marek Grze ´s, Thomas Leo McCluskey, Mark Roberts, Scott Sanner, et al. The 2014 international planning competition: Progress and trends. Ai Magazine, 36(3): 90–98,

    2014

  7. [20]

    Watch every step! llm agent learning via iterative step-level process refinement

    Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. Watch every step! llm agent learning via iterative step-level process refinement. arXiv preprint arXiv:2406.11176,

    arXiv

  8. [21]

    Wizardlm: Empowering large language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244,

    arXiv

  9. [22]

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600,

    arXiv

  10. [24]

    org/abs/2408.16293

    URL https://arxiv. org/abs/2408.16293. Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. Agent lumos: Unified and modular training for open-source language agents. In Proceedings of the 62nd Annual Meeting of the Asso...

    arXiv

  11. [25]

    Agenttun- ing: Enabling generalized agent abilities for llms

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. Agenttun- ing: Enabling generalized agent abilities for llms. arXiv preprint arXiv:2310.12823,

    arXiv

  12. [26]

    Agentohana: Design unified data and training pipeline for effective agent learning

    Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Juntao Tan, Thai Hoang, Liang- wei Yang, Yihao Feng, Zuxin Liu, et al. Agentohana: Design unified data and training pipeline for effective agent learning. arXiv preprint arXiv:2402.15506,

    arXiv

  13. [27]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372,

    arXiv

  14. [28]

    Lan- guage agent tree search unifies reasoning acting and planning in language models

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Lan- guage agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406,

    arXiv

  15. [29]

    Research on active semantic discovery technology based on SG-CIM and its application in power grid equipment supply chain optimization

    ACKNOWLEDGMENT This work was partially supported by the State Key Laboratory of Massive Personalized Customiza- tion System and Technology (No. H&C-MPC-2023-02-07(Q)), State Grid Technology Project (5700-202416236A-1-1-ZN) “Research on active semantic discovery technology base...

    2023

  16. [30]

    Aligned with Agent-FLAN, we choose AgentRefine with 32000 data for the default training setting

    usage. Aligned with Agent-FLAN, we choose AgentRefine with 32000 data for the default training setting. Aligned with AgentGen (Hu et al., 2024), we train our model for 10 epochs and select the checkpoint with the best average results to report. We also modified the LLaMA-Facto...

    2024

  17. [31]

    Other settings are aligned with LLaMA-Factory’s default settings. 14 Published as a conference paper at ICLR 2025 E C OMPARISON AMONG AGENT DATASETS Table 7 compares the number of trajectories, the methods to obtain environments and trajectories, the held-in tasks in the Agent...

    2025

  18. [32]

    Thought" : (string, compulsory)

    to test mod- els’ instruction-following ability and use gpt-4o- 2024-05-13 to judge the score. The score of AgentRefine is approximately 0.2 points higher than that of Agent-FLAN regardless of whether ShareGPT is incorporated. After incor- porating ShareGPT, both show an impro...

    2024

  19. [2008]

    Scienceworld: Is your agent smarter than a 5th grader? arXiv preprint arXiv:2203.07540,

    Ruoyao Wang, Peter Jansen, Marc-Alexandre Cˆot´e, and Prithviraj Ammanabrolu. Scienceworld: Is your agent smarter than a 5th grader? arXiv preprint arXiv:2203.07540,

    arXiv

  20. [2018]

    React: Synergizing reasoning and acting in language models

    12 Published as a conference paper at ICLR 2025 Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629,

    2025 arXiv

  21. [2020]

    Agentgen: Enhancing planning abilities for large language model based agent via environment and task generation

    Mengkang Hu, Pu Zhao, Can Xu, Qingfeng Sun, Jianguang Lou, Qingwei Lin, Ping Luo, Saravan Rajmohan, and Dongmei Zhang. Agentgen: Enhancing planning abilities for large language model based agent via environment and task generation. arXiv preprint arXiv:2408.00764,

    arXiv

  22. [2022]

    Agentgym: Evolving large language model-based agents across diverse environments

    Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He, et al. Agentgym: Evolving large language model-based agents across diverse environments. arXiv preprint arXiv:2406.04151,

    arXiv

  23. [2023]

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su

    URL https: //lmsys.org/blog/2023-03-30-vicuna/ . Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36,

    2023

  24. [2024]

    Agent-flan: Designing data and methods of effective agent tuning for large language models

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-flan: Designing data and methods of effective agent tuning for large language models. arXiv preprint arXiv:2403.12881,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.