Pith. sign in

REVIEW 3 major objections 7 minor 42 references

Evolving data-curation code from agent failures beats fixed pipelines

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 15:15 UTC pith:3VKKQVT7

load-bearing objection CurateEvo treats data-curation code as the optimization target for agentic post-training, iteratively rewriting it using dev-set failure feedback. The idea is new and the results are consistent, but one missing control leaves the core mechanism question open. the 3 major comments →

arxiv 2607.06140 v1 pith:3VKKQVT7 submitted 2026-07-07 cs.CL

CurateEvo: Data-Curation Evolving for Agentic Post-Training

Dingzirui Wang , Xuanliang Zhang , Keyan Xu , Qingfu Zhu , Wanxiang Che This is my paper
classification cs.CL
keywords data curationagentic post-trainingcode evolutionfailure-driven optimizationLLM agentsreinforcement learningcost-aware optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

CurateEvo argues that the way we prepare training data for LLM agents should not be a fixed preprocessing step but an evolving program. The framework represents the entire curation strategy — how raw interaction trajectories are filtered, refined, augmented, and split into supervised fine-tuning data, reinforcement learning data, and an inference-time memory bank — as executable code. An LLM-based agent iteratively rewrites this code using failed trajectories from a held-out development set: at each epoch, it diagnoses recurring failure modes (such as incorrect tool selection, weak multi-turn planning, or poor error recovery), revises the curation code to target those gaps, then prunes redundant or low-utility training turns under a cost-aware objective. The base model, training recipe, and raw corpus all stay fixed; only the curation code changes. Across three agent benchmarks (ACEBench-Agent, BFCL-V4, and τ²-Bench) and two data settings (human-annotated and wild user trajectories), CurateEvo outperforms prior curation methods by 3.2 and 2.7 average points respectively, while halving curation overhead. The paper also shows the curated resources transfer to different post-training recipes, yielding an average additional gain of 21.3 points.

Core claim

The central mechanism is the cost-aware evolution of executable curation code. The curation strategy ρ is treated as a program that transforms a fixed raw corpus into training and inference resources. At each epoch, failed dev-set trajectories are diagnosed into failure modes, and an LLM-based code-evolution agent rewrites ρ in two ordered steps: first an effectiveness revision that augments, filters, or refines data to cover diagnosed capability gaps, then an efficiency revision that removes redundant turns and low-utility samples while preserving coverage of rare failures. The optimization objective J(ρ) = P(ρ) − λC(ρ) balances downstream dev performance against normalized training-turn成本.

What carries the argument

executable curation strategy ρ; cost-aware objective J(ρ) = P(ρ) − λC(ρ); failure-mode diagnosis from dev-set trajectories; two-phase code revision (effectiveness then efficiency); SFT + GRPO fixed training recipe; inference-time memory bank M_mem

Load-bearing premise

The held-out development set is assumed to be a reliable proxy for test-set generalization throughout the evolution process — the curation code is rewritten to fix dev-set failures and the final strategy is selected by dev-set performance, but the paper does not report dev-set size, analyze the dev-test gap, or use any held-out validation beyond the dev set itself.

What would settle it

If the dev set is too small or unrepresentative, the evolved curation code could overfit to dev-set-specific failure patterns rather than improving generalizable agent capabilities, and the test-set gains would not replicate on a different benchmark distribution.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If curation code can be evolved from failures, the same principle could apply to other stages of the ML pipeline — reward shaping, prompt engineering, or evaluation rubrics could all be represented as code and iteratively rewritten using failure feedback.
  • The separation of policy data from memory data suggests that not all useful information in agent trajectories belongs in model parameters; some is better stored as retrievable context, which has implications for RAG-vs-finetuning trade-offs.
  • The finding that a 4B model with evolved curation outperforms an 8B model with fixed curation suggests that data quality engineering may substitute for scale in agent training, at least within the ranges tested.
  • The failure-mode taxonomy (State Binding Drift, Adaptive Recovery Failure, Grounded Execution Gap) provides a diagnostic vocabulary that could guide targeted data collection even outside the evolution framework.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 3-epoch evolution with dev-set selection creates a risk of overfitting to dev-set failure patterns; a natural extension would be to add a held-out validation split between dev and test, or to use cross-validation across multiple dev folds during strategy selection.
  • The cost-aware objective's λ parameter is tuned on the same benchmarks used for final evaluation; an adaptive λ schedule that responds to the saturation curve observed in Figure 4 could reduce this sensitivity.
  • The code-evolution agent uses a proprietary LLM (GPT-5.4), which means the curation strategy's quality is partly bounded by that model's code-editing capability; testing with open-source code-editing models would clarify whether the framework's value is robust to the choice of evolution agent.
  • The framework evolves curation independently for each data setting; a shared evolution starting point or transfer of evolved strategies across settings could test whether failure patterns generalize across data distributions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes CurateEvo, a framework that represents agentic post-training data curation as executable code and iteratively rewrites it using failed trajectories from a held-out development set. At each evolution epoch, the curation strategy transforms a fixed raw corpus into SFT data, RL data, and an inference-time memory bank. The evolution first targets effectiveness by diagnosing recurring failure modes and augmenting, filtering, or refining data accordingly, then targets efficiency by pruning redundant or low-utility training turns under a cost-aware objective. Experiments on ACEBench-Agent, BFCL-V4, and τ²-Bench under labeled and wild-data settings show CurateEvo outperforming prior curation methods by 3.2 and 2.7 average points respectively. Ablations isolate the effectiveness and efficiency revisions, and additional analyses demonstrate compatibility with different post-training recipes and reduced curation overhead.

Significance. The paper addresses a genuine gap: most agentic post-training curation methods treat data processing as a fixed pipeline rather than an adaptive, failure-driven target. The formulation of curation as executable code that is evolved via dev-set feedback is a clean and potentially impactful idea. The experimental results are internally consistent across three benchmarks and two data settings, and the ablation in Table 2 supports each component of the objective J(ρ) = P(ρ) − λC(ρ). The efficiency analysis (Figure 3) and the cross-recipe compatibility study (Table 3) add practical value. The failure mode analysis in Appendix D is a useful qualitative contribution. However, two methodological gaps—one concerning baseline comparability and one concerning the isolation of the evolution mechanism from frontier-model code-writing ability—currently limit confidence in the central causal claim.

major comments (3)
  1. §5.1, Table 1: Several baselines (MUA-RL, EnvScaler, AWM) use QWEN3-8B while CurateEvo uses QWEN3-4B. Although CurateEvo outperforming larger-model baselines is presented as a strength, the comparison is confounded by model size. The paper does not report CurateEvo results with QWEN3-8B, nor does it report the 4B baselines for MUA-RL, EnvScaler, and AWM. Without at least one matched-model comparison for these baselines, it is unclear whether the gains come from the curation evolution or from other differences in the training pipeline. This is load-bearing for the central claim that evolving curation code outperforms prior curation methods. The authors should either (a) run the strongest baselines with QWEN3-4B, or (b) run CurateEvo with QWEN3-8B, to enable at least one apples-to-apples comparison.
  2. §4.2, §5.1: The central claim is that *evolving* curation code through iterative, failure-driven revision yields better training data than fixed pipelines. However, the curation code is rewritten by GPT-5.4, a frontier model, and the final evolved strategy (Appendix C) includes sophisticated techniques (fine-grained trajectory decomposition, policy/memory separation, failure-family-oriented signal construction) that could reflect GPT-5.4's general code-writing ability rather than the iterative refinement process itself. Figure 4 shows epoch-over-epoch improvement, but this could be because GPT-5.4 receives increasingly specific information at each step, not because the evolution loop per se drives improvement. Table 2's ablation removes the failure-driven revision entirely, conflating 'no evolution' with 'no failure feedback.' The critical missing control is a single-pass condition: GPT-
  3. §3, §4.1, §5.1: The dev set Q_dev is the sole proxy for generalization throughout the evolution process, and the final strategy is selected based on dev-set performance. The paper does not report the dev-set size, does not analyze the dev-test gap, and uses only 3 evolution epochs with no held-out validation beyond Q_dev itself for strategy selection. If Q_dev is too small or not representative, the evolved curation code could overfit to dev-set failure patterns. The paper should report |Q_dev| for each benchmark and ideally include a dev-set vs. test-set performance comparison across epochs to demonstrate that dev-set improvements transfer to the held-out test sets.
minor comments (7)
  1. §4.2: The cost term C(ρ) uses tanh(log(1 + Cost_train(ρ)) / log(1 + N_ref)) with N_ref = 10^5. The sensitivity to N_ref is not analyzed. A brief note on how this choice was made or whether results are robust to it would strengthen the presentation.
  2. Figure 4: The x-axis extends to 8 epochs, but the main experiments use 3 epochs (§5.1). The relationship between the extended curve and the main-experiment configuration should be clarified—e.g., are epochs 4–8 from a separate extended run?
  3. Table 1: The GRPO-without-curation baseline includes both QWEN3-4B and QWEN3-8B rows, but the prior-method baselines mix model sizes without a clear rationale for which baseline uses which model. A footnote or table note explaining the model-size assignment would help.
  4. §5.1: The raw corpus is split 9:1 into training and development portions. The absolute sizes (number of trajectories or turns) of D_raw, Q_dev, and the curated datasets at each epoch are not reported. These numbers would help readers assess the scale of the experiments.
  5. Appendix C describes the final evolved curation strategy qualitatively but does not include the actual code. For reproducibility, the code (or a representative excerpt) should be provided, either in the appendix or as supplementary material.
  6. §5.5, Table 3: The compatibility study reports gains of 21.3 points on average when combining CurateEvo with different post-training recipes. This number appears to compare against GRPO-without-curation rather than against each recipe without CurateEvo. The comparison should be clarified.
  7. Institution name on title page: 'Harbin Institue of Technology' should be 'Harbin Institute of Technology.'

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises three major concerns: (1) baseline comparability due to mismatched model sizes (QWEN3-4B vs. QWEN3-8B), (2) isolation of the evolution mechanism from frontier-model code-writing ability via a missing single-pass control, and (3) potential dev-set overfitting given unreported dev-set sizes and no dev-test gap analysis. We agree that all three points identify genuine gaps that, if addressed, would strengthen the causal claims of the paper. We commit to revisions and additional experiments for points 1 and 3, and we agree in principle with point 2 but face a practical constraint on the single-pass control that we explain below.

read point-by-point responses

  1. Referee: §5.1, Table 1: Several baselines (MUA-RL, EnvScaler, AWM) use QWEN3-8B while CurateEvo uses QWEN3-4B. The comparison is confounded by model size. The paper does not report CurateEvo results with QWEN3-8B, nor does it report the 4B baselines for MUA-RL, EnvScaler, and AWM. Without at least one matched-model comparison for these baselines, it is unclear whether the gains come from the curation evolution or from other differences in the training pipeline. The authors should either (a) run the strongest baselines with QWEN3-4B, or (b) run CurateEvo with QWEN3-8B, to enable at least one apples-to-apples comparison.

    Authors: The referee is correct that the current Table 1 confounds model size with method for the MUA-RL, EnvScaler, and AWM baselines. We acknowledge this is a genuine gap. We note that two baselines (RODS and FunReason-MT) already use QWEN3-4B, providing matched comparisons for those methods. However, the referee's point specifically targets the three 8B baselines, and we agree that at least one matched comparison with those methods is needed. We will address this by running CurateEvo with QWEN3-8B (option b), which allows direct comparison against the existing 8B baseline results. We will also attempt to run at least the strongest 8B baseline (MUA-RL) with QWEN3-4B (option a) if the baseline code and configurations are available to us. We will add whichever matched comparisons we can produce to the revised Table 1 and discuss the implications. We will also add an explicit note in the table caption and text acknowledging which comparisons are matched and which are not, rather than presenting the cross-size comparison as a clean strength without qualification. revision: yes

  2. Referee: §4.2, §5.1: The central claim is that evolving curation code through iterative, failure-driven revision yields better training data than fixed pipelines. However, the curation code is rewritten by GPT-5.4, a frontier model, and the final evolved strategy (Appendix C) includes sophisticated techniques that could reflect GPT-5.4's general code-writing ability rather than the iterative refinement process itself. Figure 4 shows epoch-over-epoch improvement, but this could be because GPT-5.4 receives increasingly specific information at each step, not because the evolution loop per se drives improvement. Table 2's ablation removes the failure-driven revision entirely, conflating 'no evolution' with 'no failure feedback.' The critical missing control is a single-pass condition: GPT-5.4 writes the curation code once with all available failure information, without iterative refinement.

    Authors: The referee identifies a genuine confound in our ablation design. The current Table 2 '- Effectiveness' ablation removes failure-driven revision entirely, which conflates two factors: (a) whether failure feedback is used at all, and (b) whether the code is evolved iteratively versus written in a single pass. We agree that a single-pass condition—where GPT-5.4 writes the curation code once with access to all failure information from an initial evaluation, without the iterative loop—would isolate the contribution of the evolution mechanism itself. We will add this control to the revised manuscript. Specifically, we will run a 'single-pass' condition where the code-evolution agent receives the initial failed trajectories and writes the curation code in one shot, then the resulting data is used for training and evaluation without further iteration. This will be compared against both the full 3-epoch evolution and the no-evolution baseline. We note one practical constraint: the single-pass condition requires the agent to have failed trajectories available, which means an initial model must still be trained and evaluated on the dev set. This is feasible and we commit to running it. If the single-pass condition performs comparably to the full evolution, we will revise our claims accordingly to attribute the gains to failure-informed code writing rather than iterative evolution per se. revision: yes

  3. Referee: §3, §4.1, §5.1: The dev set Q_dev is the sole proxy for generalization throughout the evolution process, and the final strategy is selected based on dev-set performance. The paper does not report the dev-set size, does not analyze the dev-test gap, and uses only 3 evolution epochs with no held-out validation beyond Q_dev itself for strategy selection. If Q_dev is too small or not representative, the evolved curation code could overfit to dev-set failure patterns. The paper should report |Q_dev| for each benchmark and ideally include a dev-set vs. test-set performance comparison across epochs to demonstrate that dev-set improvements transfer to the held-out test sets.

    Authors: The referee is correct that we do not report |Q_dev| and do not analyze the dev-test gap across evolution epochs. This is a legitimate concern: if dev-set improvements do not transfer to the held-out test sets, the evolution process may be overfitting to dev-set failure patterns. We will address this in two ways. First, we will report |Q_dev| for each benchmark in the revised §5.1. The dev sets are derived from a 9:1 train/dev split of the raw corpus, so |Q_dev| is approximately 10% of each raw corpus; we will provide exact numbers. Second, we will add a dev-set vs. test-set performance comparison across evolution epochs, extending Figure 4 to show both dev and test performance at each epoch. This directly addresses the overfitting concern: if test performance tracks dev performance across epochs, the evolution is generalizing; if test performance diverges or degrades while dev performance improves, that would indicate overfitting. We note that in the current experimental protocol, the test set is never used during evolution, so reporting test performance per epoch is a post-hoc analysis that does not violate our protocol. We commit to adding this analysis. One caveat: because each epoch requires a full SFT+GRPO training run and evaluation on three benchmarks, the per-epoch test evaluation adds non-trivial compute, but it is feasible for the 3 epochs already reported. revision: yes

Circularity Check

0 steps flagged

No circularity found: dev-set selection and test-set evaluation are properly separated

full rationale

The paper's derivation chain is self-contained against external benchmarks. The curation strategy ρ is evolved using failed trajectories from a held-out dev set Q_dev (§4.1), and the final strategy is selected based on dev-set performance (§4.3). The main evaluation is on three held-out test benchmarks — ACEBench-Agent, BFCL-V4, and τ²-Bench — which are explicitly stated to be 'never used during evolution' (§4, §5.1). There is no self-citation chain: the method is defined from first principles in §3–§4, with no invocation of a prior uniqueness theorem or ansatz from the authors' own work. The effectiveness term P(ρ) = Perf_dev(π_ρ, M_ρ) is defined in terms of dev-set performance, and while the paper acknowledges it uses 'failed dev trajectories from the current epoch as an actionable surrogate for improving this term' (§4.2), this is a heuristic approximation for the optimization search, not a definitional reduction of the final reported result. The final reported numbers in Table 1 are on held-out test sets, not on the dev set used for evolution. The ablation in Table 2 removes components and measures test-set impact, which is a standard experimental control. The reader's concern about dev-set overfitting and the missing single-pass ablation are correctness risks (generalization gaps, confounding of evolution vs. model capability), not circularity. No step in the paper's claimed derivation chain reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

CurateEvo introduces three free parameters (λ, N_ref, epochs) all selected empirically. The core axioms center on dev-set representativeness and the code-evolution agent's competence. The memory bank is the main invented entity, with ablation evidence but limited implementation detail.

free parameters (3)
  • λ (cost weight) = 0.3
    Selected by sweep over {0.0, 0.1, ..., 1.0} on dev set (Figure 5). Controls effectiveness-efficiency trade-off in J(ρ) = P(ρ) − λC(ρ).
  • N_ref (reference constant for cost normalization) = 100000
    Stated in §5.1 as stabilizing the scale across datasets with long-tailed trajectory-length distributions. No justification given for this specific value.
  • Number of evolution epochs = 3
    Stated in §5.1. Figure 4 shows performance saturates after ~3 epochs, but the choice of 3 is post hoc based on observed saturation.
axioms (4)
  • domain assumption The held-out dev set Q_dev is a reliable proxy for generalization performance on unseen test benchmarks.
    Invoked in §3 (objective formulation) and §4.1 (feedback collection). The entire evolution loop depends on dev-set failures being representative of test-time weaknesses.
  • domain assumption An LLM-based code-evolution agent (GPT-5.4) can effectively diagnose failure modes and rewrite curation code to address them.
    Invoked in §4.2. The quality of evolved curation strategies depends entirely on the code-evolution agent's ability to produce meaningful code revisions from failure analysis.
  • domain assumption The fixed SFT+GRPO training recipe is sufficient to reflect the quality differences in curated data.
    Invoked in §3 and §4. If the training recipe is a bottleneck, improvements in data quality may not be reflected in final performance.
  • domain assumption Failed trajectories contain actionable signal about missing capabilities that can be addressed through data curation.
    Invoked in §4.1-4.2. The failure-mode diagnosis and curation revision assume that failures are attributable to data deficiencies rather than model capacity or training recipe limitations.
invented entities (1)
  • Inference-time memory bank M_mem independent evidence
    purpose: Stores task-relevant knowledge extracted from curated trajectories, retrieved at inference time to augment the agent's context.
    The memory bank is ablated in Table 2 (removing it causes consistent drops), providing falsifiable evidence of its contribution. However, the retrieval mechanism and memory construction logic are only described qualitatively in Appendix C.

pith-pipeline@v1.1.0-glm · 21064 in / 3236 out tokens · 352028 ms · 2026-07-08T15:15:44.898449+00:00 · methodology

0 comments
read the original abstract

Large language model (LLM) agents require post-training methods that can improve long-horizon decision making from environment feedback. However, existing agentic post-training pipelines often treat data curation as a fixed preprocessing step, focusing mainly on data augmentation while neglecting filtering, refinement, and adaptation to downstream failures. We propose CurateEvo, a failure-driven dynamic evolution framework for agentic post-training data curation. CurateEvo represents the curation strategy as executable code and iteratively rewrites it using failed trajectories from a held-out development set. At each epoch, the evolved strategy transforms a fixed raw corpus into supervised fine-tuning data, reinforcement learning data, and an inference-time memory bank. The evolution process first improves effectiveness by diagnosing recurring failure modes and augmenting, filtering, or refining data accordingly, and then improves efficiency by pruning redundant or low-utility training turns under a cost-aware objective. Experiments on ACEBench-Agent, BFCL-V4, and {\tau}^2-Bench under both labeled and wild-data settings show that CurateEvo consistently outperforms prior curation methods, improving average scores by 3.2 and 2.7 points, respectively. Further analyses demonstrate that CurateEvo is compatible with different post-training recipes and substantially reduces curation overhead.

Figures

Figures reproduced from arXiv: 2607.06140 by Dingzirui Wang, Keyan Xu, Qingfu Zhu, Wanxiang Che, Xuanliang Zhang.

Figure 1
Figure 1. Figure 1: Two limitations of existing agen￾tic post-training data curation methods. Curation Neglect denotes overlooking the curation operation like filter and refine, mixing low quality data. Adaption Rigidity denotes the curation process is fixed, which cannot be adapted to different applications flexibly. Despite their effectiveness, existing data curation methods for agentic post-training still suffer from two l… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of CURATEEVO. At evolution epoch e, the executable curation strategy ρe transforms the fixed raw corpus into an SFT dataset, an RL dataset, and an inference-time memory bank. Starting from the same base model, we train an agent using a fixed SFT+GRPO recipe and evaluate it on a held-out dev set. An LLM-based code-evolution agent diagnoses recurring failure modes from failed dev trajectories, first… view at source ↗
Figure 3
Figure 3. Figure 3: Averaged token and wall-clock overhead of each curation method per [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Averaged performance and curated training scale across evolution epochs on three bench [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effect of the cost weight λ on averaged performance and curated training scale on three benchmarks. cost-aware evolution objective further balances effectiveness and efficiency by first targeting recur￾ring failure modes and then reducing redundant or low-utility training turns. Experiments across ACEBench-Agent, BFCL-V4, and τ 2 -Bench show that CURATEEVO achieves consistent improve￾ments under both label… view at source ↗
Figure 6
Figure 6. Figure 6: The prompt to transfer failed trajectories into failed points. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The prompt to optimize curation code based on failed points and existing training data. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages · 8 internal anchors

  1. [1]

    Openhands feedback dataset

    All Hands AI. Openhands feedback dataset. https://huggingface.co/datasets/all-hands/ openhands-feedback, 2024

    work page 2024

  2. [2]

    $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment

    Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan.τ 2-bench: Evaluating conversational agents in a dual-control environment, 2025. URL https://arxiv.org/ abs/2506.07982

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    SWE-chat: Coding agent interactions from real users in the wild

    Joachim Baumann, Vishakh Padmakumar, Xiang Li, John Yang, Diyi Yang, and Sanmi Koyejo. SWE-chat: Coding agent interactions from real users in the wild. InDeep Learning for Code: Towards Human-Centered Coding Agents, 2026. URL https://openreview.net/forum? id=DzF20wT9hn

    work page 2026

  4. [4]

    Fireact: Toward language agent finetuning, 2024

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik R Narasimhan, and Shunyu Yao. Fireact: Toward language agent finetuning, 2024. URL https://openreview.net/forum?id= RqUMWdDg52

    work page 2024

  5. [5]

    ACEBench: A comprehensive evaluation of LLM tool usage

    Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Yuefeng Huang, Xiangcheng Liu, Wang Xinzhi, and Wu Liu. ACEBench: A comprehensive evaluation of LLM tool usage. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the Association for Computational Linguistics: EMNLP 2025, pa...

    work page doi:10.18653/v1/2025.findings-emnlp.697 2025

  6. [6]

    Agent-FLAN: Designing data and methods of effective agent tuning for large language models

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Wenwei Zhang, Jiangning Liu, Dahua Lin, Kai Chen, and Feng Zhao. Agent-FLAN: Designing data and methods of effective agent tuning for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics: ACL 2024, pages 9354–9366, Bangkok, Thailand,...

    work page doi:10.18653/v1/2024.findings- 2024

  7. [7]

    ATLAS: Agent tuning via learning critical steps

    Zhixun Chen, Ming Li, Yuxuan Huang, Yali Du, Meng Fang, and Tianyi Zhou. ATLAS: Agent tuning via learning critical steps. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors,Findings of the Association for Computational Linguistics: ACL 2025, pages 25334–25349, Vienna, Austria, July 2025. Association for Computational L...

    work page doi:10.18653/v1/2025.findings-acl.1299 2025

  8. [8]

    Mind2web: Towards a generalist agent for the web

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. InThirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https: //openreview.net/forum?id=kiYqbO3wqw

    work page 2023

  9. [9]

    RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents

    Ruishan Fang, Siyuan Lu, Chenyi Zhuang, and Tao Lin. Rods: Reward-driven online data synthesis for multi-turn tool-use agents, 2026. URL https://arxiv.org/abs/2606.19047

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  10. [10]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInter- national Conference on Learning Representations, 2022. URL https://openreview.net/forum? id=nZeVKeeFYf9

    work page 2022

  11. [11]

    WebCoT: Enhancing web agent rea- soning by reconstructing chain-of-thought in reflection, branching, and rollback

    Minda Hu, Tianqing Fang, Jianshu Zhang, Jun-Yu Ma, Zhisong Zhang, Jingyan Zhou, Hong- ming Zhang, Haitao Mi, Dong Yu, and Irwin King. WebCoT: Enhancing web agent rea- soning by reconstructing chain-of-thought in reflection, branching, and rollback. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Findings of the A...

    work page 2025

  12. [12]

    URL https://aclanthology.org/2025.findings- emnlp.276/

    doi: 10.18653/v1/2025.findings-emnlp.276. URL https://aclanthology.org/2025.findings- emnlp.276/. 12

    work page doi:10.18653/v1/2025.findings-emnlp.276 2025

  13. [13]

    Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan O Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning. InSecond Conference on Language Modeling, 2025. URL https: //openreview.net/forum?id=Rwhi91ideu

    work page 2025

  14. [14]

    Dataenvgym: Data gen- eration agents in teacher environments with student feedback

    Zaid Khan, Elias Stengel-Eskin, Jaemin Cho, and Mohit Bansal. Dataenvgym: Data gen- eration agents in teacher environments with student feedback. InThe Thirteenth Interna- tional Conference on Learning Representations, 2025. URL https://openreview.net/forum? id=00SnKBGTsz

    work page 2025

  15. [15]

    Autowebglm: A large language model-based web navigating agent

    Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, and Jie Tang. Autowebglm: A large language model-based web navigating agent. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, page 5295–5306, New York, NY , USA,

    work page

  16. [16]

    ISBN 9798400704901

    Association for Computing Machinery. ISBN 9798400704901. doi: 10.1145/3637528. 3671620. URL https://doi.org/10.1145/3637528.3671620

    work page doi:10.1145/3637528

  17. [17]

    Exploring expert failures improves LLM agent tuning, 2026

    Li-Cheng Lan, Andrew Bai, Minhao Cheng, Cho-Jui Hsieh, and Tianyi Zhou. Exploring expert failures improves LLM agent tuning, 2026. URL https://openreview.net/forum?id= 4fh0Z9nwjx

    work page 2026

  18. [18]

    Efficient multi-turn RL for GUI agents via decoupled training and adaptive data curation, 2026

    Pengxiang Li, zechenhu, Zirui Shang, Jingrong Wu, Yang Liu, Hui Liu, Zhi Gao, Chenrui Shi, Bofei Zhang, Zihao Zhang, Xiaochuan Shi, Zedong YU, Yuwei Wu, Xinxiao Wu, Yunde Jia, Liuyu Xiang, Zhaofeng He, and Qing Li. Efficient multi-turn RL for GUI agents via decoupled training and adaptive data curation, 2026. URL https://openreview.net/forum?id= k6AJ1N7BA2

    work page 2026

  19. [19]

    Agentre- wardbench: Evaluating automatic evaluations of web agent trajectories

    Xing Han Lù, Amirhossein Kazemnejad, Nicholas Meade, Arkil Patel, Dongchan Shin, Ale- jandra Zambrano, Karolina Stanczak, Peter Shaw, Christopher Pal, and Siva Reddy. Agentre- wardbench: Evaluating automatic evaluations of web agent trajectories. InSecond Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=fQcUZMPIvu

    work page 2025

  20. [20]

    Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E

    Shishir G. Patil, Huanzhi Mao, Charlie Cheng-Jie Ji, Fanjia Yan, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. InForty-second International Conference on Machine Learning, 2025

    work page 2025

  21. [21]

    WebRL: Training LLM web agents via self-evolving online curriculum reinforcement learning

    Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Jiadai Sun, Xinyue Yang, Yu Yang, Shuntian Yao, Wei Xu, Jie Tang, and Yuxiao Dong. WebRL: Training LLM web agents via self-evolving online curriculum reinforcement learning. InThe Thirteenth Interna- tional Conference on Learning Representations, 2025. URL https://openreview.net/forum?id= oVKEAFjEqv

    work page 2025

  22. [22]

    ToolLLM: Facilitating large lan- guage models to master 16000+ real-world APIs

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, dahai li, Zhiyuan Liu, and Maosong Sun. ToolLLM: Facilitating large lan- guage models to master 16000+ real-world APIs. InThe Twelfth International Conference on Learni...

    work page 2024

  23. [23]

    EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis

    Xiaoshuai Song, Haofei Chang, Guanting Dong, Yutao Zhu, Zhicheng Dou, and Ji-Rong Wen. Envscaler: Scaling tool-interactive environments for llm agent via programmatic synthesis. arXiv preprint arXiv:2601.05808, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  24. [24]

    AgentBank: Towards generalized LLM agents via fine-tuning on 50000+ interaction trajectories

    Yifan Song, Weimin Xiong, Xiutian Zhao, Dawei Zhu, Wenhao Wu, Ke Wang, Cheng Li, Wei Peng, and Sujian Li. AgentBank: Towards generalized LLM agents via fine-tuning on 50000+ interaction trajectories. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2124–2141, Miami,...

    work page 2024

  25. [25]

    Trial and error: Exploration-based trajectory optimization of LLM agents

    Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. Trial and error: Exploration-based trajectory optimization of LLM agents. In Lun-Wei Ku, Andre Martins, 13 and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7584–7600, Bangkok, Thailand,...

    work page doi:10.18653/v1/2024.acl-long.409 2024

  26. [26]

    Xu, Shuyan Zhou, Xingyao Wang, Xiang Yue, Tao Yu, Huan Sun, Yu Su, and Graham Neubig

    Yueqi Song, Ketan Ramaneti, Zaid Sheikh, Ziru Chen, Boyu Gou, Tianbao Xie, Yiheng Xu, Danyang Zhang, Apurva Gandhi, Fan Yang, Joseph Liu, Tianyue Ou, Zhihao Yuan, Frank F. Xu, Shuyan Zhou, Xingyao Wang, Xiang Yue, Tao Yu, Huan Sun, Yu Su, and Graham Neubig. Agent data protocol: Unifying datasets for diverse, effective fine-tuning of LLM agents. In The Fou...

    work page 2026

  27. [27]

    InSTA: Towards internet-scale training for agents, 2026

    Brandon Trabucco, Gunnar A Sigurdsson, Robinson Piramuthu, and Ruslan Salakhutdinov. InSTA: Towards internet-scale training for agents, 2026. URL https://openreview.net/forum? id=zibL9wu2LI

    work page 2026

  28. [28]

    Learning from failure: Integrating negative examples when fine-tuning large language models as agents, 2024

    Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Timothy Baldwin. Learning from failure: Integrating negative examples when fine-tuning large language models as agents, 2024

    work page 2024

  29. [29]

    NAT: En- hancing agent tuning with negative samples

    Renxi Wang, Xudong Han, Yixuan Zhang, Timothy Baldwin, and Haonan Li. NAT: En- hancing agent tuning with negative samples. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages...

    work page doi:10.18653/v1/2025.naacl-long.378 2025

  30. [30]

    Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning

    Zhaoyang Wang, Canwen Xu, Boyi Liu, Yite Wang, Siwei Han, Zhewei Yao, Huaxiu Yao, and Yuxiong He. Agent world model: Infinity synthetic environments for agentic reinforcement learning, 2026. URL https://arxiv.org/abs/2602.10090

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  31. [31]

    RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning

    Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding self- evolution in llm agents via multi-turn reinforcement learning, 2025. URL https://arxiv.org/abs...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  32. [32]

    WebAgent-r1: Training web agents via end-to-end multi-turn reinforcement learning

    Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. WebAgent-r1: Training web agents via end-to-end multi-turn reinforcement learning. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empir...

    work page doi:10.18653/v1/2025.emnlp-main.401 2025

  33. [33]

    Agentgym-RL: An open-source framework to train LLM agents for long-horizon decision making via multi-turn RL

    Zhiheng Xi, Jixuan Huang, Chenyang Liao, Baodai Huang, Jiaqi Liu, Honglin Guo, yajie yang, Rui Zheng, Junjie Ye, Jiazheng Zhang, Wenxiang Chen, Wei He, Yiwen Ding, Guanyu Li, Zehui Chen, Zhengyin Du, Xuesong Yao, Yufei Xu, Jiecao Chen, Tao Gui, Zuxuan Wu, Qi Zhang, Xuanjing Huang, and Yu-Gang Jiang. Agentgym-RL: An open-source framework to train LLM agent...

    work page 2026

  34. [34]

    LESS: Selecting influential data for targeted instruction tuning

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. InForty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=PG5fV50maR

    work page 2024

  35. [35]

    Funreason-mt technical report: Advanced data synthesis solution for real-world multi-turn tool-use, 2025

    Zengzhuang Xu, Bingguang Hao, Zechuan Wang, Yuntao Wen, Xinyi Xu, Yang Liu, Long Chen, Dong Wang, Maolin Wang, Tong Zhao, Yicheng Chen, Cunyin Peng, Jinjie Gu, Leilei Gan, Xiangyu Zhao, Chenyi Zhuang, and Shi Gu. Funreason-mt technical report: Advanced data synthesis solution for real-world multi-turn tool-use, 2025. URL https://arxiv.org/abs/ 2510.24645. 14

    work page arXiv 2025

  36. [36]

    SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

    John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated soft- ware engineering. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://arxiv.org/abs/2405.15793

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  37. [37]

    Webshop: Towards scalable real-world web interaction with grounded language agents

    Shunyu Yao, Howard Chen, John Yang, and Karthik R Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors,Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=R9KnuFlvnU

    work page 2022

  38. [38]

    Agent lumos: Unified and modular training for open-source lan- guage agents

    Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. Agent lumos: Unified and modular training for open-source lan- guage agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers...

    work page doi:10.18653/v1/2024.acl-long.670 2024

  39. [39]

    Envgen: Generating and adapting environments via LLMs for training embodied agents

    Abhay Zala, Jaemin Cho, Han Lin, Jaehong Yoon, and Mohit Bansal. Envgen: Generating and adapting environments via LLMs for training embodied agents. InFirst Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=F9tqgOPXH5

    work page 2024

  40. [40]

    AgentTuning: Enabling generalized agent abilities for LLMs

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. AgentTuning: Enabling generalized agent abilities for LLMs. In Lun-Wei Ku, Andre Mar- tins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics: ACL 2024, pages 3053–3077, Bangkok, Thailand, August 2024. Association for Computa- tional Lingu...

    work page doi:10.18653/v1/2024.findings-acl.181 2024

  41. [41]

    AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning

    Jianguo Zhang, Tian Lan, Rithesh Murthy, Zhiwei Liu, Weiran Yao, Ming Zhu, Juntao Tan, Thai Hoang, Zuxin Liu, Liangwei Yang, Yihao Feng, Shirley Kokane, Tulika Awalgaonkar, Juan Carlos Niebles, Silvio Savarese, Shelby Heinecke, Huan Wang, and Caiming Xiong. Agentohana: Design unified data and training pipeline for effective agent learning, 2024. URL https...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  42. [42]

    MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use

    Weikang Zhao, Xili Wang, Chengdi Ma, Lingbin Kong, Zhaohua Yang, Mingxiang Tuo, Xi- aowei Shi, Yitao Zhai, and Xunliang Cai. Mua-rl: Multi-turn user-interacting agent reinforce- ment learning for agentic tool use, 2025. URL https://arxiv.org/abs/2508.18669. 15 A Prompt Failed Point Generation from Failed Trajectories You are a failure analysis agent for i...

    work page internal anchor Pith review Pith/arXiv arXiv 2025