LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents

Masafumi Oyamada; Taro Yano; Yoichi Ishibashi

REVIEW 3 major objections 5 minor 14 references

LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM agent that sequences fine-tuning and model-merging steps can beat hand-built post-training pipelines, reporting gains of 3.7 math points and 9.0 tool-use points in the paper's experiments.

desk verdict LaMDAgent is a sensible agentic post-training search framework with a well-controlled first experiment, but the headline +9.0 tool-use gain is the best-of-100 on the reward benchmark, so the empirical claim is optimistic until held-out evaluation is added. read the letter →

arxiv 2505.21963 v1 pith:Y4FHJ6IG submitted 2025-05-28 cs.CL cs.AI

Taro Yano , Yoichi Ishibashi , Masafumi Oyamada This is my paper

classification cs.CLcs.AI

keywords LLMagentspost-trainingpipelinesupervisedfine-tuningTIES-Mergingmodelmergingtool-useabilityinstructionfollowingautomaticsearch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the whole post-training pipeline for an LLM — which actions to run, on which data, in which order — can be discovered autonomously by an LLM agent rather than designed by human experts. The agent enumerates actions from a pool of fine-tuning and model-merging operations, runs one chosen action, scores the resulting model on target benchmarks, and updates a textual memory that guides future choices. Two experiments report that this loop outperforms hand-built baselines: a 3.7-point average gain on math tasks with other skills intact, and a 9.0-point gain in tool-use accuracy on an already instruction-tuned model while instruction-following is preserved. If the claim is right, adapting a model to a new task reduces to specifying an action pool and a benchmark, lowering the specialized knowledge needed for post-training.

What carries the argument

The carrying mechanism is a four-step loop: (1) enumerate all concrete actions from action types (SFT, TIES-Merging) and an object pool of models, datasets, and hyperparameters; (2) have an LLM choose an action type and then the objects, guided by a textual memory and by explicit exploration directives; (3) run the action to produce a new model and score it on target tasks, aggregating metrics of different scales through a weighted sum $s_{\mathrm{multi}}=\sum_k \alpha_k s_k^{\mathrm{single}}$; (4) use a second LLM call to rewrite the memory from the latest action, its score, and all previous trials. The memory is the component that accumulates experience, and the exploration directives are what prevent the agent from collapsing onto one repeated action.

What would settle it

Hold out an AceBench split that the agent never observes, rerun the 100-iteration search with the same action pool, and compare the best model's held-out score with the base model's score; if the 9.0-point gap does not reproduce on the held-out split, the central tool-use claim is a selection artifact rather than a general gain. A second check is to rerun Experiment 1 with freshly drawn validation folds and report the mean test score of the top-ranked pipelines instead of the single best one.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that an LLM-driven search over SFT and TIES-Merging actions, scored by downstream tasks and remembered as text, can discover post-training pipelines that outperform strong baselines. In Experiment 1 the best pipeline improves average math accuracy by 3.7 points over a fully fine-tuned model trained on all data, while keeping other task scores comparable. In Experiment 2 the best pipeline raises AceBench accuracy from 0.410 to 0.500 while the MT-Bench score stays essentially flat (0.810 versus 0.804). The paper also claims the search is not brute force: ablations show that random action selection underperforms agent selection, and removing SFT from the action space costs 12.9 test-set points in Experiment 1. A further claim is that pipelines ranked best at small data size keep their ranking when the data is scaled up, whereas small score gaps between pipelines can reverse when the base model is scaled up.

Load-bearing premise

The load-bearing premise is that the benchmark scores fed back to the agent are unbiased estimates of the capability a practitioner cares about, so that picking the best of roughly 100 explored pipelines does not overfit the evaluation metric.

Editorial extensions

If this is right

Post-training becomes a delegation problem: a practitioner supplies an action pool and a benchmark, and the loop proposes, executes, and scores candidate pipelines without hand-crafting curricula.
Discovered pipelines can include strategies human experts tend to overlook, such as merging specialist models before fine-tuning on all data, widening the set of viable recipes.
Because data-size scaling preserves the ranking of explored pipelines, cheap small-data exploration can select pipelines for expensive larger-data runs.
For instruction-tuned models, the loop can add a new skill such as tool use without the catastrophic forgetting seen under naive full fine-tuning, provided the action space permits enough data-ordering freedom.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 9.0-point tool-use gain is the maximum AceBench score along the search trajectory, and the paper does not describe a held-out AceBench split; a conservative reading treats the gain as measured on the evaluation set used, and a definite claim of generalization would require re-evaluating the discovered pipeline on a disjoint benchmark split.
The same memory-update loop should extend to any model-improvement operation expressible as typed actions, such as preference learning or synthetic-data generation; the paper lists these directions as future work.
The data-scaling result suggests a practical budgeting recipe—explore on roughly 1,000 examples per dataset, then scale up the winning pipelines—while the model-size caveat implies that transferring pipelines across base-model sizes should rely on large score gaps rather than small ones.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Plainly: the framework is a sensible agentic search over post-training actions, and Experiment 1 is honestly controlled. The problem is Experiment 2. The +9.0 AceBench gain is the maximum score along a 100-step search on the same benchmark used as the reward, with no held-out split and no variance estimate. The stress-test note lands on reading the paper; the concern is real.

What's actually new: LaMDAgent unifies SFT and TIES-Merging as actions in an LLM-agent loop and updates a textual memory from task scores. That is a natural extension of the authors' earlier self-improvement work, and it makes a concrete claim: an agent can compose fine-tuning and merging into pipelines that beat simple alternatives. The paper is also careful in places, fixing hyperparameters, using temperature 0, and including a random-policy ablation in Experiment 1. That ablation is the strongest evidence in the paper—the agent's selection beats random selection on held-out test tasks. The data-size scaling result is a useful practical finding, and the discovered pipelines are interpretable.

The soft spots are proportionate to the claims. Experiment 2 is not a controlled evaluation. AceBench is the reward and the reported metric; the best model is selected from 100 scored iterations. Picking the maximum of 100 evaluations will inflate the gain even without deliberate overfitting. The paper does not describe a held-out AceBench split, does not report repeated runs, and does not include a random-policy baseline there. The Limitations section does not flag this. That is not a minor omission, because the abstract's headline claim rests on it. The novelty claim also needs sharper separation from the same authors' NAACL 2025 paper, though the concrete framework and results here are new.

Who gets value: practitioners and researchers working on automated post-training, agentic model development, and benchmark-driven search. The paper is worth a serious referee, but the revision should be held to a clear standard: a held-out tool-use evaluation (or another tool-use benchmark), repeated seeds or confidence intervals, and a random-policy control in Experiment 2. If the held-out result holds, this is a solid contribution. If not, it is still a useful framework with a weaker empirical claim.

Referee Report

3 major / 5 minor

Summary. The paper introduces LaMDAgent, an LLM-agent-driven framework that automates post-training pipeline construction by iteratively enumerating candidate actions (SFT, TIES-Merging), selecting actions via an LLM informed by memory, evaluating the resulting models on target-task scores, and updating memory from that feedback. Two experimental settings are presented: Experiment 1 teaches arithmetic, commonsense, and reading-comprehension skills to Gemma2 2B and reports a 3.7-point average improvement on math-related tasks over a fully fine-tuned baseline, with a held-out test split and a random-policy ablation; Experiment 2 enhances tool-use capability of Gemma2 2B Instruct and reports a 9.0-point AceBench accuracy improvement while preserving MT-Bench score. The paper also studies data-size and model-size scaling as cost-reduction strategies, concluding that data-size scaling helps while model-size scaling introduces challenges.

Significance. If the reported results hold, LaMDAgent would be a practically useful step toward automating post-training pipeline design, reducing the manual expertise currently required. The framework is clearly described, and Experiment 1 has several strengths: a held-out test set, a random-policy ablation, and comparison against grid-searched TIES and full fine-tuning. The data-size scaling evidence is also a useful preliminary result. However, the headline claim of a 9.0-point tool-use improvement rests on an uncontrolled in-sample selection procedure in Experiment 2, where the best score of a 100-step search on the same AceBench metric used as reward is reported without a held-out split or repeated runs. This needs to be addressed before the central claim is fully supported.

major comments (3)

[Section 4.1, Section 4.2, Figure 5] The reported +9.0 AceBench improvement in Experiment 2 is an in-sample selection result, not an out-of-sample estimate. Section 4.1 states only that AceBench is used to evaluate tool usage and does not describe a held-out split for model selection or final reporting; Section 4.2 reports the best model generated by LaMDAgent, and Figure 5 confirms this is the maximum score along the 100-iteration search trajectory. Since the same AceBench metric is used both as the reward fed back to the agent and as the reported evaluation, selecting the maximum of 100 evaluations is susceptible to best-of-N selection bias, and the +9.0 figure cannot currently be distinguished from an optimization artifact. The authors should report performance on a held-out AceBench split (or a separate tool-use benchmark) for the final model, or provide repeated-seed runs with selection-corrected estimates, and the Limitations section should acknowledge this missing held-out evaluation.
[Section 3.1, Table 1] Experiment 1's central quantitative claim is based on a single search trajectory without repeated runs, confidence intervals, or significance testing. Although temperature is set to 0 and a seed is listed in the example config, the stochasticity of LLM-based action selection and SFT training is not otherwise controlled, and the selection of Top-1, Top-2, and Top-3 models by validation score means the reported test-set margins (e.g., 1.9 points average over Fully Fine-Tuned) could reflect selection noise. The authors should run the framework multiple times (or at least provide evidence that the temperature-0 setting produces deterministic trajectories) and report the variance of the reported averages.
[Section 5, Table 3] The conclusion that model-size scaling 'introduces new challenges' is supported by only a single transfer from Gemma2 2B to Gemma2 9B, with no repeated trials or error bars. Table 3 reports single numbers for top-1, top-50, top-80, top-90, and top-100 pipelines, and the reversal between Top-1 and Top-50 may be within noise. To make the scaling claim load-bearing, the authors should either add repeated runs with confidence intervals or temper the conclusion to reflect the preliminary nature of the observation.

minor comments (5)

[Section 2.4 and Figure 11] Equation (3) defines a weighted multi-task score with hand-set scaling factors alpha_k, but the example config in Figure 11 lists "score_aggregation": "mean". Please clarify which aggregation was actually used in each experiment and how alpha_k were chosen in Experiment 1, where all metrics are accuracies in a similar range.
[Figure 1 caption] The caption contains the typo 'Action Eunumeration'; it should read 'Action Enumeration'.
[Section 5] The phrase 'trainining' appears in the first paragraph of Section 5; please correct it. Similarly, 'target tasks' is misspelled as 'taget tasks' in Section 6 (Related Work).
[Section 6] The claim 'to our knowledge, this is the first work to automate post-training using LLM agents' is difficult to verify and should be softened or supported by a more systematic search of concurrent work on LLM-driven pipeline optimization.
[Section 4.1] In Experiment 2, only Gemma2 2B Instruct is listed as an initial model, yet TIES-Merging requires at least two models; please explain how models for merging were initialized (e.g., through the SFT actions themselves) so that the action space is fully specified.

Circularity Check

1 steps flagged · score 6.0 of 10

Experiment 2's headline +9.0 AceBench gain is the maximum of 100 evaluations on the same benchmark used as the agent's reward, so the tool-use result is an in-sample selection artifact; Experiment 1 remains independently controlled.

fitted input called prediction [Section 2.4 Eq. (3); Section 4.1; Section 4.2 Results (Figures 4 and 5)]
"To account for different scales of evaluation metrics across tasks, we define the multi-task score smulti using the following formula: smulti = X_k α_k · s^k_single. ... For example, MT-Bench metrics range up to 10, while AceBench metrics reach 1, so we set the weights for each score as α_MT = 1/10, α_Ace = 1. ... The best model generated by LaMDAgent achieves an MT-Bench score of 0.810, comparable to Gemma2 2B Instruct (0.804), while improving AceBench accuracy from 0.410 to 0.500—a 9.0 point improvement."

AceBench enters the selection objective directly (Eq. 3 with α_Ace = 1) and is also the headline metric in Figures 4 and 5. The 'best model generated by LaMDAgent' is the maximum, over the 100 explored pipelines, of the same AceBench-inclusive score used as reward; Figure 5 tracks 'Max Score' along the search trajectory and Section 4.2 reports the end-to-end score evolution of the Top-1 pipeline. Thus the reported 0.410 → 0.500 (+9.0) is the optimized objective's best observed value, not an out-of-sample estimate from a held-out split or repeated seeds.

full rationale

The framework's core loop (Section 2) is an explicit score-based search, so Experiment 1's controlled design — validation split, held-out test set, random-policy and action-space ablations, temperature 0 — gives genuine evidence for the method: the reported math gain is measured on a held-out test set. No load-bearing self-citation or imported uniqueness theorem is used; the citation to Ishibashi et al. (2025) is only a related-work pointer and does not reduce any derivation. The circularity is concentrated in Experiment 2: AceBench is both the reward and the reported metric, no held-out split is described, and the +9.0 point gain is the maximum along the search trajectory (Figure 5). Since the abstract and contributions lead with this number, one central advertised result reduces to an in-sample optimization outcome, warranting a 6 rather than a 0–2. The Limitations section does not flag the missing held-out tool-use evaluation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on a small, hand-configured search space (SFT and TIES, fixed hyperparameters, one 100-step run), on the assumption that the gpt-4o agent's memory and selection improve over random choice, and on the assumption that the benchmark scores used as rewards are unbiased estimates of target performance. No new physical entities are introduced; the 'Object' and 'Action' terms are bookkeeping labels for the search state.

free parameters (4)

multi-task score weights alpha_k = alpha_MT = 1/10, alpha_Ace = 1 in Experiment 2; uniform max contribution in Experiment 1
Eq. (3), Section 2.4: hand-chosen scaling factors determine what the agent optimizes; different weights would change discovered pipelines.
exploration budget = 100 iterations per experiment
Sections 2.5, 3.1, and 4.1: the reported best-of-trajectory result depends on this arbitrary budget.
per-dataset sample size = 1,000 examples per dataset
Sections 3.1 and 4.1: the data pool is defined by this hand-chosen size, which shapes both training and the scaling experiments.
fixed hyperparameters = SFT learning rate 1e-6; TIES weight 0.5; TIES density 0.5
Section 3.1: hyperparameters are fixed by hand rather than searched; pipelines are only optimized over action order and object choice.

assumptions (5)

domain assumption The two action types (SFT and TIES-Merging) with fixed hyperparameters are sufficient to express high-performing post-training pipelines for the target tasks.
Sections 3.1 and 4.1 define the search space this way; the central claim that LaMDAgent discovers effective pipelines is restricted to this space.
domain assumption gpt-4o, acting as the selection agent, produces action choices and memory updates that improve over random selection.
The ablation in Table 2 provides some support, but the agent's reasoning is not independently auditable, and a different agent model may change the results.
domain assumption Benchmark scores used as feedback (AceBench, MT-Bench, held-out validation in Experiment 1) are unbiased estimates of target performance, and best-of-100 selection does not overfit them.
Experiment 1 uses a held-out test split; Experiment 2 reports AceBench accuracy without describing a held-out split, so this assumption is load-bearing there.
ad hoc to paper The linear aggregation in Eq. (3) with hand-set alpha_k is an adequate scalarization of multi-task objectives.
Section 2.4: the scalar reward shapes all agent feedback, and no validation is provided that alternative weights yield the same conclusions.
domain assumption A single 100-iteration run at temperature 0 sufficiently explores the pipeline space to support the reported improvements.
Sections 3.1 and 4.1: no variance over runs is reported, so all conclusions rely on a single trajectory.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents." pith.science (2026). https://pith.science/paper/Y4FHJ6IG

@misc{pith2026250521963,
  author       = {Pith},
  title        = {Pith review of: LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y4FHJ6IG}},
  note         = {Machine review of arXiv:2505.21963}
}

read the original abstract

Large Language Models (LLMs) have demonstrated exceptional performance across a wide range of tasks. To further tailor LLMs to specific domains or applications, post-training techniques such as Supervised Fine-Tuning (SFT), Preference Learning, and model merging are commonly employed. While each of these methods has been extensively studied in isolation, the automated construction of complete post-training pipelines remains an underexplored area. Existing approaches typically rely on manual design or focus narrowly on optimizing individual components, such as data ordering or merging strategies. In this work, we introduce LaMDAgent (short for Language Model Developing Agent), a novel framework that autonomously constructs and optimizes full post-training pipelines through the use of LLM-based agents. LaMDAgent systematically explores diverse model generation techniques, datasets, and hyperparameter configurations, leveraging task-based feedback to discover high-performing pipelines with minimal human intervention. Our experiments show that LaMDAgent improves tool-use accuracy by 9.0 points while preserving instruction-following capabilities. Moreover, it uncovers effective post-training strategies that are often overlooked by conventional human-driven exploration. We further analyze the impact of data and model size scaling to reduce computational costs on the exploration, finding that model size scalings introduces new challenges, whereas scaling data size enables cost-effective pipeline discovery.

Figures

Figures reproduced from arXiv: 2505.21963 by the authors.

**Figure 3.** Top-1, Top-2, and Top-3 pipelines discovered [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

**Figure 4.** LaMDAgent significantly improves tool us [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Figures from the paper (6 more)

**Figure 6.** Figure 6: Top-1 and Top-2 pipelines discovered in ex [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Data size scaling is effective for computa [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: Prompt template to select an action type. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 9.** Figure 9: Prompt template to select objects. Prompt template to update memory You are a developer of Large Language Models ( LLMs ) that can improve models based on self reflections . You will be given results and memories of the previous improving trials . The results consist o…

**Figure 10.** Figure 10: Prompt template to update memory. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

**Figure 11.** Figure 11: An example config of our proposed method. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 9 canonical work pages

[4]

Mind2web: Towards a generalist agent for the web. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Informa- tion Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jing...

arXiv 2023
[7]

Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiaoyong Wei, Shanru Lin, Hui Liu, Philip S

OpenReview.net. Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiaoyong Wei, Shanru Lin, Hui Liu, Philip S. Yu, and Qing Li. 2025. A sur- vey of webagents: Towards next-generation AI agents for web automation with large foundation models. CoRR, abs/2503.23350. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774. Guill...

arXiv 2025
[9]

In Findings of the As- sociation for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 12891–12907

Enhancing alignment using curriculum learn- ing & ranked preferences. In Findings of the As- sociation for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 12891–12907. Association for Computational Linguistics. Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Jiadai Sun, Xinyue Yang, Yu Yang, Shuntian 11 Y...

work page 2024
[10]

Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Morgane Rivière, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léona...

arXiv 2023
[11]

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increas- ing inference time. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Bal- timore, Maryland, USA, volume 162 of Proceedings 12 of Machine Learning Research, pages 23965–23998. PMLR. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Di...

arXiv 2022
[12]

Yangyang Zhao, Zhenyu Wang, and Zhenhua Huang

OpenReview.net. Yangyang Zhao, Zhenyu Wang, and Zhenhua Huang

work page
[13]

Automatic curriculum learning with over- repetition penalty for dialogue policy learning. In Thirty-Fifth AAAI Conference on Artificial Intelli- gence, AAAI 2021, Thirty-Third Conference on In- novative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2021, Vir- tual Even...

work page 2021
[14]

seed ": 42 ,

OpenReview.net. A Templates Table 8, 9, and 10 are prompt templates for action type selection, object selection, and memory gener- ation in our proposed LaMDAgent, respectively. Table 11 shows an example of configs for LaMDAgent. 13 Prompt template to select an action type You are a developer of Large Language Models ( LLMs ) who tests model improvement s...

work page 2024

Show all 14 references

[2021]

CoRR, abs/2110.14168

Training verifiers to solve math word prob- lems. CoRR, abs/2110.14168. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingx- uan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang L...

arXiv
[2022]

Training language models to follow instruc- tions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. Shishir G. Pat...

2022
[2023]

Skill-it! A data-driven skills framework for understanding and training language models. In Ad- vances in Neural Information Processing Systems 36: Annual Conference on Neural Information Process- ing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Ze...

2023
[2024]

InFindings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 15604–15621

Merge to learn: Efficiently adding skills to lan- guage models with model merging. InFindings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 15604–15621. Association for Computational Linguistics. Rémi Munos, Mich...

2024
[2025]

Associ- ation for Computational Linguistics

Can large language models invent algorithms to improve themselves? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, NAACL 2025 - Volume 1: Long Papers, Albuquerque, New...

2025 arXiv
[9366]

Association for Computational Linguistics. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [4]

Mind2web: Towards a generalist agent for the web. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Informa- tion Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jing...

arXiv 2023

[2] [7]

Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiaoyong Wei, Shanru Lin, Hui Liu, Philip S

OpenReview.net. Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan, Xiaoyong Wei, Shanru Lin, Hui Liu, Philip S. Yu, and Qing Li. 2025. A sur- vey of webagents: Towards next-generation AI agents for web automation with large foundation models. CoRR, abs/2503.23350. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774. Guill...

arXiv 2025

[3] [9]

In Findings of the As- sociation for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 12891–12907

Enhancing alignment using curriculum learn- ing & ranked preferences. In Findings of the As- sociation for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 12891–12907. Association for Computational Linguistics. Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Jiadai Sun, Xinyue Yang, Yu Yang, Shuntian 11 Y...

work page 2024

[4] [10]

Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Morgane Rivière, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léona...

arXiv 2023

[5] [11]

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increas- ing inference time. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Bal- timore, Maryland, USA, volume 162 of Proceedings 12 of Machine Learning Research, pages 23965–23998. PMLR. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Di...

arXiv 2022

[6] [12]

Yangyang Zhao, Zhenyu Wang, and Zhenhua Huang

OpenReview.net. Yangyang Zhao, Zhenyu Wang, and Zhenhua Huang

work page

[7] [13]

Automatic curriculum learning with over- repetition penalty for dialogue policy learning. In Thirty-Fifth AAAI Conference on Artificial Intelli- gence, AAAI 2021, Thirty-Third Conference on In- novative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2021, Vir- tual Even...

work page 2021

[8] [14]

seed ": 42 ,

OpenReview.net. A Templates Table 8, 9, and 10 are prompt templates for action type selection, object selection, and memory gener- ation in our proposed LaMDAgent, respectively. Table 11 shows an example of configs for LaMDAgent. 13 Prompt template to select an action type You are a developer of Large Language Models ( LLMs ) who tests model improvement s...

work page 2024