Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

An AI agent can act as a data scientist—generating, testing, and revising synthetic training data—and can itself be optimized to make that data better for learning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 12:06 UTC pith:HUCA6C6T

load-bearing objection Solid multi-domain systems paper: agentic weak–strong data loops beat matched CoT Self-Instruct, with a real meta-optimization lift—judge coupling is the main caveat, not a collapse of the claim. the 4 major comments →

arxiv 2606.25996 v3 pith:HUCA6C6T submitted 2026-06-24 cs.AI cs.CLcs.LG

Autodata: An agentic data scientist to create high quality synthetic data

Ilia Kulikov , Chenxi Whitehouse , Tianhao Wu , Yixin Nie , Swarnadeep Saha , Eryk Helenowski , Weizhe Yuan , Olga Golovneva
show 7 more authors
Jack Lanchantin Yoram Bachrach Jakob Foerster Xian Li Han Fang Sainbayar Sukhbaatar Jason Weston
This is my paper
classification cs.AI cs.CLcs.LG
keywords synthetic dataagentic systemsself-instructdata generationreinforcement learningmeta-optimizationweak-strong solverstraining data quality
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that synthetic training and evaluation data should be built the way a data scientist works: create candidates, inspect them, measure how they separate weaker and stronger solvers, extract lessons, and revise the generation recipe until the examples are useful for learning. The authors present Autodata as that general loop and Agentic Self-Instruct as a concrete version that uses a challenger, a weak solver, a strong solver, and a judge so that accepted examples are hard enough to train on but still solvable by a stronger system. Across computer-science research questions, legal reasoning, and mathematical-object reasoning, models trained on this agentic data beat equal-budget classical Self-Instruct-style data. Meta-optimizing the data-scientist agent’s own instructions further raises the rate at which it produces discriminating examples. The practical claim is that extra inference compute can be converted into higher-quality training data rather than only into better one-shot answers.

Core claim

The authors claim that an iterative agentic data-scientist loop produces synthetic examples with higher learning value than fixed-prompt Self-Instruct pipelines, because it explicitly tunes difficulty and quality to weak-versus-strong solver behavior and judge feedback; training small models with reinforcement learning on that data improves held-out performance on CS research rubrics, legal benchmarks, and scientific reasoning, and meta-optimizing the agent’s prompts further improves its success at creating such separating examples.

What carries the argument

Agentic Self-Instruct: a main agent that drives a challenger to propose examples, weak and strong solvers to attempt them, and a verifier/judge to score quality and discrimination, then rewrites the generation recipe until acceptance criteria are met; an outer meta-optimization loop treats the agent’s prompts as editable code and accepts changes only when validation weak–strong separation improves.

Load-bearing premise

The method assumes that weak–strong score gaps and LLM-judge verdicts are faithful signals of real learning utility, not artifacts of shared model bias, noise, or agents gaming the acceptance rules.

What would settle it

Hold the source corpus, model family, and training budget fixed, generate equal-sized CoT versus agentic datasets with independent judges and solvers (or human rubrics), retrain the same weak model, and check whether the agentic advantage on held-out tests and weak–strong separation disappears or reverses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Autodata, a general agentic framework in which an LLM acts as a data scientist that iteratively generates synthetic training/evaluation data, analyzes quality via solver behavior and judges, and revises its generation recipe. The main instantiation, Agentic Self-Instruct, uses a challenger plus weak and strong solvers and a verifier/judge so that accepted examples separate weak from strong performance (or are otherwise GRPO-suitable). Experiments on CS research QA (rubric-based), legal reasoning (PRBench-Legal / Legal-Hard), and Principia-style scientific reasoning report that equal-budget Agentic data yields stronger GRPO-trained Qwen3.5-4B models than classical CoT Self-Instruct; a further outer-loop meta-optimization of the CS agent prompts raises weak–strong separation pass rate from 62.1% to 79.6%. The authors argue this converts extra inference compute into higher-quality training data.

Significance. If the measured uplifts are not largely artifacts of shared judge/orchestrator bias, the work is a clear and timely contribution: it unifies Self-Instruct, grounded generation, filtering/evolution, and weak–strong / self-challenging ideas into an explicit data-scientist loop, and shows that the same loop can correct opposite failure modes (CoT data too easy on CS, too hard on legal). The controlled head-to-heads (matched example counts, same solvers and GRPO setup), dual GPT-5/Kimi grading on legal, and the meta-optimization result are concrete strengths. The framing—that data creation, not only architecture or training recipes, is a first-class target for agentic compute—is of broad interest to the post-training community.

major comments (4)
  1. [§3.1–3.3, Tables 2, 4–6] §3.1–3.3 and Tables 2, 5–6: For CS and scientific reasoning, the orchestrator, challenger, quality/loop signals, and train/eval reward model are all Kimi-K2.6 (or the same family), and acceptance/meta-optimization objectives are defined on those scores. Legal is partly insulated by independent GPT-5 re-grading (Table 4), but CS and Principia are not. The central claim that Agentic data is better training signal—not merely better at maximizing Kimi weak–strong gaps—therefore rests on a load-bearing assumption. Please either re-grade CS held-out and Principia rollouts with a disjoint strong judge (as done for legal), or add a clear ablation/limitation quantifying how much of the Agentic–CoT gap survives under an independent grader.
  2. [§3, Tables 2, 4–6] All RL student results use Qwen3.5-4B with Qwen3.5-397B as the strong solver. The paper’s claim that Agentic Self-Instruct produces higher-quality data for model training is therefore demonstrated only within one model family and one weak–strong pairing. A second student (different family or size) on at least one domain would substantially strengthen the result; if that is out of scope, the abstract and conclusion should state the single-student limitation more explicitly rather than generalizing to “higher quality model training.”
  3. [§3.1 Criteria, §3.2 loop judge, §4, §6] §6 already notes agents attempting to game the weak solver (e.g., prompt tricks). The CS acceptance thresholds (weak_avg ≤0.5 / ≥0.65 strong / gap ≥20 pp in generation; slightly different thresholds in meta-opt §4) and the legal loop-judge’s free-form grpo_suitability verdict are free parameters of the method. The manuscript should report sensitivity of accepted-set statistics and downstream RL to modest threshold changes, or freeze thresholds before the main corpus run and document that protocol, so that reported gains are not confounded with post-hoc criterion tuning.
  4. [Abstract, §4, Table 7] Meta-optimization (§4, Table 7) is reported only for CS pass rate under the same weak–strong rubric criteria used in the inner loop, not as a full outer-loop improvement of downstream RL reward on held-out tests. The abstract’s claim that meta-optimizing the data scientist “delivers an even larger performance uplift” is therefore stronger than the evidence (pass-rate 62.1%→79.6%, not a second RL table). Either run a small RL comparison with baseline vs meta-optimized agent data, or rephrase the abstract/conclusion to match what Table 7 actually measures.
minor comments (5)
  1. [Figure 4] Figure 1 and Figure 2 captions are clear, but Figure 4’s trajectory example is hard to read in the manuscript text; ensure the final PDF renders the multi-round progression at readable font size.
  2. [Appendix C] Appendix C system prompts are valuable for reproducibility; a short main-text pointer to which acceptance criteria live in code vs. free-form judge text would help readers who skip the appendix.
  3. [§1, §5] Related Work cites a concurrent “AutoData” multi-agent web-collection system (Ma et al., 2026); a one-sentence naming distinction in the introduction would reduce title collision confusion.
  4. [Tables 1, 3] Table 1 vs Table 3: reporting weak-rollout std only for legal is fine, but adding the same statistic for CS would make the “just right” narrative fully parallel.
  5. [Front matter] Date line “July 7, 2026” and arXiv versioning are fine for the preprint; ensure camera-ready uses the journal’s date style.

Circularity Check

0 steps flagged

Empirical methods paper: no derivation reduces to its inputs by construction; claims are measured RL outcomes on held-out and external benchmarks.

full rationale

Autodata / Agentic Self-Instruct is an empirical ML systems paper. Its load-bearing claims are that equal-budget Agentic Self-Instruct data yields stronger GRPO-trained Qwen3.5-4B models than CoT Self-Instruct on held-out CS rubrics (Table 2), PRBench-Legal / Legal-Hard (Table 4, with dual GPT-5 and Kimi grading), and Principia-style scientific reasoning plus the external Principia benchmark (Tables 5–6), and that meta-optimizing the agent harness raises CS weak–strong separation pass rate from 62.1% to 79.6% (Table 7). None of these steps is a first-principles derivation, fitted identity, uniqueness theorem, or renamed known result. Acceptance criteria (weak/strong gaps, loop-judge GRPO-suitability) and the meta-optimizer’s objective are design choices that define what data is kept; they do not algebraically force the downstream RL numbers. Self-citations (Self-Instruct, CoT Self-Instruct, Self-Challenging, Absolute Zero, SPICE, Meta-Harness, etc.) are related-work positioning, not load-bearing uniqueness imports that forbid alternatives. Shared Kimi-K2.6 roles as orchestrator/challenger/judge and optimization against the same weak–strong proxy raise validity and reward-hacking concerns (authors note prompt-gaming in §6), but those are confounding/proxy-fidelity issues, not circularity by construction. No quote exhibits Eq. X = Eq. Y by definition or a fitted parameter renamed as prediction. Score 0 is the honest finding.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 2 invented entities

Load-bearing content is mostly methodological choices and empirical thresholds, not physical free parameters. The central claim rests on the weak–strong learning-utility axiom, fixed acceptance/judge policies, and the assumption that LLM rubric judges track true quality. No new physical entities; invented constructs are software roles and loops.

free parameters (3)
  • CS acceptance thresholds (weak_avg, strong_avg, gap)
    Hand-chosen gates (e.g. weak <0.5, strong ≥0.65, gap ≥20pp in generation; related thresholds in meta-opt) that define which examples enter training; not derived from first principles.
  • GRPO / training hyperparameters
    Learning rate, batch size, group size, rollout counts, and step budgets are chosen settings that affect reported lifts.
  • Meta-optimizer temperature and accept rule
    Boltzmann T=0.1 and strict validation improvement for population acceptance shape which prompt diffs survive.
axioms (3)
  • domain assumption Examples that separate a weak solver from a strong solver (or that a loop-judge deems GRPO-suitable) provide a better learning signal than unfiltered CoT Self-Instruct examples.
    Core design hypothesis of Agentic Self-Instruct; supported empirically in the paper but not proved generally.
  • domain assumption LLM-as-judge rubric scores are sufficiently reliable for both data acceptance and RL reward.
    Used throughout CS/legal/scientific experiments; partially stress-tested via GPT-5 re-grading on legal only.
  • standard math Standard GRPO and instruction-following LLM behavior as in cited prior work.
    Training and evaluation rest on established RLVR / LLM tooling assumptions rather than new theory.
invented entities (2)
  • Autodata / Agentic Self-Instruct agent loop no independent evidence
    purpose: Orchestrate challenger, weak/strong solvers, and judge to iteratively produce training/eval data.
    Software architecture introduced by the paper; evaluated only via downstream model scores, not independent physical evidence.
  • Outer meta-optimizer over data-scientist prompts no independent evidence
    purpose: Evolve agent harness/prompts using the same weak–strong separation objective.
    New optimization object in this work; success measured on held-out papers under the same proxy metric.

pith-pipeline@v1.1.0-grok45 · 31035 in / 3203 out tokens · 33521 ms · 2026-07-12T12:06:48.653369+00:00 · methodology

0 comments
read the original abstract

We introduce Autodata, a general method that enables AI agents to act as data scientists who build high quality training and evaluation data. We show how to train (meta-optimize) such a data scientist agent, so that it learns to create even stronger data. We describe the overall formulation, and a specific practical implementation, Agentic Self-Instruct. We conduct experiments on computer science research tasks, legal reasoning tasks and reasoning with mathematical objects, where we obtain improved results compared to classical synthetic dataset creation methods. Further, meta-optimizing the data scientist agent itself delivers an even larger performance uplift. Agentic data creation provides a way to convert increased inference compute into higher quality model training. Overall, we believe this direction has the potential to change the way we build AI data.

Review history (3 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A Vocabulary for Multi-Agent Automated Research Systems

    cs.AI 2026-07 accept novelty 6.0

    A new 8-part tuple formally describes any multi-agent research automation system so that design choices can be isolated and compared.

  2. Execution-First Synthetic Tool-Use Trace Generation for LLM Agents

    cs.SE 2026-07 conditional novelty 5.0

    An execution-first data generator that validates tool traces before synthesizing user tasks improves Qwen agents' tool use and answer quality, with masked reasoning supervision beating full-trace supervision.

Reference graph

Works this paper leans on

38 extracted references · 15 linked inside Pith · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Reasoning over mathematical objects: on-policy reward modeling and test time aggregation

    Pranjal Aggarwal, Marjan Ghazvininejad, Seungone Kim, Ilia Kulikov, Jack Lanchantin, Xian Li, Tianjian Li, Bo Liu, Graham Neubig, Anaelia Ovalle, et al. Reasoning over mathematical objects: on-policy reward modeling and test time aggregation. arXiv preprint arXiv:2603.18886, 2026

    arXiv 2026

  3. [3]

    Gepa: Reflective prompt evolution can outperform reinforcement learning

    Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, et al. Gepa: Reflective prompt evolution can outperform reinforcement learning. arXiv preprint arXiv:2507.19457, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Prbench: Large-scale expert rubrics for evaluating high-stakes professional reasoning

    Afra Feyza Aky \"u rek, Advait Gosai, Chen Bo Calvin Zhang, Vipul Gupta, Jaehwan Jeong, Anisha Gunjal, Tahseen Rabbani, Maria Mazzone, David Randolph, Mohammad Mahmoudi Meymand, et al. Prbench: Large-scale expert rubrics for evaluating high-stakes professional reasoning. arXiv preprint arXiv:2511.11562, 2025

    arXiv 2025

  5. [5]

    Ultrafeedback: Boosting language models with high-quality feedback

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Qiang Yue, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with high-quality feedback. 2023

    2023

  6. [6]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3029--3051, 2023

    2023

  7. [7]

    Promptbreeder: Self-referential self-improvement via prompt evolution

    Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Ds-agent: Automated data science by empowering large language models with case-based reasoning

    Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. Ds-agent: Automated data science by empowering large language models with case-based reasoning. arXiv preprint arXiv:2402.17453, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset

    Peter Henderson, Mark Krass, Lucia Zheng, Neel Guha, Christopher D Manning, Dan Jurafsky, and Daniel Ho. Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset. Advances in Neural Information Processing Systems, 35: 0 29217--29234, 2022

    2022

  10. [10]

    Data interpreter: An llm agent for data science

    Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, et al. Data interpreter: An llm agent for data science. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19796--19821, 2025

    2025

  11. [11]

    autoresearch: Ai agents running research on single-gpu nanochat training automatically

    Andrej Karpathy. autoresearch: Ai agents running research on single-gpu nanochat training automatically. https://github.com/karpathy/autoresearch, 2026. GitHub repository

    2026

  12. [12]

    Meta-harness: End-to-end optimization of model harnesses

    Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-harness: End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052, 2026

    Pith/arXiv arXiv 2026

  13. [13]

    Textbooks are all you need ii: phi-1.5 technical report

    Yuanzhi Li, S \'e bastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Spice: Self-play in corpus environments improves reasoning

    Bo Liu, Chuanyang Jin, Seungone Kim, Weizhe Yuan, Wenting Zhao, Ilia Kulikov, Xian Li, Sainbayar Sukhbaatar, Jack Lanchantin, and Jason Weston. Spice: Self-play in corpus environments improves reasoning. arXiv preprint arXiv:2510.24684, 2025

    arXiv 2025

  15. [15]

    S2orc: The semantic scholar open research corpus

    Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel S Weld. S2orc: The semantic scholar open research corpus. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4969--4983, 2020

    2020

  16. [16]

    The ai scientist: Towards fully automated open-ended scientific discovery

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Source2synth: Synthetic data generation and curation grounded in real data sources

    Alisia Lupidi, Carlos Gemmell, Nicola Cancedda, Jane Dwivedi-Yu, Jason Weston, Jakob Foerster, Roberta Raileanu, and Maria Lomeli. Source2synth: Synthetic data generation and curation grounded in real data sources. arXiv preprint arXiv:2409.08239, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Autodata: A multi-agent system for open web data collection

    Tianyi Ma, Yiyue Qian, Zheyuan Zhang, Zehong Wang, Xiaoye Qian, Feifan Bai, Yifan Ding, Xuwei Luo, Shinan Zhang, Keerthiram Murugesan, et al. Autodata: A multi-agent system for open web data collection. Advances in Neural Information Processing Systems, 38: 0 173416--173448, 2026

    2026

  19. [19]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems, 36: 0 46534--46594, 2023

    2023

  20. [20]

    Agentinstruct: Toward generative teaching with agentic flows

    Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-ge Chen, Olga Vrousgos, Corby Rosset, et al. Agentinstruct: Toward generative teaching with agentic flows. arXiv preprint arXiv:2407.03502, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Orca: Progressive learning from complex explanation traces of gpt-4

    Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707, 2023

    Pith/arXiv arXiv 2023

  22. [22]

    Ai-assisted generation of difficult math questions

    Vedant Shah, Dingli Yu, Kaifeng Lyu, Simon Park, Jiatong Yu, Yinghui He, Nan Rosemary Ke, Michael Mozer, Yoshua Bengio, Sanjeev Arora, et al. Ai-assisted generation of difficult math questions. arXiv preprint arXiv:2407.21009, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Deepseekmath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the Limits of Mathematical Reasoning in Open Language Models . arXiv preprint arXiv:2402.03300, 2024. URL https://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  24. [24]

    Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data

    Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data. In International Conference on Learning Representations, volume 2025, pages 19243--19275, 2025

    2025

  25. [25]

    Self-instruct: Aligning language models with self-generated instructions

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), pages 13484--13508, 2023

    2023

  26. [26]

    Ai & human co-improvement for safer co-superintelligence

    Jason Weston and Jakob Foerster. Ai & human co-improvement for safer co-superintelligence. arXiv preprint arXiv:2512.05356, 2025

    arXiv 2025

  27. [27]

    Wizardlm: Empowering large pre-trained language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. Wizardlm: Empowering large pre-trained language models to follow complex instructions. In International Conference on Learning Representations, volume 2024, pages 30745--30766, 2024

    2024

  28. [28]

    Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing. In International Conference on Learning Representations, volume 2025, pages 76346--76382, 2025

    2025

  29. [29]

    Large language models as optimizers

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In International Conference on Learning Representations, volume 2024, pages 12028--12068, 2024

    2024

  30. [30]

    Metamath: Bootstrap your own mathematical questions for large language models

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. In International Conference on Learning Representations, volume 2024, pages 45040--45061, 2024

    2024

  31. [31]

    Cot-self-instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks

    Ping Yu, Jack Lanchantin, Tianlu Wang, Weizhe Yuan, Olga Golovneva, Ilia Kulikov, Sainbayar Sukhbaatar, Jason Weston, and Jing Xu. Cot-self-instruct: Building high-quality synthetic prompts for reasoning and non-reasoning tasks. arXiv preprint arXiv:2507.23751, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Self-rewarding language models

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. Self-rewarding language models. In Forty-first International Conference on Machine Learning, 2024

    2024

  33. [33]

    Naturalreasoning: Reasoning in the wild with 2.8 m challenging questions

    Weizhe Yuan, Jane Yu, Song Jiang, Karthik Padthe, Yang Li, Ilia Kulikov, Kyunghyun Cho, Dong Wang, Yuandong Tian, Jason E Weston, et al. Naturalreasoning: Reasoning in the wild with 2.8 m challenging questions. arXiv preprint arXiv:2502.13124, 2025

    arXiv 2025

  34. [34]

    Mammoth: Building math generalist models through hybrid instruction tuning

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. In International Conference on Learning Representations, volume 2024, pages 40320--40341, 2024

    2024

  35. [35]

    Star: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35: 0 15476--15488, 2022

    2022

  36. [36]

    Absolute zero: Reinforced self-play reasoning with zero data

    Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.03335, 2025 a

    Pith/arXiv arXiv 2025

  37. [37]

    The majority is not always right: Rl training for solution aggregation

    Wenting Zhao, Pranjal Aggarwal, Swarnadeep Saha, Asli Celikyilmaz, Jason Weston, and Ilia Kulikov. The majority is not always right: Rl training for solution aggregation. arXiv preprint arXiv:2509.06870, 2025 b

    Pith/arXiv arXiv 2025

  38. [38]

    Self-challenging language model agents

    Yifei Zhou, Sergey Levine, Jason Weston, Xian Li, and Sainbayar Sukhbaatar. Self-challenging language model agents. arXiv preprint arXiv:2506.01716, 2025

    Pith/arXiv arXiv 2025