Pith. sign in

REVIEW 4 major objections 7 minor 43 references

Language Models as Continuous Self-Evolving Data Engineers

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A language model can act as its own data engineer, generating, reviewing, and preference-annotating new training data in a loop, and in experiments this lifts Qwen2-7B's average benchmark score by 3.64 points and Qwen2-7B-Instruct's by…

desk verdict A plausible self-training pipeline with modest gains, undercut by unvalidated self-rewards and an ablation that doesn't cleanly support the full design. read the letter →

arxiv 2412.15151 v3 pith:OCV6PYIZ submitted 2024-12-19 cs.CL cs.AI

Peidong Wang , Ming Wang , Zhiming Ma , Xiaocui Yang , Shi Feng , Daling Wang , Yifei Zhang , Kaisong Song This is my paper · ORCID
classification cs.CLcs.AI
keywords LANCEself-evolvinglanguagemodelssyntheticdatagenerationfilteringdirectpreferenceoptimizationinstructiontuningLLM-as-judgeself-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes LANCE, a training paradigm in which the language model itself generates, cleans, reviews, and preference-annotates its own post-training data, and shows on Qwen2-7B and Qwen2-7B-Instruct that iterating this cycle improves average benchmark scores by 3.64 and 1.75 points respectively. The point of the claim is that high-quality expert-labeled data is the limiting resource for further LLM progress; if a model can make its own data, the bottleneck moves from annotation cost to the model's own generative and evaluative capacity. The experiments run four iterations and compare LANCE with self-instruction, self-play, and self-alignment baselines, finding continuous gains where the baselines stall or decline. A distinctive result is that the gains concentrate in mathematical reasoning, including transfer to Latin-script multilingual math, even though the seed data is general-purpose.

What carries the argument

The load-bearing object is the review score $\bar{S}_i$ that model $M_t$ assigns to each data item against a hand-written constitution (clarity, usefulness, challenge, safety, professionalism, guidance). Items with $\bar{S}_i < V = 7$ are treated as low-quality and trigger few-shot chain-of-thought generation of new instruction-response pairs for SFT; items with $\bar{S}_i \geq V$ are treated as high-quality and trigger generation of deliberately flawed responses, which are scored and paired with the original as preference data. Cleaning uses length constraints and ROUGE-L similarity filtering, and training alternates negative log-likelihood loss with the preference-driven likelihood ratio loss of DPO, where the implicit reward is $\hat{r}(x,y) = \beta \log \frac{M^\theta_t(y\mid x)}{M^S_t(y\mid x)}$, after which the fine-tuned model becomes $M_{t+1}$. The same model thus serves as generator, judge, cleaner, and annotator, closing the loop.

What would settle it

Take a held-out sample of LANCE-generated instruction and preference pairs, have human experts rate them, and compare those ratings with the constitution-based self-review scores; then train one model on data selected by the self-review scores and another on data selected by the human ratings. If the self-review selection does not beat random selection, or agrees only weakly with human ratings, the claimed self-evolution signal is not doing the work.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a single LLM can run the full post-training data construction lifecycle without human experts or an external reward model: it reviews seed data against a constitution, splits examples by a threshold, generates new instruction data for weak spots and adversarial preference pairs for strong examples, filters and re-scores them, then fine-tunes itself with SFT followed by DPO. In the reported experiments this loop produces monotone or near-monotone average gains over four iterations on Qwen2-7B and a late-iteration jump on Qwen2-7B-Instruct, with the largest gains in math (GSM8K up 21.15 on Qwen2-7B), and it is the only tested method that improves on every iteration round. The paper interprets this as evidence that self-evolution can extend beyond the seed distribution rather than collapsing toward it, and that SFT and DPO play synergistic roles: SFT stabilizes, while DPO accelerates and sustains improvement.

Load-bearing premise

The whole loop depends on the model's own review scores genuinely separating good data from bad; if those scores are miscalibrated, filtering and preference-pairing would amplify the model's existing mistakes instead of correcting them.

Editorial extensions

If this is right

  • If LANCE's results generalize, self-evolution can push a model beyond its original seed-data distribution, unlike methods that the paper says converge back toward the seed distribution.
  • The average benchmark gains persist across four iterations on Qwen2-7B, with the final iteration on Qwen2-7B-Instruct reaching the highest performance, suggesting further iterations could yield more.
  • The largest measured gains are in math reasoning (GSM8K up 21.15 on Qwen2-7B), and the gains transfer to Latin-script multilingual math even though the seed data is English-only and general-purpose.
  • Ablations indicate that neither SFT nor DPO alone suffices: removing SFT makes iteration unstable, while removing DPO slows improvement and lets it decline by the fourth iteration.
  • The paper's own limitation is that knowledge-dependent tasks improve only slightly, because self-evolved data cannot introduce knowledge beyond what the model already knows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a direct stress test would replace the model's self-review scores with an independent reward model on the same generated data and rerun the loop; if gains persist, the constitution-based self-judgment is not essential, and if they vanish, it is doing the causal work.
  • Beyond the paper: the reported multilingual math transfer covers Latin-script languages, so a natural extension is to test non-Latin scripts such as Chinese or Arabic, where the paper makes no claim of transfer.
  • Beyond the paper: since the paper says knowledge-dependent abilities are not improved, a promising extension would inject retrieval or fact-checking into the data-generation step, letting the model verify content before scoring it.
  • Beyond the paper: the threshold $V=7$ and the hand-written constitution are free parameters; varying the threshold across a grid and measuring downstream benchmark performance would show how sensitive the loop is to the review calibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes LANCE, an iterative post-training paradigm in which an LLM carries out the full data-construction cycle for its own continued training. Starting from a small seed set of instruction-response pairs plus a separately annotated review set, the model M_t (i) reviews each seed example against a hand-written 'constitution' and produces 0-10 scores with rationales (Eq. 4); (ii) for items scoring below threshold V=7, generates K new instructions and responses via few-shot and chain-of-thought prompting, and for items at or above V, generates deliberately flawed responses; (iii) filters generated data by length and ROUGE-L similarity; (iv) retains instruction data whose review score exceeds V and constructs preference pairs by comparing self-scores of the original and flawed responses (Eq. 7); and (v) trains by SFT followed by DPO, repeating for four iterations. LANCE is evaluated on Qwen2-7B and Qwen2-7B-Instruct over six Open LLM Leaderboard tasks plus MATH, Olympiad Bench, MGSM, and Minerva Math, against SFT, Self-Instruct, SPIN, and I-SHEEP. The reported results show average gains of 3.64 and 1.75 points over the SFT baseline at the best (final) iteration, concentrated in mathematical reasoning, e.g., GSM8K +21.15 on Qwen2-7B.

Significance. Should the empirical claims withstand the robustness checks described below, the paper would be a useful data point for the self-evolving LLM literature: the idea of closing the loop over generation, cleaning, review, and preference construction is coherent, the fully reported per-iteration trajectories (Tables 2, 7, 8) and released code make the results checkable, and the math-reasoning gains (GSM8K +21.15, MATH +6.38, cross-lingual MGSM transfer from an English-only general seed) are genuinely interesting. The paper also names its own limitations explicitly in Section 6: knowledge gains are bounded by the model's existing knowledge, and the computational overhead is significant. The significance is tempered, however, by three addressable gaps: the review signal that drives the whole pipeline is never validated against human judgments or an independent reward model; the ablation evidence indicates the SFT-only variant is competitive or better in the first iterations, so the preference-learning component's contribution is not established; and all conclusions rest on single runs without error bars.

major comments (4)
  1. [Section 4.4, Tables 3 and 8] The ablation evidence does not support the conclusion that the DPO/preference component is necessary for the reported gains. Taking the average rows at face value, removing DPO yields higher scores than the full pipeline in Iterations 1 and 2 (66.89 vs 65.58; 67.08 vs 65.65), a near-tie in Iteration 3 (67.85 vs 67.92), and the full pipeline's only clear advantage is 0.45 points at Iteration 4; the claimed 'decline' of the SFT-only variant at Iteration 4 is a change of 0.06 points (67.85 to 67.79), which is within run-to-run noise. Table 8 likewise shows the DPO stage lowering the average relative to the preceding SFT stage in three of four iterations (66.89 to 65.58; 67.94 to 65.65; 68.09 to 67.92), with a +0.04 gain in Iteration 4. Consequently the headline average score enhancement of 3.64 cannot be attributed to the preference-learning component, and the claims that DPO accelerates performance gains and is essential for sustaining long-term improvements require multi-seed comparisons showing a statistically reliable advantage at the later iterations before they can be accepted.
  2. [Sections 3.3.1-3.3.3 and 4.1, Eqs. (4)-(7)] The constitution-based review score is the pipeline's only quality signal: it determines the V=7 filter that admits generated instruction data (Section 3.3.3) and orders the preference pairs in Eq. (7). The paper never validates this signal against human judgments or an independent reward model, and the initial review capability is bootstrapped from an external model plus human labels (Section 4.1: Llama3-70B generates the reward rationales and scores for the review seed, retaining only items consistent with the human-labeled OpenAssistant scores). The Introduction's and Abstract's framing about eliminating the need for human involvement or external models is therefore stronger than the setup warrants, and the loop is at risk of self-confirmation: the same model generates, scores, filters, and is trained on its own data, so high benchmark scores could in principle reflect selection for the model's own stylistic preferences rather than for data quality. The paper would be substantially strengthened by a concrete validation: report per-iteration agreement between M_t's scores and held-out human scores or an independent judge, and ablate the reward by replacing the V=7 filter with volume-matched random selection to show that the review signal, rather than the extra training volume, is what drives the gains.
  3. [Section 4.2, Tables 1 and 7, Figure 3] The central empirical claims rest on single runs without seeds, error bars, or significance tests, and several adjacent-iteration differences are extremely small: on Qwen2-7B the Iteration-1-to-2 average gain is 0.07 (65.58 to 65.65), and on Qwen2-7B-Instruct the average declines in Iteration 2 (68.72 to 68.36), so the caption claim that LANCE consistently shows performance gains across iterations holds only for the base model. The comparison with baselines is also endpoint-dependent: on Qwen2-7B, LANCE's average is below SPIN's in Iterations 1 and 2 (65.58 and 65.65 vs 68.00 and 67.86) and only overtakes SPIN from Iteration 3, so 'outperforming other self-evolution methods' describes the best and final iterations, not the trajectory as a whole. Per-benchmark, ARC-C remains below the SFT starting point at every LANCE iteration (51.11 to 50.68, Table 7), which also contradicts the Section 1 claim that individual metrics remain stable or show improvement. At minimum, please report seeds and error bars for the four-iteration trajectories and the main ablation, and report the variance associated with the GSM8K jump from 66.64 at Iteration 2 to 80.14 at Iteration 3, since a single-iteration jump of this size currently carries much of the headline result.
  4. [Appendix E (Table 8), Appendix F (Table 9), Section 4.4 (Table 3)] The ablation tables contain inconsistencies that prevent an unambiguous reconstruction of the ablation trajectory. Table 3 lists the SFT starting-point average as 61.42, whereas Tables 1, 7, and 9 all report 64.60. In Table 8, the SFT rows' per-benchmark scores are swapped between the two models (the Qwen2-7B row carries the Instruct model's benchmark scores and vice versa), and the SFT-stage averages in Table 8 (66.89, 67.94, 68.09, 68.20) do not match the corresponding 'w/o dpo' averages in Table 9 (66.89, 67.08, 67.85, 67.79) from Iteration 2 onward, even though the two tables are meant to describe the same SFT-only stages. In Table 9, the LANCE Iter1 row appears to have the GSM8K and Winogrande columns swapped relative to Tables 2 and 7 (GSM8K shown as 72.38 instead of 67.32). These numbers need to be reconciled before the ablation-based conclusions in Section 4.4 can be evaluated.
minor comments (7)
  1. [Section 3.3.1] The text states that a regular expression f extracts the scores from the review output, but the review format and the regular expression are never specified; please provide both for reproducibility.
  2. [Section 4.2] The sentence 'we conducted four rounds of iterative experiments with N = 5' is ambiguous; given that the methodology defines the loop from t=0 to t=N, please clarify the intended N and whether the stopping point was fixed in advance.
  3. [Table 2] The column headers are garbled ('MA TH', 'MGSM_latin', 'Minerva Mathde sw fr average'); please reformat the table and state precisely which subtasks the MGSM and Minerva columns contain and how the Average column is computed.
  4. [Tables 2 and 7] Labels such as 'LANCEIter1' lack spaces, and more importantly, the per-iteration tables (Tables 7, 8, and 9) should use one fixed column order to avoid the column-swap confusion noted in the major comments.
  5. [Abstract and Section 6] The Abstract claims LANCE significantly reduces the time and cost of post-training data construction, but no wall-clock, GPU-hour, or cost measurements are provided anywhere; given the Limitation section's acknowledgement of significant computational overhead, please either quantify the cost or qualify the claim.
  6. [Sections 3.3.3 and 4.1] Please report the number of generated instructions, the number of preference pairs, and the pass rates of the length, similarity, and V=7 filters at each iteration, so that readers can assess whether gains scale with data volume or with the filtering mechanism.
  7. [Figure 3] The curves and axis labels are not legible at print size in the submitted rendering; please regenerate the figure at higher resolution.

Circularity Check

1 steps flagged · score 3.0 of 10

Self-referential preference construction is partially circular, but the headline benchmark gains are externally grounded.

  1. self definitional [Section 3.3.3, Eq. 7 (with Eq. 4); abstract claim of human-preference alignment]
    "Let ¯S a i and ¯S b i denote the reward values of the two responses for the i-th instruction. If ¯S a i > ¯S b i , the first item is used as the preferred response and the second as the dispreferred response; otherwise, the roles are reversed."

    Here ¯S_i = f(M_t(x_i, y_i; C)) is the same model's own review score from Eq. 4, so the DPO label y_w is defined as whichever response the model itself scores higher. Eq. 2 then trains the model to increase the likelihood ratio of y_w over y_l. The preference signal is therefore the model's own reward by construction, not an independent human judgment. The abstract's assertion that LANCE 'ensures that the data aligns with human preferences' and the introduction's promise of 'accurate preference pairs' are not derived from any external measurement; they are definitionally satisfied by the self-score ordering. Thus the preference-alignment sub-claim is self-confirming, although the headline benchmark improvements are still evaluated on external test sets and retain independent content.

full rationale

I found no load-bearing self-citation chain and no fitted parameter renamed as a prediction; the central empirical claim, the 3.64/1.75 average gains on the Open LLM Leaderboard suite, is measured on external benchmarks and is therefore not circular. The one genuinely self-definitional step is the DPO data construction: Eq. 7 orders preference pairs by the model's own Eq. 4 scores, so the model is trained to prefer what it already scores higher. That makes the 'aligns with human preferences' claim tautological at the level of data construction, but it does not by itself force the external benchmark outcome, which is why the score is moderate rather than high. I also considered the ablation discrepancy in Table 3 and the Llama3-70B bootstrap of the review seed data in Section 4.1; these are empirical consistency and autonomy-overclaim concerns, not circular reductions, so they are not counted in the score.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on the self-review loop being a reliable quality signal, plus several standard assumptions about seed data and benchmarks. No new physical or mathematical entities are introduced.

free parameters (6)
  • Reward threshold V = 7.0
    Splits seed and generated data into low/high quality; hand-set in Section 3.3.2 based on empirical observation, no tuning procedure described.
  • Sample count K = 4
    Number of new instructions and flawed responses generated per seed example (Section 3.3.2, Appendix C).
  • DPO beta = 0.2
    Strength of preference optimization relative to the SFT reference model (Section 3.1.2, Appendix C).
  • Sampling temperature and top-p = 0.7, 0.9
    Controls diversity of generated data (Appendix C).
  • Training learning rates = SFT 3e-5; DPO 5e-6
    Fine-tuning hyperparameters (Appendix C).
  • Iteration count N = 5 (results through Iter4)
    Stopping criterion; the paper reports Iter1-4 due to compute limits (Section 4.2).
assumptions (5)
  • domain assumption Self-review scores from the constitution are a valid measure of data quality and correlate with downstream usefulness.
    The pipeline filters all training data and constructs preference pairs using Mt's own scores (Sections 3.3.1-3.3.3); this is not validated against human judgments or an independent reward model.
  • domain assumption The small seed set (8,816 examples) is sufficiently representative for broad capability improvement via self-generation.
    Seed is sampled from UltraChat and OpenAssistant (Section 4.1); no analysis of topic coverage limits.
  • domain assumption Llama3-70B-generated reward rationales retained after consistency filtering are accurate enough to initialize the reviewer.
    Section 4.1 uses Llama3-70B to label seed review data; this external dependence is excluded from the 'autonomous' claim.
  • standard math Benchmark evaluations are correctly configured and free of contamination.
    Evaluation uses LM Evaluation Harness, OpenCompass, and Qwen2.5-Math with version numbers (Appendix B); contamination is not assessed.
  • domain assumption ROUGE-L similarity filtering preserves necessary diversity and does not remove beneficial novel data.
    Section 3.3.3 applies ROUGE-L to remove similar data; no analysis of what is discarded.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language Models as Continuous Self-Evolving Data Engineers." pith.science (2026). https://pith.science/paper/OCV6PYIZ

@misc{pith2026241215151,
  author       = {Pith},
  title        = {Pith review of: Language Models as Continuous Self-Evolving Data Engineers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OCV6PYIZ}},
  note         = {Machine review of arXiv:2412.15151}
}
read the original abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities on various tasks, while the further evolvement is limited to the lack of high-quality training data. In addition, traditional training approaches rely too much on expert-labeled data, setting a ceiling on the performance of LLMs. To address this issue, we propose a novel paradigm named LANCE (LANguage models as Continuous self-Evolving data engineers) that enables LLMs to train themselves by autonomously generating, cleaning, reviewing, and annotating data with preference information. Our approach demonstrates that LLMs can serve as continuous self-evolving data engineers, significantly reducing the time and cost of the post-training data construction. Through iterative fine-tuning on Qwen2 series models, we validate the effectiveness of LANCE across various tasks, showing that it can maintain high-quality data generation and continuously improve model performance. Across multiple benchmark dimensions, LANCE results in an average score enhancement of 3.64 for Qwen2-7B and 1.75 for Qwen2-7B-Instruct. This training paradigm with autonomous data construction not only reduces the reliance on human experts or external models but also ensures that the data aligns with human preferences, paving the way for the development of future superintelligent systems that can exceed human capabilities. Codes are available at: https://github.com/Control-derek/LANCE.

Figures

Figures reproduced from arXiv: 2412.15151 by the authors.

Figure 1
Figure 1. An illustration of our methodology. Tradi￾tional ML focuses on the setting where humans super￾vise models that are weaker than humans. Our methodol￾ogy explores the scenario where models self-supervise, which may be a reliable path to superintelligence. attributed to instruction tuning (Wei et al., 2022a) and Reinforcement Learning from Human Feed￾back (RLHF) (Stiennon et al., 2020), which have significantly improve… view at source ↗
Figure 2
Figure 2. Overview of LANCE. The cycle begins at t = 0 with pre-annotated seed dataset Seed0. At each time step t, model Mt generates new instruction and preference data from Seedt via Post-training data construction full-cycle. Mt is fine-tuned on instruction data (NLL) to create MS t , then on preference data (PLR) to produce MD t . In the next iteration, MD t becomes Mt+1, and new samples are merged into Seedt to form Seed… view at source ↗
Figure 3
Figure 3. Various self-evolution methods show av￾erage scores across benchmarks. The Self-Instruct method, without iterative processes, sampled 50k exam￾ples for self-training. "Iter t" denotes the t-th iteration. Bench (He et al., 2024), MGSM (Shi et al., 2023), and Minerva Math (Lewkowycz et al., 2022). The specific evaluation settings for these benchmarks are detailed in Appendix B. Additionally, Appendix C details the hyp… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of the distribution of seed data [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: An example of SFT data generation based on seed data using [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: An example of DPO data generation based on seed data using [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 1 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

    arXiv 2020

  3. [3]

    Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, et al. 2023. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312.09390

    arXiv 2023

  4. [4]

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. 2024 a . https://doi.org/10.48550/arXiv.2307.08701 AlpaGasus : Training A Better Alpaca with Fewer Data . arXiv preprint. ArXiv:2307.08701 [cs]

    arXiv doi:10.48550/arxiv.2307.08701 2024

  5. [5]

    Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024 b . https://doi.org/10.48550/arXiv.2401.01335 Self- Play Fine-Tuning Converts Weak Language Models to Strong Language Models . arXiv preprint

    arXiv doi:10.48550/arxiv.2401.01335 2024

  6. [6]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/1803.05457

    arXiv 2018

  7. [7]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . Preprint, arXiv:2110.14168

    arXiv 2021

  8. [8]

    Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Lichao Sun, Quanzheng Li, Dinggang Shen, Tianming Liu, and Xiang Li. 2023. https://doi.org/10.48550/arXiv.2302.13007 AugGPT : Leveraging ChatGPT for Text Data Augmentation . arXiv preprint. ArXiv:2...

    arXiv doi:10.48550/arxiv.2302.13007 2023

Show all 43 references
  1. [9]

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233

    2023 arXiv

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. 2023. https://doi.org/10.48550/arXiv.2308.08998 R...

    2023 doi arXiv

  4. [12]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  5. [13]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874

    2021 arXiv

  6. [14]

    o pf, Yannic Kilcher, Dimitri von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri von R \"u tte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich \'a rd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. Advances in Neura...

    2024

  7. [15]

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. 2024. https://arxiv.org/abs/2205.13147 Matryoshka representation learning . Preprint, arXiv:2205.13147

    2024 arXiv

  8. [16]

    Mahoney, Kurt Keutzer, and Amir Gholami

    Nicholas Lee, Thanakul Wattanawong, Sehoon Kim, Karttikeya Mangalam, Sheng Shen, Gopala Anumanchipali, Michael W. Mahoney, Kurt Keutzer, and Amir Gholami. 2024. https://doi.org/10.48550/arXiv.2403.15042 LLM2LLM : Boosting LLMs with Novel Iterative Data Enhancement . arXiv prep...

    2024 doi arXiv

  9. [17]

    Minzhi Li, Taiwei Shi, Caleb Ziems, Min-Yen Kan, Nancy F Chen, Zhengyuan Liu, and Diyi Yang. 2023. Coannotating: Uncertainty-guided work allocation between human and large language models for data annotation. arXiv preprint arXiv:2310.15638

    2023 arXiv

  10. [18]

    Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. 2024. https://doi.org/10.48550/arXiv.2308.06259 Self- Alignment with Instruction Backtranslation . arXiv preprint

    2024 doi arXiv

  11. [19]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  12. [20]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/v1/2022.acl-long.229 T ruthful QA : Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

    2022 doi

  13. [21]

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. 2024. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434

    2024 arXiv

  14. [22]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  15. [23]

    Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. ar...

    2023 arXiv

  16. [24]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  17. [25]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Winogrande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641

    2019 arXiv

  18. [26]

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008--3021

    2020

  19. [27]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261

    2022 arXiv

  20. [28]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  21. [29]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  22. [30]

    Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn. 2024. https://arxiv.org/abs/2211.04325 Will we run out of data? limits of llm scaling based on human-generated data . Preprint, arXiv:2211.04325

    2024 arXiv

  23. [31]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 a . https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual ...

    2023 doi

  24. [32]

    Zige Wang, Wanjun Zhong, Yufei Wang, Qi Zhu, Fei Mi, Baojun Wang, Lifeng Shang, Xin Jiang, and Qun Liu. 2023 b . Data management for large language models: A survey. arXiv preprint arXiv:2312.01700

    2023 arXiv

  25. [33]

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652

    2021 arXiv

  26. [34]

    Ting Wu, Xuefeng Li, and Pengfei Liu. 2024 a . Progress or regress? self-improvement reversal in post-training. arXiv preprint arXiv:2407.05013

    2024 arXiv

  27. [35]

    Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, and Quanquan Gu. 2024 b . https://doi.org/10.48550/arXiv.2405.00675 Self- Play Preference Optimization for Language Model Alignment . arXiv preprint

    2024 doi arXiv

  28. [36]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  29. [37]

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. 2024. https://doi.org/10.48550/arXiv.2401.10020 Self- Rewarding Language Models . arXiv preprint

    2024 doi arXiv

  30. [38]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476--15488

    2022

  31. [39]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

    2019

  32. [40]

    Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024. https://doi.org/10.48550/arXiv.2406.03816 ReST - MCTS *: LLM Self - Training via Process Reward Guided Tree Search . arXiv preprint

    2024 doi arXiv

  33. [41]

    Gonzalez, and Ion Stoica

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://openreview.net/forum?id=uccHPGDlao Judging LLM-as-a-Judge with MT-Bench and Chatbot Ar...

    2023

  34. [42]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.