Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Aligning Instruction Tuning with Pre-training

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that instruction-tuning gains come from identifying and rewriting pre-training text that the SFT data underrepresents, then fine-tuning on the combined set.

desk verdict A practical data-curation recipe with a plausible but unproven selection mechanism—missing a random baseline, but worth reviewing. read the letter →

arxiv 2501.09368 v4 pith:JEEPV7J6 submitted 2025-01-16 cs.AI

Yiming Liang , Tianyu Zheng , Xinrun Du , Ge Zhang , Jiaheng Liu , Xingwei Qu , Wenqiang Zu , Xingrun Xing
show 7 more authors
Chujie Zheng Lei Ma Guoyin Wang Zhaoxiang Zhang Wenhao Huang Xiang Yue Jiajun Zhang
This is my paper · ORCID
classification cs.AI
keywords instructiontuningpre-trainingalignmentdatadistributionkerneldensityestimationsupervisedfine-tuningsyntheticselectionlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Instruction-tuning datasets are far narrower than the corpora models were pre-trained on, and this paper argues that the gap is a fixable cause of underperformance. The proposed method, Aligning Instruction Tuning with Pre-training (AITP), finds pre-training texts that are under-represented in the supervised fine-tuning (SFT) data, rewrites those raw texts into instruction-response pairs with a stronger model, and adds the pairs to the original SFT data before fine-tuning. Across three open models and eight benchmarks, the paper reports average gains over the SFT baselines of 3.8, 1.1, and 0.9 points, with as little as a few percent of rewritten data. A same-size ablation indicates the gain is not simply the benefit of more data, and a distillation ablation indicates it is not merely the benefit of stronger-model rewriting. If the claim holds, it shows a cheap way to make instruction tuning cover the knowledge the model already has.

What carries the argument

The machinery is a "difference set" computed by density comparison. Each document from the pre-training corpus and each instruction-response pair from the SFT set is embedded, projected into two dimensions with PCA, and a kernel density estimate is computed; a pre-training point is selected when its SFT density $\hat{f}_{\mathrm{SFT}}(x_i,y_i)$ falls below a threshold $\tau$, or when the ratio $\hat{f}_{\mathrm{Pre}}(x_i,y_i)/\hat{f}_{\mathrm{SFT}}(x_i,y_i)$ exceeds a threshold. Selected raw texts are rewritten by a stronger instruction model through generate-score-answer prompts, then mixed with the original SFT set for full-parameter fine-tuning. The density estimate does the identification work; the rewriting does the format conversion.

What would settle it

Train the same three models on a rewritten random sample of the pre-training corpus matched in size to AITP's difference set; if that control reproduces the reported gains, the density-based selection is not doing the work.

Watch

Extended reading notes

Core claim

The central claim is that the distributional mismatch between instruction-tuning data and pre-training data can be detected and mended: choose raw texts from the pre-training corpus that fall in low-density regions of the SFT data (or regions where pre-training density exceeds SFT density), convert them into questions and answers, and train the original base model on the combined set. The paper reports that this consistently improves instruction-following, knowledge, coding, and reasoning benchmarks on OLMo, MAP-Neo, and Pythia, and that the improvement arrives with a small amount of rewritten difference data. The paper attributes the gain to expanding the coverage of the SFT set toward the pre-training distribution while preserving the original high-quality instruction data.

Load-bearing premise

The load-bearing assumption is that a two-dimensional density estimate of embedding vectors honestly captures which pre-training texts are missing from the instruction data and worth adding; the paper does not validate this proxy against simply sampling raw pre-training text at random.

Editorial extensions

If this is right

  • If AITP is right, a few percent of rewritten difference data can close most of the coverage gap, since performance peaks below a 10% ratio and declines as more rewritten data is added.
  • If AITP is right, gains are not attributable to dataset size, because matching the combined set's size to the original SFT set still leaves an average improvement of 2.1 on OLMo.
  • If AITP is right, gains are not attributable to distillation from a stronger model, because replacing the original SFT set with rewritten data from a similar distribution does not beat the baseline.
  • If AITP is right, the recipe transfers across model families and corpora, since the same procedure improves OLMo, MAP-Neo, and Pythia with their respective pre-training corpora.
  • If AITP is right, the density-comparison variant with the stronger embedding model yields the largest average gain (4.3 on OLMo), making the choice of embedding and selection rule a tunable part of the pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never tests whether its density proxy beats random selection from the pre-training corpus; a matched random-rewrite control would be the direct test of whether the difference-set mechanism is what causes the gain.
  • The method's success suggests that any large text collection, not only the original pre-training corpus, could serve as a source of underrepresented coverage as long as its distribution roughly matches the model's training data.
  • Because performance declines when more rewritten data is added, the quality of the rewriting step is likely the binding constraint; a better rewriting model or stricter query filtering might shift the peak to higher ratios.
  • A practical byproduct would be a diagnostic: the KDE map could be used to audit an SFT dataset for missing domains before training, in the same spirit as the paper's visualization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes AITP, a pipeline that selects pre-training documents that are underrepresented in an instruction-tuning dataset, rewrites them into instruction-response pairs with a large instruct model, and adds the rewritten examples to the original SFT dataset before full-parameter fine-tuning. The selection step projects text embeddings from bge-m3 or all-MiniLM-L6-v2 to two dimensions with PCA, estimates densities of the SFT and pre-training projections with KDE, and retains pre-training points whose SFT density is low (Eq. 7) or whose pre-training-to-SFT density ratio exceeds a threshold (Eq. 14). The method is evaluated on OLMo-7B, MAP-Neo-7B, and Pythia-12B across IFEval, MMLU, ARC, GPQA, HumanEval, MBPP, HellaSwag, and GSM8K, reporting average improvements of +3.8, +1.1, and +0.9 over the corresponding baselines, plus ablations on dataset size, distillation, and rewriting ratio.

Significance. If the reported gains are robust, the paper addresses a practically relevant problem: instruction-tuning datasets often have narrow coverage relative to the broad pre-training distribution, and leveraging openly released pre-training corpora to fill coverage gaps is a sensible and cost-conscious idea. The experimental setup has real strengths: it uses three fully open models with released pre-training corpora, a principled reservoir-sampling approximation of the huge corpora, and an objective (non-LLM-judged) IFEval metric alongside standard benchmarks. The same-size ablation in Table 3 is a useful control for total example count. However, the paper's central mechanism—that adaptive KDE/PCA-based selection of underrepresented pre-training text is what drives the gains—is not actually tested against a random-selection baseline, and several benchmark-level deltas are negative. The contribution is therefore plausible but not yet established; the manuscript needs one additional load-bearing control and more careful reporting of variability and parameters before the core claim can be accepted.

major comments (4)
  1. [§4.4, Table 3] The ablation labeled 'Same Size' does not test whether the adaptive selection rule matters. It subsamples the combined set (original SFT plus already-selected rewritten set) back to the original SFT size, so it controls for total example count but not for whether the KDE/PCA-based difference set is better than a random sample of the pre-training corpus. The 'Distillation' ablation replaces the original SFT data with rewritten same-distribution data and also does not vary the selection rule. Consequently, all positive results are consistent with the simpler hypothesis that rewriting any modest, randomly chosen sample of pre-training text into instruction-response pairs and mixing it into SFT improves these benchmarks, due to added diversity or format, independent of the density estimates. Since the stated contribution is alignment with pre-training distributions, the paper should add a control that samples the same number of documents uniformly at random from the pre-training corpus, rewrites them with the identical prompt pipeline, and mixes them into the original SFT dataset; without this control the central mechanism is unexamined.
  2. [Table 1] The claim of 'consistent performance improvements' is stronger than the data support. On MAP-Neo, HumanEval drops by 6.1 points and HellaSwag by 3.9 points; on Pythia, ARC-c drops by 4.7 points and GSM8K by 0.1 points. The average Pythia improvement is only +0.9, and no repeated seeds, error bars, or significance tests are reported anywhere. To support a claim of consistent improvement across three models and eight benchmarks, the paper should report variance across at least a few fine-tuning seeds and discuss the negative deltas rather than averaging them away.
  3. [§4.2, Table 1, §4.5] The manuscript never states the rewritten-to-original ratio used to produce the main results in Table 1. Section 4.5 and Figure 4 show performance for ratios from 0.01 to 0.7 and assert that 'less than 10%' is enough, but Table 1 does not say which ratio generated the reported +3.8/+1.1/+0.9 deltas. Since the ratio is a key free parameter of the method, the main results must name the exact ratio used, and the claim that AITP is effective with less than 10% added data should be tied to the specific configuration behind Table 1.
  4. [§3.3, Eqs. (7), (14)] The threshold values τ=0.7 and τ=1.0 are stated without any selection criterion, and the KDE bandwidth parameters hx, hy, σ are left unspecified; the rewriting ratio is also varied in Figure 4 and Table 5. Because these parameters appear to be chosen after observing the evaluation results, the paper should either provide a principled criterion for setting them or report sensitivity of the main conclusions to them. Without this, the possibility that the reported gains come from post hoc selection of thresholds and ratio for the evaluation benchmarks cannot be excluded.
minor comments (4)
  1. [§4.3, paragraph after Table 2] The sentence describing the four variants is inconsistent with Table 2: it says 'using MiniLM with density comparison results in an improvement of 3.0' and 'using bge with density comparison achieves an improvement of 3.1,' but Table 2 reports MiniLM-estimation as 3.0 and MiniLM-comparison as 3.1. The text should be corrected to match the table.
  2. [§4.3, paragraph after Table 2] The sentence 'the four AITP variants show improvements over the baseline model OLMo-SFT across various settings' is inaccurate because the MiniLM-estimation row in Table 2 has negative deltas on IFEval (P-S -0.7, P-L -2.0, I-L -0.9). The claim should be qualified to note that some variants underperform on the chat benchmark.
  3. [§2.1.2, Eqs. (5) and (6)] The KDE notation is internally inconsistent: Eq. (5) uses separate bandwidths hx and hy in the kernel argument, while Eq. (6) defines the kernel with a single bandwidth σ, and the denominator hxhy is not matched to the kernel expression. Please clarify the bandwidth convention and give the actual values used for hx, hy, and σ.
  4. [§3.1] The text says 'seven standard benchmarks' but then lists eight benchmarks (IFEval plus MMLU, ARC-c, GPQA-diamond, HumanEval, MBPP, HellaSwag, GSM8K). Please correct the count or the list.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AITP's central claims rest on held-out benchmark evaluations, and the KDE/PCA selection mechanism is an unvalidated input rather than a prediction derived from the method.

full rationale

Walking the derivation chain: Ddiff (Eqs. 1, 7, 14) is a data-selection construction that flags pre-training points with low estimated density in the SFT data; it is not claimed as a predicted outcome. The rewriting stage (Eqs. 9-11) and the training loss (Eq. 12) are standard supervised fine-tuning operations. All reported gains in Tables 1-3 and Figure 4 are measured on held-out benchmarks (IFEval, MMLU, ARC-c, GPQA, HumanEval, MBPP, HellaSwag, GSM8K), so no claimed result is fitted from or definitionally identical to the method's selection rule. The Same Size and Distillation ablations directly address the alternative explanations of increased dataset size and teacher-model distillation. The skeptical concern that there is no random-pre-training-sample control is a genuine evidential gap about whether the KDE/PCA density estimates are what drives the gains, but it is not circularity: the density proxy is an assumption fed into the pipeline, not a quantity whose value is recovered as the paper's conclusion. Self-citations (MAP-Neo as the model family, I-SHEEP in related work) are descriptive and are not load-bearing for the claimed derivation. Section 4.2's 'supposition' that improvement comes from filling coverage gaps is partly supported by figures showing that Ddiff occupies low-density SFT regions; that support is definitional, but the benchmark improvements themselves do not reduce to that definition. No circular step was found.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the validity of density-based selection in 2D embedding space, the choice of tau and ratio, and the quality of teacher rewriting. No new physical or theoretical entities are introduced.

free parameters (5)
  • threshold tau (density estimation) = 0.7
    Eq. 7 defines Ddiff via f_SFT < tau; chosen without sensitivity analysis or justification.
  • threshold tau (density comparison) = 1.0
    Eq. 14 uses f_Pre/f_SFT > tau; no sensitivity analysis is presented.
  • KDE bandwidth hx, hy, sigma = not reported
    Eqs. 5 and 6 require bandwidth parameters that control which points count as underrepresented; values are not given.
  • rewritten-to-original ratio = not stated for main results; Figure 4 suggests under 0.1
    Section 4.5 explores ratios 0.01 to 0.7 and reports best in a range; the main experiment does not specify the ratio used.
  • query quality and difficulty filtering thresholds = not reported
    Section 2.2 filters low-quality queries based on scores, but no cutoff is specified.
assumptions (3)
  • domain assumption 2D PCA plus KDE density is a faithful proxy for coverage differences between SFT and pre-training data.
    Section 2.1 reduces high-dimensional embeddings to two dimensions and measures density there; no validation that this preserves the structure relevant to instruction coverage.
  • domain assumption Pre-training corpus distribution is the correct reference distribution for what SFT should cover.
    The paper's premise is that SFT datasets are misaligned with pre-training distributions and fixing this improves generalization; this is asserted rather than derived.
  • domain assumption Qwen2.5-72B-Instruct rewriting produces instruction-response pairs of sufficient quality.
    Section 3.4 assumes the teacher model's generated pairs are high quality; the distillation ablation shows full replacement with rewritten data degrades performance, partially supporting the limits of this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning Instruction Tuning with Pre-training." pith.science (2026). https://pith.science/paper/JEEPV7J6

@misc{pith2026250109368,
  author       = {Pith},
  title        = {Pith review of: Aligning Instruction Tuning with Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JEEPV7J6}},
  note         = {Machine review of arXiv:2501.09368}
}
read the original abstract

Instruction tuning enhances large language models (LLMs) to follow human instructions across diverse tasks, relying on high-quality datasets to guide behavior. However, these datasets, whether manually curated or synthetically generated, are often narrowly focused and misaligned with the broad distributions captured during pre-training, limiting LLM generalization and effective use of pre-trained knowledge. We propose Aligning Instruction Tuning with Pre-training (AITP), a method that bridges this gap by identifying coverage shortfalls in instruction-tuning datasets and rewriting underrepresented pre-training data into high-quality instruction-response pairs. This approach enriches dataset diversity while preserving task-specific objectives. Evaluations on three fully open LLMs across eight benchmarks demonstrate consistent performance improvements with AITP. Ablations highlight the benefits of adaptive data selection, controlled rewriting, and balanced integration, emphasizing the importance of aligning instruction tuning with pre-training distributions to unlock the full potential of LLMs.

Figures

Figures reproduced from arXiv: 2501.09368 by the authors.

Figure 1
Figure 1. Visualization of Projections. The red regions at the bottom represent the pre-training corpus, while the light blue regions above represent the SFT datasets. Darker areas indicate a higher concentration of data points, whereas lighter areas represent sparser distributions. Additional pro￾jections are shown in Appendix A. on the other hand, frequently depend on expensive APIs of strong models and are tightly coupled … view at source ↗
Figure 2
Figure 2. The pipeline of AITP. AITP first generates a difference set, then rewrites the raw text into instruction-response pairs to form a rewritten set, and finally combines the rewritten set with the original SFT dataset for model training. three stages: (1) generating a difference set based on density comparisons, (2) rewriting raw text into instruction-response pairs, and (3) integrating these pairs into the original dat… view at source ↗
Figure 3
Figure 3. Data Distribution Changes in AITP. Subfigures (a)-(d) and (e)-(h) illustrate the distribution changes of the datasets under density estimation and the density comparison settings. The red region at the bottom represents the pre￾training corpus, Dolma, while the blue regions in the subfigures represent the projections of Tulu V2, the difference set, the rewritten set, and the combined set, respectively. Darker areas … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Line graph across different ratios. The x-label represents the ratio of the rewritten set to the original SFT dataset, while the y-label shows accuracy across different benchmarks. More results can be found in Appendix E. 150 100 50 0 50 100 150 t-SNE Component 1 150 1…
Figure 5
Figure 5. Figure 5: The t-SNE Visualization of SFT and Rewritten Data. The red points and blue points represent the original SFT data and the rewritten data, respectively. the same dataset size. Comparing the third and fourth rows, the improvement for the same dataset size setting is smal…
Figure 6
Figure 6. Figure 6: Visualization of data distribution changes in AITP. The red regions at the bottom denote the pre-training corpus, while the light blue regions above represent the SFT datasets. Darker areas indicate a higher concentration of data points, whereas lighter areas signify s…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach

    cs.AI 2025-02 conditional novelty 6.0 of 10

    A theory-grounded EMI metric and an upper bound on its distribution-shift gap are proposed for evaluating multimodal LLM robustness.

  2. Backbone Augmented Training for Adaptations

    cs.LG 2025-06 reject novelty 4.0 of 10

    Selecting backbone pre-training data with an influence-based score and adding it to the adaptation set can improve LoRA and DreamBooth training when adaptation data are scarce.

Reference graph

Works this paper leans on

70 extracted references · 5 canonical work pages · cited by 2 Pith papers

  1. [1]

    Yi: Open foundation models by 01.ai

    AI, ., :, Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Li, H., Zhu, J., Chen, J., Chang, J., Yu, K., Liu, P., Liu, Q., Yue, S., Yang, S., Yang, S., Yu, T., Xie, W., Huang, W., Hu, X., Ren, X., Niu, X., Nie, P., Xu, Y., Liu, Y., Wang, Y., Cai, Y., Gu, Z., Liu, Z., and Dai, Z. Yi: Open foundation models by 01.ai. arXiv preprint arXiv: 2403....

    arXiv 2024

  2. [2]

    I., Babaei, H., LeJeune, D., Siahkoohi, A., and Baraniuk, R

    Alemohammad, S., Casco-Rodriguez, J., Luzi, L., Humayun, A. I., Babaei, H., LeJeune, D., Siahkoohi, A., and Baraniuk, R. G. Self-consuming generative models go mad. arXiv preprint arXiv: 2307.01850, 2023

    arXiv 2023

  3. [3]

    Claude 3 haiku: Our fastest model yet, 2024

    Anthropic. Claude 3 haiku: Our fastest model yet, 2024. Available at: https://www.anthropic.com/news/claude-3-haiku

    work page 2024

  4. [4]

    Self-rag: Learning to retrieve, generate, and critique through self-reflection

    Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv: 2310.11511, 2023

    arXiv 2023

  5. [5]

    Program synthesis with large language models

    Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., and Sutton, C. Program synthesis with large language models. arXiv preprint arXiv: 2108.07732, 2021

    arXiv 2021

  6. [6]

    A., Purohit, S., Prashanth, U

    Biderman, S., Schoelkopf, H., Anthony, Q., Bradley, H., O'Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., Skowron, A., Sutawika, L., and van der Wal, O. Pythia: A suite for analyzing large language models across training and scaling. arXiv preprint arXiv: 2304.01373, 2023

    arXiv 2023

  7. [7]

    Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

    Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

    2024

  8. [8]

    Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Ch...

    arXiv 2021

Show all 70 references
  1. [9]

    W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al

    Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25 0 (70): 0 1--53, 2024

    2024

  2. [10]

    Chung, J. J. Y., Kamar, E., and Amershi, S. Increasing diversity while maintaining accuracy: Text data generation with large language models and human interventions. arXiv preprint arXiv: 2306.04140, 2023

    2023 arXiv

  3. [11]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: 1803.05457, 2018

    2018 arXiv

  4. [12]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. arXiv preprint arXiv: 2110.14168, 2021

    arXiv

  5. [13]

    Opencompass: A universal evaluation platform for foundation models

    Contributors, O. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass, 2023

    2023

  6. [14]

    K., Liang, W., Lin, F., Liu, A

    DeepSeek-AI, :, Bi, X., Chen, D., Chen, G., Chen, S., Dai, D., Deng, C., Ding, H., Dong, K., Du, Q., Fu, Z., Gao, H., Gao, K., Gao, W., Ge, R., Guan, K., Guo, D., Guo, J., Hao, G., Hao, Z., He, Y., Hu, W., Huang, P., Li, E., Li, G., Li, J., Li, Y., Li, Y. K., Liang, W., Lin, F...

    2024 arXiv

  7. [15]

    Interpretable contrastive monte carlo tree search reasoning

    Gao, Z., Niu, B., He, X., Xu, H., Liu, H., Liu, A., Hu, X., and Wen, L. Interpretable contrastive monte carlo tree search reasoning. arXiv preprint arXiv: 2410.01707, 2024

    2024 arXiv

  8. [16]

    Scaling synthetic data creation with 1,000,000,000 personas

    Ge, T., Chan, X., Wang, X., Yu, D., Mi, H., and Yu, D. Scaling synthetic data creation with 1,000,000,000 personas. arXiv preprint arXiv: 2406.20094, 2024

    2024 arXiv

  9. [17]

    GLM, T., :, Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., Zhao, H., Lai, H., Yu, H., Wang, H., Sun, J., Zhang, J., Cheng, J., Gui, J., Tang, J., Zhang, J., Sun, J., Li, J., Zhao, L., Wu, L., Zhong, L., Liu, M., Huang, M., Zhang, P., Zheng, Q.,...

    2024 arXiv

  10. [18]

    R., Cohan, A., Dumas, J., Elazar, Y., Gu, Y., Hessel, J., Khot, T., Merrill, W., Morrison, J

    Groeneveld, D., Beltagy, I., Walsh, P., Bhagia, A., Kinney, R., Tafjord, O., Jha, A., Ivison, H., Magnusson, I., Wang, Y., Arora, S., Atkinson, D., Authur, R., Chandu, K. R., Cohan, A., Dumas, J., Elazar, Y., Gu, Y., Hessel, J., Khot, T., Merrill, W., Morrison, J. D., Muennigh...

    2024 doi arXiv

  11. [19]

    The curious decline of linguistic diversity: Training language models on synthetic text

    Guo, Y., Shang, G., Vazirgiannis, M., and Clavel, C. The curious decline of linguistic diversity: Training language models on synthetic text. In Duh, K., Gomez, H., and Bethard, S. (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp.\ 3589--3604, ...

    2024 doi

  12. [20]

    Measuring massive multitask language understanding

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  13. [21]

    Large language models can self-improve

    Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., and Han, J. Large language models can self-improve. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp.\ 1051--1068, Singapore, December...

    2023 doi

  14. [22]

    A., Beltagy, I., and Hajishirzi, H

    Ivison, H., Wang, Y., Pyatkin, V., Lambert, N., Peters, M., Dasigi, P., Jang, J., Wadden, D., Smith, N. A., Beltagy, I., and Hajishirzi, H. Camels in a changing climate: Enhancing lm adaptation with tulu 2. arXiv preprint arXiv: 2311.10702, 2023

    2023 arXiv

  15. [23]

    Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de las Casas, D., Hanna, E. B., Bressand, F., Lengyel, G., Bour, G., Lample, G., Lavaud, L. R., Saulnier, L., Lachaux, M.-A., Stock, P., Subramanian, S., Yang, S., Antoniak, S., Scao...

    2024 arXiv

  16. [24]

    Importance weighting can help large language models self-improve

    Jiang, C., min Chan, C., Xue, W., Liu, Q., and Guo, Y. Importance weighting can help large language models self-improve. arXiv preprint arXiv: 2408.09849, 2024 b

    2024 arXiv

  17. [25]

    Teaching language models to hallucinate less with synthetic tasks

    Jones, E., Palangi, H., Simões, C., Chandrasekaran, V., Mukherjee, S., Mitra, A., Awadallah, A., and Kamar, E. Teaching language models to hallucinate less with synthetic tasks. arXiv preprint arXiv: 2310.06827, 2023

    2023 arXiv

  18. [26]

    Self-specialization: Uncovering latent expertise within large language models

    Kang, J., Luo, H., Zhu, Y., Hansen, J., Glass, J., Cox, D., Ritter, A., Feris, R., and Karlinsky, L. Self-specialization: Uncovering latent expertise within large language models. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational...

    2024 doi

  19. [27]

    Unifiedqa: Crossing format boundaries with a single qa system

    Khashabi, D., Min, S., Khot, T., Sabharwal, A., Tafjord, O., Clark, P., and Hajishirzi, H. Unifiedqa: Crossing format boundaries with a single qa system. arXiv preprint arXiv: 2005.00700, 2020

    2005 arXiv

  20. [28]

    H., Gonzalez, J

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    2023

  21. [29]

    Longform: Effective instruction tuning with reverse instructions

    Köksal, A., Schick, T., Korhonen, A., and Schütze, H. Longform: Effective instruction tuning with reverse instructions. Conference on Empirical Methods in Natural Language Processing, 2023. doi:10.18653/v1/2024.findings-emnlp.414

    2023 doi

  22. [30]

    W., Keutzer, K., and Gholami, A

    Lee, N., Wattanawong, T., Kim, S., Mangalam, K., Shen, S., Anumanchipalli, G., Mahoney, M. W., Keutzer, K., and Gholami, A. Llm2llm: Boosting llms with novel iterative data enhancement. arXiv preprint arXiv: 2403.15042, 2024

    2024 arXiv

  23. [31]

    Selective reflection-tuning: Student-selected data recycling for LLM instruction-tuning

    Li, M., Chen, L., Chen, J., He, S., Gu, J., and Zhou, T. Selective reflection-tuning: Student-selected data recycling for LLM instruction-tuning. In Ku, L., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thai...

    2024 doi

  24. [32]

    Self-alignment with instruction backtranslation

    Li, X., Yu, P., Zhou, C., Schick, T., Levy, O., Zettlemoyer, L., Weston, J., and Lewis, M. Self-alignment with instruction backtranslation. arXiv preprint arXiv: 2308.06259, 2023

    2023 arXiv

  25. [33]

    Openorca: An open dataset of gpt augmented flan reasoning traces

    Lian, W., Goodson, B., Pentland, E., Cook, A., Vong, C., and "Teknium". Openorca: An open dataset of gpt augmented flan reasoning traces. https://https://huggingface.co/Open-Orca/OpenOrca, 2023

    2023

  26. [34]

    I-sheep: Self-alignment of llm from scratch through an iterative self-enhancement paradigm

    Liang, Y., Zhang, G., Qu, X., Zheng, T., Guo, J., Du, X., Yang, Z., Liu, J., Lin, C., Ma, L., Huang, W., and Zhang, J. I-sheep: Self-alignment of llm from scratch through an iterative self-enhancement paradigm. arXiv preprint arXiv: 2408.08072, 2024

    2024 arXiv

  27. [35]

    Liu, R., Wei, J., Liu, F., Si, C., Zhang, Y., Rao, J., Zheng, S., Peng, D., Yang, D., Zhou, D., and Dai, A. M. Best practices and lessons learned on synthetic data. arXiv preprint arXiv: 2404.07503, 2024

    2024 arXiv

  28. [36]

    Liu, Z., Qiao, A., Neiswanger, W., Wang, H., Tan, B., Tao, T., Li, J., Wang, Y., Sun, S., Pangarkar, O., Fan, R., Gu, Y., Miller, V., Zhuang, Y., He, G., Li, H., Koto, F., Tang, L., Ranjan, N., Shen, Z., Ren, X., Iriondo, R., Mu, C., Hu, Z., Schulze, M., Nakov, P., Baldwin, T....

    2023 arXiv

  29. [37]

    Self: Self-evolution with language feedback

    Lu, J., Zhong, W., Huang, W., Wang, Y., Zhu, Q., Mi, F., Wang, B., Wang, W., Zeng, X., Shang, L., Jiang, X., and Liu, Q. Self: Self-evolution with language feedback. arXiv preprint arXiv: 2310.00533, 2023

    2023 arXiv

  30. [38]

    Source2synth: Synthetic data generation and curation grounded in real data sources

    Lupidi, A., Gemmell, C., Cancedda, N., Dwivedi-Yu, J., Weston, J., Foerster, J., Raileanu, R., and Lomeli, M. Source2synth: Synthetic data generation and curation grounded in real data sources. arXiv preprint arXiv: 2409.08239, 2024

    2024 arXiv

  31. [39]

    P., Hermann, K., Welleck, S., Yazdanbakhsh, A., and Clark, P

    Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., and Clark, P. Self-refine: Iterative refinement with self-feedback. arXiv preprint ar...

    2023 arXiv

  32. [40]

    Steering language generation: Harnessing contrastive expert guidance and negative prompting for coherent and diverse synthetic data generation

    O'Neill, C., Ting, Y.-S., Ciuca, I., Miller, J., and Bui, T. Steering language generation: Harnessing contrastive expert guidance and negative prompting for coherent and diverse synthetic data generation. arXiv preprint arXiv: 2308.07645, 2023

    2023 arXiv

  33. [41]

    OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., Bello, I., Berdine, J., Bernadett-Shapi...

    2023 arXiv

  34. [42]

    Instruction tuning with gpt-4

    Peng, B., Li, C., He, P., Galley, M., and Gao, J. Instruction tuning with gpt-4. arXiv preprint arXiv: 2304.03277, 2023

    2023 arXiv

  35. [43]

    and Gurevych, I

    Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/abs/1908.10084

    2019 arXiv

  36. [44]

    L., Stickland, A

    Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv: 2311.12022, 2023

    2023 arXiv

  37. [45]

    and Guven, E

    Renze, M. and Guven, E. Self-reflection in llm agents: Effects on problem-solving performance. arXiv preprint arXiv: 2405.06682, 2024

    2024 arXiv

  38. [46]

    H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T

    Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., Dey, M., Bari, M. S., Xu, C., Thakker, U., Sharma, S. S., Szczechla, E., Kim, T., Chhablani, G., Nayak, N., Datta, D., Chang, J., Jiang, M. T.-J., Wang,...

    arXiv

  39. [47]

    A., Chan, J

    Scheurer, J., Campos, J. A., Chan, J. S., Chen, A., Cho, K., and Perez, E. Training language models with language feedback. arXiv preprint arXiv: 2204.14146, 2022

    2022 arXiv

  40. [48]

    The curse of recursion: Training on generated data makes models forget

    Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., and Anderson, R. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv: 2305.17493, 2023

    2023 arXiv

  41. [49]

    Principle-driven self-alignment of language models from scratch with minimal human supervision

    Sun, Z., Shen, Y., Zhou, Q., Zhang, H., Chen, Z., Cox, D., Yang, Y., and Gan, C. Principle-driven self-alignment of language models from scratch with minimal human supervision. arXiv preprint arXiv: 2305.03047, 2023

    2023 arXiv

  42. [50]

    Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023

    2023

  43. [51]

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., Silver, D., Johnson, M., Antonoglou, I., Schrittwieser, J., Glaese, A., Chen, J., Pitler, E., Lillicrap, T., Lazaridou, A., Firat, O., Molloy, J., Isard, ...

    2023 arXiv

  44. [52]

    Qwen2.5: A party of foundation models, September 2024

    Team, Q. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  45. [53]

    Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023

    Teknium. Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023. URL https://huggingface.co/datasets/teknium/OpenHermes-2.5

    2023

  46. [54]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 2302.13971, 2023

    2023 arXiv

  47. [55]

    Vitter, J. S. Random sampling with a reservoir. ACM Trans. Math. Softw., 11 0 (1): 0 37–57, March 1985. ISSN 0098-3500. doi:10.1145/3147.3165. URL https://doi.org/10.1145/3147.3165

    1985

  48. [56]

    A., Khashabi, D., and Hajishirzi, H

    Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv: 2212.10560, 2022 a

    2022 arXiv

  49. [57]

    S., Naik, A., Stap, D., et al

    Wang, Y., Mishra, S., Alipoormolabashi, P., Kordi, Y., Mirzaei, A., Arunkumar, A., Ashok, A., Dhanasekaran, A. S., Naik, A., Stap, D., et al. Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks. In EMNLP, 2022 b

    2022

  50. [58]

    W., Lester, B., Du, N., Dai, A

    Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. Finetuned language models are zero-shot learners. International Conference on Learning Representations, 2021

    2021

  51. [59]

    P., Kawaguchi, K., and Shieh, M

    Xie, Y., Goyal, A., Zheng, W., Kan, M.-Y., Lillicrap, T. P., Kawaguchi, K., and Shieh, M. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv: 2405.00451, 2024

    2024 arXiv

  52. [60]

    Qwen2 technical report

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Yang, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M....

    2024 arXiv

  53. [61]

    Self-distillation bridges distribution gap in language model fine-tuning

    Yang, Z., Pang, T., Feng, H., Wang, H., Chen, W., Zhu, M., and Liu, Q. Self-distillation bridges distribution gap in language model fine-tuning. arXiv preprint arXiv: 2402.13669, 2024 b

    2024 arXiv

  54. [62]

    T., Li, Z., Weller, A., and Liu, W

    Yu, L., Jiang, W., Shi, H., Yu, J., Liu, Z., Zhang, Y., Kwok, J. T., Li, Z., Weller, A., and Liu, W. Metamath: Bootstrap your own mathematical questions for large language models. International Conference on Learning Representations, 2023. doi:10.48550/arXiv.2309.12284

    2023 doi arXiv

  55. [63]

    Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., and Weston, J

    Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., and Weston, J. Self-rewarding language models. arXiv preprint arXiv: 2401.10020, 2024

    2024 arXiv

  56. [64]

    Hellaswag: Can a machine really finish your sentence? Annual Meeting of the Association for Computational Linguistics, 2019

    Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. Hellaswag: Can a machine really finish your sentence? Annual Meeting of the Association for Computational Linguistics, 2019. doi:10.18653/v1/P19-1472

    2019 doi

  57. [65]

    Zhang, G., Qu, S., Liu, J., Zhang, C., Lin, C., Yu, C. L., Pan, D., Cheng, E., Liu, J., Lin, Q., Yuan, R., Zheng, T., Pang, W., Du, X., Liang, Y., Ma, Y., Li, Y., Ma, Z., Lin, B., Benetos, E., Yang, H., Zhou, J., Ma, K., Liu, M., Niu, M., Wang, N., Que, Q., Liu, R., Liu, S., G...

    2024 arXiv

  58. [66]

    Sled: Self logits evolution decoding for improving factuality in large language models

    Zhang, J., Juan, D.-C., Rashtchian, C., Ferng, C.-S., Jiang, H., and Chen, Y. Sled: Self logits evolution decoding for improving factuality in large language models. arXiv preprint arXiv: 2411.02433, 2024 b

    2024 arXiv

  59. [67]

    Kun: Answer polishment for chinese self-alignment with instruction back-translation

    Zheng, T., Guo, S., Qu, X., Guo, J., Du, X., Jia, Q., Lin, C., Huang, W., Fu, J., and Zhang, G. Kun: Answer polishment for chinese self-alignment with instruction back-translation. arXiv preprint arXiv: 2401.06477, 2024

    2024 arXiv

  60. [68]

    Lima: Less is more for alignment

    Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. Lima: Less is more for alignment. arXiv preprint arXiv: 2305.11206, 2023 a

    2023 arXiv

  61. [69]

    Instruction-following evaluation for large language models

    Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., and Hou, L. Instruction-following evaluation for large language models. arXiv preprint arXiv: 2311.07911, 2023 b

    2023 arXiv

  62. [70]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.