Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

On the Effect of Instruction Tuning Loss on Generalization

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that standard instruction tuning, which trains only on response tokens, is suboptimal; giving prompt tokens a low-to-moderate loss weight and response tokens a moderate-to-high weight yields the best models, with an…

desk verdict A genuinely large and open empirical sweep of instruction-tuning loss weights, but the headline gains are selected maxima and a normalization confound means the 'prompt weight' may not be the active ingredient. read the letter →

arxiv 2507.07817 v2 pith:QQRGEENY submitted 2025-07-10 cs.CL cs.AIcs.LG

Anwoy Chatterjee , H S V N S Kowndinya Renduchintala , Sumit Bhatia , Tanmoy Chakraborty This is my paper
classification cs.CLcs.AIcs.LG
keywords instructiontuninglossweightingprompttokensresponsegeneralizationpreferencealignmentDPOrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Instruction tuning of language models normally computes loss only on response tokens and completely ignores prompt tokens. This paper argues that the standard choice is rarely optimal and proposes Weighted Instruction Tuning (WIT), which trains with weight $\lambda_p$ on prompt-token loss and $\lambda_r$ on response-token loss. Across five models, three finetuning datasets, and five benchmarks, the best WIT configurations beat the conventional $(0,1)$ loss in every model-dataset combination, averaging about 6.55% relative improvement. The paper also finds that WIT-tuned models are better starting points for DPO (Direct Preference Optimization) preference alignment and are less sensitive to intent-preserving prompt rewording. If correct, the result means a two-scalar change to a standard loss is a nearly free way to improve instruction-following models and their downstream alignment.

What carries the argument

The central object is the WIT loss, which interpolates between conventional instruction tuning at $(\lambda_p, \lambda_r) = (0,1)$ and continual pre-training at $(1,1)$ by weighting the prompt and response log-likelihood terms separately and normalizing by the number of tokens with nonzero weight. The argument is carried by sweeping these two weights on a $6 \times 6$ grid across five models, three finetuning datasets, and five evaluation benchmarks, then comparing average performance, DPO transfer, and POSIX robustness across the grid. The paper also uses correlation analysis between the optimal $\lambda_p$ and prompt characteristics (length, n-gram diversity, parse-tree depth) and model perplexity to explain when a nonzero prompt weight helps.

What would settle it

Rerun the best $(\lambda_p, \lambda_r)$ cell and the conventional $(0,1)$ baseline for the same model and dataset across, say, ten random seeds, and check whether the average advantage holds outside one standard error; if the best cell falls within noise, as several reported gains already are below 1.5%, the central claim would not survive.

Watch

Extended reading notes

Core claim

The paper's central claim is that the conventional response-only instruction tuning objective, which sums the negative log-likelihood of response tokens given the prompt and zeroes out prompt tokens, is suboptimal. It proposes WIT, $\mathcal{L}_{\mathrm{WIT}} = -\frac{1}{\sum_i (I(\lambda_p \neq 0)|P_i| + I(\lambda_r \neq 0)|R_i|)} \sum_i \left(\lambda_p \sum_j \log P(p_i^{(j)} \mid p_i^{(<j)}) + \lambda_r \sum_j \log P(r_i^{(j)} \mid P_i, r_i^{(<j)})\right)$, sweeping $(\lambda_p, \lambda_r)$ over a $6 \times 6$ grid. In all 15 model-dataset combinations the best cell has low-to-moderate $\lambda_p$ (0 to 0.6) and moderate-to-high $\lambda_r$ (0.4 to 1), and the per-setting best cells average 6.55% relative gain over conventional instruction tuning. The same weighting pattern survives DPO, where WIT initializations give roughly 8% average relative gain, and moderate response weights also reduce prompt sensitivity as measured by the Prompt Sensitivity Index (POSIX), a metric for robustness to intent-preserving prompt rewording. The paper interprets this as evidence that a partial prompt-token learning signal aids instruction comprehension, while full response-token weight encourages overfitting to response patterns.

Load-bearing premise

The announced gains rest on the assumption that the best weight pair, chosen after looking at results on the same five benchmarks, reflects a stable real advantage over the conventional $(0,1)$ baseline rather than run-to-run noise, because each configuration was trained once with no error bars.

Editorial extensions

If this is right

  • Switching from the conventional $(0,1)$ loss to a low-to-moderate prompt weight and moderate-to-high response weight improves average benchmark performance by about 6.55% relative to conventional instruction tuning.
  • WIT-tuned models are better DPO starting points: preference alignment on top of optimal WIT weights beats DPO on conventional instruction tuning by roughly 8.01%, and still by about 2.44% when the instruction-tuning-optimal weights are reused.
  • Moderate response-token weights improve robustness to intent-preserving prompt variations, while the conventional response-only loss is rarely the most robust configuration.
  • Training on prompt tokens alone, with $\lambda_r = 0$, can already improve instruction following over the base model when the training data is large and diverse, suggesting unannotated prompts carry usable signal.
  • The optimal weights are not universal: they correlate with training-data prompt length, diversity, syntactic complexity, and model perplexity, so weights should be chosen for the specific model and data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the best $(\lambda_p, \lambda_r)$ cell is selected post hoc on the same five benchmarks where the gains are reported, the 6.55% figure probably overstates what a practitioner would see on new benchmarks; a held-out evaluation or multi-seed check is needed to confirm the effect size.
  • If the effect is real, weighted instruction tuning is a nearly free improvement: it requires no new data or architecture, only two scalar weights, and it can be stacked with existing data-curation and preference-alignment recipes.
  • The robustness result suggests a testable connection to the memorization-versus-generalization debate: lower response weight should reduce verbatim memorization of training responses, which could be checked directly with memorization probes on the finetuning data.
  • The paper's own limitation about fixed weights points to the natural next experiment: adaptive per-token or per-step weighting based on the model's current likelihoods might outperform any fixed grid.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Weighted Instruction Tuning (WIT), a modification of the standard instruction-tuning objective that assigns separate weights λp and λr to prompt-token and response-token losses, respectively. It evaluates WIT over a 6×6 grid of (λp, λr) values for five language models (Llama-3.2-1B/3B, Gemma-2-2B, Mistral-7B, Llama-3-8B) and three instruction-tuning datasets (LIMA, Alpaca-Cleaned, Tülu-v2), measuring average performance on MMLU, BBH, AlpacaEval, IFEval, and MT-Bench. The authors report that the conventional (0,1) configuration is rarely optimal, that low-to-moderate prompt-token weights together with moderate-to-high response-token weights give the best average performance (a mean relative gain of 6.55% in Table 1), that WIT-tuned models serve as better DPO starting points (a mean relative gain of 8.01% in Table 3), and that prompt-only tuning can improve instruction-following. The paper also presents a post hoc correlation analysis linking optimal prompt-token weights to prompt length, diversity, and model perplexity in Section 5.1.

Significance. If the central finding were robust, it would be practically valuable: a simple change to the instruction-tuning loss—requiring no extra data or model modifications—could improve average benchmark performance and downstream DPO starting points across model families and data scales. The paper's strengths are its broad experimental grid (five models × three datasets × 36 weight configurations, plus 525 DPO runs), its coverage of multiple model families and scales, and its release of open-source code. The observation that prompt-only training can improve instruction-following on IFEval is also interesting and potentially useful for low-annotation settings. However, the quantitative headline claims are not well supported as they stand, because the best cell is selected post hoc on the same benchmarks used to report the gains, runs are single-seed with no error bars, and the loss formulation in Eq. (2) introduces a normalization confound that is never ablated. These issues are load-bearing for the paper's main claims, so the correct assessment is that the study is valuable and worth pursuing, but the current evidence is incomplete.

major comments (3)
  1. [§4, Table 1, Figure 2] The headline quantitative claim—an average relative gain of 6.55% for WIT over conventional instruction tuning—is obtained by selecting the best (λp, λr) cell from a 36-point grid for each (model, training dataset) on the same five benchmarks that are then used to report the gain, with a single seed per cell. Under run-to-run noise, the maximum of 36 draws is an upward-biased estimate of the advantage of any fixed configuration, and several reported gains are smaller than typical benchmark noise (e.g., Llama-3.2-3B on Tülu-v2: +0.58%; Llama-3.2-3B on Alpaca-Cleaned: +0.62%). Without multiple seeds or a held-out validation split, the statement in Section 4 that "conventional instruction tuning is never the optimal choice" is not quantitatively supported.
  2. [§2, Eq. (2)] The normalization in LWIT conflates the prompt-token weight with a global rescaling of the response-token loss. For any configuration with λp > 0 and λr = 1, the denominator becomes Σ_i (|Pi| + |Ri|) rather than Σ_i |Ri|, so the effective contribution of each response token is multiplied by Σ_i |Ri| / Σ_i (|Pi| + |Ri|), which is substantially less than 1 for prompt-heavy datasets. The observed improvements relative to (0,1) could therefore be caused by this implicit down-weighting of response-token gradients rather than by the added prompt-token loss. No experiment in Section 4 compares a candidate (λp, λr) against an alternative that keeps the response-only normalization while adding prompt-token loss (e.g., normalizing by Σ_i |Ri| regardless of λp). Without this ablation, the paper's central attribution of the gains to the prompt-token weight is not established.
  3. [§4.2, Table 3, Figure 3] The DPO-transfer result for Llama-3-8B on LIMA is driven by a collapsed baseline: DPO on conventional instruction tuning scores 28.94, whereas the pre-DPO conventional instruction-tuned model scores 45.48 (Figure 2, LIMA panel). A drop of this magnitude indicates DPO training instability for that baseline, and the reported +29.23% relative gain is not a meaningful measure of WIT's benefit. The paper does not report DPO runs with multiple seeds or training curves, and it does not explain this baseline collapse. The DPO-transfer claim should be re-evaluated excluding or properly accounting for such unstable runs.
minor comments (4)
  1. [§4, Tables 1 and 3] The selected optimal (λp, λr) values are not shown in Tables 1 and 3; reporting the chosen grid cell in each row would let readers verify the result without cross-referencing the heatmaps in Figures 2 and 3.
  2. [§4.1, Figures 6–8] The percentages in Section 4.1 (e.g., 61/75, 43/75, 18/75) are computed from per-benchmark heatmaps in the appendix, but the underlying numeric scores are not provided in machine-readable form; adding a supplementary table would improve verifiability.
  3. [§5.1, Table 5] The correlation analysis is based on very small samples (n=3 for finetuning datasets; n=5 for evaluation benchmarks and models) and is reported without confidence intervals or p-values; these correlations should be labeled as exploratory rather than as established relationships.
  4. [Throughout] There are minor typographical inconsistencies, including 'Tulu-v2' versus 'Tülu-v2' and 'BBH-Hard' in Section 6; please harmonize these and check the appendix captions for consistent notation for λp and λr.

Circularity Check

2 steps flagged · score 6.0 of 10

Headline gains are selected maxima over the same 36-cell grid used for evaluation, so the central empirical claim reduces to post hoc fitting rather than independent prediction.

  1. fitted input called prediction [Section 4 (Results), Table 1 and Figure 2]
    "To study the role of prompt and response tokens in instruction tuning, we finetune five language models ... by varying the prompt and response weight configurations (λp, λr) in {0, 0.2, 0.4, 0.6, 0.8, 1.0}. ... As summarized in Table 1, WIT yields consistent improvements in average benchmark performance, achieving an average relative gain of around 6.55%."

    The 'WIT Loss (Optimal λp, λr)' column of Table 1 is the argmax of the same five-benchmark average shown in Figure 2, and the relative gain is max_cell(score) - conventional(score) on that same evaluation set. The 6.55% figure is a selected maximum over 36 configurations, not a prediction on held-out benchmarks or a parameter-free claim; with single-seed runs and no error bars, the reported advantage is forced upward by the selection procedure. The qualitative summary that low-to-moderate λp and moderate-to-high λr are best is likewise a description of the same post hoc maxima, so the central quantitative claim reduces to the fitting procedure rather than being independently confirmed.

  2. fitted input called prediction [Section 4.2, Table 3]
    "Table 3 shows the relative performance gain on downstream tasks for DPO on top of WIT (with optimal setting of prompt and response token weights) over DPO on conventional instruction tuning. ... We find that the optimal configuration of prompt and response token weights for DPO yields a relative gain of nearly 8%."

    The DPO transfer claim repeats the same post hoc selection: the 'optimal (λp, λr)' for DPO is chosen from the same grid and evaluated on the same five benchmarks used to select it. The 8.01% average is therefore again a maximum over noisy cells; several entries are near zero (+0.00%, +0.27%, +0.81%), while the largest gains arise from a collapsed conventional-DPO baseline (Llama-3-8B on LIMA: 28.94 after DPO versus 45.48 for pre-DPO conventional IT). The reported transfer gain is thus partly a selected-maximum artifact over an unstable baseline, not an independent validation that WIT is a better DPO starting point.

full rationale

The loss formulation itself is not circular: Eq. (2) is an explicit weighted cross-entropy that contains the conventional LIT as the (λp, λr) = (0, 1) corner, and the paper does not derive WIT from the benchmarks. However, the paper's headline quantitative claims reduce to a fitting procedure. The optimal weights in Tables 1 and 3 are selected post hoc from a 36-cell grid evaluated on the same five benchmarks where the gains are reported, so the 6.55% and 8.01% averages are maxima-of-grid comparisons rather than independent predictions; this is the fitted-input-called-prediction pattern. The robustness section relies on POSIX (Chatterjee et al., 2024) by the same four authors, but POSIX is an externally published metric and is not defined in terms of the present results, so that self-citation is not load-bearing. No uniqueness theorem, ansatz-via-citation, or renaming-of-known-result chain is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim carries two fitted weights (λp, λr) chosen per setting from a grid and evaluated on the same benchmarks used for the gains, plus schedule choices (epochs, learning rate) inherited from prior work. The main axioms are domain assumptions: that fixed scalar reweighting of the CE loss is a sufficient loss space, that equal-weight averaging of five heterogeneous benchmarks is a meaningful objective, and that single-seed runs are informative. No invented entities are introduced.

free parameters (3)
  • lambda_p (prompt token weight) = selected per (model, dataset, benchmark) from {0, 0.2, 0.4, 0.6, 0.8, 1.0}; average optima 0.17 (BBH) to 0.48 (IFEval)
    The central knob of WIT. The 'optimal' value is chosen by evaluating all grid cells on the same five benchmarks used to compute the reported gains, so the quantitative headline depends directly on this fitted choice.
  • lambda_r (response token weight) = selected per setting from the same grid; average optima 0.43 (IFEval) to 0.64 (AlpacaEval)
    The second knob. Reported headline gains and DPO-transfer claims are the gains of the best (lambda_p, lambda_r) cell, i.e., a maximum over the grid.
  • epochs per dataset = 5 (LIMA), 2 (Alpaca-Cleaned), 1 (Tülu-v2)
    Schedule chosen per dataset; if the schedule interacts with loss weighting (small data needs more epochs), the optimal-weight comparisons are confounded across datasets.
assumptions (4)
  • domain assumption Weighting the prompt-token and response-token cross-entropies with fixed scalars is a valid and sufficient way to search the loss space for instruction tuning.
    Eq. 2 defines the entire method; the paper does not justify this family against other loss forms (e.g., adaptive token weighting or margin-based losses).
  • domain assumption Equal-weight averaging of MMLU, BBH, IFEval, AlpacaEval (GPT-4o-mini judged) and MT-Bench (Llama-3.3-70B judged, scaled by 10) is a meaningful single objective.
    Section 3.3 and Figures 2 and 3: 'average performance across five benchmarks' mixes accuracy, win-rate, and 1-10 scores; optimal weights shift across benchmarks (Table 2), so the averaged optimum is a modeling choice.
  • domain assumption Single-seed, single-run differences of 0.5 to 2 benchmark points are treated as informative signal.
    Section 4: no standard deviations or seeds are reported; e.g., LIMA gains of +0.45% to +2.15% and Llama-3.2-3B gains of +0.58% are used as evidence for WIT.
  • standard math Cross-entropy is computed and normalized per token class as in Eq. 2 with the indicator-function denominator.
    The loss algebra is standard weighted CE; the indicator-function normalization is the paper's choice to make (0,1) reduce exactly to conventional LIT.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Effect of Instruction Tuning Loss on Generalization." pith.science (2026). https://pith.science/paper/QQRGEENY

@misc{pith2026250707817,
  author       = {Pith},
  title        = {Pith review of: On the Effect of Instruction Tuning Loss on Generalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QQRGEENY}},
  note         = {Machine review of arXiv:2507.07817}
}
read the original abstract

Instruction Tuning has emerged as a pivotal post-training paradigm that enables pre-trained language models to better follow user instructions. Despite its significance, little attention has been given to optimizing the loss function used. A fundamental, yet often overlooked, question is whether the conventional auto-regressive objective - where loss is computed only on response tokens, excluding prompt tokens - is truly optimal for instruction tuning. In this work, we systematically investigate the impact of differentially weighting prompt and response tokens in instruction tuning loss, and propose Weighted Instruction Tuning (WIT) as a better alternative to conventional instruction tuning. Through extensive experiments on five language models of different families and scale, three finetuning datasets of different sizes, and five diverse evaluation benchmarks, we show that the standard instruction tuning loss often yields suboptimal performance and limited robustness to input prompt variations. We find that a low-to-moderate weight for prompt tokens coupled with a moderate-to-high weight for response tokens yields the best-performing models across settings and also serve as better starting points for the subsequent preference alignment training. These findings highlight the need to reconsider instruction tuning loss and offer actionable insights for developing more robust and generalizable models. Our code is open-sourced at https://github.com/kowndinya-renduchintala/WIT.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Toward Cybersecurity-Expert Small Language Models

    cs.CL 2025-10 conditional novelty 5.0 of 10

    A family of 4B–20B cybersecurity models fine-tuned on an enriched, expert-steered reasoning dataset matches or beats larger frontier models on core CTI benchmarks.

Reference graph

Works this paper leans on

59 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Seokhyun An and Hyounghun Kim. 2024. https://arxiv.org/abs/2410.02465 Response tuning: Aligning large language models without instruction . arXiv preprint arXiv:2410.02465v1

    work page Pith review arXiv 2024

  4. [4]

    Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, and Donald Metzler

    Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, and Donald Metzler. 2022. https://openreview.net/forum?id=Vzh1BFUCiIX ExT5 : Towards extreme multi-task scaling for transfer learning . In International Conference on Learn...

    work page 2022

  5. [5]

    Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, and Christopher Re

    Simran Arora, Avanika Narayan, Mayee F. Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, and Christopher Re. 2023. https://openreview.net/forum?id=bhUPJnS2g0X Ask me anything: A simple strategy for prompting language models . In The Eleventh International Conference on Learning Representations

    work page 2023

  6. [6]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    work page 2020

  7. [7]

    Anwoy Chatterjee, H S V N S Kowndinya Renduchintala, Sumit Bhatia, and Tanmoy Chakraborty. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.852 POSIX : A prompt sensitivity index for large language models . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14550--14565, Miami, Florida, USA. Association for Computational ...

    doi:10.18653/v1/2024.findings-emnlp.852 2024

  8. [8]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing GPT-4 with 90\ See https://vicuna.lmsys.org (accessed 14 April 2023)

    work page 2023

Show all 59 references
  1. [9]

    Le, Sergey Levine, and Yi Ma

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. 2025. https://openreview.net/forum?id=dYur3yabMj SFT memorizes, RL generalizes: A comparative study of foundation model post-training . In Forty-second I...

    2025

  2. [10]

    Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinso...

    2024

  3. [11]

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm Free dolly: Introducing the world's ...

    2023

  4. [12]

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. https://proceedings.mlr.press/v235/cui24f.html ULTRAFEEDBACK : Boosting language models with scaled AI feedback . In Procee...

    2024

  5. [13]

    Tri Dao. 2024. https://openreview.net/forum?id=mZn2Xyh9Ec FlashAttention-2 : Faster attention with better parallelism and work partitioning . In The Twelfth International Conference on Learning Representations

    2024

  6. [14]

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.183 Enhancing chat language models by scaling high-quality instructional conversations . In Proceedings of the 2023 Conference ...

    2023 doi

  7. [15]

    Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2024. https://doi.org/10.18653/v1/2024.acl-long.12 How abilities in large language models are affected by supervised fine-tuning data compositio...

    2024 doi

  8. [16]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024 doi

  9. [17]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In International Conference on Learning Representations

    2021

  10. [18]

    Liu, Percy Liang, and Christopher D

    John Hewitt, Nelson F. Liu, Percy Liang, and Christopher D. Manning. 2024. https://arxiv.org/abs/2409.14254 Instruction following without instruction tuning . arXiv preprint arXiv:2409.14254v1

    2024 arXiv

  11. [19]

    Mathew Huerta-Enochian and Seung Yong Ko. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1267 Instruction fine-tuning: Does prompt loss matter? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22771--22795, Miami, Florida, USA...

    2024 doi

  12. [20]

    Smith, Iz Beltagy, and Hannaneh Hajishirzi

    Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2311.10702 Camels in a changing climate: Enhancing lm adaptation with tulu 2...

    2023 arXiv

  13. [21]

    Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein

    Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R. Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. https://openreview.net/forum?id=0bMmZ3fkCk NEFT une: No...

    2024

  14. [22]

    Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. 2023. https://arxiv.org/abs/2303.14742 Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases . arXiv preprint arXiv:...

    2023 arXiv

  15. [23]

    o pf, Yannic Kilcher, Dimitri von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri von R \"u tte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Rich \'a rd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu N...

    2023

  16. [24]

    Alina Leidinger, Robert van Rooij, and Ekaterina Shutova. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.618 The language of prompting: What linguistic properties make a prompt successful? In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9...

    2023 doi

  17. [25]

    Weston, and Mike Lewis

    Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason E. Weston, and Mike Lewis. 2024. https://openreview.net/forum?id=1oijHJBRsT Self-alignment with instruction backtranslation . In The Twelfth International Conference on Learning Representations

    2024

  18. [26]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval

    2023

  19. [27]

    Wing Lian, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". 2023. OpenOrca : An open dataset of gpt augmented flan reasoning traces. https://https://huggingface.co/datasets/Open-Orca/OpenOrca

    2023

  20. [28]

    Le, Barret Zoph, Jason Wei, and Adam Roberts

    Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. https://proceedings.mlr.press/v202/longpre23a.html The flan collection: Designing data and methods for effective instruction tuning . ...

    2023

  21. [29]

    Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. https://doi.org/10.1162/tacl_a_00536 Locally typical sampling . Transactions of the Association for Computational Linguistics, 11:102--121

    2023 doi

  22. [30]

    Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, and Ahmed Awadallah. 2023. https://arxiv.org/abs/2311.110...

    2023 arXiv

  23. [31]

    Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. 2024. https://doi.org/10.1162/tacl_a_00681 State of what art? a call for multi-prompt LLM evaluation . Transactions of the Association for Computational Linguistics, 12:933--949

    2024 doi

  24. [32]

    Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. https://arxiv.org/abs/2306.02707 Orca: Progressive learning from complex explanation traces of GPT-4 . arXiv preprint arXiv:2306.02707v1

    2023 arXiv

  25. [33]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022

  26. [34]

    Wei Pang, Chuan Zhou, Xiao-Hua Zhou, and Xiaojie Wang. 2024. https://doi.org/10.18653/v1/2024.findings-acl.341 Phased instruction fine-tuning for large language models . In Findings of the Association for Computational Linguistics: ACL 2024, pages 5735--5748, Bangkok, Thailand...

    2024 doi

  27. [35]

    Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang, Krishnateja Killamsetty, Shivchander Sudalairaj, Wenlong Zhao, Seungwook Han, Abhishek Bhandwaldar, Guangxuan Xu, Kai Xu, Ligong Han, Luke Inglis, and Akash Srivastava. 2025. https://openreview.net/forum?id=eENHKMTOfW Unveiling t...

    2025

  28. [36]

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. https://arxiv.org/abs/2304.03277 Instruction tuning with gpt-4 . arXiv preprint arXiv:2304.03277v1

    2023 arXiv

  29. [37]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--53741

    2023

  30. [38]

    H S V N S Kowndinya Renduchintala, Sumit Bhatia, and Ganesh Ramakrishnan. 2024. https://doi.org/10.18653/v1/2024.findings-acl.766 SMART : Submodular data mixture strategy for instruction tuning . In Findings of the Association for Computational Linguistics: ACL 2024, pages 129...

    2024 doi

  31. [39]

    Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajy...

    2022

  32. [40]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. https://openreview.net/forum?id=RIu5lyNXjT Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting . In The Twelfth International...

    2024

  33. [41]

    Zhengxiang Shi, Adam Yang, Bin Wu, Laurence Aitchison, Emine Yilmaz, and Aldo Lipani. 2025. Instruction tuning with loss over instructions. Advances in Neural Information Processing Systems, 37:69176--69205

    2025

  34. [42]

    Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, and Guoyin Wang. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.603 Text classification via large language models . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 899...

    2023 doi

  35. [43]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. https://doi.org/10.18653/v1/2023.findings-acl.824 Challenging BIG -bench tasks and whether chain-of-thought c...

    2023 doi

  36. [44]

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: An instruction-following llama model

    2023

  37. [45]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  38. [46]

    Anton Voronov, Lena Wolf, and Max Ryabinin. 2024. https://doi.org/10.18653/v1/2024.findings-acl.375 Mind your format: Towards consistent evaluation of in-context learning improvements . In Findings of the Association for Computational Linguistics: ACL 2024, pages 6287--6310, B...

    2024 doi

  39. [47]

    Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, and Sadao Kurohashi. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.214 GPT - RE : In-context learning for relation extraction using large language models . In Proceedings of the 2023 Conference on Empi...

    2023 doi

  40. [48]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022 a . https://arxiv.org/abs/2203.11171 Self-consistency improves chain of thought reasoning in language models . arXiv preprint arXiv:2203.11171v4

    2022 arXiv

  41. [49]

    Smith, Iz Beltagy, and Hannaneh Hajishirzi

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neur...

    2023

  42. [50]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuzni...

    2022

  43. [51]

    Zige Wang, Wanjun Zhong, Yufei Wang, Qi Zhu, Fei Mi, Baojun Wang, Lifeng Shang, Xin Jiang, and Qun Liu. 2024. https://arxiv.org/abs/2312.01700 Data management for training large language models: A survey . arXiv preprint arXiv:2312.01700v3

    2024 arXiv

  44. [52]

    Dai, and Quoc V

    Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. https://openreview.net/forum?id=gEZrGCozdqR Finetuned language models are zero-shot learners . In International Conference on Learning Representations

    2022

  45. [53]

    Juncheng Xie, Shensian Syu, and Hung-yi Lee. 2024. https://arxiv.org/abs/2409.00096 Non-instructional fine-tuning: Enabling instruction-following capabilities in pre-trained language models without instruction-following data . arXiv preprint arXiv:2409.00096v1

    2024 arXiv

  46. [54]

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023. https://arxiv.org/abs/2308.01825 Scaling relationship on learning mathematical reasoning with large language models . arXiv preprint arXiv:2308.01825v2

    2023 arXiv

  47. [55]

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2023. https://arxiv.org/abs/2308.10792 Instruction tuning for large language models: A survey . arXiv preprint arXiv:2308.10792v8

    2023

  48. [56]

    Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. https://proceedings.mlr.press/v139/zhao21c.html Calibrate before use: Improving few-shot performance of language models . In Proceedings of the 38th International Conference on Machine Learning, volume 139 ...

    2021

  49. [57]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  50. [58]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023 a . LIMA : Less is more for alignment. Advances in Neural Information Processing Systems...

    2023

  51. [59]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023 b . https://arxiv.org/abs/2311.07911 Instruction-following evaluation for large language models . arXiv preprint arXiv:2311.07911v1

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.