Pith. sign in

REVIEW 6 major objections 6 minor 61 references

ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis

T0 review · 6 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ResoFilter claims that a sample's value for fine-tuning is visible in how little it moves the model's deep weights, and that keeping only the low-motion half of a math dataset matches full fine-tuning.

desk verdict A genuinely new per-sample weight-delta filter that shows a believable 50%-data effect on math, but the headline number is selected on GSM8k and the per-sample fine-tuning budget is unspecified. read the letter →

arxiv 2412.14809 v3 pith:4TG2TGAX submitted 2024-12-19 cs.CL

Zeao Tu , Xiangdi Meng , Yu He , Zihan Yao , Tianyu Qi , Jun Liu , Ming Li This is my paper · ORCID
classification cs.CL
keywords dataselectioninstructiontuningsyntheticfilteringparameterdifferenceweightanalysislargelanguagemodelsmathematicalreasoningfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ResoFilter proposes a fine-grained way to prune synthetic fine-tuning data for large language models: measure how much each training example moves the model's weights when the base model is fine-tuned on that example alone, then keep the examples that move the weights least. The paper argues that these low-disruption examples are the useful ones, while high-disruption examples are likely noise, outliers, or off-task content. On the MetaMath mathematics dataset, using the 50% of data selected this way matches full-dataset fine-tuning on GSM8k, and using 75% slightly beats full fine-tuning on two of three tested models. If this holds, data filtering becomes a property of the data-model pair rather than an external quality score, which would make synthetic data pipelines cheaper and more interpretable.

What carries the argument

The central object is the per-sample parameter-difference score, the Data-Parameter feature. It is computed by fine-tuning a fresh copy of the base model on a single sample $d_i$ to obtain $M_i$, taking $\Delta W = M_i - M_0$, and averaging the mean of $\Delta W$ over the up-projection modules of the last $n=3$ layers. This score ranks every data point, and the method keeps the bottom $k$ samples while restoring their original order for training. The score is the bridge between data and task: it says a useful sample is one that changes deep model weights little, while a disruptive sample is treated as low-quality.

What would settle it

On a small MetaMath subset, train the base model on the full subset, then retrain with each of the lowest-scored samples removed one at a time and with each of the highest-scored samples removed one at a time; if removing low-scored samples does not consistently hurt more than removing randomly chosen samples, the per-sample weight-change ranking is not measuring marginal contribution.

Watch

Extended reading notes

Core claim

The central discovery is that the magnitude of the per-sample parameter change during single-example fine-tuning is a usable data-quality signal. For each sample, the method trains a fresh copy of the base model on that one sample, subtracts the original weights, and averages the mean absolute difference over the up-projection (Wup) matrices of the last three layers; this value is the sample's score. Samples with the smallest scores are retained and the rest discarded, preserving original order for training. On MetaMath, retaining the bottom 50% by this score gives GSM8k accuracy essentially equal to full SFT (for example, 0.6497 versus 0.649 for Gemma2-2B), and retaining 75% outperforms the full dataset on Gemma2-2B and Llama2-7B. Feature analysis characterizes low-score ('good') data as longer, more repetitive, internally similar samples and high-score ('dirty') data as short, rare-vocabulary, diverse samples.

Load-bearing premise

The load-bearing premise is that the weight change from fine-tuning a fresh copy of the base model on one example, averaged over the last three layers' up-projection matrices, predicts that example's contribution when thousands of examples are fine-tuned together; the paper does not test this premise directly.

Editorial extensions

If this is right

  • Retaining only 50% of MetaMath by ResoFilter matches full-dataset SFT on GSM8k across Gemma2-2B, Llama2-7B, and Llama2-13B; at 75% it slightly beats full SFT on Gemma2-2B and Llama2-7B.
  • The selection transfers across model scales: data filtered by Gemma2-2B improves Gemma2-9B, and data filtered by Llama2-7B improves Llama2-70B, with relative gains comparable to those seen on the smaller models.
  • The signal transfers across domains: on HumanEval-CodeX, 50% of filtered code data gives 0.425 versus 0.3549 for random sampling, while on MMLU and BBH the results are comparable or mixed with full fine-tuning often below the base model.
  • The retained low-diff subset has a consistent profile: longer sequences, common vocabulary, low unique-token ratio, and high internal query similarity, suggesting the score selects for representative, low-noise examples rather than rare or outlier content.
  • Because the filtering can be run once on a small model and reused on larger ones, the method offers a practical filter-once-apply-many workflow for synthetic data construction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors do not state is that the score should behave like an expectedness measure, so a direct test is to compare ResoFilter rankings with base-model loss or perplexity to see how much overlap exists.
  • The paper does not specify the single-sample fine-tuning budget (number of steps, learning rate, optimizer); a stress test is whether rankings survive changes to that protocol, since the method depends on those scores being stable.
  • The authors' own limitation section notes untested regimes (models above 70B parameters, Mixture-of-Experts, and conversational systems); the filter-once-apply-many pattern would be on firmer ground if tested there.
  • If the mechanism is causal, the low-diff subset should also show up in training dynamics as flatter loss curves or less forgetting; the paper does not report such a test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. This paper introduces ResoFilter, a data-filtering method for fine-tuning large language models. For each training sample, the method fine-tunes a fresh copy of the base model on that single sample, computes the mean absolute parameter difference in the up-projection matrices of the last three transformer layers before and after this one-sample update, and uses that scalar as a 'diff' score. The final fine-tuning set keeps the samples with the lowest diff scores, preserving their original dataset order, and discards the highest-scoring samples, which the authors interpret as 'dirty' data. The authors evaluate ResoFilter on MetaMath with GSM8k accuracy across Gemma2-2B, Llama2-7B, and Llama2-13B, reporting that using only 50% of the data selected by the method matches or slightly exceeds full-data SFT in two of three models. They also report cross-domain results on code (HumanEval-CodeX) and general knowledge (MMLU, BBH), plus a cross-model transfer experiment where data filtered by a small model is used to fine-tune larger models. The paper includes ablations over weight modules, statistical aggregators, layer positions, and data ordering, all evaluated on GSM8k.

Significance. If the 50% result survives a held-out configuration test, ResoFilter would be a practical contribution: it is conceptually simple, interpretable, and its 'filter-once-apply-many' protocol reduces the cost of data selection for larger models. The feature analysis in Section 5.2, which ties low-diff samples to longer, more repetitive, and more self-similar text, is a useful qualitative insight. The paper also reports cross-model scaling to 9B and 70B parameters, which is a concrete strength, and Appendix C provides three-seed variance for one model, which is a welcome reproducibility step. However, the current evidence is weakened by test-set-selected configurations, missing per-sample training details, single-run comparisons, and a contradictory generalization claim; these issues are addressable in revision, so the result is not yet ready for acceptance.

major comments (6)
  1. [Section 5.1, Tables 9/4, Figure 2, Table 1] The default configuration (Wup module, last-3 layers, mean statistic) is selected by comparing GSM8k scores across ablations, and the same GSM8k benchmark is then used in Table 1 to report the headline that 50% selected data matches full SFT. This constitutes selection on the test set: the reported 50% scores are the best among a menu of variants screened on GSM8k, so the advantage over random/full SFT may be inflated by noise-driven choices. For Llama2-13B, the 50% score (0.6853) is already below full SFT (0.6935), so the 'comparable' claim is not adversarially tested. I request a held-out configuration selection: choose the configuration on a separate validation set or a different task, then report Table 1 with that pre-registered configuration, or at least show that the conclusion is stable across the menu of configurations.
  2. [Algorithm 1, line 14, and Section 4.1] There is an inconsistency between p as the fraction of data retained and p as the filtering ratio. Section 4.1 states that filtering methods produce 25%, 50%, and 75% of the total data, so p=0.25 means retaining 25%. Algorithm 1 computes k = ⌊|D|×(1−p)⌋ and its comment says 'this effectively removes the top p% of samples,' which for p=0.25 retains 75% and removes 25%. Equation (4) is consistent with p as the retained fraction, but Algorithm 1 is not. Please align the pseudocode, the comments, and the experimental description, and state explicitly whether p denotes the retained fraction or the removed fraction throughout.
  3. [Tables 1, 3, 4, 9, 10 and Appendix C] Table 8 reports three seeds only for Gemma2-2B and shows run-to-run variation of roughly ±0.003 to ±0.01 across ratios. The main comparisons in Tables 1, 3, 4, 9, and 10 are single runs, and many differences between methods are within this noise band (e.g., in Table 1 our method at 25% equals random at 25% for Gemma2-2B, both 0.6042). The claim that ResoFilter consistently outperforms other selection approaches is therefore not supported for those cells. Please report per-seed results or error bars for all headline tables, at least for one model per domain.
  4. [Section 3.3 and Appendix F, Algorithm 1] The per-sample fine-tuning that produces the scores diff_i is not specified: the number of optimization steps, learning rate, optimizer, batch size, sequence length, and compute budget are all omitted. Without these details the scores are not reproducible, and the core assumption that a single-sample fine-tuned weight change predicts a sample's contribution in a multi-sample run is never tested. Please specify the per-sample training protocol and include a sensitivity analysis over its key hyperparameters.
  5. [Section 4.2, Table 1] The generalization claim that the method 'consistently outperforms random sampling across different domains' is contradicted by the lower half of Table 1: on MMLU at 50% our method (0.4893) is below random (0.5152), and on BBH at 50% our method (0.3620) is below random (0.3759). Please correct the claim, and if the method is not intended to improve on general knowledge benchmarks, state this clearly and provide a test of the hypothesis that the method does not harm beyond random.
  6. [Section 3.2, Eqs. (1)-(3)] The objective function E is presented as capturing an optimal trade-off, but Fcharacteristic(p) is the average of f(j,M) over the selected set Dp while f(j,M) is never defined independently of the selection rule. Since the algorithm selects exactly the samples with the smallest s(j,M), any ranking of 'characteristic intensity' that is monotone in s makes the claimed optimum in p true by construction. The parameters β and λ are not estimated or used. Please either show how Eq. (1) guides the algorithm or restate it clearly as a descriptive model rather than an optimization objective.
minor comments (6)
  1. [Throughout] There are several typos and formatting issues: 'Alabtion' in Section 5.1 should be 'Ablation'; 'Nuggest' in Table 1 should be 'Nuggets'; 'examned' in Section 5.2 should be 'examined'; 'MaxLegnth' in Table 2 should be 'MaxLength'; and 'V olume' appears in two references and should be 'Volume'.
  2. [Section 4.1, Baselines] The description of the Loss and PPL baselines says the data are 'ranked from high to low based on loss and PPL, and then selected the top 25%, 50%, 75%,' but the ranking direction determines whether high-loss or low-loss samples are selected; please clarify whether the selected samples are those with the highest or lowest loss/PPL, and confirm this matches the code.
  3. [Section 3.3] The statement that data points causing smaller differences are 'potentially more valuable' appears before the ablation evidence; it would be clearer to present the ablation results in Section 5.1 as the basis for this choice.
  4. [Figure 2] The x-axis is described as 'the first layer to the 26th layer'; please specify the model and layer indexing (0-based or 1-based) and note the total number of layers in that model.
  5. [Abstract and Appendix C] The abstract states that code and data will be released upon acceptance, while the paper also states that the source code is publicly available in a GitHub repository; please reconcile these statements and include a version or commit hash.
  6. [Table 3] The 'Base Score' column lists Gemma2-9B base 0.69 and Llama2-70B base 0.61, but the text says '3.2% vs. 3.4% average gain at p25'; please define how these percentage gains are computed from the table values.

Circularity Check

2 steps flagged · score 5.0 of 10

The low-diff selection rule makes the 'high-quality data' conclusion true by definition, and the headline GSM8k result is configuration-selected on the same test set; the core benchmark is external but not independent.

  1. self definitional [Section 3.2-3.3 (Eqs. 1-4), Algorithm 1 (Appendix F), and Section 5.2 (Feature Analysis)]
    "We consider data points causing smaller differences in the last n layers of the model as potentially more valuable, as they are less likely to disrupt previously acquired knowledge. The High Diff Value set i.e 'dirty sample' is the top 1% of samples with the highest mean difference, while the Low Diff Value set i.e 'good sample' means the bottom 1% with the lowest mean difference."

    Equation (4) selects Dp using rank(s(j,M)), while Equation (3) defines Fcharacteristic as the average of f(j,M) over Dp; Algorithm 1 uses diff_i (mean Wup difference in the last n layers) as that score and keeps the bottom-k samples. Thus a low-diff subset has a high Fcharacteristic by construction, so the 'objective' E in Equation (1) merely restates the selection rule rather than deriving a trade-off. Section 5.2 then labels the top/bottom diff groups 'dirty'/'good' using the same score and interprets their lexical features as explaining why low-diff data is 'crucial,' using the score as both the grouping variable and the quality label.

  2. fitted input called prediction [Section 3.3 and Section 5.1 (module/statistic/layer ablations) vs. Table 1 / Section 4.2 (main GSM8k results)]
    "Based on our comprehensive analysis (detailed in Appendix D), we identified Wup as the most effective weight module for data filtering. As shown in Table 9, Wup consistently outperforms other modules across different data ratios... Unless otherwise specified, we use the mean difference of weights in the last three layers as our default method for the following experiments."

    The default module (Wup), statistic (mean), and layer range (last three layers) are selected by inspecting GSM8k scores in Table 9, Table 4, and Figure 2, and the headline claim that 50% of ResoFilter-selected MetaMath matches full SFT is then evaluated on the same GSM8k benchmark in Table 1. The reported 50% GSM8k number is therefore a configuration-screened result rather than an out-of-sample prediction; its advantage over random or full SFT is not independent of the choices made after seeing GSM8k.

full rationale

The paper contains no load-bearing self-citation chain: the only overlapping-author reference (Yao et al., 2024) appears in background discussion and is not used to justify the method. However, two steps are circular or in-sample. First, the low-diff selection rule and Fcharacteristic are the same quantity: keeping bottom-diff samples maximizes Fcharacteristic by definition, so the claimed 'optimal trade-off' in Equation (1) is a tautology, and the Section 5.2 'dirty/good' feature analysis re-describes groups formed by that same score rather than independently validating it. Second, the Wup/last-3/mean configuration is selected from GSM8k ablations, and the main 50%-matches-full-SFT claim is measured on that same GSM8k set; the benchmark numbers are external and nontrivial, but the headline evaluation is not held out from configuration selection. The method is also tested on independent domains (code, MMLU, BBH) where it underperforms random at several ratios, which limits the generalization claim but does not make it circular. Overall, the central benchmark has independent content, but the definitional selection objective and the test-set-selected configuration justify a score of 5 rather than 0-2.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or model entities; the 'resonance' in the title is a metaphor for the weight-difference score. The load-bearing ingredients are hand-picked components (module, depth, statistic, ratio) plus an unstated per-sample fine-tuning budget, and an objective function that is defined but never used.

free parameters (6)
  • number of layers n = 3
    Set to last 3 layers after layer-sweep experiments (Figure 2, Appendix D); no held-out selection.
  • module choice = Wup (up_proj)
    Chosen because Table 9 shows Wup best at 50/75%; this is a post-hoc choice on the evaluation benchmark.
  • statistic aggregator = mean
    Table 4 shows p99 best at 25% and p90 best at 50/75%; mean is adopted as default despite not being best in any ratio.
  • filtering ratio p = 25%, 50%, 75% in experiments
    User-chosen; the objective function in Eq 1 is not used to derive an optimal p.
  • beta and lambda = not specified
    Eq 1-2 define these as tuning parameters but no values are given and the objective is never optimized.
  • per-sample fine-tuning budget = not stated
    Algorithm 1 says 'Fine-tune M0 using di' but does not specify steps, optimizer, or learning rate for the per-sample pass; this controls the diff scores.
assumptions (4)
  • domain assumption Per-sample fine-tuning weight changes proxy the influence of a sample in joint training.
    Algorithm 1 computes diff_i by fine-tuning M0 on each di in isolation; the paper asserts without evidence that this reveals the sample's contribution when many samples are trained together (Section 3.3).
  • domain assumption Samples causing small parameter changes in the last layers are higher quality.
    Section 3.3: 'We consider data points causing smaller differences in the last n layers of the model as potentially more valuable.' This is the core prior; it is validated only indirectly through benchmark accuracy.
  • domain assumption The configuration fixed after ablations (Wup, last 3 layers, mean) transfers across models and domains.
    The module and layer choices were selected on GSM8k (Tables 4, 9, Figure 2) and then applied to code and general QA; transferability is assumed rather than derived.
  • ad hoc to paper Fine-tuning cannot improve general knowledge benchmarks like MMLU/BBH, so performance drops are expected rather than method failures.
    Appendix B explains away negative MMLU/BBH results by asserting fundamental limits of fine-tuning, citing Sun and Dredze 2024; this claim is used to insulate the method from negative evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis." pith.science (2026). https://pith.science/paper/4TG2TGAX

@misc{pith2026241214809,
  author       = {Pith},
  title        = {Pith review of: ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4TG2TGAX}},
  note         = {Machine review of arXiv:2412.14809}
}
read the original abstract

Large language models (LLMs) have shown remarkable effectiveness across various domains, with data augmentation methods utilizing GPT for synthetic data generation becoming prevalent. However, the quality and utility of augmented data remain questionable, and current methods lack clear metrics for evaluating data characteristics. To address these challenges, we propose ResoFilter, a novel method that integrates models, data, and tasks to refine datasets. ResoFilter leverages the fine-tuning process to obtain Data-Parameter features for data selection, offering improved interpretability by representing data characteristics through model weights. Our experiments demonstrate that ResoFilter achieves comparable results to full-scale fine-tuning using only half the data in mathematical tasks and exhibits strong generalization across different models and domains. This method provides valuable insights for constructing synthetic datasets and evaluating high-quality data, offering a promising solution for enhancing data augmentation techniques and improving training dataset quality for LLMs. For reproducibility, we will release our code and data upon acceptance.

Figures

Figures reproduced from arXiv: 2412.14809 by the authors.

Figure 1
Figure 1. Workflow for our method. The left side of the figure illustrates the detailed process of calculating [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Here we used filtering based on different [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 2
Figure 2. We analyzed the Wup weights of the model from the first layer to the 26th layer. The 25% shows a continuous upward trend, while the 75% fluctuates within a certain range. results, significantly better than the random base￾line. The mean method performs stably at all data ratios and consistently outperforms random selec￾tion. In contrast, the performance of cosine simi￾larity and Pearson correlation coefficient metho… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Illustrates the token Frequency distribution [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: To evaluate the lexical diversity of the dataset, [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: In order to gain a deeper understanding of the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 6 canonical work pages

  1. [1]

    Yelaman Abdullin, Diego Molla-Aliod, Bahadorreza Ofoghi, John Yearwood, and Qingyang Li. 2024. Synthetic dialogue dataset generation using llm agents. arXiv preprint arXiv:2401.17461

    arXiv 2024

  2. [2]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

    arXiv 2022

  3. [3]

    Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. 2023. https://arxiv.org/abs/2307.06290 Instruction mining: When data mining meets large language model finetuning . Preprint, arXiv:2307.06290

    arXiv 2023

  4. [4]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2023 a . https://arxiv.org/abs/2309.07597 Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . Preprint, arXiv:2309.07597

    arXiv 2023

  5. [5]

    Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. 2023 b . Alpagasus: Training a better alpaca with fewer data. arXiv preprint arXiv:2307.08701

    arXiv 2023

  6. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

    arXiv 2021

  7. [7]

    Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. 2023. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883

    arXiv 2023

  8. [8]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

Show all 61 references
  1. [9]

    Alexis Conneau and Guillaume Lample. 2019. https://proceedings.neurips.cc/paper_files/paper/2019/file/c04c19c2c2474dbf5f7ac4372c5b9af1-Paper.pdf Cross-lingual language model pretraining . In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc

    2019

  2. [10]

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. Company Blog of Databricks

    2023

  3. [11]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2022 doi

  4. [12]

    Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Wei Liu, Ninghao Liu, et al. 2023. Auggpt: Leveraging chatgpt for text data augmentation. arXiv preprint arXiv:2302.13007

    2023 arXiv

  5. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://aclanthology.org/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . NAACL

    2019

  6. [14]

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027

    2020 arXiv

  7. [15]

    Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. https://arxiv.org/abs/2305.00586 How does gpt-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model . Preprint, arXiv:2305.00586

    2023 arXiv

  8. [16]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  9. [17]

    Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023. https://doi.org/10.18653/v1/2023.acl-long.806 Unnatural instructions: Tuning language models with (almost) no human labor . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics...

    2023 doi

  10. [18]

    Mojan Javaheripi, S \'e bastien Bubeck, Marah Abdin, Jyoti Aneja, Sebastien Bubeck, Caio C \'e sar Teodoro Mendes, Weizhu Chen, Allie Del Giorno, Ronen Eldan, Sivakanth Gopi, et al. 2023. Phi-2: The surprising power of small language models. Microsoft Research Blog

    2023

  11. [19]

    o pf, Yannic Kilcher, Dimitri von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri von R \"u tte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich \'a rd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. Advances in Neura...

    2024

  12. [20]

    Po-Nien Kung, Fan Yin, Di Wu, Kai-Wei Chang, and Nanyun Peng. 2023. Active instruction tuning: Improving cross-task generalization by training on prompt sensitive tasks. arXiv preprint arXiv:2311.00288

    2023 arXiv

  13. [21]

    Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2022. https://doi.org/10.18653/v1/2022.acl-long.577 Deduplicating training data makes language models better . In Proceedings of the 60th Annual Meeting of t...

    2022 doi

  14. [22]

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2024 a . Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36

    2024

  15. [23]

    Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. 2024 b . Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. arXiv preprint arXiv:2402.00530

    2024 arXiv

  16. [24]

    Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. 2023 a . From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning. arXiv preprint arXiv:2308.12032

    2023 arXiv

  17. [25]

    Yunshui Li, Binyuan Hui, Xiaobo Xia, Jiaxi Yang, Min Yang, Lei Zhang, Shuzheng Si, Junhao Liu, Tongliang Liu, Fei Huang, et al. 2023 b . One shot learning as instruction data prospector for large language models. arXiv preprint arXiv:2312.10302

    2023 arXiv

  18. [26]

    Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2023. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312.15685

    2023 arXiv

  19. [27]

    Ziche Liu, Rui Ke, Feng Jiang, and Haizhou Li. 2024. https://api.semanticscholar.org/CorpusID:270620248 Take the essence and discard the dross: A rethinking on data selection for fine-tuning large language models . ArXiv, abs/2406.14115

    2024 arXiv

  20. [28]

    Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. 2023. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688

    2023 arXiv

  21. [29]

    Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2023. \# instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Conference on Learning Representations

    2023

  22. [30]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568

    2023 arXiv

  23. [31]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359--17372

    2022

  24. [32]

    Kshitij Mishra, Tamer Soliman, Anil Ramakrishna, A. G. Galstyan, and Anoop Kumar. 2024. https://api.semanticscholar.org/CorpusID:268417156 Correcting language model outputs by editing salient layers . In Findings

    2024

  25. [33]

    Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021. Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773

    2021 arXiv

  26. [34]

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. Generative representational instruction tuning. arXiv preprint arXiv:2402.09906

    2024 arXiv

  27. [35]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  28. [36]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  29. [37]

    Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

    Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. https://aclanthology.org/N18-1202 Deep contextualized word representations . NAACL

    2018

  30. [38]

    Alec Radford and Karthik Narasimhan. 2018. https://api.semanticscholar.org/CorpusID:49313245 Improving language understanding by generative pre-training

    2018

  31. [39]

    Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446

    2021 arXiv

  32. [40]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290

    2023 arXiv

  33. [41]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. https://jmlr.org/papers/volume21/20-074/20-074.pdf Exploring the limits of transfer learning with a unified text-to-text transformer . J. Mach. L...

    2020

  34. [42]

    Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyo...

    2022

  35. [43]

    Tobias Schimanski, Jingwei Ni, Mathias Kraus, Elliott Ash, and Markus Leippold. 2024. https://arxiv.org/abs/2402.08277 Towards faithful and robust llm specialists for evidence-based question-answering . Preprint, arXiv:2402.08277

    2024 arXiv

  36. [44]

    Kaiser Sun and Mark Dredze. 2024. https://arxiv.org/abs/2408.06663 Amuro & char: Analyzing the relationship between pre-training and fine-tuning of large language models . Preprint, arXiv:2408.06663

    2024 arXiv

  37. [45]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261

    2022 arXiv

  38. [46]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  39. [47]

    The Mistral AI Team. 2024. Mistral-7B-v0.3 . https://huggingface.co/mistralai/Mistral-7B-v0.3. Accessed: 2024-09-24

    2024

  40. [48]

    I Tenney. 2019. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950

    2019 arXiv

  41. [49]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  42. [50]

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. 2023 a . How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neural Information P...

    2023

  43. [51]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual ...

    2023 doi

  44. [52]

    Lai Wei, Zihao Jiang, Weiran Huang, and Lichao Sun. 2023. https://arxiv.org/abs/2308.12067 Instructiongpt-4: A 200-instruction paradigm for fine-tuning minigpt-4 . Preprint, arXiv:2308.12067

    2023 arXiv

  45. [53]

    Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzm \'a n, Armand Joulin, and Edouard Grave. 2020. https://aclanthology.org/2020.lrec-1.494 CCN et: Extracting high quality monolingual datasets from web crawl data . In Proceedings of the Twel...

    2020

  46. [54]

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. https://arxiv.org/abs/2402.04333 Less: Selecting influential data for targeted instruction tuning . Preprint, arXiv:2402.04333

    2024 arXiv

  47. [55]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244

    2023 arXiv

  48. [56]

    Zihan Yao, Yu He, Tianyu Qi, and Ming Li. 2024. https://api.semanticscholar.org/CorpusID:268876196 Scalable model editing via customized expert networks . ArXiv, abs/2404.02699

    2024 arXiv

  49. [57]

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284

    2023 arXiv

  50. [58]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2024. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36

    2024

  51. [59]

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593

    2019 arXiv

  52. [60]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.