Pith. sign in

REVIEW 3 major objections 4 minor 51 references

FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes FisherSFT, a greedy sentence-selection method that maximizes a log-determinant surrogate for the Fisher information of the softmax log-likelihood, and claims lower prediction error and better generated text at fixed…

desk verdict Useful heuristic with a genuinely fast lazy-greedy implementation, but the theory as stated does not hold; treat this as an empirical paper with a broken Theorem. read the letter →

arxiv 2505.14826 v1 pith:SLVOXWTK submitted 2025-05-20 cs.LG cs.CLstat.ML

Rohan Deb , Kiran Thekumparampil , Kousha Kalantari , Gaurush Hiranandani , Shoham Sabach , Branislav Kveton This is my paper
classification cs.LGcs.CLstat.ML MSC 62K0568T0790C27
keywords data-efficientfine-tuningsupervisedFisherinformationoptimaldesignsubmodularmaximizationlog-determinantdataselectionlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Supervised fine-tuning of a language model costs roughly in proportion to the number of training sentences, so choosing which sentences to spend that budget on is a direct path to cheaper adaptation. This paper proposes FisherSFT, which treats the fine-tuning loss as a product of multinomial logistic regressions over the model's pre-logit embeddings and selects sentences that maximize a tractable surrogate for the Fisher information of those regressions. The surrogate is the log determinant of the accumulated token-embedding covariance matrix, and the selection is done greedily, exploiting the monotone submodularity of log determinant for speed. The paper reports that, at the same budget, models fine-tuned on FisherSFT-selected sentences have lower maximum and mean prediction error than uniform, density, clustering, and sentence-level optimal-design baselines on synthetic and word2vec tasks, and that GPT-2 fine-tuned on its selections produces text an LLM judge prefers. If the claim holds, fine-tuning budgets can be spent on fewer, more informative sentences without sacrificing quality.

What carries the argument

The workhorse is the design matrix $V = I_d + \sum_{(i,j) \in \text{selected sentences}} x_{i,j}x_{i,j}^\top$, built from pre-logit embeddings $x_{i,j}$ of each token. Greedily choosing the sentence that maximally increases $\log \det V$, with $V$ updated after each pick, is a monotone submodular maximization, so a caching scheme (Algorithm 2) can skip recomputing gains that cannot improve and evaluate candidate sentences in parallel batches. The log-determinant criterion is the volume of the ellipsoid defined by $V$, so selection favors sentences whose tokens explore new embedding directions and disfavors redundancy. The theory attaches to this procedure a maximum-prediction-error bound via a strong-convexity argument that uses the softmax Hessian's Kronecker form and a concentration bound on the score gradient.

What would settle it

Take any vocabulary size $L$ and any softmax vector $p$ with equal entries, $p=(1/L,\dots,1/L)$. A direct eigenvalue calculation shows that $\mathrm{diag}(p)-pp^\top$ has a zero eigenvalue with eigenvector $(1,\dots,1)$, so Lemma 3.1's $\gamma$ condition is violated at the very point where the softmax is most uniform; hence the stated lower bound cannot be established by that lemma. An empirical check would be to run FisherSFT on a dataset whose token embeddings are all identical, where the log-det objective gives every sentence the same score; if the $O(1/\sqrt{n})$ error bound still holds in that setting, some other mechanism must be responsible.

Watch

Extended reading notes

Core claim

The central claim is that information gain for supervised fine-tuning can be measured almost entirely through the covariance of token embeddings in the last layer. The paper shows that the Hessian of the softmax negative log-likelihood has a Kronecker structure, with one factor depending on softmax probabilities and the other on the outer products $x_{i,j}x_{i,j}^\top$; Lemma 3.1 asserts that, under a uniform positive-definiteness condition on the probability factor, the log determinant of the full Hessian is bounded below by $d$ times the log determinant of the averaged embedding covariance. Maximizing that lower bound reduces to a $d\times d$ greedy optimal-design problem, over sentences rather than individual tokens. The paper's Theorem 4.3 then bounds the maximum prediction error of the maximum-likelihood estimator on the selected sentences by $\tilde{O}(dL/\sqrt{n})$ up to problem constants, under Assumption 4.2's per-step approximation ratio. Empirically, the method outperforms the compared sampling baselines on synthetic data, on word2vec features, and in a GPT-4o-judged text-generation comparison, where human-visible samples show less repetitive output than uniform sampling.

Load-bearing premise

The proof that the greedy log-determinant objective is a valid proxy for Fisher information rests on a per-token assumption that the matrix $\mathrm{diag}(p)-pp^\top$ is uniformly positive definite, but for a softmax probability vector this matrix always has a zero eigenvalue along the all-ones direction, so the assumed constant cannot exist.

Editorial extensions

If this is right

  • At a fixed budget of $n$ sentences, FisherSFT spends the budget on sentences whose token embeddings explore new directions, so redundant sentences do not consume fine-tuning compute.
  • The maximum prediction error of the selected-data estimator is proved to decay as $O(1/\sqrt{n})$ up to log factors, matching the rate of recent optimal-design bounds.
  • On the synthetic and word2vec tasks, FisherSFT reaches the best baseline's error with fewer sentences; in the synthetic setup, the best baseline's error at $n=2000$ is matched by FisherSFT at $n=1000$.
  • In the GPT-2 Shakespeare experiments, an LLM judge preferred text from models fine-tuned on FisherSFT-selected data over uniform, density, and AskLLM baselines at every tested budget from 100 to 5000 sentences.
  • Algorithm 2 produces the same selected subset as the naive greedy Algorithm 1 while exploiting monotone submodularity and batch parallelism to cut computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension, not pursued in the paper, is to apply the same greedy log-det selection to embeddings extracted at earlier layers or from adapter weights; the selection objective is layer-agnostic, though the theory's proof would need re-checking for that setting.
  • Because the selection score uses only embedding geometry and never the label tokens, the method's predicted advantage is largest when last-layer softmax structure dominates the task; domains where reasoning happens in earlier layers may show smaller gains.
  • The proof's $O(e^{2L})$ vocabulary factor is an artifact of a uniform strong-convexity bound; a corrected analysis on the $L-1$-dimensional quotient space after removing the all-ones logit direction could plausibly replace it with a factor of $L-1$.
  • An empirical check that would sharpen the practical claim is to compare FisherSFT against random selection on datasets with many near-duplicate sentences; the log-det objective should be robust to such redundancy, which would confirm that the gain comes from information content rather than coverage.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FisherSFT, a data-selection method for supervised fine-tuning of LLMs. It formulates the last layer of an autoregressive model as a multinomial logistic regression, approximates the Hessian of the negative log-likelihood by a per-token covariance matrix, and greedily selects sentences that maximize the log-determinant of the accumulated embedding covariance. The authors present a fast lazy-greedy implementation, a theoretical O(1/sqrt(n)) maximum-prediction-error bound (Theorem 4.3), and experiments on synthetic data, word2vec embeddings, and GPT-2 fine-tuning on TinyShakespeare, where FisherSFT outperforms uniform, density, clustering, sentence-level optimal design, and AskLLM baselines. The central theoretical justification for the selection objective is Lemma 3.1, which is used to lower-bound the log-determinant of the true Hessian by the log-determinant of the tractable design matrix.

Significance. If the empirical findings are robust, the paper contributes a simple, computationally efficient connection between LLM fine-tuning data selection and D-optimal experimental design. The synthetic experiments are well designed because the ground-truth softmax parameter is known, and they provide direct evidence that the greedy log-det rule is a reasonable heuristic at fixed sample budgets. The use of a held-out LLM judge for the GPT-2 experiments is a useful complement to the synthetic metrics. The method has no fitted selection constants apart from the ridge regularization sigma0, and the lazy-greedy acceleration is a practical contribution. However, the advertised theoretical guarantee is currently unsupported: the key matrix inequality in Lemma 3.1 is unsatisfiable, and the determinant identity used in its proof is misstated. The empirical value of the heuristic is not necessarily affected by these issues, but the claim that the method is 'analyzable' with a valid information-gain bound is not established by the manuscript as written.

major comments (3)
  1. [Lemma 3.1 / Section 3.3] The assumption diag(p(y|xi,j;Theta)) - p(y|xi,j;Theta)p(y|xi,j;Theta)^T >= gamma I_L for gamma > 0 is unsatisfiable for any probability vector p: multiplying by the all-ones vector gives (diag(p)-pp^T)1 = p - p(p^T1) = 0. The matrix is always singular, so the inequality cannot hold. This invalidates the step in Section 3.3 that concludes Hessian >= (gamma/n) sum_i sum_j I_L (x) xi,j xi,j^T. Because this step is the sole bridge from the true Hessian to the tractable objective log det(sum xi,j xi,j^T), the theoretical motivation for the selection rule is not established. Restricting attention to the subspace {Theta : Theta 1 = 0} may salvage a positive-definiteness statement, but that restriction is not used in Lemma 3.1.
  2. [Section 3.3, determinant identity] The proof of Lemma 3.1 misapplies the Kronecker determinant identity. For A in R^{p x p} and B in R^{q x q}, det(A (x) B) = det(A)^q det(B)^p. With A = I_L and B = (gamma/n) sum xi,j xi,j^T, the correct lower bound is det(B)^L, not det(B)^d. The manuscript writes det(I_L)^L det(B)^d and concludes log det(Hessian) >= d log det((gamma/n) sum xi,j xi,j^T). Since L is the vocabulary size and generally L >> d, the stated lower bound does not follow; the proof would yield an L log det bound, which is a different and much larger quantity. This is a second load-bearing error in the derivation of the objective.
  3. [Assumption 4.2 and Theorem 4.3] Theorem 4.3 depends on Assumption 4.2, a per-step diversity ratio kappa that is not verified in the experiments or argued from the greedy choice. As stated in the main text, the assumption quantifies over i in S_{t-1}, which appears to be a typo for the unselected set; either way, it is not justified. The greedy rule only guarantees that the selected sentence has the largest log-det increment among the remaining sentences at each step, so a version of the inequality with kappa = 1 holds for unselected sentences by definition. If the intended assumption is different, the manuscript needs to state it correctly and justify why kappa is bounded. Without this, the displayed O(1/sqrt(n)) bound is not a standalone guarantee on the algorithm's behavior.
minor comments (4)
  1. [Algorithm 1] The input line writes {x_i = (x_{i,j})_{N_i}_{j=1}}_i, but the length of sentence i is denoted M_i in the body; the notation should be made consistent.
  2. [Lemma 4.4] The statement of Lemma 4.4 omits the regularization term: it defines Sigma_S as a sum of xi,j xi,j^T only, while Assumption 4.1 and the proof use Sigma_S = sigma_0 I + sum xi,j xi,j^T. This should be corrected.
  3. [Table 1] The LLM-judge results are reported as point estimates without confidence intervals, and one entry is 0.52 with a reported position bias of 0.54. A short discussion of statistical significance or a bias-corrected estimate would make the comparison more convincing.
  4. [Figures 1 and 2] The legend labels are inconsistent between the two figures, e.g., 'Greedy Sentence-OD' versus 'SentenceOD' and 'DensitySampling' versus 'Density Sampling'. Please unify the naming in the figures and the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the greedy log-det selection rule is an independently motivated D-optimal design relaxation, evaluated against external baselines, and the overlapping-author citations are comparative rather than load-bearing.

full rationale

The derivation chain is not circular. The selection rule, greedy maximization of log det of the accumulated token-embedding covariance, is not defined in terms of the claimed error bound; it is presented as a D-optimal-design relaxation of log det of the Fisher/Hessian matrix, with the surrogate obtained through a lower bound rather than by identity. The theoretical result is explicitly conditional on Assumptions 4.1 and 4.2; Assumption 4.2 quantifies the greedy approximation ratio and is a stated assumption, not a fitted parameter, a renamed prediction, or an imported uniqueness theorem. Prior work of overlapping authorship (Mukherjee et al. 2024, Thekumparampil et al. 2024) is cited only to contrast token-level selection with sentence-level selection, and the sentence-level approach is actually evaluated as the SentenceOD baseline in Section 5.1 rather than being used as evidence. The supporting machinery is external (Nemhauser et al. 1978 for submodularity, standard Bernstein concentration, and Hajek et al. 2014 for a convexity claim), and the empirical evaluation is self-contained: synthetic MLE errors and GPT-4o judged text comparisons are computed separately from the selection objective. The serious flaw in Lemma 3.1 and Appendix B—the unsatisfiable assumption diag(p)-pp^T ⪰ gamma I for a categorical covariance that is singular along the all-ones vector, and the apparent exponent swap in the Kronecker determinant identity of Section 3.3—is a mathematical-validity defect, not a circularity, because the method's empirical content does not presuppose the lemma's conclusion. No step reduces to its own input by construction, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central theoretical claim rests on a multinomial model of the last layer, boundedness assumptions, a positive-definite margin that is actually unsatisfiable, and a strong per-step greedy approximation ratio. No fitted parameters are used to produce the reported gains, but the theory as written does not establish the advertised rate.

free parameters (1)
  • regularization constant sigma0 = unspecified
    Used in the design matrix initialization and error bound as sigma0 I plus the empirical covariance; no value or selection procedure is given in the paper.
assumptions (6)
  • domain assumption Multinomial logistic model for the last layer of the LLM.
    The method treats every next-token prediction as multinomial logistic regression on pre-logit embeddings; this is an approximation to the actual LLM last layer.
  • domain assumption Bounded features and parameters (Assumption 4.1).
    The theorem assumes norm-bounded feature vectors and model parameters to obtain concentration and strong-convexity arguments; this is conventional but not verified in experiments.
  • ad hoc to paper Positive-definite margin gamma in Lemma 3.1.
    Lemma 3.1 assumes diag(p) - pp^T is bounded below by gamma I_L for gamma > 0, but this matrix is always singular along the all-ones vector, so the assumption is unsatisfiable.
  • ad hoc to paper Per-step kappa diversity ratio (Assumption 4.2).
    Theorem 4.3 requires every remaining sentence's log-det gain to be within a factor kappa of the greedy sentence at every step; this is a strong assumption that is neither proved nor empirically validated.
  • standard math Hajek et al. 2014 spectral claim for multinomial covariance.
    Lemma 4.5 uses a cited matrix inequality to show strong convexity of the multinomial loss; the correctness of that external claim is assumed.
  • standard math Kronecker determinant identity.
    The proof uses det(A tensor B) = det(A)^q det(B)^p; the paper's application appears to swap p and q, which is one source of the determinant factor error.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain." pith.science (2026). https://pith.science/paper/SLVOXWTK

@misc{pith2026250514826,
  author       = {Pith},
  title        = {Pith review of: FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SLVOXWTK}},
  note         = {Machine review of arXiv:2505.14826}
}
read the original abstract

Supervised fine-tuning (SFT) is a standard approach to adapting large language models (LLMs) to new domains. In this work, we improve the statistical efficiency of SFT by selecting an informative subset of training examples. Specifically, for a fixed budget of training examples, which determines the computational cost of fine-tuning, we determine the most informative ones. The key idea in our method is to select examples that maximize information gain, measured by the Hessian of the log-likelihood of the LLM. We approximate it efficiently by linearizing the LLM at the last layer using multinomial logistic regression models. Our approach is computationally efficient, analyzable, and performs well empirically. We demonstrate this on several problems, and back our claims with both quantitative results and an LLM evaluation.

Figures

Figures reproduced from arXiv: 2505.14826 by the authors.

Figure 1
Figure 1. Comparison of maximum and mean prediction errors on synthetic token vectors. The x axis shows the number of sentences selected to train the model. The y axis shows the corresponding error averaged over 20 runs. 1000 1250 1500 1750 2000 Number of samples 40 50 60 Maximum prediction error word2vec FisherSFT Uniform Greedy Sentence-OD ClusteredSampling DensitySampling 1000 1250 1500 1750 2000 Number of samples 12.5 15.… view at source ↗
Figure 2
Figure 2. Comparison of maximum and mean prediction errors on word2vec token vectors. The x axis shows the number of sentences selected to train the model. The y axis shows the corresponding error averaged over 20 runs. You are a judge of Shakespeare text. <tag1>text1</tag1> <tag2>text2</tag2> Respond 2 if the text inside <tag2> is more fluent Shakespeare text than the text inside <tag1>. Respond 1 otherwise. The prompt does … view at source ↗
Figure 3
Figure 3. Text generated by fine-tuned GPT-2 models on sentences selected by Uniform and FisherSFT. The latter is more coherent [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 32 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Abbas, A., Tirmala, K., Simig, D., Ganguli, S., and Morcos, A. S. Semdedup: Data-efficient learning at web-scale through semantic deduplication. arXiv preprint arXiv:2303.09540, 2023

    arXiv 2023

  3. [3]

    Improved algorithms for linear stochastic bandits

    Abbasi-Yadkori, Y., Pal, D., and Szepesvari, C. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems 24, pp.\ 2312--2320, 2011

    work page 2011

  4. [4]

    P., and Wunder, M

    Axiotis, K., Cohen-Addad, V., Henzinger, M., Jerome, S., Mirrokni, V., Saulpic, D., Woodruff, D. P., and Wunder, M. Data-efficient learning via clustering-based sensitivity sampling: Foundation models and beyond. In Proceedings of the 41st International Conference on Machine Learning. PMLR, 2024

    work page 2024

  5. [5]

    Bernstein, D. S. Matrix Mathematics: Theory, Facts, and Formulas with Application to Linear Systems Theory. Princeton University Press, Princeton, NJ, 2nd edition, 2009. ISBN 978-0691118028

    work page 2009

  6. [6]

    Pattern Recognition and Machine Learning

    Bishop, C. Pattern Recognition and Machine Learning. Springer, New York, NY, 2006

    work page 2006

  7. [7]

    Bommasani, R. et al. On the opportunities and risks of foundation models. CoRR, abs/2108.07258, 2021. URL https://arxiv.org/abs/2108.07258

    arXiv 2021

  8. [8]

    Coresets via bilevel optimization for continual learning and streaming

    Borsos, Z., Mutny, M., and Krause, A. Coresets via bilevel optimization for continual learning and streaming. In Advances in Neural Information Processing Systems, volume 33, pp.\ 14879--14890, 2020

    work page 2020

Show all 51 references
  1. [9]

    Brown, T. et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33, 2020

    2020

  2. [10]

    Super-samples from kernel herding

    Chen, Y., Welling, M., and Smola, A. Super-samples from kernel herding. arXiv preprint arXiv:1203.3472, 2012

    2012 arXiv

  3. [11]

    M., Haussmann, E., and Fardet, E

    Chitta, K., \'A lvarez, J. M., Haussmann, E., and Fardet, E. Training data subset search with ensemble active learning. IEEE Transactions on Intelligent Transportation Systems, 23 0 (9): 0 14741--14752, 2021

    2021

  4. [12]

    and Shrivastava, A

    Coleman, B. and Shrivastava, A. Sub-linear race sketches for approximate kernel density estimation on streaming data. In Proceedings of The Web Conference 2020, WWW '20, pp.\ 1739–1749, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450370233. doi:10.11...

    2020

  5. [13]

    Selection via proxy: Efficient data selection for deep learning

    Coleman, C., Yeh, C., Mussmann, S., Mirzasoleiman, B., Bailis, P., Liang, P., Leskovec, J., and Zaharia, M. Selection via proxy: Efficient data selection for deep learning. In International Conference on Learning Representations, 2020

    2020

  6. [14]

    Das, N., Chakraborty, S., Pacchiano, A., and Chowdhury, S. R. Active preference optimization for sample efficient RLHF . CoRR, abs/2402.10500, 2024. URL https://arxiv.org/abs/2402.10500

    2024 arXiv

  7. [15]

    and Zhang, C

    Feldman, V. and Zhang, C. What neural networks memorize and why: discovering the long tail via influence estimation. In Advances in Neural Information Processing Systems, volume 33, pp.\ 2881--2891, 2020

    2020

  8. [16]

    On the mathematical foundations of theoretical statistics

    Fisher, R. On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London: Series A, 222: 0 309--368, 1922

    1922

  9. [17]

    Minimax-optimal inference from partial rankings

    Hajek, B., Oh, S., and Xu, J. Minimax-optimal inference from partial rankings. arXiv preprint arXiv:1406.5638, 2014. URL https://arxiv.org/abs/1406.5638

    2014 arXiv

  10. [18]

    Hastings, W. K. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57 0 (1): 0 97--109, 1970

    1970

  11. [19]

    LoRA : Low-rank adaptation of large language models

    Hu, E., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. LoRA : Low-rank adaptation of large language models. In Proceedings of the 10th International Conference on Learning Representations, 2022

    2022

  12. [20]

    Indyk, P., Mahabadi, S., Mahdian, M., and Mirrokni, V. S. Composable core-sets for diversity and coverage maximization. In Proceedings of the 33rd ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pp.\ 100--108, 2014

    2014

  13. [21]

    and Liberty, E

    Karnin, Z. and Liberty, E. Discrepancy, coreset, and sketches in machine learning. In Conference on Learning Theory, pp.\ 1975--1993. PMLR, 2019

    1975

  14. [22]

    char-rnn

    Karpathy, A. char-rnn. https://github.com/karpathy/char-rnn, 2015

    2015

  15. [23]

    and Szepesvari, C

    Lattimore, T. and Szepesvari, C. Bandit Algorithms. Cambridge University Press, 2019

    2019

  16. [24]

    Beyond scale: The diversity coefficient as a data quality metric demonstrates llms are pre-trained on formally diverse data

    Lee, A., Miranda, B., and Koyejo, S. Beyond scale: The diversity coefficient as a data quality metric demonstrates llms are pre-trained on formally diverse data. arXiv preprint arXiv:2306.13840, 2023

    2023 arXiv

  17. [25]

    Deduplicating training data makes language models better

    Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., and Carlini, N. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 8424--...

    2022

  18. [26]

    Liu, P., Shi, C., and Sun, W. W. Dual active learning for reinforcement learning from human feedback. CoRR, abs/2410.02504, 2024. URL https://arxiv.org/abs/2410.02504

    2024 arXiv

  19. [27]

    Peft: State-of-the-art parameter-efficient fine-tuning methods

    Mangrulkar, S., Gugger, S., Debut, L., Belkada, Y., Paul, S., and Bossan, B. Peft: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft, 2022

    2022

  20. [28]

    Meding, R., Buschtoff, L. M. S., Geirhos, R., and Wichmann, F. A. Trivial or impossible--dichotomous data difficulty makes model differences (on imagenet and beyond). arXiv preprint arXiv:2110.05922, 2021

    2021 arXiv

  21. [29]

    S., and Dean, J

    Mikolov, T., Chen, K., Corrado, G. S., and Dean, J. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, 2013. URL https://api.semanticscholar.org/CorpusID:5959482

    2013

  22. [30]

    Prioritized training on points that are learnable, worth learning, and not yet learnt

    Mindermann, S., Brauner, J., Razzak, M., Sharma, M., Kirsch, A., Xu, W., H \"o ltgen, B., Gomez, A., Morisot, A., Farquhar, S., et al. Prioritized training on points that are learnable, worth learning, and not yet learnt. In International Conference on Machine Learning, pp.\ 1...

    2022

  23. [31]

    M., Barak, B., Scao, T

    Muenchigoff, M., Rush, A. M., Barak, B., Scao, T. L., Piktus, T., Tazi, N., Pyysalo, S., Wolf, T., and Raffel, C. Scaling data-constrained language models. arXiv preprint arXiv:2305.10623, 2023

    2023 arXiv

  24. [32]

    Optimal design for human preference elicitation

    Mukherjee, S., Lalitha, A., Kalantari, K., Deshmukh, A., Liu, G., Ma, Y., and Kveton, B. Optimal design for human preference elicitation. In Advances in Neural Information Processing Systems 37, 2024

    2024

  25. [33]

    L., Wolsey, L

    Nemhauser, G. L., Wolsey, L. A., and Fisher, M. L. An analysis of approximations for maximizing submodular set functions - I . Mathematical Programming, 14 0 (1): 0 265--294, 1978

    1978

  26. [34]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to foll...

    2022

  27. [35]

    Paul, M., Ganguli, S., and Dziugaite, G. K. Deep learning on a data diet: Finding important examples early in training. In Advances in Neural Information Processing Systems, volume 34, pp.\ 2960--2971, 2021

    2021

  28. [36]

    Phillips, J. M. Coresets and sketches. In Handbook of discrete and computational geometry, pp.\ 1269--1288. Chapman and Hall/CRC, 2017

    2017

  29. [37]

    Optimal Design of Experiments, volume 50 of Classics in Applied Mathematics

    Pukelsheim, F. Optimal Design of Experiments, volume 50 of Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia, PA, 2006. ISBN 0898716047

    2006

  30. [38]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. OpenAI Technical Report, 2019. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

    2019

  31. [39]

    Direct preference optimization: Your language model is secretly a reward model

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36, 2023

    2023

  32. [40]

    SVP - CF : Selection via proxy for collaborative filtering data

    Sachdeva, N., Wu, C.-J., and McAuley, J. SVP - CF : Selection via proxy for collaborative filtering data. arXiv preprint arXiv:2107.04984, 2021

    2021 arXiv

  33. [41]

    H., Caverlee, J., and Cheng, D

    Sachdeva, N., Coleman, B., Kang, W.-C., Ni, J., Hong, L., Chi, E. H., Caverlee, J., and Cheng, D. Z. How to train data-efficient llms. arXiv preprint arXiv:2402.09668, 2024

    2024 arXiv

  34. [42]

    Optimal design for reward modeling in RLHF

    Scheid, A., Boursier, E., Durmus, A., Jordan, M., Menard, P., Moulines, E., and Valko, M. Optimal design for reward modeling in RLHF . CoRR, abs/2410.17055, 2024. URL https://arxiv.org/abs/2410.17055

    2024 arXiv

  35. [43]

    Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., and Morcos, A. S. Beyond neural scaling laws: beating power law scaling via data pruning. In Advances in Neural Information Processing Systems, volume 35, pp.\ 19523--19536, 2022

    2022

  36. [44]

    and Yang, M

    Stufken, J. and Yang, M. Optimal designs for generalized linear models. In Design and Analysis of Experiments, pp.\ 137--164. John Wiley & Sons, 2012

    2012

  37. [45]

    Comparing few to rank many: Active human preference learning using randomized Frank-Wolfe

    Thekumparampil, K., Hiranandani, G., Kalantari, K., Sabach, S., and Kveton, B. Comparing few to rank many: Active human preference learning using randomized Frank-Wolfe . CoRR, abs/2412.19396, 2024. URL https://arxiv.org/abs/2412.19396

    2024 arXiv

  38. [46]

    Tirmala, K., Simig, D., Aghajanyan, A., and Morcos, A. S. D4 : Improving lm pre-training via document de-duplication and diversification. arXiv preprint arXiv:2308.12284, 2023

    2023 arXiv

  39. [47]

    On coresets for support vector machines

    Tukan, M., Baykal, C., Feldman, D., and Rus, D. On coresets for support vector machines. Theoretical Computer Science, 890: 0 171--191, 2021

    2021

  40. [48]

    W., Lester, B., Du, N., Dai, A., and Le, Q

    Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A., and Le, Q. Finetuned language models are zero-shot learners. In Proceedings of the 10th International Conference on Learning Representations, 2022

    2022

  41. [49]

    Ccnet: Extracting high quality monolingual datasets from web crawl data

    Wenzek, G., Lachaux, M.-A., Conneau, A., Chaudhary, V., Guzm \'a n, F., Joulin, A., and Grave, E. Ccnet: Extracting high quality monolingual datasets from web crawl data. arXiv preprint arXiv:1911.00359, 2019

    1911 arXiv

  42. [50]

    L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Huggingface's tr...

    2020 arXiv

  43. [51]

    Principled reinforcement learning with human feedback from pairwise or K -wise comparisons

    Zhu, B., Jiao, J., and Jordan, M. Principled reinforcement learning with human feedback from pairwise or K -wise comparisons. CoRR, abs/2301.11270, 2023. URL https://arxiv.org/abs/2301.11270

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.