Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations

Benjamin Roth; Loris Schoenegger; Lukas Hinterleitner

REVIEW 3 major objections 5 minor 8 references

A greedily selected subset of a language model's components outperforms the full gradient and random projection at retrieving the training examples that influenced a prediction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:29 UTC pith:CG22D2SS

load-bearing objection The benchmark and single-component analyses are useful, but the headline 'select beats project' claim is an in-sample fit. the 3 major comments →

arxiv 2601.16651 v3 pith:CG22D2SS submitted 2026-01-23 cs.CL

Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations

Lukas Hinterleitner , Loris Schoenegger , Benjamin Roth This is my paper

classification cs.CL

keywords training data attributioninfluence estimationgradient selectionrandom projectioninstance-based explanationlarge language modelsretrieval benchmarkcomponent selection

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks how to compress a billion-parameter model's gradients for training-data attribution: pick a few informative components or project the whole vector into a lower-dimensional space? It builds a re-training-free retrieval benchmark—match a paraphrased or model-generated query back to its original training example—and finds that a greedily chosen subset of components outperforms both the full gradient and random projection while costing far less compute. The result matters because gradient-based explanation methods, such as influence estimation, are bottlenecked by gradient size; a roughly four-hour selection run replaces hundreds of hours of projection, making such explanations practical. The paper also finds that a component's function matters more than its parameter count, and that including more components can add noise rather than signal.

Core claim

The paper's central claim is that when gradients must be compressed to make instance-based explanations tractable, deliberately choosing a small set of model components beats both the uncompressed full gradient and indiscriminate random projection. To test this, the authors construct a retrieval benchmark: a query sample (a paraphrase of a training point, or a model-generated completion on a paraphrased prompt) must be matched to its original among five candidates. They then greedily add the component—of 113 distinct tensors in a 1.2B-parameter decoder-only transformer—that most improves retrieval accuracy, exploiting the fact that the cosine similarity of any subset of components can be rec

What carries the argument

The central mechanism is the linearity of the dot product over concatenated component gradients: the cosine similarity for any subset of components can be reconstructed exactly by summing precomputed per-component dot products. This lets the greedy forward search—which iteratively adds the component yielding the largest retrieval-accuracy improvement—operate entirely on scalar values, avoiding materialization or storage of high-dimensional gradient vectors.

Load-bearing premise

The load-bearing premise is that higher cosine similarity between a query's gradient and its true original's gradient—compared with other training samples—actually indicates training-data influence; the paper states this assumption in Section 3.2, and if it fails, retrieval accuracy measures generic gradient similarity rather than explanation quality.

What would settle it

Take a fixed query and its known original among five candidates and run the greedy selection; if a randomly chosen subset with the same parameter budget reaches the same retrieval accuracy as the greedy subset, the selection signal is doing no work. Alternatively, if the full gradient retrieves the true original accurately in cases where the selected subset fails, the noise-filtering claim is not general.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

For retrieval-style training-data attribution, targeted component selection is preferable to random projection and to the full gradient as a source of gradient representations.
The full gradient is not the best available representation; filtering out noisy components improves retrieval accuracy, nearly doubling it in the harder model-generated setting.
A component's functional role matters more than its parameter count: MLP gate/up and attention Q/O projections carry the signal, while attention K/V projections perform near or below random chance.
The proposed benchmark is re-training free, local, static, and model-invariant, so it can be reused to evaluate other compression and attribution methods.
Architecture-aware selection reduces compute by orders of magnitude relative to random projection, making gradient-based explanations of large models more feasible in practice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the same greedy selection strategy could be applied to other gradient-comparison methods, such as influence functions, data Shapley, or data pruning, where full-gradient comparisons are prohibitive; the retrieval proxy would then need to be validated against true leave-one-out influence.
Beyond the paper: the experiments cover only one 1.2B-parameter model and one fine-tuning dataset; a natural test is whether the selected components (early/late layers, MLP-heavy) transfer across architectures, scales, and domains, or whether selection must be re-run per task.
Beyond the paper: the non-monotonic accuracy-versus-parameter-budget curve suggests an optimal budget per task; a practical extension would be a stopping rule that selects components until retrieval accuracy plateaus rather than using a fixed parameter fraction.
Beyond the paper: random projection preserves the full gradient's geometry but not retrieval performance, so the benchmark measures task-specific discriminative signal rather than geometric fidelity; making this distinction explicit could guide future compression objectives.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The benchmark and single-component analyses are useful, but the headline 'select beats project' claim is an in-sample fit.

read the letter

Quick take: the headline claim is not substantiated. The greedy selector in Algorithm 2 is optimized on the exact query and candidate sets used in Figure 4, and no held-out split is described. So the reported accuracy curves are training-set fits. The stress-test note is correct, and it lands squarely on the central comparison.

What the paper does well: it builds a clean retrieval benchmark—paraphrased and model-generated queries, BM25 pre-filtered candidates, a clear accuracy metric—and it ships code and pre-computed dot products. The single-component analysis is genuinely informative: the finding that MLP gate/up projections carry more signal than K/V projections, and that performance varies with depth, is a useful empirical contribution that doesn't depend on the greedy selection. The greedy algorithm itself is also elegant: reconstructing cosine similarity for any subset from scalar dot-products is a neat computational trick.

The soft spots, in proportion: the main flaw is circularity in the comparison. There is no train/validation split, so the improvement of selection over projection may be noise fitting. With 113 components and 988 queries, the selector can exploit idiosyncrasies of the particular paraphrases and candidate sets. The projection baseline is also not given the same label-informed optimization—it is a fixed, unsupervised baseline, so the comparison is not entirely fair. That asymmetry is a secondary issue; the missing holdout is primary. The efficiency comparison is asymmetric as well: the greedy method uses precomputed dot products, while the projection numbers include on-the-fly projection; their caching estimate helps, but the comparison is not apples-to-apples.

One thing worth saying: the benchmark's validity rests on the assumption that paraphrases have higher gradient cosine similarity with their originals than with other training samples. That assumption is plausible for the paraphrased setting (full gradient hits 0.993) and weaker for the model-generated setting (0.218, near chance), which the authors acknowledge. That isn't fatal, but it means the model-generated results should be interpreted cautiously.

Bottom line: this is a useful benchmark and a well-executed single-component study, but the central 'select beats project' conclusion needs a proper train/validation split before it can be trusted. It deserves a serious referee; with a held-out evaluation and a task-aligned projection baseline, the result might well hold. As is, I would not cite the headline claim, but I would use the benchmark.

Referee Report

3 major / 5 minor

Summary. The paper addresses the practical problem that gradient-based training data attribution for LLMs is computationally prohibitive in full parameter space. It proposes a retrieval benchmark in which the gradient of a paraphrased or model-generated query must be matched to its original training sample among BM25-selected distractors. The authors compare three representations: the full gradient, a random projection baseline, and a greedy component-selection method (Select) that incrementally adds the model component that most improves retrieval accuracy. Using a 1.2B OLMo model and 988 LIMA samples, they report that the greedy selection outperforms both the full gradient and random projection in retrieval accuracy, and that it is computationally cheaper than random projection. The paper also provides per-component analyses showing that MLP and attention Q/O components are more informative than K/V components, and that early and late layers dominate in the model-generated setting.

Significance. If the central comparison were valid, this would be a useful contribution to instance-based explainability: it systematically evaluates a practical design choice (select vs. project), introduces a reusable benchmark, releases code, and leverages the linearity of dot products to make subset selection efficient. The per-component findings are interesting and could guide practitioners. However, the headline claim currently rests on an evaluation protocol that fits the selection criterion to the same data used to report accuracy. The evidence for the claimed superiority of Select over full gradients and random projection is therefore not established by the current experiments. A held-out evaluation is needed before the results can be interpreted as generalizable findings.

major comments (3)

[§3.3 (Algorithm 2) and §5.3 (Figure 4)] The reported retrieval accuracies for Select are in-sample fits. Algorithm 2 greedily chooses each component by maximizing acc(b) on the exact query sets Dp/Dm and the exact candidate sets Ci(b) that are later used to plot Figure 4. No train/validation split is described in Section 4 or 5, and the paper does not mention any form of nested evaluation. With 113 components and 988 queries, the selector can exploit idiosyncrasies of the particular paraphrase/generation instances. The contrast with the model-generated setting—full gradient 0.218 vs. random baseline 0.2, selected subset 0.36—is especially suggestive of noise fitting. The authors must evaluate Select on held-out queries/candidates (e.g., select on one subset, report on a disjoint subset) before claiming that a selected subset 'captures information' better than full gradients or random projection.
[§3.3 vs. §3.4 and Figure 4] The comparison is not apples-to-apples. The greedy selector is explicitly label-informed: it is optimized on the same retrieval task used for reporting. Random projection and the full gradient are fixed representations with no access to the task labels or the evaluation set. The observed gain may therefore be the value of optimizing on the evaluation set rather than an intrinsic advantage of component selection over projection. The authors should either give the baselines the same type of label-informed selection (e.g., select projection dimensions by accuracy on a training split) or, more simply, report the selected subset's accuracy on a split not used during greedy selection. Without this, the central claim in the Abstract and Conclusion is unsupported.
[§3.2 (retrieval task definition)] The benchmark's validity depends on the assumption in Section 3.2 that cosine similarity between the gradient of a paraphrased sample and its original is higher than to other training samples. The paper does not independently validate this assumption; it merely reports retrieval accuracy under the candidate set construction that always inserts the original sample (Appendix B.1, Algorithm 1). If the assumption fails, the retrieval task may be measuring arbitrary gradient similarity rather than training data influence, and the comparison would say little about explanations. A validation using a known influence signal (e.g., fine-tuning on inserted facts, or agreement with an established influence method) would strengthen the claim that the benchmark measures what the paper says it measures.

minor comments (5)

[§4.1] Typo: 'We provides 0 as an example below' should be 'We provide s0 as an example below.'
[§2, Related Work] 'proximal Bergmann response function' appears to be a typo for 'proximal Bregman response function.'
[§5.3 and Figures 4–5] The accuracy curves are shown without error bars or repeated runs. Since the greedy selection is deterministic but the random projection may introduce variance, reporting standard errors over multiple projection seeds would clarify the comparison.
[Table 2] The computational cost comparison is reported for the paraphrased setting only. It would be helpful to state explicitly whether the model-generated setting has the same profile, and to clarify the 'with caching' scenario's assumptions about memory and I/O.
[Algorithm 2] The epsilon stability constant is added only to the denominator in the reconstructed similarity. Its effect on selected subsets and accuracy is not discussed; a small sensitivity analysis would be useful.

Circularity Check

1 steps flagged

Greedy selection is optimized on the exact evaluation set, making the headline retrieval gain a fitted objective rather than an out-of-sample result.

specific steps

fitted input called prediction [Section 3.2 (retrieval metric), Section 3.3 / Algorithm 2 (selection objective), Section 5.3 (reported result)]
"At each step, we choose the next component index(l ∗, k∗)to add: (l∗, k∗) = arg max (l,k)∈W\S acc(b) S∪{(l,k)} (Dp, D) ... a← 1 N PN i=1 I[γ̂ S′ i,i > maxj∈Ci(b)\{i} γ̂ S′ i,j] ... Figure 4 shows that greedily selecting components based on retrieval accuracy identifies a small subset that significantly outperforms both random projection and the full gradient."

The component subset S is selected by Algorithm 2 to maximize acc(b) on the exact query set Dp (or Dm) and the exact candidate sets Ci(b) used for the results in Section 5. The reported accuracies are therefore the value of the selection objective, not held-out predictions. No train/validation split is described in Section 4 or 5; all 988 queries and their candidates are used both to choose components and to score them. The gain of 'Select' over 'Project' is the output of a search over 113 components on the evaluation data, so the comparison is between an optimized selector and a non-optimized baseline.

full rationale

The paper's central empirical claim—that greedily selected components capture training-data-influence information more effectively than the full gradient or random projection—rests on an in-sample optimization. Section 3.2 defines the retrieval accuracy acc(b)_θ(Dp,D) over query set Dp and BM25 candidate sets Ci(b). Section 3.3 and Algorithm 2 then choose components by maximizing exactly this acc(b) on the same Dp and Ci(b), using precomputed dot products. Section 5.3 reports those same accuracy curves as evidence of superiority. Because Algorithm 2 optimizes the evaluation metric itself and no held-out split is described, the reported 'prediction' that selection outperforms projection is the fitted objective value. This is a clear instance of fitted-input-called-prediction: the selected subset is a parameter fit to the evaluation data. The non-circular contributions—full-gradient behavior, single-component analyses, component-type/depth patterns, and the computational-cost comparison—may still stand, and the benchmark is a reasonable proposal, but the headline superiority claim requires held-out queries/candidates before it can support generalization. The Limitations section also concedes that generalization across architectures, scales, and domains was not explored. There is no load-bearing self-citation or definitional equivalence beyond this in-sample selection, so the score is 7 rather than higher.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central comparison rests on the untested assumption that gradient similarity to a paraphrase identifies the true training source, on the choice of candidate set size (b=5), and on the fitted component subset. The greedy subset is the largest free object: it is selected by optimizing the exact metric reported, which is the main circularity.

free parameters (4)

candidate set size b = 5
Candidate set size chosen for tractability; defines chance level of 0.2 and strongly affects absolute accuracies.
random projection dimension d = 1% and 5% of total parameters (not tuned)
Used for projection baseline; fidelity and cost scale with d; not selected to optimize retrieval.
epsilon stability constant = unspecified small positive
Added to denominator in Algorithm 2 to avoid division by zero.
selected component subset S = e.g., <5% of parameters in paraphrased setting; achieving 0.998 accuracy in paraphrased and ~0.36 in model-generated
Greedy forward selection maximizes retrieval accuracy on the evaluation set; this is the core fitted object and the main circularity.

axioms (4)

domain assumption Cosine similarity between gradients of a paraphrase and its original training sample is higher than to unrelated samples
Section 3.2 states this as the initial assumption; the entire benchmark's validity as a measure of training data influence depends on it.
domain assumption Retrieval accuracy on paraphrased/model-generated queries is a valid proxy for instance-based explanation quality
No validation against known influence ground truth is provided; the paper itself notes that influence evaluation is an open problem (discussion in Related Work of Grosse et al., 2023).
standard math Component-wise dot products sum to reconstruct cosine similarity for any subset
Appendix A.2 derives this from linearity of the dot product; it is exact and not a source of error.
domain assumption Results on a single 1.2B model and a 988-sample LIMA subset generalize
Acknowledged in Limitations; only one model and one dataset are used.

pith-pipeline@v1.3.0-alltime-deepseek · 11809 in / 14094 out tokens · 138558 ms · 2026-08-03T08:29:17.886963+00:00 · methodology

0 comments

read the original abstract

Gradient-based methods for instance-based explanation for large language models (LLMs) are hindered by the immense dimensionality of model gradients. In practice, influence estimation is restricted to a subset of model parameters to make computation tractable, but this subset is often chosen ad hoc and rarely justified by systematic evaluation. This paper investigates if it is better to create low-dimensional representations by selecting a small, architecturally informed subset of model components or by projecting the full gradients into a lower-dimensional space. Using a novel benchmark, we show that a greedily selected subset of components captures the information about training data influence needed for a retrieval task more effectively than either the full gradient or random projection. We further find that this approach is more computationally efficient than random projection, demonstrating that targeted component selection is a practical strategy for making instance-based explanations of large models more computationally feasible.

Figures

Figures reproduced from arXiv: 2601.16651 by Benjamin Roth, Loris Schoenegger, Lukas Hinterleitner.

**Figure 2.** Figure 2: Component accuracy vs. parameter count. Each boxplot groups all components of a specific type, which share the same size. Left: Attention projections. Center: MLP components. Right: Embedding layer. Performance by model depth The accuracy distribution of influential gradient components across model depth differs substantially between settings. While accuracy is high and stable across all layers in the pa… view at source ↗

**Figure 3.** Figure 3: Accuracy vs. layer depth for the model-generated setting. Each color represents a layer component. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

**Figure 4.** Figure 4: Greedy component selection (optimized for accuracy) vs. Random Projection. The x-axis shows the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Greedy component selection vs. random projection, optimizing for similarity to the full model gradient. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

**Figure 6.** Figure 6: System prompt for Dp B.3 Greedy Selection by Accuracy The Select method employs a greedy forward algorithm to find a subset of components that maximizes retrieval accuracy. Algorithm 2 provides a formal specification. The algorithm iteratively adds the component that yields the greatest improvement to the accuracy metric, operating entirely on the precomputed dot products δ (l,k) i,j . We introduce the no… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

8 extracted references · 2 linked inside Pith

[2]

Association for Computational Linguistics

What does BERT learn about the structure of language? InProceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Vol- ume 1: Long Papers, pages 3651–3657. Association for Computational Linguistics. William Johnson and Joram Lindenstrauss. 1984. Ex- tensions of lipschitz maps i...

2019
[7]

Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue

Mammoth2: Scaling instructions from the web.Advances in Neural Information Processing Systems, 37:90629–90660. Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. 10
[8]

InFindings of the Association for Computational Linguistics: ACL 2024, pages 12834–12859, Bangkok, Thailand

OpenCodeInterpreter: Integrating code gener- ation with execution and refinement. InFindings of the Association for Computational Linguistics: ACL 2024, pages 12834–12859, Bangkok, Thailand. As- sociation for Computational Linguistics. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Sus...

2024
[2019]

Data cleansing for models trained with SGD. InAdvances in Neural Information Processing Sys- tems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 4215– 4224. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew E. Peters, Pradeep Dasigi, Joel Jang, Davi...

Pith/arXiv arXiv 2019
[2023]

In The Twelfth International Conference on Learning Representations

DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models. In The Twelfth International Conference on Learning Representations. Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. 2023. Does circuit analysis interpretability scale? evidence from multiple choice capabilities...

Pith/arXiv arXiv 2023
[2024]

InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 841–860, Bangkok, Thailand

Token-wise influential training data retrieval for large language models. InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 841–860, Bangkok, Thailand. Association for Com- putational Linguistics. Jiang Liu, Jialian Wu, Prakamya Mishra, Zicheng Liu, Sudhanshu Ranjan, Pratik Prabhanja...

2024
[2025]

Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen

OpenReview.net. Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. LESS: Select- ing Influential Data for Targeted Instruction Tuning. InProceedings of the 41st International Conference on Machine Learning, pages 54104–54132. PMLR. Chih-Kuan Yeh, Ankur Taly, Mukund Sundararajan, Frederick Liu, and Pradeep Ravikumar. 202...

2024
[8186]

AAAI Press. Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bo- gin, Khyathi Raghavi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, and 17 others. 2024. Dolma: an open corpus of t...

2024

[1] [2]

Association for Computational Linguistics

What does BERT learn about the structure of language? InProceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Vol- ume 1: Long Papers, pages 3651–3657. Association for Computational Linguistics. William Johnson and Joram Lindenstrauss. 1984. Ex- tensions of lipschitz maps i...

2019

[2] [7]

Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue

Mammoth2: Scaling instructions from the web.Advances in Neural Information Processing Systems, 37:90629–90660. Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. 10

[3] [8]

InFindings of the Association for Computational Linguistics: ACL 2024, pages 12834–12859, Bangkok, Thailand

OpenCodeInterpreter: Integrating code gener- ation with execution and refinement. InFindings of the Association for Computational Linguistics: ACL 2024, pages 12834–12859, Bangkok, Thailand. As- sociation for Computational Linguistics. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Sus...

2024

[4] [2019]

Data cleansing for models trained with SGD. InAdvances in Neural Information Processing Sys- tems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 4215– 4224. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew E. Peters, Pradeep Dasigi, Joel Jang, Davi...

Pith/arXiv arXiv 2019

[5] [2023]

In The Twelfth International Conference on Learning Representations

DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models. In The Twelfth International Conference on Learning Representations. Tom Lieberum, Matthew Rahtz, János Kramár, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. 2023. Does circuit analysis interpretability scale? evidence from multiple choice capabilities...

Pith/arXiv arXiv 2023

[6] [2024]

InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 841–860, Bangkok, Thailand

Token-wise influential training data retrieval for large language models. InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 841–860, Bangkok, Thailand. Association for Com- putational Linguistics. Jiang Liu, Jialian Wu, Prakamya Mishra, Zicheng Liu, Sudhanshu Ranjan, Pratik Prabhanja...

2024

[7] [2025]

Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen

OpenReview.net. Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. LESS: Select- ing Influential Data for Targeted Instruction Tuning. InProceedings of the 41st International Conference on Machine Learning, pages 54104–54132. PMLR. Chih-Kuan Yeh, Ankur Taly, Mukund Sundararajan, Frederick Liu, and Pradeep Ravikumar. 202...

2024

[8] [8186]

AAAI Press. Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bo- gin, Khyathi Raghavi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, and 17 others. 2024. Dolma: an open corpus of t...

2024