QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning

Alham Fikri Aji; Ayu Purwarianti; Moses Ananta; Muhammad Farid Adilazuarda; Zayd Muhammad Kawakibi Zuhri

REVIEW 4 major objections 5 minor 42 references

QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read QLESS claims that quantizing LoRA gradients to as few as 1 bit preserves data-selection quality in LLM fine-tuning while cutting gradient storage up to 16x.

desk verdict Useful empirical study of quantized gradient datastores for data selection, but the 1-bit result is unreproducible as written because Eq. (5) makes all 1-bit vectors zero. read the letter →

arxiv 2502.01703 v1 pith:MJFQDPEH submitted 2025-02-03 cs.LG cs.AIcs.CL

Moses Ananta , Muhammad Farid Adilazuarda , Zayd Muhammad Kawakibi Zuhri , Ayu Purwarianti , Alham Fikri Aji This is my paper · ORCID

classification cs.LGcs.AIcs.CL

keywords datavaluationselectiongradientquantizationinstructiontuninglargelanguagemodelsLoRArandomprojectioninfluencefunctions

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QLESS (Quantized Low-rank Gradient Similarity Search) extends the LESS data-selection framework by quantizing the stored gradient features, replacing floating-point vectors with low-bit integer codes plus one scale factor per vector. The paper's central claim is that this compression preserves enough directional information for influence-based data valuation: across several LLM families and benchmarks, models fine-tuned on QLESS-selected data match or nearly match those trained on LESS-selected data, while the gradient datastore shrinks by up to 16x. The headline result is that 1-bit quantized gradients, where each dimension is essentially a sign, still produce competitive selections, suggesting that coarse directional agreement rather than precise magnitudes drives the value of gradient similarity for instruction-tuning data selection. The paper also shows that absmean quantization outperforms absmax at low bit widths because it avoids collapsing most entries into the zero bin, and that stacking QLoRA model quantization on top of gradient quantization keeps performance close to full-precision baselines.

What carries the argument

The central object is the quantized-normalized gradient vector $\hat{q}_{z,i} = q_{z,i}/\|q_{z,i}\|$, where $q_{z,i}$ is the absmax-quantized random projection of a LoRA training gradient. Influence is scored by the learning-rate-weighted cosine similarity $\sum_i \eta_i \langle \hat{q}_{z',i}, \hat{q}_{z,i}\rangle$, mirroring LESS but operating on integer codes. The two-step compression, LoRA-based random projection down to 8192 dimensions followed by absmax quantization to $b$ bits, is what carries the memory savings, while the normalization of the quantized vector rather than the dequantized float is the mechanism that allegedly keeps cosine similarities meaningful at extreme compression.

What would settle it

One concrete test: on the paper's own gradient datastore, compute the rank correlation (for example Kendall's tau) between LESS influence scores and QLESS influence scores for each bit width, and re-run fine-tuning on the top 5% selected by a dequantize-then-normalize ablation. If that ablation consistently outperforms QLESS, or if rank correlation collapses at 1-bit while performance stays high, then the claimed preservation of valuation quality is not explained by the quantized-normalized cosine similarity itself.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a simple absmax-based quantization of randomly projected LoRA gradients, storing k b-bit integers and one scale per vector instead of k 16-bit floats, retains the information needed for LESS-style influence estimation. QLESS computes influence as the learning-rate-weighted cosine similarity between normalized integer vectors, and the experiments show that 8-bit QLESS performs on par with LESS, 4-bit and 2-bit remain competitive, and even 1-bit quantization (sign-only codes) often beats a random 5% selection baseline and stays within a few points of 16-bit performance. This holds across LLaMA-2, LLaMA-3, Mistral, and Qwen models on MMLU, BBH, and TyDiQA, with the gradient datastore reduced from 16.54 GB to 1.03 GB at 1-bit. A follow-on analysis finds that absmean quantization is more robust than absmax at low bit widths because absmax shifts most small gradient values into the zero bin, and that combining QLoRA with QLESS preserves most of the selection quality while further cutting the memory footprint.

Load-bearing premise

The load-bearing premise is that normalizing the quantized integer vector, rather than dequantizing back to floats and then normalizing, still ranks training examples in nearly the same order as LESS would, so the top 5% selection stays useful.

Editorial extensions

If this is right

Gradient datastores for data selection can be stored in 1-8 bits, enabling selection over much larger instruction-tuning corpora within a fixed memory budget.
8-bit QLESS can replace 16-bit LESS as a drop-in memory-saving step with no expected performance loss on the evaluated benchmarks.
Even 1-bit sign-based gradients pick training subsets that outperform random selection, so extreme compression is a viable regime for influence-based data valuation.
Combining QLoRA model quantization with QLESS gradient quantization reduces the total memory footprint to roughly 22-26 GB for 7B models with only minor performance degradation.
Low-bit selections are qualitatively consistent: 16/8/4/1-bit variants select similar top examples, whereas 2-bit can shift because of sparsity, indicating bit-width calibration matters at intermediate precision.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: if 1-bit quantization preserves selection quality, then the effective signal in gradient-based data valuation may be mostly the sign pattern of the projected gradient, which would connect QLESS to sign-based and binary-hash similarity methods and could be tested by comparing QLESS against pure sign random projections without learned scales.
Editorial extension: the paper does not analyze why normalizing the integer vector preserves ranking; a direct rank-correlation study between dequantized-normalized and quantized-normalized influence scores across bit widths would separate the contribution of the scale factor from the contribution of the quantization grid.
Editorial extension: the absmean-vs-absmax trade-off suggests an adaptive or learned quantization scale per gradient vector could improve low-bit performance further, a testable variant the paper does not run.
Editorial extension: because the selected-data fraction experiments show performance plateauing at 0.5-5%, the practical value of QLESS may be in enabling much smaller but better-curated training sets rather than only in compressing storage.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

QLESS is a simple, sensible extension of LESS: quantize the stored projected gradients. The paper's main value is the empirical sweep—8-, 4-, 2-, 1-bit across five models and three benchmarks—and the memory numbers are real: 16.54 GB down to 1.03 GB for the same datastore. The 8-bit and 4-bit results mostly track LESS, and the code and data are promised, which makes this reproducible in principle. That is worth having.

The soft spots are real, though. The biggest is the 1-bit definition. Eq. (5) sets alpha = 2^{b-1} - 1, so for b=1 alpha=0 and every stored vector is identically zero. Eq. (6) then divides by ||q||=0. Yet Tables 1, 2, and 4 report 1-bit averages from 58.59 to 71.21. The obvious intended fix is alpha = 2^b - 1 (sign quantization), and Table 3 has a separate "Sign 1-bit" row, which suggests that is what was run—but the paper never says so. As written, the headline claim that 1-bit preserves quality cannot be reproduced from the stated algorithm. This is more than a typo; it sits at the center of the paper's most interesting claim.

Second, Mistral 7B is a counterexample. In Table 4, every QLESS variant (8-bit through 1-bit) averages below random 5% selection, and LESS itself is barely above random. The text says QLESS "performs competitively" across models, but this model family says the opposite. Maybe it's variance—the standard deviations overlap—but no significance testing is reported anywhere, so we don't know.

Third, the normalization choice in Eq. (6) normalizes the quantized integer vector, not the dequantized float vector. This changes cosine similarity in a way that isn't analyzed, and there's no ablation against normalizing after dequantization. It might be fine, but it's untested.

These are fixable. The 8-bit/4-bit story is solid and worth publishing. The 1-bit claim needs either a corrected formula and a clear specification of the sign variant, or removal of the "1-bit works" narrative. A referee should be asked to check that.

I'd send this to peer review with the expectation of major revision. A careful reader can get value from the comparison, but only after the 1-bit definition is sorted out.

Referee Report

4 major / 5 minor

Summary. The paper proposes QLESS, a memory-efficient extension of the LESS data-valuation framework. QLESS first maps LoRA-based gradients through a random projection and then quantizes the projected vectors with an absmax uniform quantizer at bit widths {1,2,4,8}; influence scores are computed by cosine similarity between the normalized quantized vectors. Experiments on five LLMs (LLaMA-2, LLaMA-3.1/3.2, Mistral, Qwen) and three benchmarks (TyDiQA, MMLU, BBH) report that QLESS achieves performance comparable to LESS while reducing gradient-datastore memory by up to 16x, and that 1-bit quantization preserves data-valuation quality. The paper also includes QLoRA ablations, a comparison of absmax vs. absmean quantization, and a qualitative analysis of selected examples.

Significance. If the central claim holds, QLESS is a practical, low-cost ingredient for making gradient-based data selection feasible at large scale, and its 1-bit result would be a notable empirical finding about the robustness of influence computation under extreme compression. The paper ships code, follows the LESS pipeline closely, and reports results across multiple model families and benchmarks, which is a genuine strength. However, the most striking claim (1-bit viability) is currently unreproducible as written because the stated quantization formula degenerates at b=1, and the aggregate 'comparable' claim is not backed by significance testing. The contribution is incremental but useful; the experimental breadth is adequate for a systems-oriented venue if the algorithmic defect and statistical gaps are fixed.

major comments (4)

[Section 3.1, Eq. (5)] The quantization formula defines alpha = 2^{b-1} - 1, which gives alpha = 0 for b=1. Under this definition every stored q_{z,i,m} is zero, so Eq. (6)'s division by ||q_{z,i}|| is undefined and Eq. (7) produces no signal. Yet Tables 1, 2, and 4 report nontrivial 1-bit results (e.g., Qwen 1-bit avg 70.72 and Llama 3.1 1-bit avg 65.93). The paper therefore does not actually specify the 1-bit algorithm whose results it reports. Please state the exact 1-bit mapping (e.g., alpha = 2^b - 1, which reduces to sign quantization, as in the separate 'Sign 1-bit' row of Table 3) and re-run or explicitly re-derive the 1-bit experiments under that definition.
[Section 4.1 / Table 4] The aggregate claim that QLESS is 'comparable to LESS' is not supported for Mistral 7B: QLESS 8-bit (58.45), 4-bit (57.76), 2-bit (58.52), and 1-bit (58.59) all fall below the random 5% baseline (59.41), and LESS itself also falls below random 5% (59.70 vs. 59.41). Because the headline comparison averages across very different per-model outcomes, at least one per-model discussion and a paired statistical test across seeds (or a clear statement of seed counts) should be provided; otherwise the 'comparable' claim holds only as an unverified aggregate.
[Section 3.2, Eqs. (6)-(7)] QLESS normalizes the quantized integer vector q rather than the dequantized float vector, and then computes cosine similarity in Eq. (7). This is a specific algorithmic choice, but the paper provides no analysis of it: the cited QRP guarantees (Section 2.4) concern distance preservation of the underlying continuous vector after projection and quantization, not the cosine of the normalized integer vector. Please either provide a short proof or an ablation comparing (a) cosine on normalized dequantized vectors, (b) cosine on normalized raw quantized integers, and (c) the sign-only variant, for at least one model and benchmark; this would also help isolate the sparsity effect discussed in Section 5.
[Section 4.2 and Tables 1-5] No significance tests are reported anywhere in the paper. The reported standard deviations (in parentheses) are large relative to the differences that support the 'comparable' claim (e.g., Table 1's average gaps of 0.1-0.6 points vs. stds of 0.1-2.6). Please add paired significance tests (e.g., paired t-test or Wilcoxon over seeds) or report per-seed results so the reader can assess whether the QLESS-vs-LESS differences and the QLESS-vs-random differences are real.

minor comments (5)

[Section 2.4] The phrase 'The resulting codes qi = Q(yi)' uses a subscript inconsistency with the main text's q_{z,i,m}; please unify the notation.
[Section 1 / Figure 1] Figure 1 lists 'Random 100%' and 'Random 5%' as baseline methods but does not define their markers in the caption; please clarify which bars are which and add error bars if available.
[Section 4.1 / Appendix A] The claimed 'three trials using distinct random seeds' should specify whether the standard deviations in Tables 1-5 are over the three trials, and should state the seed values or a seeding protocol for reproducibility.
[Section 5] The sentence 'Interestingly, 1-bit quantization does not exhibit the extreme sparsity problem because its representation inherently omits a zero bin' is not consistent with the stated formula (which contains a zero bin when alpha=0); please reconcile once the 1-bit definition is corrected.
[References] The reference list is missing the QLESS-specific comparison to the very recent LESS follow-ups; please cite the relevant 2024-2025 data-selection work for context and to avoid overclaiming novelty of the quantization-into-datastore idea.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: QLESS empirically extends LESS with swept quantization bit widths and validates against external benchmarks; no fitted parameter is renamed as a prediction.

full rationale

The paper's central claim is that quantizing the LESS gradient datastore preserves data-selection quality. This is an empirical result tested on five model architectures and three external benchmarks, not a derivation from fitted constants. The quantization bit width b is swept over {8,4,2,1} rather than tuned to the reported outcomes, so no 'prediction' is forced by construction. The absmax quantization scheme and the QRP background are cited from external literature (Bernstein et al.; Alistarh et al.; Li et al.; Jacques), and LESS itself is cited from external authors (Xia et al.), so the load-bearing framework is not a self-citation chain. The only self-referential citation in the bibliography is Aji & Heafield (2017), a related-work example of gradient compression in distributed learning; no argument or experimental claim depends on it. The reported limitation that sequential random projection followed by quantization 'may not optimally preserve influence relationships' is an honest scope statement, not a circular step. The skeptic's observation that Eq. (5) sets alpha=0 for b=1, making the stored 1-bit vector zero and Eq. (6) undefined, is a genuine correctness/reproducibility concern about the stated algorithm, but it is not circularity: it does not make any output equivalent to an input by definition. Accordingly, the derivation chain is self-contained with respect to the paper's own equations, and the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

QLESS introduces no new entities. It relies on existing assumptions from LESS and QRP: gradient similarity indicates influence, LoRA gradients are representative, and quantized random projection preserves directional structure. The only parameter is the quantization bit width, which is swept, not fitted.

free parameters (1)

Quantization bit width b = swept over {1, 2, 4, 8}
Chosen by hand as the compression level to evaluate. The paper does not fit this value to data, but reports results for each setting.

assumptions (3)

domain assumption Quantized random projection preserves inner products or angles sufficiently for influence estimation.
Section 2.4 invokes QRP literature (Jacques 2015, Li et al. 2016) to justify the quantization step, but does not verify that the guarantees apply to the specific quantize-then-normalize pipeline used in Eq. (6)-(7).
domain assumption Gradient similarity is a valid proxy for training data influence.
Inherited from LESS (Section 2.2). The paper does not re-derive this and relies on the LESS validation.
domain assumption LoRA warmup gradients during 4 epochs are representative of the fine-tuning dynamics.
Inherited from LESS experimental protocol (Section 4.1). No analysis is provided for this assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning." pith.science (2026). https://pith.science/paper/MJFQDPEH

@misc{pith2026250201703,
  author       = {Pith},
  title        = {Pith review of: QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MJFQDPEH}},
  note         = {Machine review of arXiv:2502.01703}
}

read the original abstract

Fine-tuning large language models (LLMs) is often constrained by the computational costs of processing massive datasets. We propose \textbf{QLESS} (Quantized Low-rank Gradient Similarity Search), which integrates gradient quantization with the LESS framework to enable memory-efficient data valuation and selection. QLESS employs a two-step compression process: first, it obtains low-dimensional gradient representations through LoRA-based random projection; then, it quantizes these gradients to low-bitwidth representations. Experiments on multiple LLM architectures (LLaMA, Mistral, Qwen) and benchmarks (MMLU, BBH, TyDiQA) show that QLESS achieves comparable data selection performance to LESS while reducing memory usage by up to 16x. Even 1-bit gradient quantization preserves data valuation quality. These findings underscore QLESS as a practical, scalable approach to identifying informative examples within strict memory constraints.

Figures

Figures reproduced from arXiv: 2502.01703 by the authors.

**Figure 1.** Data selection method comparison on average model performance across various models and benchmarks. The x-axis shows the average performance (%), while the y-axis represents different data selection methods. QLESS with various quantization levels achieves comparable performance to LESS. Park et al., 2023), which estimate the contribution of individual training samples to model performance, have proven effective in … view at source ↗

**Figure 2.** Overview of the data valuation and selection process. LESS and QLESS differ from each other in step 2 and 3. and stores gˆz,i in a gradient datastore. During inference, the influence of z on a validation sample z ′ is then computed by summing the learning-rate-weighted cosine similarities across N checkpoints: InfLESS(z, z′ ) = X N i=1 ηi ⟨∇ℓ(z ′ ; θi), Γ(z; θi)⟩ ∥∇ℓ(z ′ ; θi)∥ ∥Γ(z; θi)∥ = X N i=1 ηi [PITH_FULL_IM… view at source ↗

**Figure 3.** Absmax- vs absmean-based quantization comparison based on the distribution of values in the gradient tensors. Analyzing Sparsity Effects of Absmax Quantization While absmax-based quantization effectively compresses gradients, it can induce significant sparsity as bit-width decreases. In particular, most gradient values collapse into the zero bin for 2-bit and 4-bit settings, leading to a heavily sparse representati… view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Subset distribution of the top 5% selected examples for different quantization levels. Distribution of Data Sources. From [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 24 canonical work pages

[1]

Database-friendly random projections: Johnson-lindenstrauss with binary coins

Achlioptas, D. Database-friendly random projections: Johnson-lindenstrauss with binary coins. Journal of Computer and System Sciences, 66 0 (4): 0 671--687, 2003. ISSN 0022-0000. doi:https://doi.org/10.1016/S0022-0000(03)00025-4. URL https://www.sciencedirect.com/science/article/pii/S0022000003000254. Special Issue on PODS 2001

doi:10.1016/s0022-0000(03)00025-4 2003
[2]

Aji, A. F. and Heafield, K. Sparse communication for distributed gradient descent. In Palmer, M., Hwa, R., and Riedel, S. (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp.\ 440--445, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi:10.18653/v1/D17-1045. URL https://aclant...

doi:10.18653/v1/d17-1045 2017
[3]

Z., Tomioka, R., and Vojnovic, M

Alistarh, D., Grubic, D., Li, J. Z., Tomioka, R., and Vojnovic, M. Qsgd: communication-efficient sgd via gradient quantization and encoding. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 1707–1718, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964

work page 2017
[4]

signsgd: Compressed optimisation for non-convex problems

Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., and Anandkumar, A. signsgd: Compressed optimisation for non-convex problems. In International Conference on Machine Learning, pp.\ 560--569. PMLR, 2018

2018
[5]

Charikar, M. S. Similarity estimation techniques from rounding algorithms. In Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, STOC '02, pp.\ 380–388, New York, NY, USA, 2002. Association for Computing Machinery. ISBN 1581134959. doi:10.1145/509907.509965. URL https://doi.org/10.1145/509907.509965

work page arXiv 2002
[6]

K., Ahn, H., Bae, J., Zhao, K., Kang, M., Chung, Y., Pratapa, A., Neiswanger, W., Strubell, E., Mitamura, T., Schneider, J., Hovy, E., Grosse, R., and Xing, E

Choe, S. K., Ahn, H., Bae, J., Zhao, K., Kang, M., Chung, Y., Pratapa, A., Neiswanger, W., Strubell, E., Mitamura, T., Schneider, J., Hovy, E., Grosse, R., and Xing, E. What is your data worth to gpt? llm-scale data valuation with influence functions, 2024. URL https://arxiv.org/abs/2405.13954

arXiv 2024
[7]

H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., and Palomaki, J

Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., and Palomaki, J. TyDi QA : A benchmark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, 2020

work page 2020
[8]

Free Dolly : Introducing the world's first truly open instruction-tuned LLM , 2023

Conover, M., Hayes, M., Mathur, A., Xie, J., Wan, J., Shah, S., Ghodsi, A., Wendell, P., Zaharia, M., and Xin, R. Free Dolly : Introducing the world's first truly open instruction-tuned LLM , 2023

work page 2023

Show all 42 references

[9]

8-bit approximations for parallelism in deep learning

Dettmers, T. 8-bit approximations for parallelism in deep learning. In ICLR (Poster), 2016. URL http://arxiv.org/abs/1511.04561

2016 arXiv
[10]

Llm.int8(): 8-bit matrix multiplication for transformers at scale, 2022

Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. Llm.int8(): 8-bit matrix multiplication for transformers at scale, 2022. URL https://arxiv.org/abs/2208.07339

2022 arXiv
[11]

Qlora: efficient finetuning of quantized llms

Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: efficient finetuning of quantized llms. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2024. Curran Associates Inc

2024
[12]

The llama 3 herd of models

Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

2024 arXiv
[13]

Estimating training data influence by tracing gradient descent

Garima, Liu, F., Kale, S., and Sundararajan, M. Estimating training data influence by tracing gradient descent. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546

2020
[14]

Measuring massive multitask language understanding

Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2020

2020
[15]

J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

2021 arXiv
[16]

Fedpara: Low-rank hadamard product for communication-efficient federated learning

Hyeon-Woo, N., Ye-Bin, M., and Oh, T.-H. Fedpara: Low-rank hadamard product for communication-efficient federated learning. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=d71n4ftoCBy

2022
[17]

A quantized johnson–lindenstrauss lemma: The finding of buffon’s needle

Jacques, L. A quantized johnson–lindenstrauss lemma: The finding of buffon’s needle. IEEE Transactions on Information Theory, 61 0 (9): 0 5012--5027, 2015. doi:10.1109/TIT.2015.2453355

2015
[18]

and Cambareri, V

Jacques, L. and Cambareri, V. Time for dithering: fast and quantized random embeddings via the restricted isometry property. Information and Inference: A Journal of the IMA, 6 0 (4): 0 441--476, 04 2017. ISSN 2049-8764. doi:10.1093/imaiai/iax004. URL https://doi.org/10.1093/im...

2017 doi
[19]

Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

2023 arXiv
[20]

Johnson, W. B. and Lindenstrauss, J. Extensions of lipschitz mappings into hilbert space. Contemporary mathematics, 26: 0 189--206, 1984

1984
[21]

Koh, P. W. and Liang, P. Understanding black-box predictions via influence functions. In International conference on machine learning, pp.\ 1885--1894. PMLR, 2017

2017
[22]

o pf, A., Kilcher, Y., von R \

K \"o pf, A., Kilcher, Y., von R \"u tte, D., Anagnostidis, S., Tam, Z.-R., Stevens, K., Barhoum, A., Duc, N. M., Stanley, O., Nagyfi, R., et al. OpenAssistant conversations--democratizing large language model alignment. 2023

2023
[23]

Datainf: Efficiently estimating data influence in lo RA -tuned LLM s and diffusion models

Kwon, Y., Wu, E., Wu, K., and Zou, J. Datainf: Efficiently estimating data influence in lo RA -tuned LLM s and diffusion models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=9m02ib92Wz

2024
[24]

Quantized embeddings of scale-invariant image features for mobile augmented reality

Li, M., Rane, S., and Boufounos, P. Quantized embeddings of scale-invariant image features for mobile augmented reality. In 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP), pp.\ 1--6, 2012. doi:10.1109/MMSP.2012.6343406

2012
[25]

J., and Church, K

Li, P., Hastie, T. J., and Church, K. W. Very sparse random projections. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '06, pp.\ 287–296, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595933395....

2006
[26]

Coding for random projections

Li, P., Mitzenmacher, M., and Shrivastava, A. Coding for random projections. In Xing, E. P. and Jebara, T. (eds.), Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp.\ 676--684, Bejing, China, 22--24 ...

2014
[27]

Quantized random projections and non-linear estimation of cosine similarity

Li, P., Mitzenmacher, M., and Slawski, M. Quantized random projections and non-linear estimation of cosine similarity. In Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc...

2016
[28]

J., Weller, A., and Sch \"o lkopf, B

Liu, W., Qiu, Z., Feng, Y., Xiu, Y., Xue, Y., Yu, L., Feng, H., Liu, Z., Heo, J., Peng, S., Wen, Y., Black, M. J., Weller, A., and Sch \"o lkopf, B. Parameter-efficient orthogonal finetuning via butterfly factorization. In The Twelfth International Conference on Learning Repre...

2024
[29]

W., Tay, Y., Zhou, D., Le, Q

Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., Zhou, D., Le, Q. V., Zoph, B., Wei, J., et al. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688, 2023

2023 arXiv
[30]

M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A

Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A. Trak: attributing model behavior at scale. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023

2023
[31]

Controlling text-to-image diffusion by orthogonal finetuning

Qiu, Z., Liu, W., Feng, H., Xue, Y., Feng, Y., Liu, Z., Zhang, D., Weller, A., and Sch \"o lkopf, B. Controlling text-to-image diffusion by orthogonal finetuning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=K...

2023
[32]

Qwen2.5 technical report, 2025

Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin,...

2025 arXiv
[33]

U., Cordonnier, J.-B., and Jaggi, M

Stich, S. U., Cordonnier, J.-B., and Jaggi, M. Sparsified sgd with memory. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp.\ 4452–4463, Red Hook, NY, USA, 2018. Curran Associates Inc

2018
[34]

W., Chowdhery, A., Le, Q., Chi, E., Zhou, D., et al

Suzgun, M., Scales, N., Sch \"a rli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q., Chi, E., Zhou, D., et al. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pp.\ ...

2023
[35]

Llama 2: Open foundation and fine-tuned chat models

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

2023 arXiv
[36]

Gradient sparsification for communication-efficient distributed optimization

Wangni, J., Wang, J., Liu, J., and Zhang, T. Gradient sparsification for communication-efficient distributed optimization. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volu...

2018
[37]

V., Zhou, D., et al

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35: 0 24824--24837, 2022

2022
[38]

Terngrad: ternary gradients to reduce communication in distributed deep learning

Wen, W., Xu, C., Yan, F., Wu, C., Wang, Y., Chen, Y., and Li, H. Terngrad: ternary gradients to reduce communication in distributed deep learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 1508–1518, Red Hook, N...

2017
[39]

Less: selecting influential data for targeted instruction tuning

Xia, M., Malladi, S., Gururangan, S., Arora, S., and Chen, D. Less: selecting influential data for targeted instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2025

2025
[40]

YEH, S.-Y., Hsieh, Y.-G., Gao, Z., Yang, B. B. W., Oh, G., and Gong, Y. Navigating text-to-image customization: From ly CORIS fine-tuning to model evaluation. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=wfzXa8e783

2024
[41]

Adaptive budget allocation for parameter-efficient fine-tuning

Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., and Zhao, T. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=lq62uWRJjiY

2023
[42]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Database-friendly random projections: Johnson-lindenstrauss with binary coins

Achlioptas, D. Database-friendly random projections: Johnson-lindenstrauss with binary coins. Journal of Computer and System Sciences, 66 0 (4): 0 671--687, 2003. ISSN 0022-0000. doi:https://doi.org/10.1016/S0022-0000(03)00025-4. URL https://www.sciencedirect.com/science/article/pii/S0022000003000254. Special Issue on PODS 2001

doi:10.1016/s0022-0000(03)00025-4 2003

[2] [2]

Aji, A. F. and Heafield, K. Sparse communication for distributed gradient descent. In Palmer, M., Hwa, R., and Riedel, S. (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp.\ 440--445, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi:10.18653/v1/D17-1045. URL https://aclant...

doi:10.18653/v1/d17-1045 2017

[3] [3]

Z., Tomioka, R., and Vojnovic, M

Alistarh, D., Grubic, D., Li, J. Z., Tomioka, R., and Vojnovic, M. Qsgd: communication-efficient sgd via gradient quantization and encoding. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 1707–1718, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964

work page 2017

[4] [4]

signsgd: Compressed optimisation for non-convex problems

Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., and Anandkumar, A. signsgd: Compressed optimisation for non-convex problems. In International Conference on Machine Learning, pp.\ 560--569. PMLR, 2018

2018

[5] [5]

Charikar, M. S. Similarity estimation techniques from rounding algorithms. In Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, STOC '02, pp.\ 380–388, New York, NY, USA, 2002. Association for Computing Machinery. ISBN 1581134959. doi:10.1145/509907.509965. URL https://doi.org/10.1145/509907.509965

work page arXiv 2002

[6] [6]

K., Ahn, H., Bae, J., Zhao, K., Kang, M., Chung, Y., Pratapa, A., Neiswanger, W., Strubell, E., Mitamura, T., Schneider, J., Hovy, E., Grosse, R., and Xing, E

Choe, S. K., Ahn, H., Bae, J., Zhao, K., Kang, M., Chung, Y., Pratapa, A., Neiswanger, W., Strubell, E., Mitamura, T., Schneider, J., Hovy, E., Grosse, R., and Xing, E. What is your data worth to gpt? llm-scale data valuation with influence functions, 2024. URL https://arxiv.org/abs/2405.13954

arXiv 2024

[7] [7]

H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., and Palomaki, J

Clark, J. H., Choi, E., Collins, M., Garrette, D., Kwiatkowski, T., Nikolaev, V., and Palomaki, J. TyDi QA : A benchmark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, 2020

work page 2020

[8] [8]

Free Dolly : Introducing the world's first truly open instruction-tuned LLM , 2023

Conover, M., Hayes, M., Mathur, A., Xie, J., Wan, J., Shah, S., Ghodsi, A., Wendell, P., Zaharia, M., and Xin, R. Free Dolly : Introducing the world's first truly open instruction-tuned LLM , 2023

work page 2023