Pith. sign in

REVIEW 3 major objections 5 minor 60 references

LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims LowRA is the first framework to make LoRA fine-tuning accurate below 2 bits per parameter, staying usable at 1.15 bits while cutting memory by up to 50 percent.

desk verdict LowRA is a plausible sub-2-bit LoRA system, but the paper overstates its own mechanism in one important place and leaves reproducibility gaps; worth reading, not yet trustworthy as-is. read the letter →

arxiv 2502.08141 v1 pith:PDYDMJIX submitted 2025-02-12 cs.LG cs.ARcs.CLcs.PF

Zikai Zhou , Qizheng Zhang , Hermann Kumbong , Kunle Olukotun This is my paper · ORCID
classification cs.LGcs.ARcs.CLcs.PF
keywords LowRALofine-tuningsub-2-bitquantizationmixed-precisionassignmentintegerlinearprogrammingLloyd-MaxalgorithmCUDAkernelsmemory-efficientLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LowRA is a framework with a single target: keeping the fine-tuning of large language models accurate when the frozen base weights are stored at under 2 bits per parameter, using LoRA (low-rank adaptation), the technique that freezes base weights and trains small trainable adapters. It combines three coordinated choices: per-output-channel quantization mappings and thresholds found by a weighted Lloyd-Max search, a two-level integer-linear-programming step that assigns each output channel a 1-, 2-, or 4-bit precision under a fixed bit budget, and real CUDA kernels that pack and unpack the low-bit weights instead of simulating quantization. Across LLaMA-2-7B, LLaMA-2-13B, BART-large, and LLaMA-30B on four datasets, the paper reports that LowRA beats the QLoRA and LoftQ baselines at every tested width below 4 bits, matches them at 4 bits, and stays accurate at 1.75 bits per parameter (1.15 bits on LLaMA-30B). If these results hold, ultra-low-bit LoRA becomes practical on devices with very limited memory, such as a Raspberry Pi or a single 16 GB GPU.

What carries the argument

The load-bearing machinery is a three-part quantization pipeline. A weighted Lloyd-Max learner extends the classic Lloyd-Max quantizer by weighting each 64-element, absmax-normalized block of weights by its own scale, producing per-output-channel mappings and thresholds that minimize weighted mean-squared error. A two-level integer linear program then assigns each output channel a precision of 1, 2, or 4 bits: cluster-level ILPs decide how many channels in each cluster receive each precision under the global bit budget, and intra-cluster ILPs pick the specific channels, all minimizing the summed squared error $\mathrm{SSE}=\sum_i (x_i-\hat{x}_i)^2$. Custom CUDA quantize and dequantize kernels provide the third pillar, storing the mixed-precision weights as genuinely packed bits rather than simulating them in floating point. The shared objective binding all three pieces is SSE minimization, which the paper uses as a proxy for preserving fine-tuned task performance.

What would settle it

Fine-tune LLaMA-2-7B on WikiText-2 at 1.75 and 2.0 bits with LowRA's mappings, thresholds, and LoftQ initialization, but replace the ILP's SSE-minimizing precision assignment with a uniform per-channel assignment at the same total bit budget, or with an assignment chosen to minimize validation perplexity instead of SSE; if the uniform or task-loss assignment matches or beats LowRA's perplexity, the SSE proxy is not doing the load-bearing work.

Watch

Extended reading notes

Core claim

The paper's central claim is that the 2-bit floor on quantized LoRA fine-tuning is not fundamental but an artifact of coarse, globally shared quantization choices. The discovery is that three levels of fine-grained control — per-output-channel mappings and thresholds, per-channel mixed precision chosen by a hierarchical ILP, and kernel support for genuinely packed 1-, 2-, and 4-bit representations — jointly keep task performance intact far below 2 bits. LowRA reports the first accurate LoRA fine-tuning at 1.75 bits per parameter on LLaMA-2-7B, LLaMA-2-13B, and BART-large, and at 1.15 bits on LLaMA-30B, with perplexity and ROUGE scores that beat QLoRA and LoftQ at every width below 4 bits and match them at 4 bits, while using about 0.86 fewer bits per parameter at matched quality. The quantization is data-free and one-shot, leaving the base weights task-agnostic and reusable across multiple LoRA adapters, which is exactly what multi-tenant serving setups require.

Load-bearing premise

The load-bearing assumption is that minimizing the summed squared error of the quantized base weights reliably predicts how well the fine-tuned model will perform; if that proxy diverges from real task quality at very low precision, LowRA's carefully chosen per-channel bit assignments could be no better than a uniform scheme.

Editorial extensions

If this is right

  • Fine-tuning stays accurate at 1.75 bits per parameter on LLaMA-2-7B, LLaMA-2-13B, and BART-large, and at 1.15 bits on LLaMA-30B.
  • At equal bit widths between 2 and 3 bits, LowRA reports lower perplexity and higher ROUGE than QLoRA and LoftQ, matching them at 4 bits; at matched quality it needs about 0.86 fewer bits per parameter on average.
  • Memory for fine-tuning and inference falls by 30–50%, which the paper claims makes LLaMA-2-7B deployable on a Raspberry Pi 4 with 4 GB of RAM and LLaMA-30B fine-tunable on a single 16 GB T4 GPU.
  • Because quantization is data-free and one-shot, one set of quantized base weights can serve many LoRA adapters, which fits multi-tenant serving.
  • The CUDA kernels replace simulated quantization, so the reported savings are realized in actual training and deployment memory rather than only in theory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The SSE proxy is asserted but not demonstrated at sub-2-bit precision: the paper never shows that minimizing quantized-base-weight error tracks downstream perplexity or ROUGE, so the ILP's precision choices could underperform a uniform scheme on models or tasks outside the four tested.
  • The same mapping/threshold search and per-channel precision pipeline should transfer to other frozen-weight compression settings, such as encoder-only models, vision-language backbones, or activation quantization, but the paper does not test those cases.
  • A discriminating stress test would replace the ILP's SSE-minimizing assignment with a uniform or random assignment at the same average bit count; if task quality does not drop, the reported gains come from the mappings, thresholds, or kernels rather than from the precision assignment.
  • The 'first below 2 bits' claim is relative to the baselines compared here (QLoRA, LoftQ, PiSSA); other sub-2-bit quantization or PEFT schemes outside that set could predate or match the result.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. LowRA proposes a framework for LoRA fine-tuning of LLMs at sub-2-bit precision. It combines a weighted Lloyd-Max algorithm for learning quantization mappings and thresholds, a hierarchical ILP-based channelwise precision assigner, and custom CUDA kernels for low-bit/mixed-precision quantization and dequantization. Experiments on LLaMA-2-7B, LLaMA-2-13B, BART-large, and LLaMA-30B report perplexity and ROUGE improvements over QLoRA and LoftQ at equal bit widths, and claim the first accurate LoRA fine-tuning below 2 bits per parameter, down to 1.15 bits on LLaMA-30B.

Significance. If the claims are substantiated, LowRA addresses a real gap: existing quantized-LoRA methods generally operate at 2-4 bits, and sub-2-bit LoRA fine-tuning with acceptable task performance would be a significant step for resource-constrained fine-tuning and deployment. The ILP formulation for precision assignment and the weighted Lloyd-Max formulation are coherent, and the CUDA kernel implementation is a practical system contribution that addresses the simulated-quantization gap noted in Section 2.3. The paper also includes honest discussion of LoftQ's reproducibility issues and reports overhead measurements. However, the current manuscript contains an internal inconsistency between the claimed per-output-channel design and the implemented global scheme, an ablation table with unexplained exact matches to LoftQ, and no validation of the SSE proxy that motivates the precision assigner. These issues prevent the central claims from being accepted at face value.

major comments (3)
  1. [Sections 4.1 and 5] The paper claims per-output-channel thresholds and mappings, but the implementation uses a single global learned set. Section 4.1 states that 'LowRA allows each output channel to adopt a different combination of mappings and thresholds,' and Section 5 says the learner produces 'per-output-channel mappings and thresholds.' Yet Section 4.1 later says 'we adopt the same set of learned thresholds and learned mappings for a single base weight for the proof of concept,' and Section 5 states 'we take the average of all thresholds to preserve distribution and prevent instability.' The evaluated system therefore does not implement per-output-channel quantization as motivated in Section 2.3 (limitation L2). The authors must clarify what the implementation actually does. If the system uses global mappings/thresholds, the per-channel motivation and Figure 3 are misleading; if it uses per-channel mappings, the averaging step needs a detailed explanation of how it is reconciled with per-channel thresholds.
  2. [Appendix B, Tables 4 and 5] The 'PA Only' ablation rows report exactly the same perplexity, accuracy, and ROUGE scores as LoftQ at 2.0 and 4.0 bpp. For example, Table 5 shows perplexity 8.63 and accuracy 0.536 for both LoftQ and PA Only at 2.0 bpp, and Table 4 shows ROUGE1 31.8941 for both at 2.0 bpp on XSUM. The paper does not report seeds, determinism settings, or the concrete precision assignments used by PA Only. If PA Only reuses LoftQ's assignment at these budgets, the ablation is circular and does not isolate the precision assigner. If it does not reuse LoftQ's assignment, the exact equality suggests the rows were not independently measured. Either way, the component-level evidence for the precision assigner is void. The authors must disclose the assignments, seeds, and reproduction code, and independently re-run these configurations to confirm the reported numbers.
  3. [Section 6 and Table 5] The precision assignment objective is minimization of SSE, which Section 6 calls 'an effective proxy' for downstream task performance, but no experiment validates this proxy. The ablation in Table 5 shows that PA Only yields minimal improvement over LoftQ on LLaMA-2-7B/WikiText-2 (e.g., at 2.5 bpp: 7.75 vs 7.72 perplexity; at 3.0 bpp: 6.75 vs 6.87 perplexity), so the benefit of the SSE-based assigner is not self-evident. Please add an experiment that compares the SSE-based assignment against a uniform assignment or LoftQ's layerwise assignment at the same bit budget, reporting downstream perplexity or ROUGE on at least one model and dataset. Without such a comparison, the reported gains cannot be attributed to the precision assigner.
minor comments (5)
  1. [Table 3 caption] The caption reads 'LLaMA-33B' but the text and elsewhere refer to LLaMA-30B; please correct this typo.
  2. [Section 7.5 and Appendix C] The memory estimates in Appendix C are explicitly 'linear layer only (not attention),' but Section 7.5 uses these figures to claim deployment on a Raspberry Pi 4 (4GB RAM) and fine-tuning LLaMA-30B on a Tesla T4 (16GB). The non-linear-layer memory (attention, embeddings, activations) should be accounted for before making these deployment claims, or the claims should be reworded as estimates for linear layers only.
  3. [Section 5 and Appendix F] The initialization code in Appendix F creates per-channel mapping and threshold tensors, but the text in Section 5 says the thresholds are averaged. Please clarify whether the mappings are also averaged and how the 'per-output-channel' description in Figure 3 and Section 4.1 relates to the actual tensors used in the kernel.
  4. [Abstract and Section 7.4] The claim of being 'the first framework to enable LoRA fine-tuning below 2 bits per parameter' should be accompanied by a precise definition of what is included in 'bits per parameter' (base weights only, or adapters and optimizer state) and a discussion of the closest prior sub-2-bit works, such as those cited as LoRA-GA [46] and IR-QLoRA [37], to justify the novelty claim.
  5. [Section 5 and Appendix E] The paper states that two Lloyd-Max iterations and 128 K-Means clusters are sufficient, but no sensitivity analysis is provided. A brief ablation varying these values would strengthen the claim that the method is robust to these hyperparameters.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LowRA's reconstruction-error objectives are decoupled from downstream evaluation; ablation equalities are budget-degenerate, not self-referential.

full rationale

LowRA's two quantitative components (the weighted Lloyd-Max mapping/threshold learner and the two-level ILP precision assigner) both minimize reconstruction error (MSE/SSE) on the frozen pretrained weights. The paper's headline results—better perplexity/ROUGE at equal bits, or equal quality at lower bits—are measured on held-out task metrics that never appear in these objectives, so no fitted parameter is renamed as a prediction and no quantity is defined in terms of the claimed outcome. There are no self-citations and no imported uniqueness theorems. The apparent exact equality of the 'PA Only' ablation row with LoftQ at 2.0 and 4.0 bpp in Tables 4-5 is consistent with the precision budget degenerating to uniform 2-bit/4-bit assignment (available precisions are {1,2,4}), and the paper openly notes that the precision assigner 'only yields minimal advantage' on Llama-2-7B/WikiText; at intermediate budgets the rows differ, so the ablation is not constructing the central result from LoftQ's outputs. The SSE-as-proxy assumption in Section 6 is an unverified modeling assumption and a correctness risk, but it is not circular because downstream quality is never used to fit the proxy.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claims rest on the SSE-as-proxy assumption, the choice of LoftQ initialization, and several hand-set hyperparameters (Lloyd-Max iterations, K-Means clusters, precision set restrictions). No new physical or mathematical entities are invented; the method is data-free for quantization but depends on the pretrained weight distributions. The most consequential unstated premise is that minimizing weight reconstruction error at sub-2-bit precision leads to downstream task accuracy.

free parameters (4)
  • Lloyd-Max iterations = 2
    Hand-set in Section 5 to cap preprocessing cost; no ablation on iteration count.
  • K-Means clusters per parameter-count group = 128
    Chosen in Section 6.1 for scalability; no ablation reported.
  • Precision set for bpp >= 2.0 = {2, 4} bits
    Section 6.2 footnote: including 1-bit underperformed for bpp >= 2.0, so 1-bit is excluded in that regime; a data-influenced configuration choice.
  • Groupwise normalization group size = 64 elements
    Inherited from QLoRA (Section 4.1); not tuned in this paper.
assumptions (6)
  • domain assumption The SSE of quantized base weights is a valid proxy for downstream task performance.
    Drives the ILP precision assigner (Section 6); no experiment validates the SSE-task performance correlation at sub-2-bit precision.
  • domain assumption Output-channel grouping captures the dominant variation in LLM linear layer weights.
    Justified by a 2.20x standard deviation ratio measured only on Llama2-7b (Appendix H) and assumed across architectures.
  • domain assumption NormalFloat thresholds and mappings are a good initialization for the Lloyd-Max learner.
    Listing 1 initializes 2/4-bit mappings with NF4 values; alternative initializations are not explored.
  • ad hoc to paper Averaging per-channel thresholds into a single global set preserves the benefit of the learned mappings.
    Section 5 says 'we take the average of all thresholds' and Section 4.1 says 'we adopt the same set of learned thresholds and learned mappings for a single base weight'; this contradicts the per-output-channel design and is asserted without ablation.
  • ad hoc to paper Two Lloyd-Max iterations and 128 K-Means clusters are sufficient for near-optimal quantization quality.
    Section 5 sets iterations to 2 and Section 6.1 sets 128 clusters; no convergence analysis or sensitivity study is provided.
  • domain assumption The two-level ILP decomposition closely approximates the optimal global mixed-precision assignment.
    The hierarchical ILP in Section 6 is a heuristic to keep solving tractable; the optimality gap is not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits." pith.science (2026). https://pith.science/paper/PDYDMJIX

@misc{pith2026250208141,
  author       = {Pith},
  title        = {Pith review of: LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PDYDMJIX}},
  note         = {Machine review of arXiv:2502.08141}
}
read the original abstract

Fine-tuning large language models (LLMs) is increasingly costly as models scale to hundreds of billions of parameters, and even parameter-efficient fine-tuning (PEFT) methods like LoRA remain resource-intensive. We introduce LowRA, the first framework to enable LoRA fine-tuning below 2 bits per parameter with minimal performance loss. LowRA optimizes fine-grained quantization - mapping, threshold selection, and precision assignment - while leveraging efficient CUDA kernels for scalable deployment. Extensive evaluations across 4 LLMs and 4 datasets show that LowRA achieves a superior performance-precision trade-off above 2 bits and remains accurate down to 1.15 bits, reducing memory usage by up to 50%. Our results highlight the potential of ultra-low-bit LoRA fine-tuning for resource-constrained environments.

Figures

Figures reproduced from arXiv: 2502.08141 by the authors.

Figure 1
Figure 1. End-to-end workflow of LowRA. 2.1 Low-Rank Adaptation (LoRA) of LLMs Fine-tuning large language models (LLMs) allows us to adapt pre-trained LLMs to particular tasks or domains [50, 48, 55]. This process usually requires changing all model parameters, which can be prohibitively expensive (in terms of compute and memory) when the number of model parameters increases. Low-Rank Adaptation (LoRA) [14] tackles this by fr… view at source ↗
Figure 2
Figure 2. Distributions of normalized parameters in different output channels sampled from the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Roles of mappings and thresholds in quantization. Circles represent thresholds whereas crosses represent mappings. Colored Triangles represent the process of converting a range of original/unquantized real values - partitioned by thresholds - to the mapped values corresponding to each quantization level. methods require a calibration set for quantization scheme learning [24, 16]. In contrast, LowRA uses data-free on… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Two-step ILP-based Workflow for Channelwise Precision Assignment [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Overview of Kernel for Low-Bit Fine-Grained [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Overview of Kernel for Low-Bit Fine-Grained [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Decomposition of finetuning memory footprint for Llama-2 7B and 13B under different bits per parameter. 1.75 1.8 1.9 2.0 2.25 2.5 3.0 4.0 Bits per Parameter (BPP) 0.0 0.2 0.4 0.6 0.8 1.0 Memory Usage (Normalized to Max) 3309 3348 3425 3502 3695 3888 4274 5046 306 306 3…
Figure 8
Figure 8. Figure 8: Decomposition of inference memory footprint for Llama-2 7B and 13B under different bits per parameter. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Decomposition of memory footprint for Llama-33B under different bits per parameter. [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: Standard Deviations Along Output Channel Dimension vs Along Input Channel Dimen [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 28 canonical work pages

  1. [1]

    A general language assistant as a laboratory for alignment

    Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861 , 2021

    arXiv 2021

  2. [2]

    Binarybert: Pushing the limit of bert quantization

    Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. Binarybert: Pushing the limit of bert quantization. arXiv preprint arXiv:2012.15701, 2020

    arXiv 2012

  3. [3]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 , 2022

    arXiv 2022

  4. [4]

    Flexquant: Elastic quantization framework for locally hosted llm on edge devices, 2025

    Yuji Chai, Mujin Kwen, David Brooks, and Gu-Yeon Wei. Flexquant: Elastic quantization framework for locally hosted llm on edge devices, 2025

    work page 2025

  5. [5]

    Punica: Multi-tenant lora serving

    Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krishnamurthy. Punica: Multi-tenant lora serving. Proceedings of Machine Learning and Systems , 6:1–13, 2024

    work page 2024

  6. [6]

    Tesla P100 GPU Accelerator

    NVIDIA Corporation. Tesla P100 GPU Accelerator. https://www.nvidia.com/en-in/ data-center/tesla-p100/, 2016. Accessed: 2025-01-29

    work page 2016

  7. [7]

    Tesla V100 GPU Accelerator Datasheet

    NVIDIA Corporation. Tesla V100 GPU Accelerator Datasheet. https://images.nvidia.com/ content/technologies/volta/pdf/tesla-volta-v100-datasheet-letter-fnl-web.pdf ,

    work page

  8. [8]

    NVIDIA T4 Virtualization Datasheet

    NVIDIA Corporation. NVIDIA T4 Virtualization Datasheet. https://www.nvidia. com/content/dam/en-zz/Solutions/design-visualization/solutions/resources/ documents1/Datasheet_NVIDIA_T4_Virtualization.pdf, 2021. Accessed: 2025-01-29

    work page 2021

Show all 60 references
  1. [9]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems , 36, 2024

    2024

  2. [10]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024. 14

    2024 arXiv

  3. [11]

    Learned step size quantization

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. arXiv preprint arXiv:1902.08153 , 2019

    1902 arXiv

  4. [12]

    Raspberry Pi 4 Model B

    Raspberry Pi Foundation. Raspberry Pi 4 Model B. https://www.raspberrypi.com/ products/raspberry-pi-4-model-b/ , 2019. Accessed: 2025-01-29

    2019

  5. [13]

    Teaching machines to read and comprehend

    Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. Advances in neural information processing systems, 28, 2015

    2015

  6. [14]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  7. [15]

    Mitigating large language model hallucination with faithful finetuning

    Minda Hu, Bowei He, Yufei Wang, Liangyou Li, Chen Ma, and Irwin King. Mitigating large language model hallucination with faithful finetuning. arXiv preprint arXiv:2406.11267 , 2024

    2024 arXiv

  8. [16]

    Accurate post training quantization with small calibration sets

    Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets. In International Conference on Machine Learning, pages 4466–4475. PMLR, 2021

    2021

  9. [17]

    L4q: Parameter efficient quantization-aware training on large language models via lora-wise lsq

    Hyesung Jeon, Yulhwa Kim, and Jae-joon Kim. L4q: Parameter efficient quantization-aware training on large language models via lora-wise lsq. arXiv preprint arXiv:2402.04902 , 2024

    2024 arXiv

  10. [18]

    The singular value decomposition: Its computation and some applications

    Virginia Klema and Alan Laub. The singular value decomposition: Its computation and some applications. IEEE Transactions on automatic control , 25(2):164–176, 1980

    1980

  11. [19]

    Openassistant conversations-democratizing large language model alignment

    Andreas K¨ opf, Yannic Kilcher, Dimitri von R¨ utte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich´ ard Nagyfi, et al. Openassistant conversations-democratizing large language model alignment. Advances in Neural Informatio...

    2024

  12. [20]

    Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    Mike Lewis. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 , 2019

    1910 arXiv

  13. [21]

    BART-Large Model Card

    Mike Lewis and Facebook AI. BART-Large Model Card. https://huggingface.co/facebook/ bart-large, 2023. Accessed: [Jan 26th 2025]

    2023

  14. [22]

    Svdqunat: Absorbing outliers by low-rank components for 4-bit diffusion models

    Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. Svdqunat: Absorbing outliers by low-rank components for 4-bit diffusion models. arXiv preprint arXiv:2411.05007 , 2024

    2024

  15. [23]

    Loftq: Lora-fine-tuning-aware quantization for large language models

    Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao. Loftq: Lora-fine-tuning-aware quantization for large language models. arXiv preprint arXiv:2310.08659, 2023

    2023 arXiv

  16. [24]

    Apiq: Finetuning of 2-bit quantized large language model

    Baohao Liao, Christian Herold, Shahram Khadivi, and Christof Monz. Apiq: Finetuning of 2-bit quantized large language model. arXiv preprint arXiv:2402.05147 , 2024

    2024 arXiv

  17. [25]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. 15

    2024 arXiv

  18. [26]

    Mitigating hallucination in large multi-modal models via robust instruction tuning

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representations , 2023

    2023

  19. [27]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems , 35:1950–1965, 2022

    1950

  20. [28]

    Nonuniform- to-uniform quantization: Towards accurate quantization via generalized straight-through estimation

    Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P Xing, and Zhiqiang Shen. Nonuniform- to-uniform quantization: Towards accurate quantization via generalized straight-through estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages...

    2022

  21. [29]

    Least squares quantization in pcm

    Stuart Lloyd. Least squares quantization in pcm. IEEE transactions on information theory , 28(2):129–137, 1982

    1982

  22. [30]

    Unipelt: A unified framework for parameter-efficient language model tuning

    Yuning Mao, Lambert Mathias, Rui Hou, Amjad Almahairi, Hao Ma, Jiawei Han, Wen-tau Yih, and Madian Khabsa. Unipelt: A unified framework for parameter-efficient language model tuning. arXiv preprint arXiv:2110.07577 , 2021

    2021 arXiv

  23. [31]

    Quantizing for minimum distortion

    Joel Max. Quantizing for minimum distortion. IRE Transactions on Information Theory , 6(1):7–12, 1960

    1960

  24. [32]

    Pissa: Principal singular values and singular vectors adaptation of large language models

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948 , 2024

    2024 arXiv

  25. [33]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843 , 2016

    2016 arXiv

  26. [34]

    Pulp: a linear programming toolkit for python

    Stuart Mitchell, Michael OSullivan, and Iain Dunning. Pulp: a linear programming toolkit for python. The University of Auckland, Auckland, New Zealand , 65:25, 2011

    2011

  27. [35]

    Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization

    Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745, 2018

    2018 arXiv

  28. [36]

    Towards modular llms by building and reusing a library of loras

    Oleksiy Ostapenko, Zhan Su, Edoardo Maria Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, and Alessandro Sordoni. Towards modular llms by building and reusing a library of loras. arXiv preprint arXiv:2405.11157 , 2024

    2024 arXiv

  29. [37]

    Accurate lora-finetuning quantization of llms via information retention

    Haotong Qin, Xudong Ma, Xingyu Zheng, Xiaoyang Li, Yang Zhang, Shouda Liu, Jie Luo, Xianglong Liu, and Michele Magno. Accurate lora-finetuning quantization of llms via information retention. arXiv preprint arXiv:2402.05445 , 2024

    2024 arXiv

  30. [38]

    Coin-or: an open-source library for optimization

    Matthew J Saltzman. Coin-or: an open-source library for optimization. Programming languages and systems in computational economics and finance , pages 3–32, 2002

    2002

  31. [39]

    Not all bits have equal value: Heterogeneous precisions via trainable noise

    Pedro Savarese, Xin Yuan, Yanjing Li, and Michael Maire. Not all bits have equal value: Heterogeneous precisions via trainable noise. Advances in Neural Information Processing Systems, 35:35769–35782, 2022. 16

    2022

  32. [40]

    Q-bert: Hessian based ultra low precision quantization of bert

    Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Q-bert: Hessian based ultra low precision quantization of bert. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 8815–8821, 2020

    2020

  33. [41]

    Agile-quant: Activation-guided quantization for faster inference of llms on the edge, 2023

    Xuan Shen, Peiyan Dong, Lei Lu, Zhenglun Kong, Zhengang Li, Ming Lin, Chao Wu, and Yanzhi Wang. Agile-quant: Activation-guided quantization for faster inference of llms on the edge, 2023

    2023

  34. [42]

    Slora: Scalable serving of thousands of lora adapters

    Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, et al. Slora: Scalable serving of thousands of lora adapters. Proceedings of Machine Learning and Systems , 6:296–311, 2024

    2024

  35. [43]

    Mobilequant: Mobile-friendly quantization for on-device language models, 2024

    Fuwen Tan, Royson Lee, Lukasz Dudziak, Shell Xu Hu, Sourav Bhattacharya, Timothy Hospedales, Georgios Tzimiropoulos, and Brais Martinez. Mobilequant: Mobile-friendly quantization for on-device language models, 2024

    2024

  36. [44]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ ee Lacroix, Baptiste Rozi` ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023

    2023 arXiv

  37. [45]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023

    2023 arXiv

  38. [46]

    Lora-ga: Low-rank adaptation with gradient approxi- mation

    Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approxi- mation. arXiv preprint arXiv:2407.05000 , 2024

    2024 arXiv

  39. [47]

    Bitstack: Any-size compression of large language models in variable memory environments, 2025

    Xinghao Wang, Pengyu Wang, Bo Wang, Dong Zhang, Yunhua Zhou, and Xipeng Qiu. Bitstack: Any-size compression of large language models in variable memory environments, 2025

    2025

  40. [48]

    Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arXiv pre...

    2022 arXiv

  41. [49]

    Attention is all you need

    A Waswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A Gomez, L Kaiser, and I Polosukhin. Attention is all you need. In NIPS, 2017

    2017

  42. [50]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  43. [51]

    Bsq: Exploring bit-level sparsity for mixed-precision neural network quantization

    Huanrui Yang, Lin Duan, Yiran Chen, and Hai Li. Bsq: Exploring bit-level sparsity for mixed-precision neural network quantization. arXiv preprint arXiv:2102.10462 , 2021

    2021 arXiv

  44. [52]

    Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models

    Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199 , 2021

    2021

  45. [53]

    Atom: Low-bit quantization for efficient and accurate llm serving

    Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, and Baris Kasikci. Atom: Low-bit quantization for efficient and accurate llm serving. Proceedings of Machine Learning and Systems , 6:196–209, 2024. 17

    2024

  46. [54]

    Sysmol: A hardware-software co-design framework for ultra-low and fine-grained mixed-precision neural networks

    Cyrus Zhou, Vaughn Richard, Pedro Savarese, Zachary Hassman, Michael Maire, Michael DiBrino, and Yanjing Li. Sysmol: A hardware-software co-design framework for ultra-low and fine-grained mixed-precision neural networks. arXiv preprint arXiv:2311.14114 , 2023

    2023

  47. [55]

    ND” indicates no data, “-

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 , 2019. 18 A LowRA System Support for Low-Bit Fine-Grained LoRA Fine...

    1909 arXiv

  48. [57]

    Low-Rank Decomposition: A singular value decomposition (SVD) of the current weight (or an updated version of it) is performed to factor out a low-rank approximation

  49. [58]

    initializes

    Residual Quantization: The remaining component (i.e., the difference between the original weight and the low-rank approximation) is quantized to preserve overall model capacity with fewer bits. The key distinction lies in how these two steps are ordered: • LoftQ first quantize...

    1919

  50. [59]

    Assignment (E-step): Assign each data point xi to the cluster center closest in Euclidean distance: c(i) ← arg min 1≤j≤K xi − yj

  51. [60]

    (M-step) These steps are repeated until convergence or until a stopping criterion (e.g., a maximum number of iterations) is met

    Update (M-step): Recompute each cluster center yj as the weighted centroid of the points assigned to it: yj ← P i:c(i)=j wi xi P i:c(i)=j wi . (M-step) These steps are repeated until convergence or until a stopping criterion (e.g., a maximum number of iterations) is met. H Par...

  52. [2017]

    Accessed: 2025-01-29

    2025

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.