Pith. sign in

REVIEW 11 cited by

QuIP: 2-Bit Quantization of Large Language Models With Guarantees

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2307.13304 v2 pith:5SZUXPGZ submitted 2023-07-25 cs.LG cs.CL

Jerry Chee , Yaohui Cai , Volodymyr Kuleshov , Christopher De Sa This is my paper · ORCID
classification cs.LGcs.CL
keywords quantizationquipincoherenceweightexistingfirsthessianlanguage
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

This work studies post-training parameter quantization in large language models (LLMs). We introduce quantization with incoherence processing (QuIP), a new method based on the insight that quantization benefits from $\textit{incoherent}$ weight and Hessian matrices, i.e., from the weights being even in magnitude and the directions in which it is important to round them accurately being unaligned with the coordinate axes. QuIP consists of two steps: (1) an adaptive rounding procedure minimizing a quadratic proxy objective; (2) efficient pre- and post-processing that ensures weight and Hessian incoherence via multiplication by random orthogonal matrices. We complement QuIP with the first theoretical analysis for an LLM-scale quantization algorithm, and show that our theory also applies to an existing method, OPTQ. Empirically, we find that our incoherence preprocessing improves several existing quantization algorithms and yields the first LLM quantization methods that produce viable results using only two bits per weight. Our code can be found at https://github.com/Cornell-RelaxML/QuIP.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAs

    cs.AR 2025-04 conditional novelty 7.0 of 10

    TeLLMe is the first edge FPGA accelerator that runs a 1.58-bit ternary LLM end-to-end, including prefill and decoding, achieving 9.51 tokens/s and 0.55 to 1.15 second prefill under 7 watts.

  2. Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

    cs.LG 2024-11 conditional novelty 7.0 of 10

    A new theorem and method (HIGGS) make per-layer quantization error a reliable predictor of final model perplexity, enabling state-of-the-art data-free and dynamic bit-width LLM compression.

  3. BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration

    cs.LG 2024-11 conditional novelty 7.0 of 10

    BitMoD quantizes LLM weights to 3-4 bits using per-group custom floating-point formats with a repurposed zero value, and accelerates them with a bit-serial PE and dequantization unit.

  4. When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Under dynamic W4A4KV4 quantisation, a head-shared RoPE-aligned pairwise rotation that exactly minimizes a pooled variance surrogate still yields higher perplexity than full-head Hadamard mixing in all evaluated comparisons.

  5. SoftWater: Class-Aware Rate Allocation for Softmax Quantization

    cs.LG 2026-08 conditional novelty 6.0 of 10

    SoftWater, a KL-divergence-based quantizer for LLM softmax heads, allocates bit rate by class frequency and variance and beats WaterSIC at matched head rates on 59 of 60 test points.

  6. Price of metric universality in vector quantization is at most 0.11 bit

    cs.IT 2026-02 conditional novelty 6.0 of 10

    A universal vector-quantization codebook exists that is within 0.11 bit/coordinate of covariance-adaptive waterfilling simultaneously for all input covariances, for Gaussian weights.

  7. ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    ROSAQ uses PCA projection to find salient transformer weight channels, keeps those in FP16, quantizes the rest to INT3/4, and reports modest gains over previous quantization methods.

  8. PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling

    cs.LG 2025-06 conditional novelty 6.0 of 10

    PCDVQ compresses LLM weights to 2 bits by quantizing vector directions and magnitudes separately with distribution-matched codebooks, reporting modest zero-shot accuracy gains over prior vector quantization baselines.

  9. OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting

    cs.LG 2025-01 conditional novelty 6.0 of 10

    OSTQuant quantizes LLM weights, activations, and KV cache to 4 bits using learnable orthogonal and scaling transformations plus a new KL-Top loss, reporting near-lossless W4-only and strong W4A4KV4 results on LLaMA models.

  10. Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs

    cs.LG 2026-08 conditional novelty 5.0 of 10

    A 2-bit base plus three 2-bit residual stages gives one checkpoint that runs at 2, 4, 6, or 8 bits, matching a prior multi-precision baseline at 6-8 bits in most tested models.

  11. Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoring

    cs.LG 2025-01 conditional novelty 5.0 of 10

    A post-training quantization scheme that uses per-group leading-one detection to keep four salient bits from an 8/16-bit integer base, achieving 4-bit weights, activations, and KV cache without fine-tuning or rotation.

Pith tools