Pith. sign in

REVIEW 12 cited by

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.04396 v2 pith:XAEMDIAU submitted 2024-02-06 cs.LG cs.AIcs.CL

Albert Tseng , Jerry Chee , Qingyao Sun , Volodymyr Kuleshov , Christopher De Sa This is my paper
classification cs.LGcs.AIcs.CL
keywords quipquantizationachievesbettercodebookshadamardincoherenceintroduce
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing their weights to low-precision. In this work, we introduce QuIP#, a weight-only PTQ method that achieves state-of-the-art results in extreme compression regimes ($\le$ 4 bits per weight) using three novel techniques. First, QuIP# improves QuIP's (Chee et al., 2023) incoherence processing by using the randomized Hadamard transform, which is faster and has better theoretical properties. Second, QuIP# uses vector quantization to take advantage of the ball-shaped sub-Gaussian distribution that incoherent weights possess: specifically, we introduce a set of hardware-efficient codebooks based on the highly symmetric $E_8$ lattice, which achieves the optimal 8-dimension unit ball packing. Third, QuIP# uses fine-tuning to improve fidelity to the original model. Our experiments show that QuIP# outperforms existing PTQ methods, enables new behaviors in PTQ scaling, and supports fast inference. Our code can be found at https://github.com/Cornell-RelaxML/quip-sharp.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Leech Lattice Vector Quantization for Efficient LLM Compression

    cs.LG 2026-03 conditional novelty 6.5 of 10

    Leech-lattice vector quantization with codebook-free indexing and shape-gain search reaches state-of-the-art ~2-bit post-training LLM compression and reduces reliance on Hadamard rotations.

  2. TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A task guide acts as a meta-query to compress the KV cache once per task, so one compressed memory serves many future questions.

  3. GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference

    cs.AR 2026-07 conditional novelty 6.0 of 10

    Rotation and fine-grained group quantization can work together if rotation spans several quantization groups and outlier channels are permuted onto harmonic Hadamard rows, enabling 4-bit LLM inference with integer-onl...

  4. A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Motion-aware vector quantization plus centroid-reuse GEMM and a custom accelerator yield multi-fold VLA speedups over GPU and prior accelerators with ~2.5% average success-rate drop.

  5. Reliability Scaling Laws for Quantized Large Language Models

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Reliability of quantized LLMs peaks nonlinearly at 4-bit precision under fixed total model bits, while accuracy scales monotonically, and quantization can improve robustness to natural perturbations.

  6. FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    FAIR-Calib is a frontier-aware instability-reweighted calibration framework for PTQ of dLLMs that minimizes reweighted hidden-state MSE to reduce frontier decision flips.

  7. Price of metric universality in vector quantization is at most 0.11 bit

    cs.IT 2026-02 conditional novelty 6.0 of 10

    A universal vector-quantization codebook exists that is within 0.11 bit/coordinate of covariance-adaptive waterfilling simultaneously for all input covariances, for Gaussian weights.

  8. SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference

    cs.LG 2026-02 conditional novelty 6.0 of 10

    SALAAD uses an ADMM-style optimizer and an adaptive controller to force LLM weights into sparse-plus-low-rank form during pretraining, so one checkpoint can be elastically deployed at many parameter budgets.

  9. From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models

    cs.CV 2025-12 conditional novelty 6.0 of 10

    TAD, a 5,861-question benchmark, shows VLMs score far below humans on temporal understanding of driving videos, and an ego-trajectory text summary (TCogMap) substantially boosts their scores.

  10. CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs

    cs.LG 2025-07 conditional novelty 6.0 of 10

    CCQ compresses LLMs to 2.0-2.75 bits per weight using convolutional codes and bit-shift decoding, shrinking 671B-parameter models to under 200GB.

  11. Break Through the Compression Bottleneck: From Theory to Practice

    cs.CL 2026-05 reject novelty 5.0 of 10

    The paper asserts a first proof that low-rank decomposition and quantization are non-orthogonal tools for LLM compression, recommends low-rank-first ordering, and adds a diagonal scaling fix (DAM) that reduces the com...

  12. Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models

    cs.NE 2026-07 conditional novelty 4.0 of 10

    A whole-network final-feature and statistics matching objective improves 1.125-bit and 4.125-bit LLM quantization over layer-local and distillation baselines, but its cross-layer mechanism reduces exactly to final-fea...

Pith tools