Pith. sign in

REVIEW 4 major objections 5 minor 39 references

PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that in 2-bit vector quantization of LLM weights, direction is far more sensitive than magnitude, so splitting the two and spending more bits on direction improves zero-shot accuracy by at least 1.5% over prior methods.

desk verdict The polar-decoupling idea is genuinely worth a look, but the headline 'at least 1.5%' gain rests on a bit-rate arithmetic slip and a missing matched-budget comparison. read the letter →

arxiv 2506.05432 v2 pith:YABVYOF4 submitted 2025-06-05 cs.LG cs.AI

Yuxuan Yue , Zukang Xu , Zhihang Yuan , Dawei Yang , Jianlong Wu , Liqiang Nie This is my paper
classification cs.LGcs.AI
keywords vectorquantizationLLMweightcompressionpost-trainingpolarcoordinatescodebookdesignE8latticeLloyd-Max2-bit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the dominant error in 2-bit vector quantization of LLM weights comes from direction, not length. It reports that separately clustering the directions of LLaMA-2-7B weights costs 46.5% zero-shot accuracy, while clustering magnitudes alone costs 2.3%, and explains that Euclidean distance weights magnitude error quadratically. To fix this imbalance, it converts weight vectors to polar coordinates, quantizes direction and magnitude with separate codebooks, and gives most of the bit budget to direction. The resulting method, PCDVQ, is claimed to beat previous 2-bit quantizers by at least 1.5% average zero-shot accuracy on six LLMs. If true, the paper identifies a reusable design rule: spend quantization bits on direction, and build codebooks from the known Gaussian geometry rather than from data clustering.

What carries the argument

The polar-coordinate decoupling transforms each 8-dimensional weight vector into a unit direction and a scalar radius, quantizing the direction by cosine similarity and the radius by scalar quantizer search. The direction codebook is built by greedily sampling directions from the E8 lattice, which the paper uses because E8 gives the densest sphere packing in dimension 8 and its directions are nearly uniform in space. The magnitude codebook is built with the Lloyd-Max algorithm using the probability density function of the root chi-square distribution with 8 degrees of freedom, which is the exact magnitude distribution of a standard Gaussian vector. A randomized Hadamard transform plus per-column scaling is applied first so that every weight column is treated as approximately standard Gaussian, making the two global codebooks reusable across models and layers.

What would settle it

Take a 2-bit PCDVQ run on LLaMA-2-7B, record the actual magnitudes of rotated weight vectors, and compare their quantiles with the root chi-square cumulative distribution used to build the Lloyd-Max codebook; a systematic mismatch would show that the standard-Gaussian premise fails. Equally decisive: replace the global E8 and Lloyd-Max codebooks with per-layer cluster centers at the same bit budget and check whether the reported 1.5% advantage survives.

Watch

Extended reading notes

Core claim

The paper's central claim is that when weights of a large language model are vector-quantized at 2 bits, the direction of each weight vector carries far more information than its magnitude. On LLaMA-2-7B, quantizing directions alone costs roughly 46.5% zero-shot accuracy, whereas quantizing magnitudes alone costs only 2.3%, and the gap widens as the number of cluster centers shrinks. The authors further argue that the Euclidean distance used by existing vector quantizers over-weights magnitude error, because the magnitude gap enters the squared error quadratically while the direction gap enters approximately linearly. PCDVQ therefore splits each vector into a unit direction and a scalar magnitude, quantizes them with separate codebooks, and allocates more index bits to direction; across six LLMs and five zero-shot tasks, it reports average accuracy gains of at least 1.5% over prior 2-bit methods.

Load-bearing premise

All claimed gains rest on the premise that after a random Hadamard rotation and per-column scaling, every weight column is close enough to a standard Gaussian distribution that fixed codebooks built from that model are well calibrated for every layer and model.

Editorial extensions

If this is right

  • Allocating most of a 2-bit budget to a 14-bit direction index and only 2 bits to magnitude is sufficient to beat coupled vector quantization at equal or higher bits on LLaMA-2, LLaMA-3, and Mistral models.
  • The same precomputed codebooks can be reused for different LLMs because the Hadamard-plus-scaling regularization standardizes weights to a common Gaussian distribution, avoiding per-model codebook training.
  • Direction error dominates the total quantization error for high-dimensional weight vectors, so vector quantizers that minimize Euclidean distance on coupled vectors are systematically optimizing the wrong quantity.
  • Adding a single extra bit to the direction codebook at 2.125 bits yields substantial accuracy gains, indicating that the marginal quantization bit is best spent on direction until saturation.
  • PCDVQ inherits the inference speedups of low-bit weight-only quantization, reporting roughly a tripling of generation throughput, because the compressed weights reduce memory bandwidth.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to test whether the same direction-over-magnitude sensitivity holds in polar-coordinate KV-cache quantizers, whose phase and radius components could be given asymmetric bit budgets.
  • The fixed global codebooks rest on the standard-Gaussian premise, so a testable variant is to learn per-layer or per-head codebooks from actual rotated weights; if residual structure survives the Hadamard transform, such data-driven codebooks should outperform the fixed ones.
  • The decoupling idea could transfer to non-LLM vector quantization workloads with anisotropic data, where E8's spatial uniformity may be less helpful than data-dependent direction codebooks.
  • The reported throughput gain suggests additional engineering room in fused dequantization kernels that reconstruct vectors as a direction-magnitude product in registers, a possibility the paper measures but does not optimize.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PCDVQ, a vector-quantization scheme for LLM weights that decouples each vector into direction and magnitude, quantizes these components independently with separate codebooks, and allocates more bits to direction. The motivation is an observed higher sensitivity of direction to quantization, and the method adds distribution-aligned codebooks: greedy sampling of E8 lattice directions for the direction codebook and Lloyd-Max scalar quantization for a root-chi-square magnitude distribution. Experiments on LLaMA-2, LLaMA-3, and Mistral-7B report lower perplexity and higher zero-shot accuracy than SQ/VQ baselines, with an advertised improvement of at least 1.5% zero-shot accuracy at the 2-bit level.

Significance. If the claims hold, PCDVQ would be a useful 2-bit weight-compression method and would support a clear design principle: allocate more bits to direction and avoid Euclidean-distance coupling in vector quantization. The analytical derivation of the magnitude distribution and the offline codebook construction are elegant, and the experimental evaluation spans six LLMs and multiple zero-shot tasks. However, the central quantitative claim is currently tied to incorrect bit arithmetic and unequal-budget comparisons, and the distribution-alignment premise behind DACC is cited rather than verified on the evaluated models. The core idea is promising, but the paper needs substantial revision before the headline claim can be accepted.

major comments (4)
  1. [A.3, Tables 1 and 2] The BPW computation in Appendix A.3 is incorrect: with a=16, b=2, and k=8, (16+2)/8 = 2.25, not 2.125. The rows labeled 'PCDVQ 2.125' therefore consume the same budget as GPTVQ 2.25 and are compared against baselines at lower bit rates. Recomputing the advertised margin at matched budgets against VPTQ gives +1.08 for LLaMA-2-7B (60.44 vs 59.36), +0.55 for LLaMA-2-13B (63.66 vs 63.11), +1.24 for LLaMA-3-70B (71.98 vs 70.74), and +1.13 for Mistral-7B (64.33 vs 63.20), all below the claimed 'at least 1.5%'. The headline claim in the abstract and Section 1 is therefore unsupported by the reported data. Please either rerun the intended a=15 configuration, which changes the direction codebook size by a factor of two and will likely change accuracy, or re-baseline all comparisons at equal bit rates and adjust the claim accordingly. The memory-reduction percentages in Section 4.4 should also be recomputed for the true 2.25 BPW.
  2. [4.1 and Tables 1, 2] The main result tables contain only the mislabeled 2.125-bit PCDVQ rows; the 2.0-bit PCDVQ configuration (a=14, b=2) is reported only in Table 3 for LLaMA-2-7B. Consequently, the paper's repeated claim of outperforming baselines 'at 2-bit level' is not directly demonstrated for the other model and task combinations, and the comparisons in Tables 1 and 2 are at unequal bit rates. Please include 2.0-bit PCDVQ rows in the main tables, or explicitly restrict the claim to the evaluated bit rates.
  3. [3.2.1 and 3.2.3] The DACC codebooks are constructed from the assumption that after randomized Hadamard transform and per-column scaling, every weight column is approximately standard Gaussian, so directions are spatially uniform and squared magnitudes follow a chi-square distribution. This premise is cited to QuIP [5] but is not verified on any of the six models evaluated here. Since the E8 direction codebook and the Lloyd-Max magnitude codebook are precomputed once and reused for all models, residual correlations, outliers, or non-Gaussian column distributions would systematically miscalibrate them and could remove the claimed advantage. Please add a direct empirical check on the actual transformed weights, such as comparing the empirical distribution of r^2 to chi-square(k) or including an ablation with data-driven codebooks, to support the transferability claim.
  4. [3.2.3, Eq. (7)] There is an unresolved dimensionality mismatch in the direction codebook. The quantized direction parameter phi is defined as a (k-1)-dimensional tuple of angles in Eqs. (6)-(7), while the E8 lattice lives in R^8 and its directions are 8-dimensional unit vectors; for k=8 the codebook C_phi is said to have rows in R^(k-1)=R^7. The paper does not state how E8 directions are transformed into this angular space, nor how cosine similarity is computed between two angle tuples. Without this mapping the method is not reproducible and the claim that the E8 codebook is aligned with the direction distribution cannot be checked. Please specify the exact representation, such as converting 8D unit vectors to angular coordinates or quantizing the 8D normalized vectors directly and reconciling the codebook shape.
minor comments (5)
  1. [Throughout] Please fix typographical issues: 'access' should be 'assess' in Sections 1 and 3.1.2, and 'Llyod-max' should be 'Lloyd-Max' in Algorithms 2 and Table 4.
  2. [Section 2.2] The notation for the codebook size is inconsistent: C is written as R^{2n x k} but the number of codebook entries is later described as 2^n; use a single symbol for the bit-width, such as b, throughout.
  3. [Section 3.1.2, Eq. (5)] The definition of Delta r as sqrt((||v||-||v_hat||)^2) is redundant; defining Delta r = ||v|| - ||v_hat|| would make the decomposition of the squared Euclidean distance clearer.
  4. [Section 4.3, Figure 3] The direction-MSE definition 2||v||^2(1-cos theta) is introduced only in the ablation study; define it earlier in Section 3.1 so that the decomposition used in Figure 3 is available when the motivation is discussed.
  5. [Section 4.4] The efficiency analysis reports generation throughput on an RTX 4090 but does not measure dequantization latency or the overhead of the Hadamard transform mentioned in Appendix A.4; please clarify whether the reported 95.7 tokens/second includes these costs.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: PCDVQ's codebooks are precomputed from external mathematical results (E8 lattice, Lloyd-Max, Hadamard incoherence) and evaluated on external benchmarks; the paper's self-citations are present but not load-bearing.

full rationale

PCDVQ's derivation chain is self-contained against external benchmarks. The two codebooks are constructed offline from analytic distributions: the direction codebook by greedy sampling of E8 lattice directions (Algorithm 1, Section 3.2.3) and the magnitude codebook by Lloyd-Max on a PDF derived in Appendix A.1 from the root chi-square law (Algorithm 2). Neither codebook is fitted to the evaluated models or evaluation tasks; both depend only on the standard-Gaussian premise (Section 3.2.1) that a randomized Hadamard transform maps weight columns to approximately N(0,1), a guarantee imported from the external QuIP work [5, 30], not from the present authors' prior papers. The motivation ('direction exhibits significantly greater sensitivity to quantization compared to the magnitude', abstract) is an independent empirical measurement using K-Means clustering (Figure 1a), not a consequence implied by the method's definitions. The claimed gains are measured on external zero-shot tasks (Arc-Challenge, Arc-Easy, HellaSwag, PIQA, WinoGrande) and PPL (WikiText2, C4), so the headline comparison is not forced by construction, and the a/b bit split is a hand-chosen hyperparameter rather than a fitted parameter later reported as a prediction. Self-citations exist but are not load-bearing: [15] (OstQuant, which shares authors Dawei Yang, Zukang Xu, Zhihang Yuan) is cited only as an example of RHM-based SQ and for dataset choice (Sections 3.2.1 and 4.1); [38] (WKVQuant, sharing authors Yuxuan Yue, Zhihang Yuan, Jianlong Wu, Liqiang Nie) and [37] (LLMViewer) are cited only for the generic importance of post-training quantization (Section 2). No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation, since the Gaussian premise is attributed to external references. Two concerns are noted but are correctness risks rather than circularity: (i) Appendix A.3 states that with a=16, b=2, k=8, 'the BPW can be calculated by (a+b)/k= 2.125', but (16+2)/8 = 2.25, so the rows labeled '2.125' actually cost 2.25 bits/weight and are compared against baselines at lower bit budgets, which weakens the 'at least 1.5%' headline claim; and (ii) the standard-Gaussian premise is cited to prior work rather than verified on the evaluated models, so the DACC codebooks could be miscalibrated if residual structure remains after the Hadamard transform. Neither issue makes any result equivalent to its inputs by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the Gaussianity of RHT-transformed weights (domain assumption), the uniformity of Gaussian directions, the chi-square magnitude law (standard math), the heuristic E8 direction codebook, plus a hand-chosen bit allocation (a=14/16, b=2) and unstated Lloyd-Max and calibration-data settings. No new physical or conceptual entities are postulated.

free parameters (5)
  • a (direction codebook bits) = 14 for 2-bit, 16 for the paper's 2.125-bit (actually 2.25-bit) setting
    Bit allocation hyperparameter chosen by the authors; the 2.125-bit configuration improves QA Avg from 58.60 to 60.44 on LLaMA-2-7B, so the allocation materially affects results.
  • b (magnitude codebook bits) = 2
    Fixed to 2 (4 magnitude levels) across all models and bit settings; no sensitivity study for b is reported.
  • k (vector dimension) = 8
    Fixed vector dimension for VQ, tied to the E8 lattice geometry; a design choice not swept in the paper.
  • Lloyd-Max hyperparameters (tau, tol, M) = unspecified
    Algorithm 2 requires a maximum threshold tau, tolerance tol, and max iterations M; none are reported, so the exact magnitude codebook is not reproducible.
  • fine-tuning calibration sample count = unspecified
    Section 4.1 uses randomly selected WikiText2 and C4 samples 'consistent with original implementations' without stating counts, affecting the fine-tuned results.
assumptions (5)
  • domain assumption Randomized Hadamard transform plus per-column scaling maps weight columns approximately to N(0,1)
    Invoked in Section 3.2.1 and cited to QuIP [5]; the entire DACC codebook construction (Eqs. 9-11, Algorithms 1-2) presumes this distribution.
  • standard math Directions of Gaussian vectors are spatially uniform on the sphere
    Used in Section 3.2.3 to justify the E8 direction codebook; the derived angular marginals in Eq. 9 are stated incorrectly, as the marginal of phi_i is proportional to sin^(k-1-i)(phi).
  • standard math For k independent standard Gaussian variables, the sum of squares follows a chi-square distribution with k degrees of freedom
    Used in Eqs. 10-11 to derive the magnitude PDF and to justify Lloyd-Max as the optimal scalar quantizer.
  • ad hoc to paper Greedy max-min selection from E8 lattice directions yields a good direction codebook
    Algorithm 1 is a heuristic with no optimality guarantee, and its candidate set ('all directions of E8 lattice') is undefined in the paper.
  • domain assumption Minimizing squared error in weight space correlates with downstream task accuracy
    Standard PTQ assumption, implicit throughout; DACC optimizes weight-space MSE, not task loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling." pith.science (2026). https://pith.science/paper/YABVYOF4

@misc{pith2026250605432,
  author       = {Pith},
  title        = {Pith review of: PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YABVYOF4}},
  note         = {Machine review of arXiv:2506.05432}
}
read the original abstract

Large Language Models (LLMs) face significant challenges in edge deployment due to their massive parameter scale. Vector Quantization (VQ), a clustering-based quantization method, serves as a prevalent solution to this issue for its extremely low-bit (even at 2-bit) and considerable accuracy. Since a vector is a quantity in mathematics and physics that has both direction and magnitude, existing VQ works typically quantize them in a coupled manner. However, we find that direction exhibits significantly greater sensitivity to quantization compared to the magnitude. For instance, when separately clustering the directions and magnitudes of weight vectors in LLaMA-2-7B, the accuracy drop of zero-shot tasks are 46.5\% and 2.3\%, respectively. This gap even increases with the reduction of clustering centers. Further, Euclidean distance, a common metric to access vector similarities in current VQ works, places greater emphasis on reducing the magnitude error. This property is contrary to the above finding, unavoidably leading to larger quantization errors. To these ends, this paper proposes Polar Coordinate Decoupled Vector Quantization (PCDVQ), an effective and efficient VQ framework consisting of two key modules: 1) Polar Coordinate Decoupling (PCD), which transforms vectors into their polar coordinate representations and perform independent quantization of the direction and magnitude parameters.2) Distribution Aligned Codebook Construction (DACC), which optimizes the direction and magnitude codebooks in accordance with the source distribution. Experimental results show that PCDVQ outperforms baseline methods at 2-bit level by at least 1.5\% zero-shot accuracy, establishing a novel paradigm for accurate and highly compressed LLMs.

Figures

Figures reproduced from arXiv: 2506.05432 by the authors.

Figure 1
Figure 1. Comparison between direction and magnitude. We utilize the classic K-Means [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of PCDVQ. It contains two novel techniques. (1) We introduce the Polar Coordinate [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Ablation study on PCD. Both QuIP# and PCDVQ are in 2-bit setting. The x-axis denotes [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 7 canonical work pages

  1. [5]

    Quip: 2-bit quantization of large language models with guarantees.arXiv preprint arXiv:2307.13304, 2023

    Jerry Chee, Yaohui Cai, V olodymyr Kuleshov, and Christopher De Sa. Quip: 2-bit quantization of large language models with guarantees.arXiv preprint arXiv:2307.13304, 2023

    arXiv 2023

  2. [1]

    Quarot: Outlier-free 4-bit inference in rotated llms.arXiv preprint arXiv:2404.00456, 2024

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference in rotated llms.arXiv preprint arXiv:2404.00456, 2024

    arXiv 2024

  3. [2]

    Springer, 2006

    Christopher M Bishop and Nasser M Nasrabadi.Pattern recognition and machine learning, volume 4. Springer, 2006

    2006

  4. [3]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. InAssociation for the Advancement of Artificial Intelligence, 2020

    work page 2020

  5. [4]

    Language models are few-shot learners.Advances in Neural Information Processing Systems, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 2020

    work page 2020

  6. [6]

    Db-llm: Accurate dual-binarization for efficient llms

    Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, et al. Db-llm: Accurate dual-binarization for efficient llms. arXiv preprint arXiv:2402.11960, 2024

    arXiv 2024

  7. [7]

    Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    arXiv 2018

  8. [8]

    High-dimensional data analysis: The curses and blessings of dimension- ality.AMS math challenges lecture, 1(2000):32, 2000

    David L Donoho et al. High-dimensional data analysis: The curses and blessings of dimension- ality.AMS math challenges lecture, 1(2000):32, 2000

    work page 2000

Show all 39 references
  1. [9]

    Extreme compression of large language models via additive quantization.arXiv preprint arXiv:2401.06118, 2024

    Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, and Dan Alistarh. Extreme compression of large language models via additive quantization.arXiv preprint arXiv:2401.06118, 2024

    2024 arXiv

  2. [10]

    Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  3. [11]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  4. [12]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 10

    2025 arXiv

  5. [13]

    Polarquant: Quantizing kv caches with polar transformation.arXiv preprint arXiv:2502.02617, 2025

    Insu Han, Praneeth Kacham, Amin Karbasi, Vahab Mirrokni, and Amir Zandieh. Polarquant: Quantizing kv caches with polar transformation.arXiv preprint arXiv:2502.02617, 2025

    2025 arXiv

  6. [14]

    Springer, 2009

    Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman.The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009

    2009

  7. [15]

    Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting.arXiv preprint arXiv:2501.13987, 2025

    Xing Hu, Yuan Cheng, Dawei Yang, Zukang Xu, Zhihang Yuan, Jiangyong Yu, Chen Xu, Zhe Jiang, and Sifan Zhou. Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting.arXiv preprint arXiv:2501.13987, 2025

    2025 arXiv

  8. [16]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    arXiv

  9. [17]

    Awq: Activation-aware weight quantization for llm compression and acceleration.arXiv preprint arXiv:2306.00978, 2023

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration.arXiv preprint arXiv:2306.00978, 2023

    2023 arXiv

  10. [18]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  11. [19]

    Vptq: Extreme low-bit vector post-training quantization for large language models

    Yifei Liu, Jicheng Wen, Yang Wang, Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, and Mao Yang. Vptq: Extreme low-bit vector post-training quantization for large language models. arXiv preprint arXiv:2409.17066, 2024

    2024 arXiv

  12. [20]

    Spinquant: Llm quantization with learned rotations.arXiv preprint arXiv:2405.16406, 2024

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. Spinquant: Llm quantization with learned rotations.arXiv preprint arXiv:2405.16406, 2024

    2024 arXiv

  13. [21]

    Least squares quantization in pcm.IEEE transactions on information theory, 28(2):129–137, 1982

    Stuart Lloyd. Least squares quantization in pcm.IEEE transactions on information theory, 28(2):129–137, 1982

    1982

  14. [22]

    Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    2016 arXiv

  15. [23]

    The expectation-maximization algorithm.IEEE Signal processing magazine, 13(6):47–60, 1996

    Todd K Moon. The expectation-maximization algorithm.IEEE Signal processing magazine, 13(6):47–60, 1996

    1996

  16. [24]

    Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 2022

    2022

  17. [25]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 2020

    2020

  18. [26]

    Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 2021

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 2021

    2021

  19. [27]

    Omniquant: Omnidirectionally calibrated quantiza- tion for large language models.CoRR, abs/2308.13137, 2023

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantiza- tion for large language models.CoRR, abs/2308.13137, 2023

    2023 arXiv

  20. [28]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  21. [29]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 11

    2023 arXiv

  22. [30]

    Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks.arXiv preprint arXiv:2402.04396, 2024

    Albert Tseng, Jerry Chee, Qingyao Sun, V olodymyr Kuleshov, and Christopher De Sa. Quip#: Even better llm quantization with hadamard incoherence and lattice codebooks.arXiv preprint arXiv:2402.04396, 2024

    2024 arXiv

  23. [31]

    Qtip: Quantization with trellises and incoherence processing.Advances in Neural Information Processing Systems, 37:59597–59620, 2024

    Albert Tseng, Qingyao Sun, David Hou, and Christopher M De Sa. Qtip: Quantization with trellises and incoherence processing.Advances in Neural Information Processing Systems, 37:59597–59620, 2024

    2024

  24. [32]

    Gptvq: The blessing of dimensionality for llm quantization.arXiv preprint arXiv:2402.15319, 2024

    Mart Van Baalen, Andrey Kuzmin, Markus Nagel, Peter Couperus, Cedric Bastoul, Eric Mahurin, Tijmen Blankevoort, and Paul Whatmough. Gptvq: The blessing of dimensionality for llm quantization.arXiv preprint arXiv:2402.15319, 2024

    2024 arXiv

  25. [33]

    The sphere packing problem in dimension 8.Annals of mathematics, pages 991–1015, 2017

    Maryna S Viazovska. The sphere packing problem in dimension 8.Annals of mathematics, pages 991–1015, 2017

    2017

  26. [34]

    Polarquant: Leveraging polar transformation for efficient key cache quantization and decoding acceleration.arXiv preprint arXiv:2502.00527, 2025

    Songhao Wu, Ang Lv, Xiao Feng, Yufei Zhang, Xun Zhang, Guojun Yin, Wei Lin, and Rui Yan. Polarquant: Leveraging polar transformation for efficient key cache quantization and decoding acceleration.arXiv preprint arXiv:2502.00527, 2025

    2025 arXiv

  27. [35]

    Smoothquant: Accurate and efficient post-training quantization for large language models.arXiv preprint arXiv:2211.10438, 2022

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models.arXiv preprint arXiv:2211.10438, 2022

    2022 arXiv

  28. [36]

    Zeroquant: Efficient and affordable post-training quantization for large-scale transformers

    Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. arXiv preprint arXiv:2206.01861, 2022

    2022 arXiv

  29. [37]

    LLMViewer

    Zhihang Yuan. LLMViewer. https://github.com/hahnyuan/LLMViewer, 2024. Accessed: February 14, 2024

    2024

  30. [38]

    Wkvquant: Quantizing weight and key/value cache for large language models gains more.arXiv preprint arXiv:2402.12065, 2024

    Yuxuan Yue, Zhihang Yuan, Haojie Duanmu, Sifan Zhou, Jianlong Wu, and Liqiang Nie. Wkvquant: Quantizing weight and key/value cache for large language models gains more.arXiv preprint arXiv:2402.12065, 2024

    2024 arXiv

  31. [39]

    Hellaswag: Can a machine really finish your sentence?arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence?arXiv preprint arXiv:1905.07830, 2019. A Appendix A.1 Derivation of Root Chi-square Distribution Suppose there is a magnitude variable X such that Y=X 2...

    1905 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.