Pith. sign in

REVIEW 4 major objections 7 minor 42 references

ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Models

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PCA rotation reveals which weight channels deserve full precision, improving LLM quantization.

desk verdict ROSAQ is a plausible incremental idea—PCA-based saliency for mixed-precision quantization—but its central claim needs an ablation that isolates the eigenvalue criterion. read the letter →

arxiv 2506.13472 v2 pith:7TAGMB5V submitted 2025-06-16 cs.CL cs.AI

Junho Yoon , Geom Lee , Donghyeon Jeon , Inho Kang , Seung-Hoon Na This is my paper · ORCID
classification cs.CLcs.AI
keywords largelanguagemodelspost-trainingquantizationweightsaliency-awarePCAprojectionrotationalinvariancemixed-precisionLLMinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when quantizing a large language model, the channels to protect with higher precision are best found after rotating the weight matrix by the principal components of its input activations, not before. It proposes ROSAQ, which uses the rotational invariance of linear layers to rewrite each layer as $(XR)(R^T W)$, obtains $R$ from PCA on a calibration set, and keeps the $K$ channels with the largest eigenvalues in FP16 while quantizing the rest to INT3 or INT4. Because the top-principal dimensions concentrate activation variance, the paper claims they are naturally the salient features whose quantization error matters most. On LLaMA2-7B, LLaMA3-8B, and Qwen2-7B, ROSAQ reports perplexity and zero-shot accuracy improvements over the activation-magnitude baseline and over GPTQ, SpinQuant, and AWQ, with a fused kernel giving about 2.3x speedup over FP16. The point would matter if true: a cheap, data-only rotation could decide where to spend precision in compressed LLMs without retraining.

What carries the argument

The carrying identity is the rotational invariance of linear layers, $XW = (XR)(R^T W)$, together with the PCA eigendecomposition $X^T X = R\Lambda R^T$ used to choose $R$. The eigenvalues of the calibration activation covariance rank the rotated channels; the $K$ channels with the largest eigenvalues are designated salient, kept in FP16, while the rest are quantized to INT3/INT4 in groups of 128. This machinery transforms saliency detection from a per-channel magnitude heuristic into a variance-maximizing projection, and the head-wise variant applies the same idea separately to each attention head's representation.

What would settle it

Take a layer from LLaMA2-7B, compute the PCA rotation, and find a channel with a low eigenvalue whose weights have unusually large absolute values; quantize that channel to INT3 while keeping a top-eigenvalue channel in FP16, and compare perplexity to the ROSAQ assignment. If the low-variance, high-weight channel degrades output more than the top-eigenvalue channel, the top-K saliency rule is false.

Watch

Extended reading notes

Core claim

The central discovery is that saliency for weight quantization should be defined in the PCA-projected feature space rather than the original activation space. Concretely, the paper constructs a rotation matrix $R$ from the eigendecomposition of the calibration covariance $X^T X = R\Lambda R^T$, rotates the weights to $R^T W$, and declares the channels corresponding to the $K$ largest eigenvalues salient. Those channels are kept in FP16 and the remaining channels are quantized in INT3/INT4 groups of 128. The paper argues that because PCA maximizes variance, the projected principal channels have larger and more separated activation magnitudes than magnitude-based salient channels in the original space, and its experiments support this by showing lower perplexity and higher MMLU than the rotationless Mixed baseline and existing methods. For multi-head attention, head-wise PCA, with a separate rotation per attention head, is claimed to further improve over global PCA.

Load-bearing premise

The load-bearing premise is that a channel's importance is fully determined by the variance it explains in the calibration input activations, so a low-variance channel containing large or otherwise sensitive weights would be unprotected.

Editorial extensions

If this is right

  • If the PCA-rotation claim holds, a calibration set alone, with no re-training and no gradient information, is enough to locate the precision-critical channels in a transformer.
  • Protecting only the top-K eigenchannels in FP16 while quantizing the rest in INT3/INT4 gives lower WikiText-2 perplexity and higher zero-shot accuracy than quantizing all channels at low precision or using original-space activation magnitudes.
  • The head-wise PCA variant for multi-head attention improves over applying one global PCA to the concatenated heads in the paper's LLaMA2-7B comparison.
  • The mixed-precision layout is compatible with a fused decode kernel, yielding about 2.3x speedup over FP16 at batch size 64 and about 2x speedup at batch size 128.
  • At the aggressive INT3 setting, the method reports the highest MMLU among the compared methods on several model and task combinations, suggesting the benefit grows as precision drops.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the selection rule is testable against weight-aware criteria: one could compare the top-eigenvalue channels with the channels whose INT3 quantization causes the largest output perturbation; if overlap is low, the variance-only rule is incomplete.
  • Because the PCA rotation is computed from a calibration set, the method's stability under different calibration data and sequence lengths is an open extension; a practical deployment would want the chosen salient channels to be invariant across calibration samples.
  • The same variance-maximizing selection could be applied to activation quantization or to the key/value cache, where outlier channels also concentrate; the paper itself notes weight-activation quantization and retrieval-augmented generation as future work.
  • If rotation makes saliency more concentrated, then ROSAQ and orthogonal-projection outlier-removal rotations are likely complementary rather than competing; combining them could let even fewer channels carry FP16.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes ROSAQ, a post-training quantization method for LLMs that performs PCA on calibration activations to rotate weight matrices, selects K channels with the largest eigenvalues as salient, stores them in FP16, and quantizes the remaining channels to INT3/INT4 with per-group quantization. For multi-head attention, a head-wise PCA is used. The method is evaluated on LLaMA2-7B, LLaMA3-8B, and Qwen2-7B in terms of WikiText-2 perplexity, zero-shot common-sense reasoning, and MMLU, and is compared with GPTQ, SpinQuant, AWQ, and a rotation-less Mixed baseline. Throughput is measured with the QUICK kernel and reported as speedup over FP16.

Significance. If the central claim holds, ROSAQ offers a simple and computationally light way to identify channels worth protecting during quantization, using only calibration activations. The paper includes useful ablations (top vs bottom vs random salient channels; head-wise vs global PCA) and is transparent about its limitations, including the open question of how PCA saliency relates to outlier-removing rotations. The main contribution is plausible but not fully established because the saliency criterion is not isolated from other design choices, and several reported differences are within the noise of a single run.

major comments (4)
  1. [Section 3.1, Eq. (4)] The saliency criterion is defined solely by the eigenvalues of X^T X, i.e., the input activation covariance along rotated directions. This ignores the magnitudes of the rotated weights R^T W and the downstream loss; a direction with small activation variance but large weight entries could incur significant quantization error. The ablation in Appendix B (Top vs Bottom vs Random) tests only the choice of top versus bottom eigenvalues within the same criterion, not whether this eigenvalue-based criterion is better than a weight-aware or loss-aware criterion. To support the abstract's statement that 'projected principal dimensions are naturally considered as salient features,' please compare in the same rotated space the eigenvalue criterion against alternatives such as row-norm of R^T W or AWQ-style activation-magnitude selection, with all other components fixed.
  2. [Section 4.1 and Appendix G] The 'Mixed' baseline is defined as R=I with saliency based on activation magnitudes, but the WD layer in ROSAQ is not rotated and is instead protected by AWQ per-channel scaling. It is not clear whether the Mixed baseline also applies this scaling to WD. If it does not, the comparison conflates the effect of PCA rotation with the effect of the AWQ scaling patch. Please specify the exact configuration of Mixed and include an ablation of ROSAQ without the AWQ scaling on WD so the contribution of the rotation itself is isolated.
  3. [Tables 5-7] The reported improvements are not uniform. For example, Table 5 (LLaMA2-7B, INT4g128) shows SpinQuant achieving lower PPL (5.52 vs 5.57) and the Mixed baseline achieving higher CSR (59.62 vs 59.29) than ROSAQ. The main text's claim that ROSAQ is 'slightly superior' is therefore contradicted by some rows. The authors should qualify the claim, for example by noting that ROSAQ is usually best or tied but with exceptions, and discuss possible reasons for these cases.
  4. [Section 4.2 and Appendix G] No error bars or multiple seeds are reported, and the differences between methods are often small (e.g., PPL differences below 0.1). The number of salient channels K is set per layer type (128 for WQ/WK/WV, 32 per head for WO, 128 for WU/WG in Appendix G) with no sensitivity analysis. Since the performance of the method may depend on K, please report at least a small sweep over K or a justification for the chosen values, and include repeated runs or bootstrap confidence intervals for the main tables.
minor comments (7)
  1. [Eq. (3)] In Eq. (3), the definitions of W_S and W_N are ambiguous: they are described as sub-blocks of the weight matrix, but after rotation the relevant sub-blocks are rows of R^T W. Please clarify the notation, for example by defining W_S=(R^T W)_S and W_N=(R^T W)_N.
  2. [Section 3.1] Section 3.1 refers to 'Fig 1' for the activation magnitude plots, but the plots appear in Figure 2; please correct the reference.
  3. [Appendix B] Appendix B contains the typo 'eigenvaluesl' in place of 'eigenvalues'; please fix it.
  4. [Table 4] Table 4 is difficult to read because the decode-speed and speedup columns are not clearly separated and the device name is repeated. Also, because ROSAQ, AWQ, and GPTQ use different kernels (QUICK, AutoAWQ, and Marlin), the speed comparisons across methods reflect kernel differences rather than algorithmic differences; please state this caveat in the text.
  5. [Table 5] In Table 5, the FP16 row condenses PPL and the first accuracy score into '- 5.4779.11', making the table hard to parse; please separate these values.
  6. [Appendix F, Table 8] The eigenvalues listed in Table 8 do not appear sorted even though the channels were selected by average magnitude; please clarify that the eigenvalues correspond to the magnitude-selected channels, or sort the rows accordingly.
  7. [Section 5 and Limitations] The Limitations section admits that 'further analysis is needed' to connect saliency-aware and outlier-free rotations; this caveat should also appear in the introduction or conclusion so that the paper's claims are not overstated.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PCA-based saliency is a definition tested against independent benchmarks, not a prediction derived from its own inputs.

full rationale

The paper derives the rotation matrix R from PCA on calibration activations (Eq. 4) and defines salient channels as the K largest eigenvalue directions (Section 3.2). This is a definition or hypothesis rather than a derived result: the claim that principal dimensions are 'salient' is operationalized by construction, and its value is then judged empirically on held-out Wikitext2, common-sense reasoning, and MMLU, compared with GPTQ, SpinQuant, AWQ, and a rotation-less 'Mixed' baseline. No equation in the paper reduces a predicted quantity to a fitted input: the FP16/INT3-INT4 assignment is not fitted from the evaluation metrics; K is a manually set hyperparameter (Appendix G), which is tuning rather than circularity. The selection criterion itself uses only activation covariance X^T X and ignores weight magnitudes, but that is a correctness or robustness limitation, explicitly acknowledged in the Limitations section, not a circular reduction. There are no load-bearing self-citations or imported uniqueness theorems. The only definitional element is the name 'salient' attached to top eigenvectors; because the paper's actual claims are the resulting perplexity and accuracy improvements, this does not make the derivation circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three assumptions: exact rotational invariance of linear layers, representativeness of the calibration set for PCA, and the identification of high-variance principal channels with weight channels that must be kept precise. The number of salient channels K is hand-set per layer, group size 128 is a design choice, and the FFN down-projection protection is borrowed from AWQ with an unspecified scaling exponent. No invented entities are introduced.

free parameters (3)
  • K (number of salient channels per layer) = 128 for WQ/WK/WV/WU/WG; 32 per head for WO
    Set by hand in Appendix G without a sensitivity sweep; this is the central knob controlling how much of the weight stays in FP16.
  • Group size for per-group quantization = 128
    Chosen as a standard INT4 weight-quantization setting and used to define which channels fall in the FP16 salient group versus low-precision groups.
  • AWQ-style per-channel scaling exponent for WD = unspecified (default from AWQ)
    Applied to the FFN down-projection because the rotation cannot be absorbed cleanly; the exact scale computation is not given in the paper, so results depend on an external default.
assumptions (4)
  • standard math Rotational invariance: for any orthonormal R, XW = (XR)(R^T W)
    Used in Eq. (1) to justify applying PCA rotation to weights; exact for square rotation matrices.
  • domain assumption A small calibration set from the Pile represents the activation distributions of all evaluated tasks and models
    PCA eigenvectors and eigenvalues in Eq. (4) are computed on this set, so if it is not representative, the salient channels are wrong.
  • domain assumption Channels with the largest eigenvalues are the salient channels whose FP16 preservation most reduces quantization error
    Stated as the paper's 'key underlying expectation' in Section 1; no theorem or loss-based analysis is provided.
  • ad hoc to paper The FFN down-projection WD cannot be rotated and is instead protected by AWQ per-channel scaling
    Introduced in Appendix G as a patch; it mixes two invariance principles and is outside the core PCA derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Models." pith.science (2026). https://pith.science/paper/7TAGMB5V

@misc{pith2026250613472,
  author       = {Pith},
  title        = {Pith review of: ROSAQ: Rotation-based Saliency-Aware Weight Quantization for Efficiently Compressing Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7TAGMB5V}},
  note         = {Machine review of arXiv:2506.13472}
}
read the original abstract

Quantization has been widely studied as an effective technique for reducing the memory requirement of large language models (LLMs), potentially improving the latency time as well. Utilizing the characteristic of rotational invariance of transformer, we propose the rotation-based saliency-aware weight quantization (ROSAQ), which identifies salient channels in the projection feature space, not in the original feature space, where the projected "principal" dimensions are naturally considered as "salient" features. The proposed ROSAQ consists of 1) PCA-based projection, which first performs principal component analysis (PCA) on a calibration set and transforms via the PCA projection, 2) Salient channel dentification, which selects dimensions corresponding to the K-largest eigenvalues as salient channels, and 3) Saliency-aware quantization with mixed-precision, which uses FP16 for salient dimensions and INT3/4 for other dimensions. Experiment results show that ROSAQ shows improvements over the baseline saliency-aware quantization on the original feature space and other existing quantization methods. With kernel fusion, ROSAQ presents about 2.3x speed up over FP16 implementation in generating 256 tokens with a batch size of 64.

Figures

Figures reproduced from arXiv: 2506.13472 by the authors.

Figure 1
Figure 1. An overview diagram of ROSAQ that quan￾tizes the weights of a linear layer XW, using rotational invariance as described by Eq. (1), where X is the cali￾bration data matrix. ROSAQ first applies the PCA-based projection, taking Q as R, with eigenvectors obtained from Eq. (4). The salient channels denoted as WS, cor￾responding to the K largest eigenvalues, are represented in FP16, while the remaining non-salient channe… view at source ↗
Figure 2
Figure 2. Magnitude of the input activation values to [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 11 canonical work pages

  1. [1]

    AI@Meta. 2024. Llama 3 model card. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md. Accessed: 2024-09-15

    work page 2024

  2. [2]

    Croci, Marcelo Gennari Do Nascimento, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari Do Nascimento, Torsten Hoefler, and James Hensman. 2024 a . https://openreview.net/forum?id=vXxardq6db Slicegpt: Compress large language models by deleting rows and columns . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

  3. [3]

    Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024 b . https://arxiv.org/abs/2404.00456 Quarot: Outlier-free 4-bit inference in rotated llms . Preprint, arXiv:2404.00456

    arXiv 2024

  4. [4]

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. Piqa: Reasoning about physical commonsense in natural language. In Thirty-Fourth AAAI Conference on Artificial Intelligence

    2020

  5. [5]

    Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher De Sa. 2024. https://arxiv.org/abs/2307.13304 Quip: 2-bit quantization of large language models with guarantees . Preprint, arXiv:2307.13304

    arXiv 2024

  6. [6]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/1803.05457

    arXiv 2018

  7. [7]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. https://arxiv.org/abs/2208.07339 Llm.int8(): 8-bit matrix multiplication for transformers at scale . Preprint, arXiv:2208.07339

    arXiv 2022

  8. [8]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/1feb87871436031bdc0f2beaa62a049b-Abstract-Conference.html Qlora: Efficient finetuning of quantized llms . In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, Neur...

    2023

Show all 42 references
  1. [9]

    Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. 2024. https://openreview.net/forum?id=Q1u25ahSuy Spqr: A sparse-quantized representation for near-lossless LLM weight com...

    2024

  2. [10]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. https://arxiv.org/abs/2210.17323 Gptq: Accurate post-training quantization for generative pre-trained transformers . Preprint, arXiv:2210.17323

    2023 arXiv

  3. [11]

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. https://arxiv.org/abs/2101.00027 The pile: An 800gb dataset of diverse text for language modeling . P...

    2020 arXiv

  4. [12]

    Ziyi Guan, Hantao Huang, Yupeng Su, Hong Huang, Ngai Wong, and Hao Yu. 2024. https://doi.org/10.48550/ARXIV.2402.14866 APTQ: attention-aware post-training mixed-precision quantization for large language models . CoRR, abs/2402.14866

    2024 doi arXiv

  5. [13]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://arxiv.org/abs/2009.03300 Measuring massive multitask language understanding . Preprint, arXiv:2009.03300

    2021 arXiv

  6. [14]

    Yuxuan Hu, Jing Zhang, Zhe Zhao, Chen Zhao, Xiaodong Chen, Cuiping Li, and Hong Chen. 2024. https://aclanthology.org/2024.findings-acl.187 sp^3 : Enhancing structured pruning via PCA projection . In Findings of the Association for Computational Linguistics ACL 2024, pages 3150...

    2024

  7. [15]

    Mahoney, and Kurt Keutzer

    Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, and Kurt Keutzer. 2024 a . https://openreview.net/forum?id=0jpbpFia8m Squeezellm: Dense-and-sparse quantization . In Forty-first International Conference on Machine Learning, ICML 20...

    2024

  8. [16]

    Taesu Kim, Jongho Lee, Daehyun Ahn, Sarang Kim, Jiwoong Choi, Minkyu Kim, and Hyungjun Kim. 2024 b . https://arxiv.org/abs/2402.10076 Quick: Quantization-aware interleaving and conflict-free kernel for efficient llm inference . Preprint, arXiv:2402.10076

    2024 arXiv

  9. [17]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e...

    2020

  10. [18]

    Jun Li, Li Fuxin, and Sinisa Todorovic. 2020. https://arxiv.org/abs/2002.01113 Efficient riemannian optimization on the stiefel manifold via the cayley transform . Preprint, arXiv:2002.01113

    2020 arXiv

  11. [19]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei - Ming Chen, Wei - Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. https://proceedings.mlsys.org/paper\_files/paper/2024/hash/42a452cbafa9dd64e9ba4aa95cc1ef21-Abstract-Conference.html AWQ: activation-a...

    2024

  12. [20]

    Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, and Bohan Zhuang. 2024 a . https://openreview.net/forum?id=FIplmUWdm3 QLLM: accurate and efficient low-bitwidth quantization for large language models . In The Twelfth International Conference on Learning Representa...

    2024

  13. [21]

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023. https://arxiv.org/abs/2305.17888 Llm-qat: Data-free quantization aware training for large language models . Preprint, arXiv:2305.17888

    2023 arXiv

  14. [22]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024 b . https://arxiv.org/abs/2405.16406 Spinquant: Llm quantization with learned rotations . Preprint, arXiv:2405.16406

    2024 arXiv

  15. [23]

    Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. 2024. https://arxiv.org/abs/2402.17764 The era of 1-bit llms: All large language models are in 1.58 bits . Preprint, arXiv:2402.17764

    2024 arXiv

  16. [24]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. https://arxiv.org/abs/1609.07843 Pointer sentinel mixture models . Preprint, arXiv:1609.07843

    2016 arXiv

  17. [25]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP

    2018

  18. [26]

    Jiayi Pan, Chengcan Wang, Kaifu Zheng, Yangguang Li, Zhenyu Wang, and Bin Feng. 2023. https://arxiv.org/abs/2312.03788 Smoothquant+: Accurate and efficient 4-bit post-training weightquantization for llm . Preprint, arXiv:2312.03788

    2023 arXiv

  19. [27]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Winogrande: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641

    2019 arXiv

  20. [28]

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2024. https://openreview.net/forum?id=8Wuvhh0LYW Omniquant: Omnidirectionally calibrated quantization for large language models . In The Twelfth Internat...

    2024

  21. [29]

    Xuan Shen, Zhenglun Kong, Changdi Yang, Zhaoyang Han, Lei Lu, Peiyan Dong, Cheng Lyu, Chih hsiang Li, Xuehang Guo, Zhihao Shu, Wei Niu, Miriam Leeser, Pu Zhao, and Yanzhi Wang. 2024. https://arxiv.org/abs/2402.10787 Edgeqat: Entropy and distribution guided quantization-aware t...

    2024 arXiv

  22. [30]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://doi.org/10.18653/v1/N19-1421 C ommonsense QA : A question answering challenge targeting commonsense knowledge . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...

    2019 doi

  23. [31]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  24. [32]

    Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. 2024. https://openreview.net/forum?id=9BrydUVcoe Quip \# : Even better LLM quantization with hadamard incoherence and lattice codebooks . In Forty-first International Conference on Machine Learni...

    2024

  25. [33]

    Xiaoxia Wu, Haojun Xia, Stephen Youn, Zhen Zheng, Shiyang Chen, Arash Bakhtiari, Michael Wyatt, Reza Yazdani Aminabadi, Yuxiong He, Olatunji Ruwase, Leon Song, and Zhewei Yao. 2023 a . https://arxiv.org/abs/2312.08583 Zeroquant(4+2): Redefining llms quantization with a new fp6...

    2023 arXiv

  26. [34]

    Xiaoxia Wu, Zhewei Yao, and Yuxiong He. 2023 b . https://arxiv.org/abs/2307.09782 Zeroquant-fp: A leap forward in llms post-training w4a8 quantization using floating-point formats . Preprint, arXiv:2307.09782

    2023 arXiv

  27. [35]

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: accurate and efficient post-training quantization for large language models. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org

    2023

  28. [36]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  29. [37]

    Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. https://arxiv.org/abs/2206.01861 Zeroquant: Efficient and affordable post-training quantization for large-scale transformers . Preprint, arXiv:2206.01861

    2022 arXiv

  30. [38]

    Zhewei Yao, Xiaoxia Wu, Cheng Li, Stephen Youn, and Yuxiong He. 2023. https://arxiv.org/abs/2303.08302 Zeroquant-v2: Exploring post-training quantization in llms from comprehensive study to low rank compensation . Preprint, arXiv:2303.08302

    2023 arXiv

  31. [39]

    Zhewei Yao, Xiaoxia Wu, Cheng Li, Stephen Youn, and Yuxiong He. 2024. https://doi.org/10.1609/AAAI.V38I17.29908 Exploring post-training quantization in llms from comprehensive study to low rank compensation . In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 20...

    2024 doi

  32. [40]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

    2019

  33. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.