Pith. sign in

REVIEW 5 minor 53 references

When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation

T0 review · 0 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RoPE-commuting rotations are exactly the per-frequency-pair rotations, yet the closed-form optimum of that family fails to improve 4-bit quantisation because its surrogate is misaligned with the quantiser's tokenwise range statistic.

desk verdict A scoped negative result that holds up: the head-shared RoPE-aligned pairwise rotation does not beat full-head Hadamard under dynamic W4A4KV4, and the paper is honest about that scope. read the letter →

arxiv 2608.13365 v1 pith:DR3PV4ZT submitted 2026-08-13 cs.LG

Shuhan Wang , Yilin Luo , Nan Xu , Chi Wang Cheung This is my paper · ORCID
classification cs.LG
keywords post-trainingquantisationRoPE-alignedrotationsW4A4KV4HadamardrotationKVcachemixingsupportcovariancesurrogateperplexityevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rotation-based post-training quantisation of LLMs usually applies an orthogonal transform across an entire attention head, while rotary position embeddings (RoPE) partition each head into two-dimensional frequency pairs; this paper asks whether a rotation that respects that pair structure can beat the usual full-head Hadamard mix. It proves the converse characterisation that, for pairwise-distinct frequencies, the only single-head orthogonal maps commuting with RoPE are independent rotations inside each frequency pair, derives the exact angle that minimises the larger channel variance of a position-averaged pooled-covariance surrogate, and verifies that its implementation attains that analytic minimum. The empirical answer is negative: across four checkpoints under dynamic W4A4KV4 quantisation, the head-shared pairwise rotation — even at its verified optimum — raises perplexity relative to the full-head Hadamard at every context length tested. Composing the pairwise rotation with the Hadamard stays within a selected ±0.05 perplexity equivalence margin, and estimating the angle from the K stream alone narrows but does not close the gap. The paper's explanation is that the rotation optimises a second-moment surrogate while the dynamic quantiser sets its step from a tokenwise group range, and the two-channel mixing support of the pairwise rotation cannot redistribute a peak the way a full head can.

What carries the argument

The load-bearing objects are the single-head RoPE centraliser and the pooled-covariance surrogate it is paired with. Lemma 2 identifies $\mathcal{B}_{d_h}=\bigoplus_{k=1}^{K}G(\phi_k)\cong SO(2)^{K}$, with $K=d_h/2$, as the complete family of orthogonal maps commuting with RoPE when the frequencies are distinct, turning 'RoPE-aligned' into a sharply bounded design space whose freedom is exactly one angle per pair. On that space, the surrogate $\tilde\Sigma^{(k)}=\tfrac{1}{L}\sum_{m=0}^{L-1}R^{(k)}_m\Sigma^{(k)}(R^{(k)}_m)^\top$ position-averages the per-pair calibration covariance, and Theorem 3 solves its minimax channel-variance problem in closed form, giving the angle that equalises the two rotated diagonal entries; Proposition 4 then pins the Q/K application rule, since shared band angles are necessary and sufficient to preserve relative-position attention scores. The explanatory concept of the paper is 'mixing support' — the number $b$ of channels over which a peak can be redistributed, $b=2$ for the pairwise family versus $b=d_h$ for the Hadamard — which the block-Hadamard interpolation uses to show that range, quantisation error, and perplexity all fall as support grows.

What would settle it

Re-run the exact W4A4KV4 protocol with head-dependent per-pair angles $\phi_{h,k}$, which the paper's Lemma 2 allows: if any such configuration matches or beats the full-head Hadamard baseline on WikiText-2 perplexity, the negative result is an artefact of the head-shared constraint rather than of RoPE-aligned pairwise rotation itself. Alternatively, optimise the rotation against the quantiser's actual tokenwise group range statistic instead of the position-averaged second moment; if perplexity then reaches the Hadamard level, the surrogate mismatch — not the two-channel support — is the operative cause.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a scoped negative result: exact optimality of a RoPE-aligned local objective does not translate into quantisation accuracy. For a single head with pairwise-distinct RoPE frequencies, the commuting orthogonal maps in $O(d_h)$ are exactly the band-aligned rotations $\mathcal{B}_{d_h} \cong SO(2)^{d_h/2}$ acting independently inside each frequency pair (Lemma 2), the converse of the per-pair construction the paper attributes to earlier work. For the implemented head-shared subfamily, Theorem 3 gives the closed-form equalising angle $\phi_k^* = \mathrm{wrap}_{[-\pi/4,\pi/4)}\big[\tfrac12\operatorname{atan2}(\tilde\Sigma^{(k)}_{11}-\tilde\Sigma^{(k)}_{22},\,2\tilde\Sigma^{(k)}_{12})\big]$ that minimises $\max_{j\in\{1,2\}}\big[G(\phi_k)\tilde\Sigma^{(k)}G(\phi_k)^\top\big]_{jj}$ down to $(\sigma_1^2+\sigma_2^2)/2$, and the implementation attains this minimum to within $6.9\times10^{-7}$ against a tolerance of $5\times10^{-5}$. Yet pairwise-only replacement of the full-head Hadamard increases perplexity on all four checkpoints at every short- and long-context point, and the verified surrogate optimum $\phi_k^*$ has the largest pairwise-only gap on Llama-3.2-3B. The account offered is a double misalignment: the surrogate averages a per-pair second moment over positions, while dynamic INT4 sets $\Delta_{t,g}=(u_{t,g}-\ell_{t,g})/(2^B-1)$ from a tokenwise group range, and the pairwise transform's mixing support is only two channels, where the outlier-suppression bound is $1/\sqrt{2}$ instead of roughly $1/\sqrt{128}$ for a full head; a block-Hadamard interpolation from two channels to the full head monotonically reduces K range, relative quantisation error, and perplexity degradation.

Load-bearing premise

The load-bearing premise is the head-shared parameterisation: one rotation angle per layer and frequency pair, broadcast to every attention head, which is the only subfamily the experiments test, while the paper's own centraliser characterisation also permits head-dependent angles that remain unmeasured.

Editorial extensions

If this is right

  • A rotation for quantisation should be judged by whether its optimisation objective matches the quantiser's scale-setting statistic: commuting with RoPE is a structural constraint that, on its own, gives no accuracy benefit in the tested dynamic W4A4KV4 setting.
  • K-only estimation of the shared angle outperforms Q/K-pooled estimation on all four checkpoints, so the angle should be estimated from the stream that is actually quantised, though even this does not reach the full-head Hadamard baseline.
  • Mixing support bounds peak suppression: the post-rotation peak magnitude is at least $|x|/\sqrt{b}$, giving $|x|/\sqrt{2}$ at $b=2$ versus $|x|/\sqrt{128}$ at a full head, and the evaluated support interpolation from $b=2$ to $b=d_h$ monotonically lowers K range, relative quantisation error, and perplexity degradation.
  • Exact attainability of a surrogate optimum certifies the surrogate, not the deployment: the verified optimum $\phi_k^*$ produced the largest pairwise-only perplexity gap on Llama-3.2-3B.
  • Pairwise rotations remain usable as a layer on top of full-head mixing, since pairwise+Hadamard satisfies the selected $\pm0.05$-PPL equivalence criterion on every checkpoint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the paper's own Lemma 2 permits head-dependent angles $\phi_{h,k}$, and only the head-shared subfamily was tested, the negative conclusion may not extend to the full RoPE-commuting family; an evaluation with per-head pairwise angles would settle that boundary.
  • Editorial inference: the paper's mismatch analysis suggests a direct alternative design — choose the rotation to minimise the quantiser's actual tokenwise group range at full broadcast support rather than a position-averaged second moment — which is a testable construction the paper does not run.
  • Editorial inference: the exploratory static per-channel control reversed the perplexity ordering, so the negative result is tied to dynamic per-token quantisation and may not transfer to static scale rules or other bit widths.
  • Editorial inference: since post-RoPE Q/K maps need not commute with RoPE to preserve correctness (the same transform on both streams preserves inner products), the monotone support interpolation points toward head-wide or learned rotations as the more promising RoPE-aware direction, with commutativity available as a constraint only when it does not shrink mixing support.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. This paper studies the design of the online Q/K rotation R3 in dynamic W4A4KV4 post-training quantisation, contrasting rotations that respect the RoPE frequency-pair structure with full-head Hadamard mixing. The theoretical part characterises the single-head centraliser of RoPE as the product of independent per-pair rotations (Lemma 2, including the converse), derives a closed-form angle minimising the larger per-pair channel variance of a pooled-covariance, position-averaged surrogate (Theorem 3), and reports numerical attainment of that minimum on Llama-3.2-3B. The empirical part compares the head-shared pairwise configuration with the full-head Hadamard across four checkpoints and multiple context lengths: pairwise-only has higher perplexity in every short-context paired comparison; composing it with the Hadamard meets a pre-specified ±0.05 PPL interval criterion; K-only angle estimation improves pairwise-only without closing the gap; and a block-Hadamard support interpolation associates larger mixing support with lower K range, lower relative quantisation error, and smaller perplexity degradation.

Significance. If the results hold, the paper provides a clean, explicitly scoped negative benchmark: exact optimality for a natural RoPE-aligned surrogate does not translate into improved accuracy under the evaluated dynamic quantiser, and the discrepancy is plausibly tied to the quantiser's tokenwise range statistic and two-channel mixing support. The strengths are the clean algebra in Lemma 2 and Theorem 3, the numerical attainment check (maximum excess 6.90e-7 against a 5e-5 tolerance), the paired seed design for the primary short-context comparisons with confidence intervals, and the unusually careful boundary-drawing: the empirical conclusion is repeatedly restricted to the head-shared subfamily, with head-dependent angles and cross-head mixing explicitly listed as untested. The reproducibility apparatus, including portable drivers, manifest generators, and verification scripts, is a real asset. The principal limitation is that the negative result is for the head-shared subfamily; the paper does not overclaim beyond that.

minor comments (5)
  1. [Section 7 / Table 20] The long-context cross-model evidence in Table 20 is reported as point-estimate ranges without paired confidence intervals or standard deviations. Since the abstract and Section 7 state that pairwise-only increases perplexity at both short and long context lengths across four checkpoints, please either supply the paired interval information for these points or explicitly restrict the long-context claim to mean point estimates for the non-primary checkpoints.
  2. [Section 3, Eq. (3)] The function wrap[−π/4,π/4) is used in Theorem 3 but is never defined. Please define the residue-modulo-π/2 convention at first use; the proof in Appendix C uses it, but the body should stand alone.
  3. [Appendix C, proof of Proposition 4] The band-k components of q and k are written q(k) and k(k), which collides with the band index k. Please rename the key vector, for example as κ^(k) or kbold^(k), for readability.
  4. [Section 4 / Appendix A.5] The verification that the implementation attains the analytic minimum reconstructs the surrogate from stored per-pair covariances; this is disclosed in the appendix, but a one-sentence statement in the main text saying that the verification uses reconstructed moments rather than raw activations would prevent over-reading of the attainment claim.
  5. [Section 6.3 / Summary] In the support interpolation, block sizes b>2 deliberately leave the RoPE-commuting family, so the monotone improvement in Table 3 conflates support size with loss of commutativity. The text's caveat is explicit, but the summary's 'as support increases' phrasing invites a causal reading; consider adding 'along the evaluated non-commuting block-Hadamard path' to the summary.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the negative result rests on independent paired perplexity measurements, and the analytic surrogate is explicitly scoped and not treated as a perplexity predictor.

full rationale

The paper's central claim is a scoped empirical negative result: the evaluated head-shared pairwise rotation does not reduce perplexity relative to full-head Hadamard mixing under dynamic W4A4KV4. The derived angle (Theorem 3) minimizes a pooled-covariance, position-averaged variance surrogate by construction, and the paper consistently refuses to treat that surrogate optimality as evidence of quantisation improvement: Section 3 states the optimum 'does not characterise the tokenwise range used by the quantiser or end-to-end perplexity,' and Section 6.2 explicitly contrasts the surrogate's second-moment objective with the quantiser's tokenwise group range. The perplexity comparisons in Tables 1, 2, and 11 are independent paired measurements where only R3 varies and all other rotations, Hessian, calibration sample, and seed are fixed. The implementation check that the analytic minimum is attained reuses the same surrogate, but this is a code-correctness verification, not a predictive claim, and the paper explicitly notes it does not establish optimality over head-dependent angles. No load-bearing self-citation chain is present: FPTQuant is cited for the known per-pair family, and the converse characterization is proven in Appendix C. The paper also explicitly scopes the negative result to the head-shared subfamily and lists head-dependent angles and cross-head mixing as untested in the Limitations. No equation is equated to its own input, and no fitted parameter is renamed as a prediction. The derivation chain is therefore self-contained and non-circular.

Assumptions & free parameters 1 free parameters · 5 assumptions · 1 invented entities

The central derivation introduces a definitional surrogate objective and a head-shared angle estimator; no hidden fitted physical constants or new particles are involved. The rotation angle is openly estimated from calibration data, the surrogate is explicitly disclosed as a tractable target rather than a model of the quantizer, and the negative empirical result is measured independently of the surrogate.

free parameters (1)
  • Head-shared per-pair rotation angle phi_hat_k (and position-averaged phi_star_k) = 64 angles per layer per checkpoint, estimated from 128 WikiText-2 calibration sequences
    The evaluated transform's angles are estimated from calibration data rather than derived from an external principle; the paper compares three estimation rules. This is the method's own parameter and is openly reported, but the negative result applies to this fitted configuration.
assumptions (5)
  • standard math SO(2) rotations commute and any real orthogonal map commuting with a rotation whose angle is not a multiple of pi preserves each eigenspace plane
    Used in Lemma 2 and Proposition 4; standard linear algebra background.
  • domain assumption RoPE frequencies within a head are pairwise distinct and lie in (0, pi)
    Required for the converse centralizer theorem in Lemma 2; the paper notes that repeated frequencies would introduce extra commuting factors.
  • domain assumption Dynamic asymmetric INT4 sets its step from a tokenwise group range, and only K is quantized after R3 in the evaluated pipeline
    Central to the mismatch argument in Section 6; the conclusion is scoped to this quantizer behavior.
  • ad hoc to paper The pooled-covariance, position-averaged matrix in Eq. (2) is the optimization target
    The paper defines this surrogate and explicitly states that positional stationarity is neither assumed nor established; all optimality statements refer to this definitional objective.
  • domain assumption The empirical evaluation protocol (DartQuant whip calibration on 128 WikiText-2 sequences of length 2048, GPTQ weights, four checkpoints) is representative of the claim
    The negative result is explicitly scoped to this protocol in Appendix A and the Limitations.
invented entities (1)
  • Pooled-covariance position-averaged surrogate Sigma_tilde(k) (Eq. 2)
    purpose: Defines the analytic objective whose minimizer is derived in Theorem 3 and verified numerically; the paper argues this objective is misaligned with the quantizer scale statistic.
    This surrogate is constructed by the paper as a tractable optimization target. It has no independent falsifiable handle outside the paper, and the paper itself demonstrates its misalignment with the deployed quantizer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation." pith.science (2026). https://pith.science/paper/DR3PV4ZT

@misc{pith2026260813365,
  author       = {Pith},
  title        = {Pith review of: When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DR3PV4ZT}},
  note         = {Machine review of arXiv:2608.13365}
}
abstract

Rotation-based post-training quantisation commonly applies an orthogonal transform across an entire attention head to reduce outlier-induced error. RoPE instead partitions each head into two-dimensional frequency pairs, raising the question of whether a transform respecting this decomposition can improve on full-head mixing. Prior work has established the per-pair rotations that commute with RoPE. We state the converse result that, for distinct frequencies, no other single-head orthogonal map commutes with RoPE. For the head-shared parameterisation used in our experiments, we then derive the rotation angle that minimises the larger channel variance under a pooled-covariance, position-averaged surrogate and verify that the implementation attains its analytic minimum. The evaluated head-shared pairwise configuration does not improve accuracy in the tested dynamic W4A4KV4 setting. Across four checkpoints, replacing the full-head Hadamard with this configuration increases perplexity at both short and long context lengths. Composing the pairwise rotation with the Hadamard satisfies the selected $\pm0.05$-PPL interval criterion under the default estimator. Estimating the shared angle from K alone improves pairwise-only on every checkpoint but does not close its gap to full-head mixing. The analytic objective controls a position-averaged second moment of a pooled calibration covariance, whereas the dynamic quantiser sets its step from a tokenwise group range. The pairwise transform also has only two-channel mixing support. Along a controlled interpolation from two-channel to full-head mixing, K range, relative quantisation error, and perplexity degradation decrease as support increases. These results show that optimality for a structured surrogate need not reduce quantisation error when the surrogate and mixing support are misaligned with the quantiser's scale-setting statistic.

Figures

Figures reproduced from arXiv: 2608.13365 by the authors.

Figure 1
Figure 1. Exploratory comparison across quantiser regimes. Dynamic curves use the deployed per-token rule [PITH_FULL_IMAGE:figures/full_fig_p029_1.png] view at source ↗
Figure 2
Figure 2. Descriptive scale-setting diagnostic. The dynamic [PITH_FULL_IMAGE:figures/full_fig_p030_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 8 canonical work pages

  1. [1]

    Intriguing Properties of Quantization at Scale

    Arash Ahmadian, Saurabh Dash, Hongyu Chen, Bharat Venkitesh, Zhen Stephen Gou, Phil Blunsom, Ahmet \"U st \"u n, and Sara Hooker. Intriguing properties of quantization at scale. In Advances in Neural Information Processing Systems 36 ( NeurIPS 2023) , 2023. arXiv:2305.19268

    work page Pith review arXiv 2023

  2. [2]

    GQA : Training generalized multi-query transformer models from multi-head checkpoints

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebr \'o n, and Sumit Sanghai. GQA : Training generalized multi-query transformer models from multi-head checkpoints. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. arXiv:2305.13245

    arXiv 2023

  3. [3]

    KurTail : Kurtosis-based LLM quantization

    Mohammad Sadegh Akhondzadeh, Aleksandar Bojchevski, Evangelos Eleftheriou, and Martino Dazzi. KurTail : Kurtosis-based LLM quantization. In Findings of the Association for Computational Linguistics: EMNLP 2025 , 2025. arXiv:2503.01483

    arXiv 2025

  4. [4]

    Croci, Marcelo Gennari do Nascimento , Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento , Torsten Hoefler, and James Hensman. SliceGPT : Compress large language models by deleting rows and columns. In International Conference on Learning Representations (ICLR), 2024 a . arXiv:2401.15024

    arXiv 2024

  5. [5]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot : Outlier-free 4-bit inference in rotated LLMs . In Advances in Neural Information Processing Systems (NeurIPS), 2024 b . arXiv:2404.00456

    arXiv 2024

  6. [6]

    Proof-Pile : A pre-training dataset of mathematical texts

    Zhangir Azerbayev, Edward Ayers, and Bartosz Piotrowski. Proof-Pile : A pre-training dataset of mathematical texts. Hoskinson Center for Formal Mathematics / EleutherAI, 2022. URL https://github.com/zhangir-azerbayev/proof-pile

    work page 2022

  7. [7]

    LongBench v2 : Towards deeper understanding and reasoning on realistic long-context multitasks

    Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench v2 : Towards deeper understanding and reasoning on realistic long-context multitasks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp...

    arXiv 2025

  8. [8]

    QuIP : 2-bit quantization of large language models with guarantees

    Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher De Sa. QuIP : 2-bit quantization of large language models with guarantees. In Advances in Neural Information Processing Systems (NeurIPS), 2023. arXiv:2307.13304

    arXiv 2023

Show all 53 references
  1. [9]

    Castro, Torsten Hoefler, and Dan Alistarh

    Jiale Chen, Vage Egiazarian, Roberto L. Castro, Torsten Hoefler, and Dan Alistarh. WUSH : Near-optimal adaptive transforms for LLM quantization. In International Conference on Machine Learning (ICML), 2026 a . arXiv:2512.00956

    2026 arXiv

  2. [10]

    PrefixQuant : Eliminating outliers by prefixed tokens for large language models quantization

    Mengzhao Chen, Yi Liu, Jiahao Wang, Yi Bin, Wenqi Shao, and Ping Luo. PrefixQuant : Eliminating outliers by prefixed tokens for large language models quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026 b . arXiv:2410.05265

    2026 arXiv

  3. [11]

    Grouped sequency-arranged rotation: Optimizing rotation transformation for quantization for free

    Euntae Choi, Sumin Song, Woosang Lim, and Sungjoo Yoo. Grouped sequency-arranged rotation: Optimizing rotation transformation for quantization for free. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Student Research Workshop,...

    2025 arXiv

  4. [12]

    LLM.int8() : 8-bit matrix multiplication for transformers at scale

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8() : 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pp.\ 30318--30332. Curran Associates, Inc., 2022. doi:10.52202/068431-...

    2022 arXiv

  5. [13]

    GPTQ : Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ : Accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations (ICLR), 2023. arXiv:2210.17323

    2023 arXiv

  6. [14]

    The Llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  7. [15]

    Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. KVQuant : Towards 10 million context length LLM inference with KV cache quantization. In Advances in Neural Information Processing Systems 37 ( NeurIPS 2024) ...

    2024 arXiv

  8. [16]

    RULER : What's the real context size of your long-context language models? In Conference on Language Modeling ( COLM ) , 2024

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. RULER : What's the real context size of your long-context language models? In Conference on Language Modeling ( COLM ) , 2024. arXiv:2404.06654

    2024 arXiv

  9. [17]

    OSTQuant : Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting

    Xing Hu, Yuan Cheng, Dawei Yang, Zhixuan Chen, Zukang Xu, Jiangyong Yu, Chen Xu, Zhihang Yuan, Zhe Jiang, and Sifan Zhou. OSTQuant : Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting. In International Confere...

    2025 arXiv

  10. [18]

    Efficient attentions for long document summarization

    Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. Efficient attentions for long document summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HL...

    2021 arXiv

  11. [19]

    SAW-INT4 : System-aware 4-bit KV -cache quantization for real-world LLM serving

    Jinda Jia, Jisen Li, Zhongzhu Zhou, Jung Hwan Heo, Jue Wang, Tri Dao, Shuaiwen Leon Song, Ben Athiwaratkun, Chenfeng Xu, Tianyi Zhang, and Xiaoxia Wu. SAW-INT4 : System-aware 4-bit KV -cache quantization for real-world LLM serving. arXiv preprint arXiv:2604.19157, 2026

    2026 arXiv

  12. [20]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L \'e lio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Tho...

    arXiv

  13. [21]

    Equivalence tests: A practical primer for t tests, correlations, and meta-analyses

    Dani \"e l Lakens. Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8 0 (4): 0 355--362, 2017. doi:10.1177/1948550617697177

    2017 doi

  14. [22]

    CommVQ : Commutative vector quantization for KV cache compression

    Junyan Li, Yang Zhang, Muhammad Yusuf Hassan, Talha Chafekar, Tianle Cai, Zhile Ren, Pengsheng Guo, Foroozan Karimzadeh, Colorado Reed, Chong Wang, and Chuang Gan. CommVQ : Commutative vector quantization for KV cache compression. In International Conference on Machine Learnin...

    2025 arXiv

  15. [23]

    ParoQuant : Pairwise rotation quantization for efficient reasoning LLM inference

    Yesheng Liang, Haisheng Chen, Zihan Zhang, Song Han, and Zhijian Liu. ParoQuant : Pairwise rotation quantization for efficient reasoning LLM inference. In International Conference on Learning Representations (ICLR), 2026. arXiv:2511.10645

    2026

  16. [24]

    DuQuant : Distributing outliers via dual transformation makes stronger quantized LLMs

    Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei. DuQuant : Distributing outliers via dual transformation makes stronger quantized LLMs . In Advances in Neural Information Processing Systems (NeurIPS), 2024 a . Oral...

    2024 arXiv

  17. [25]

    AWQ : Activation-aware weight quantization for LLM compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ : Activation-aware weight quantization for LLM compression and acceleration. In Proceedings of Machine Learning and Systems (MLSys), 2024 b ...

    2024 arXiv

  18. [26]

    SpinQuant : LLM quantization with learned rotations

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. SpinQuant : LLM quantization with learned rotations. In International Conference on Learning Representations (ICLR), 2025. ...

    2025 arXiv

  19. [27]

    KIVI : A tuning-free asymmetric 2bit quantization for KV cache

    Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. KIVI : A tuning-free asymmetric 2bit quantization for KV cache. In Proceedings of the 41st International Conference on Machine Learning (ICML), pp.\ 32332--32344, 2024. ...

    2024 arXiv

  20. [28]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations ( ICLR ) , 2017. arXiv:1609.07843

    2017 arXiv

  21. [29]

    Llama 3.2 1B model card

    Meta . Llama 3.2 1B model card. Hugging Face model card, 2024 a . URL https://huggingface.co/meta-llama/Llama-3.2-1B

    2024

  22. [30]

    Llama 3.2 3B model card

    Meta . Llama 3.2 3B model card. Hugging Face model card, 2024 b . URL https://huggingface.co/meta-llama/Llama-3.2-3B

    2024

  23. [31]

    Mistral-7B-v0.3 model card

    Mistral AI . Mistral-7B-v0.3 model card. Hugging Face model card, 2024. URL https://huggingface.co/mistralai/Mistral-7B-v0.3

    2024

  24. [32]

    Influence-inspired spectral rotations for extreme low-bit LLM quantization

    Gorgi Pavlov. Influence-inspired spectral rotations for extreme low-bit LLM quantization. arXiv preprint arXiv:2605.25203, 2026

    2026 arXiv

  25. [33]

    YaRN : Efficient context window extension of large language models

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. YaRN : Efficient context window extension of large language models. In International Conference on Learning Representations ( ICLR ) , 2024. arXiv:2309.00071

    2024 arXiv

  26. [34]

    Q-ROAR : Outlier-aware rescaling for RoPE position interpolation in quantized long-context LLMs

    Ye Qiao and Sitao Huang. Q-ROAR : Outlier-aware rescaling for RoPE position interpolation in quantized long-context LLMs . In AAAI Conference on Artificial Intelligence (AAAI), Student Abstract Track, 2026. arXiv:2509.14391

    2026

  27. [35]

    Rethinking RoPE scaling in quantized LLM : Theory, outlier, and channel-band analysis with weight rescaling

    Ye Qiao, Haocheng Xu, Xiaofan Zhang, and Sitao Huang. Rethinking RoPE scaling in quantized LLM : Theory, outlier, and channel-band analysis with weight rescaling. arXiv preprint arXiv:2510.00028, 2025

    2025

  28. [36]

    Rae, Anna Potapenko, Siddhant M

    Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier, and Timothy P. Lillicrap. Compressive transformers for long-range sequence modelling. In International Conference on Learning Representations ( ICLR ) , 2020. arXiv:1911.05507; introduces the PG-19 long-range l...

    2020 arXiv

  29. [37]

    Sai Sanjeet, Ian Colbert, Pablo Monteagudo-Lago, Giuseppe Franco, Yaman Umuroglu, and Nicholas J. Fraser. Pushing the limits of block rotations in post-training quantization. arXiv preprint arXiv:2601.22347, 2026

    2026 arXiv

  30. [38]

    OmniQuant : Omnidirectionally calibrated quantization for large language models

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. OmniQuant : Omnidirectionally calibrated quantization for large language models. In International Conference on Learning Representations (ICLR), 2024. ar...

    2024 arXiv

  31. [39]

    DartQuant : Efficient rotational distribution calibration for LLM quantization

    Yuantian Shao, Yuanteng Chen, Peisong Wang, Jianlin Yu, Jing Lin, Yiwu Yao, Zhihui Wei, and Jian Cheng. DartQuant : Efficient rotational distribution calibration for LLM quantization. In Advances in Neural Information Processing Systems (NeurIPS), 2025. arXiv:2511.04063

    2025

  32. [40]

    RoFormer : Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. RoFormer : Enhanced transformer with rotary position embedding. Neurocomputing, 568: 0 127063, 2024. doi:10.1016/j.neucom.2023.127063. arXiv:2104.09864

    2024

  33. [41]

    RotateKV : Accurate and robust 2-bit KV cache quantization for LLMs via outlier-aware adaptive rotations

    Zunhai Su, Zhe Chen, Wang Shen, Hanyu Wei, Linge Li, Huangqi Yu, and Kehong Yuan. RotateKV : Accurate and robust 2-bit KV cache quantization for LLMs via outlier-aware adaptive rotations. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), ...

    2025 arXiv

  34. [42]

    FlatQuant : Flatness matters for LLM quantization

    Yuxuan Sun, Ruikang Liu, Haoli Bai, Han Bao, Kang Zhao, Yuening Li, Jiaxin Hu, Xianzhi Yu, Lu Hou, Chun Yuan, Xin Jiang, Wulong Liu, and Jun Yao. FlatQuant : Flatness matters for LLM quantization. In International Conference on Machine Learning (ICML), 2025. arXiv:2410.09426

    2025 arXiv

  35. [43]

    QuIP\# : Even better LLM quantization with Hadamard incoherence and lattice codebooks

    Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. QuIP\# : Even better LLM quantization with Hadamard incoherence and lattice codebooks. In International Conference on Machine Learning (ICML), 2024. arXiv:2402.04396

    2024 arXiv

  36. [44]

    FPTQuant : Function-preserving transforms for LLM quantization

    Boris van Breugel, Yelysei Bondarenko, Paul Whatmough, and Markus Nagel. FPTQuant : Function-preserving transforms for LLM quantization. In ICML 2025 Workshop on Efficient Systems for Foundation Models (ES-FoMo-III), 2025. Oral; arXiv:2506.04985. Subsequently published at ICML...

    2025 arXiv

  37. [45]

    SQuat : Subspace-orthogonal KV cache quantization

    Hao Wang, Ligong Han, Kai Xu, and Akash Srivastava. SQuat : Subspace-orthogonal KV cache quantization. In Conference on Language Modeling ( COLM ) , 2025. arXiv:2503.24358

    2025 arXiv

  38. [46]

    Outlier suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling

    Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, and Xianglong Liu. Outlier suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling. In Proceedings of the 2023 Conference on Empirical Methods...

    2023 arXiv

  39. [47]

    PolarQuant : Leveraging polar transformation for efficient key cache quantization and decoding acceleration

    Songhao Wu, Ang Lv, Xiao Feng, Yufei Zhang, Xun Zhang, Guojun Yin, Wei Lin, and Rui Yan. PolarQuant : Leveraging polar transformation for efficient key cache quantization and decoding acceleration. In Advances in Neural Information Processing Systems (NeurIPS), 2025. arXiv:2502.00527

    2025 arXiv

  40. [48]

    DFRot : Achieving outlier-free and massive activation-free for rotated LLMs with refined rotation

    Jingyang Xiang and Sai Qian Zhang. DFRot : Achieving outlier-free and massive activation-free for rotated LLMs with refined rotation. In Conference on Language Modeling (COLM), 2025. arXiv:2412.00648

    2025 arXiv

  41. [49]

    SmoothQuant : Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant : Accurate and efficient post-training quantization for large language models. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp.\ 38087--38099, 2023. arX...

    2023 arXiv

  42. [50]

    Outlier smoothing with closed-form rotations for W4A4 large language model quantization

    Jinying Xiao, Bin Ji, Shasha Li, Xiaodong Liu, Jun Ma, Chao Wang, Wei Li, Ye Zhong, Xuan Xie, Nyima Tashi, and Jie Yu. Outlier smoothing with closed-form rotations for W4A4 large language model quantization. arXiv preprint arXiv:2511.22316, 2025

    2025 arXiv

  43. [51]

    ButterflyQuant : Ultra-low-bit LLM quantization through learnable orthogonal butterfly transforms

    Bingxin Xu, Zhen Dong, Oussama Elachqar, and Yuzhang Shang. ButterflyQuant : Ultra-low-bit LLM quantization through learnable orthogonal butterfly transforms. arXiv preprint arXiv:2509.09679, 2025

    2025

  44. [52]

    ComRoPE : Scalable and robust rotary position embedding parameterized by trainable commuting angle matrices

    Hao Yu, Tangyu Jiang, Shuning Jia, Shannan Yan, Shunning Liu, Haolong Qian, Guanghao Li, Shuting Dong, and Chun Yuan. ComRoPE : Scalable and robust rotary position embedding parameterized by trainable commuting angle matrices. In Proceedings of the IEEE/CVF Conference on Compu...

    2025 arXiv

  45. [53]

    TurboQuant : Online vector quantization with near-optimal distortion rate

    Amir Zandieh, Majid Daliri, Majid Hadian, and Vahab Mirrokni. TurboQuant : Online vector quantization with near-optimal distortion rate. In International Conference on Learning Representations (ICLR), 2026. arXiv:2504.19874

    2026 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.