Pith. sign in

REVIEW 2 major objections 6 minor 33 references

Choosing the sign of the quantization scale places the extra signed-integer endpoint on the dominant outlier and cuts few-bit error at no runtime cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 07:34 UTC pith:OENDYB27

load-bearing objection Zero-overhead sign flip on the absmax scale recovers most of the asymmetric accuracy benefit at 2–4 bits while keeping pure symmetric kernels; theory is clean but loosest exactly where the gains are largest. the 2 major comments →

arxiv 2607.08779 v1 pith:OENDYB27 submitted 2026-06-12 cs.LG cs.AI

Signed Symmetric Quantization for Few-Bit Integers

Ian Colbert , Eashan Dash , Pablo Monteagudo-Lago , Juan Amboage , Srinidhi N , Giuseppe Franco , Nicholas J. Fraser , Arun Ramachandran This is my paper
classification cs.LG cs.AI
keywords signed symmetric quantizationfew-bit integersabsmax gridscale signLLM post-training quantizationclipping errorzero-point free
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard symmetric integer quantization forces the scale to be positive, so the extra negative level of a signed alphabet always lands on the negative tail and can clip large positive weights. At 2–4 bits that clipping is a material source of error. The paper shows a third option: keep the zero point at zero (and therefore keep the cheap symmetric kernels and metadata) but freely choose the sign of the scale so that the extra endpoint sits on whichever tail holds the largest-magnitude weight. The resulting signed absmax grid is proved conditionally bound-optimal for worst-case squared error; the mild cardinality condition holds for 88–99 % of weight groups in modern language models. Empirically the change improves WikiText-2 perplexity and few-shot recovery over ordinary positive-scale symmetric quantization while matching its memory and throughput numbers exactly, and it narrows most of the accuracy gap to full asymmetric quantization without paying the zero-point tax.

Core claim

Signed symmetric quantization—uniform integer grids on the signed alphabet with zero point fixed at zero but scale allowed to be negative—lets a closed-form sign rule place the single extra representable endpoint exactly on the dominant outlier. That choice is conditionally bound-optimal for worst-case ℓ₂ error, is analytically equivalent to a unit zero-point shift, and yields measurable model-level gains at identical inference cost.

What carries the argument

The signed absmax grid: scale magnitude α = ‖w‖_∞ / 2^{q−1} and sign γ⋆ = −sign(w_{i⋆}) where i⋆ indexes the largest-magnitude coordinate. Theorem 4.3 bounds total squared error by a quantity that depends on γ only through the size of the clipped set; Corollary 4.4 then shows that γ⋆ minimises the bound whenever |C_{γ⋆}| ≤ |C_{−γ⋆}|.

Load-bearing premise

The paper treats the worst-case error bound, which depends on the scale sign only through how many weights are clipped, as a tight enough surrogate for real squared error that shrinking the clipped set will reliably improve end-to-end model quality.

What would settle it

On a fresh set of weight groups, measure whether groups that satisfy the cardinality condition still show lower realised ℓ₂ error under the signed absmax rule than under the opposite sign; if the correlation collapses or model-level perplexity gains reverse, the surrogate fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any existing symmetric integer kernel can adopt the signed scale at zero extra memory or arithmetic cost and still recover part of the accuracy of an asymmetric format.
  • At 2-bit and 3-bit widths the sign choice becomes a first-order design parameter rather than an implementation detail.
  • Post-training pipelines that already store scales as signed reals need only flip a sign bit per group; no new metadata or kernel changes are required.
  • The analytic equivalence to a unit zero-point shift supplies a clean design space that sits strictly between pure symmetric and full asymmetric quantization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same sign freedom should transfer to activation quantization or to convolutional layers whose weight distributions also exhibit a clear dominant tail.
  • If future formats store scales in an unsigned encoding, an extra sign bit per group would still be far cheaper than a full zero-point field.
  • Joint learning of rotations and signed scales may further close the remaining gap to min-max asymmetric grids without re-introducing runtime offsets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper identifies that the conventional strictly positive scale in symmetric integer quantization assigns the extra negative level of the signed alphabet to the negative tail, which can clip positive outliers and become material at few-bit widths. It proposes signed symmetric quantization: keep z=0 and choose the scale sign so that the extra endpoint lands on the dominant-outlier tail (signed absmax). Theory gives (i) an ℓ₂ error decomposition separating rounding from clipping, a worst-case bound depending on γ only through clipped-set cardinality, and conditional bound-optimality of the proposed sign rule when |C_γ⋆|≤|C_{-γ⋆}|, and (ii) analytic equivalence of a scale sign flip to a unit zero-point shift on the same signed alphabet. Empirically, the cardinality condition holds for 88–99% of groups on Llama3/Qwen3 weights; direct casting and PTQ pipelines (rotations, Qronos, learned grids) improve WikiText2 and few-shot recovery over positive-scale symmetric formats at identical symmetric runtime cost, while remaining competitive with asymmetric formats especially at 3–4 bits.

Significance. If the results hold, the work supplies a zero-overhead degree of freedom that partially recovers asymmetric endpoint alignment without zero-point metadata or kernel cost—directly relevant to production GGUF/llama.cpp-style deployments. Strengths include complete proofs in Appendix A under standard measure-zero assumptions, a closed-form data-free sign rule, large-scale verification of the cardinality condition (>200M groups), isolation of the sign effect via direct casting (Table 2), and composition with modern PTQ (Tables 3–4) plus systems measurements (Table 1). Treating scale sign as an explicit parameter rather than a convention is a clean, falsifiable contribution with immediate practical value at 2–4 bits.

major comments (2)
  1. Corollary 4.4 and Remark 4.5 establish conditional bound-optimality only when |C_γ⋆|≤|C_{-γ⋆}|, which holds for ~88% of groups at q=2, ~96% at q=3, and ~99% at q=4 (Figure 1a). Yet the largest model-level gains appear precisely at 2 bits (e.g., Table 3: WikiText2 103.1→17.8; recovery +7.9 pp), where the condition is weakest. The paper notes the tension and invokes unconditional structural properties (exact dominant-outlier preservation; clipping confined to the opposite tail). That fallback is plausible but currently secondary in the narrative. For the theory-to-practice link to be load-bearing at the headline regime, the discussion (and abstract/intro emphasis) should more clearly rank the unconditional structural properties as the primary explanation of 2-bit gains, with conditional bound-optimality as the supporting guarantee that holds most of the time—rather than leading with the co
  2. Theorem 4.3 bounds per-clipped-coordinate squared error by α² (vs α²/4 for unclipped), so the γ-dependent term is a factor-of-4 worst-case surrogate. Figure 1 shows that the cardinality margin predicts realized ℓ₂ gain with high frequency when m(w)>0, which is reassuring, but the paper does not quantify how often γ⋆ still reduces exact Δ_γ when the cardinality condition fails (the left half of panels b–d). A short breakdown of realized g(w) on the ~12% of q=2 groups where |C_γ⋆|>|C_{-γ⋆}| would tighten the claim that the sign rule remains effective even off the conditional optimum, and would better support the 2-bit results that drive the empirical story.
minor comments (6)
  1. Abstract and §1 use “unsigned symmetric quantizer” for the conventional positive-scale case; elsewhere the alphabet is signed. Prefer “strictly positive-scale” or “conventional symmetric” throughout to avoid confusion with unsigned alphabets used in min-max asymmetric grids.
  2. Table 1 caption and body: ratios <1 for memory and >1 for throughput favor symmetric; a one-line note that Q2_0/Q2_1 are experimental formats introduced here (as in Appendix B) would help readers who know only standard GGUF types.
  3. Figure 1: quadrant percentages are from full counts while the scatter is a 1% subsample—state this more prominently in the caption so the visual density is not misread as the empirical mass.
  4. Theorem 4.6 is stated for a scalar w; the extension to vectors is immediate but a one-sentence remark that the identity holds coordinatewise under the same no-midpoint assumption would avoid a pedantic objection.
  5. §5 and Appendix C: Qronos damping λ=10^{-4}σ₁ and group size 32 are free parameters; listing them once in a small hyperparameter table would aid reproducibility without cluttering the main text.
  6. Typos / polish: “an AMD EPYC™” spacing; “Q2 0s” vs “Q2_0s” consistency in tables; “pre-trained” hyphenation is fine but keep uniform with “few-bit”.

Circularity Check

0 steps flagged

No circularity: conditional bound-optimality is derived from an explicit error decomposition and worst-case bound; the sign rule is closed-form and the equivalence is algebraic.

full rationale

The paper's central theoretical claims are self-contained. Lemma 4.2 decomposes ℓ₂ error into a γ-independent rounding term E(w) and a clipping penalty Δ_γ that depends only on the clipped set C_γ. Theorem 4.3 converts this into a deterministic upper bound whose sole γ-dependent factor is the clipped fraction ρ_γ. Corollary 4.4 then states the elementary consequence that any γ minimizing |C_γ| also minimizes the bound, and that the proposed γ⋆ = −sign(w_i⋆) is such a minimizer precisely when |C_γ⋆| ≤ |C_−γ⋆|. This is ordinary derivation, not a definitional loop: the sign rule is motivated independently by exact representation of the dominant outlier, the bound is proved from first principles under the stated no-midpoint assumption, and the condition is checked empirically rather than assumed. Theorem 4.6 is a pure algebraic identity equating a negative scale at z=0 with a unit zero-point shift on the same alphabet. Empirical sections compare the resulting format against the conventional positive-scale baseline (and against asymmetric min-max) on public models and standard benchmarks; no parameter is fitted to a subset of data and then re-presented as a prediction. Self-citations (Qronos, Brevitas, prior scale-optimization notes) supply reusable algorithmic components, not load-bearing uniqueness theorems that force the present claims. The skeptic observation that largest model-level gains occur where the cardinality condition is weakest is a legitimate question of explanatory tightness, not circularity: the paper itself flags the tension in Remark 4.5 and falls back on unconditional structural properties (exact outlier preservation, opposite-tail clipping). No step reduces a claimed prediction or optimality result to its own inputs by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 1 invented entities

The central claim rests on standard rounding arithmetic, a measure-zero no-midpoint assumption, the conventional absmax scale magnitude, and the empirical observation that LLM weight groups possess a dominant outlier. No free parameters are fitted to produce the sign rule itself; group size 32 and Qronos hyperparameters are inherited from the deployment target and prior work.

free parameters (2)
  • group size = 32
    Fixed at 32 to match the llama.cpp / GGUF block convention; not optimised for the sign rule but inherited from the target runtime.
  • Qronos damping λ = 1e-4 * σ1
    Set to 10^{-4} times the largest singular value of the covariance; standard hyper-parameter of the adaptive-rounding baseline, held fixed across formats.
axioms (3)
  • standard math Round-to-nearest is unique and odd (⌊−v⌉ = −⌊v⌉) under the no-midpoint condition wi/α ∉ ℤ + 1/2
    Invoked throughout Section 4 and Appendix A; measure-zero under continuous weight distributions.
  • domain assumption Scale magnitude is set by absmax: α = ∥w∥∞ / 2^{q−1}
    Standard symmetric-quantisation convention (Eq. 3); the paper only optimises the sign of this magnitude.
  • domain assumption LLM weight groups of size 32 typically possess a single dominant outlier whose exact representation is beneficial
    Underpins the practical value of the sign rule; supported by the 88–99 % cardinality statistics but not proved for all models or granularities.
invented entities (1)
  • signed absmax grid independent evidence
    purpose: Places the extra negative endpoint of the signed alphabet on the dominant-outlier tail while keeping zero point zero.
    Defined by Eq. 5; a design choice rather than a physical postulate. Independent evidence is the empirical accuracy gains and the conditional optimality proof.

pith-pipeline@v1.1.0-grok45 · 24293 in / 2744 out tokens · 34382 ms · 2026-07-13T07:34:01.531782+00:00 · methodology

0 comments
read the original abstract

The signed integer alphabet contains one more negative representable value than positive. Yet, by convention, the standard symmetric integer quantizer fixes its scale to be strictly positive, which assigns this extra representable value to the negative tail and can force clipping of positive outliers. In this work, we show that, at few-bit precision, such clipping is a non-trivial source of quantization error. Asymmetric quantization addresses this problem with a zero point, shifting the grid toward the observed data range; however, this flexibility is well-known to carry a runtime penalty. For example, in llama.cpp on an AMD EPYC(TM) "Turin" CPU, a 4-bit symmetric format uses up to 9% less memory with up to 2.45$\times$ higher throughput than its asymmetric counterpart. We highlight signed symmetric quantization as a third option that retains the runtime profile of symmetric quantization without the penalty of the asymmetric format: our signed absmax grid places the extra representable value on the dominant-outlier tail through a principled and lightweight sign selection rule while keeping the zero point at zero. Our theoretical analysis offers two main results. First, we establish the signed absmax grid as conditionally bound-optimal on $\ell_2$ quantization error, and show that the condition holds for 88-99% of weight groups across pre-trained large language models (LLMs) at low bit widths. Second, we show that negating the scale of a standard symmetric quantizer is analytically equivalent to a unit zero point shift on the same signed integer alphabet. We empirically validate our proposal on models from the Qwen3, Qwen3.5, and Llama3 families, and observe improvement in perplexity and downstream few-shot accuracy over the standard unsigned symmetric quantizer at no extra inference cost

Figures

Figures reproduced from arXiv: 2607.08779 by Arun Ramachandran, Eashan Dash, Giuseppe Franco, Ian Colbert, Juan Amboage, Nicholas J. Fraser, Pablo Monteagudo-Lago, Srinidhi N.

Figure 1
Figure 1. Figure 1: Empirical validation of signed symmetric quantization on Llama3 8B and Qwen3 8B, [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: We illustrated the mergeable rotation graph used by the PTQ pipelines behind Tables [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 9 linked inside Pith

  1. [1]

    Optimal post-training quantization scales and where to find them.arXiv preprint arXiv:2606.10890, 2026

    Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert, Giuseppe Franco, and Nicholas Fraser. Optimal post-training quantization scales and where to find them.arXiv preprint arXiv:2606.10890, 2026

    Pith/arXiv arXiv 2026

  2. [2]

    QuaRot: Outlier-free 4-bit inference in rotated LLMs.Advances in Neural Information Processing Systems, 37:100213– 100240, 2024

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot: Outlier-free 4-bit inference in rotated LLMs.Advances in Neural Information Processing Systems, 37:100213– 100240, 2024

    2024

  3. [3]

    Estimating or propagating gradi- ents through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradi- ents through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    Pith/arXiv arXiv 2013

  4. [4]

    LSQ+: Im- proving low-bit quantization through learnable offsets and better initialization

    Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. LSQ+: Im- proving low-bit quantization through learnable offsets and better initialization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 2978–2985, 2020

    2020

  5. [5]

    PIQA: Reasoning about phys- ical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. PIQA: Reasoning about phys- ical commonsense in natural language. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020

    2020

  6. [6]

    Think you have solved question answering? try ARC, the AI2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try ARC, the AI2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Pith/arXiv arXiv 2018

  7. [7]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    Binaryconnect: Training deep neural networks with binary weights during propagations.Advances in neural information processing systems, 28, 2015

    Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations.Advances in neural information processing systems, 28, 2015

    2015

  9. [9]

    Esser, Jeffrey L

    Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S. Modha. Learned step size quantization. InInternational Conference on Learn- ing Representations, 2020

    2020

  10. [10]

    OPTQ: Accurate quanti- zation for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quanti- zation for generative pre-trained transformers. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  11. [11]

    Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, et al. Are we done with MMLU? InProceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technolo...

    2025

  12. [12]

    llama.cpp.https://github.com/ggml-org/llama.cpp, 2023

    Georgi Gerganov. llama.cpp.https://github.com/ggml-org/llama.cpp, 2023. Ac- cessed: 2026-02-12

    2023

  13. [13]

    A survey of quantization methods for efficient neural network inference

    Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. InLow-power com- puter vision, pages 291–326. Chapman and Hall/CRC, 2022

    2022

  14. [14]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    LightEval: A lightweight framework for LLM evaluation, 2024

    Nathan Habib, Cl ´ementine Fourrier, Hynek Kydl ´ıˇcek, Thomas Wolf, and Lewis Tunstall. LightEval: A lightweight framework for LLM evaluation, 2024

    2024

  16. [16]

    GPTQ, 2022

    IST-DASLab. GPTQ, 2022. 10

    2022

  17. [17]

    Quantization and training of neural networks for ef- ficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for ef- ficient integer-arithmetic-only inference. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  18. [18]

    Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks.Proceedings of Machine Learning and Systems, 2:112–128, 2020

    Sambhav Jain, Albert Gural, Michael Wu, and Chris Dick. Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks.Proceedings of Machine Learning and Systems, 2:112–128, 2020

    2020

  19. [19]

    Quantizing deep convolutional networks for efficient inference: A whitepaper.arXiv preprint arXiv:1806.08342, 2018

    Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper.arXiv preprint arXiv:1806.08342, 2018

    Pith/arXiv arXiv 2018

  20. [20]

    SpinQuant: LLM quantization with learned rotations

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. SpinQuant: LLM quantization with learned rotations. InProceedings of the International Conference on Learn- ing Representations (ICLR), 2025

    2025

  21. [21]

    Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    Pith/arXiv arXiv 2016

  22. [22]

    The fineweb datasets: Decanting the web for the finest text data at scale.Advances in Neural Information Processing Systems, 37:30811–30849, 2024

    Guilherme Penedo, Hynek Kydl ´ıˇcek, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Lean- dro V on Werra, Thomas Wolf, et al. The fineweb datasets: Decanting the web for the finest text data at scale.Advances in Neural Information Processing Systems, 37:30811–30849, 2024

    2024

  23. [23]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026

    2026

  24. [24]

    WinoGrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. WinoGrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    2021

  25. [25]

    MixQuant: Pushing the limits of block rotations in post-training quantiza- tion.arXiv preprint arXiv:2601.22347, 2026

    Sai Sanjeet, Ian Colbert, Pablo Monteagudo-Lago, Giuseppe Franco, Yaman Umuroglu, and Nicholas J Fraser. MixQuant: Pushing the limits of block rotations in post-training quantiza- tion.arXiv preprint arXiv:2601.22347, 2026

    Pith/arXiv arXiv 2026

  26. [26]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025

    2025

  27. [27]

    Transformers: State-of-the- art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the- art n...

    2020

  28. [28]

    SmoothQuant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. InInternational conference on machine learning, pages 38087–38099. PMLR, 2023

    2023

  29. [29]

    Brevitas: Quantization-aware training in pytorch.https://github.com/Xilinx/ brevitas, 2018

    Xilinx. Brevitas: Quantization-aware training in pytorch.https://github.com/Xilinx/ brevitas, 2018. Accessed: 2026-02-12

    2018

  30. [30]

    HellaSwag: Can a machine really finish your sentence? InProceedings of the 57th annual meeting of the association for computational linguistics, pages 4791–4800, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? InProceedings of the 57th annual meeting of the association for computational linguistics, pages 4791–4800, 2019

    2019

  31. [31]

    Provable post-training quantiza- tion: Theoretical analysis of OPTQ and Qronos.arXiv preprint arXiv:2508.04853, 2025

    Haoyu Zhang, Shihao Zhang, Ian Colbert, and Rayan Saab. Provable post-training quantiza- tion: Theoretical analysis of OPTQ and Qronos.arXiv preprint arXiv:2508.04853, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Beacon: Post-training quantization with integrated grid selec- tion.IEEE Signal Processing Letters, 2026

    Shihao Zhang and Rayan Saab. Beacon: Post-training quantization with integrated grid selec- tion.IEEE Signal Processing Letters, 2026

    2026

  33. [33]

    narrow range

    Shihao Zhang, Haoyu Zhang, Ian Colbert, and Rayan Saab. Qronos: Correcting the past by shaping the future... in post-training quantization. InThe Fourteenth International Conference on Learning Representations, 2026. 11 A Proofs Throughout this appendix, we adopt the assumptions stated in Section 4: the signed integer alphabet Aq :={−2 q−1, . . . ,2q−1 −1...

    2026