Pith. sign in

REVIEW 3 major objections 4 minor 51 references

ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Post-training quantization outputs are not final: a fixed-grid discrete refinement pass can still improve them.

desk verdict ReQuant is a clean, well-executed fixed-grid refinement stage for PTQ with correct math and broad experiments; the 'consistent improvement' claim is slightly too strong and the stale-activation-statistics assumption deserves an explicit fix, but it merits serious peer review. read the letter →

arxiv 2608.07019 v1 pith:4P3RRPC4 submitted 2026-08-07 cs.AI

Yongge Ma , Guoan Wang , Feiyu Wang , Yaoming Li , Qian Zhang , Zihan Yan , Yinjun Han , Tong Yang This is my paper · ORCID
classification cs.AI
keywords post-trainingquantizationlargelanguagemodelsdiscretecoordinatedescentfixed-gridrefinementreconstructionerrorintegerweightcodeslow-bitmodelcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes ReQuant, a post-training quantization post-processing stage that keeps improving a quantized model after an initial PTQ method has produced it. The central claim is that the initial integer assignments are not final: ReQuant revisits each weight code on the fixed quantization grid, accepts only moves that strictly reduce the layer-wise mean squared reconstruction error, and preserves bit-width, scales, zero-points, and inference kernels. If true, any existing PTQ pipeline can be improved by plugging in this refinement, with the largest gains for simple initializers and low bit-widths. The paper reports that refining a simple round-to-nearest initialization across sweeps approaches or surpasses the stronger GPTAQ initializer under the same format.

What carries the argument

The load-bearing object is the exact discrete coordinate score $\Delta L(\Delta q_j) = -\Delta q_j\, g_j + (\Delta q_j)^2 \, H_{jj}$, derived from the row-wise quadratic reconstruction loss. Here $\mathbf{H} = \widetilde{X}\widetilde{X}^\top$ is the row-level Hessian of the quantized-branch activations and $\mathbf{B} = \Delta X \widetilde{X}^\top$ captures the activation mismatch between full-precision and quantized prefixes; both are precomputed once. This score lets ReQuant evaluate every $K$-neighborhood grid move in constant time, update the gradient incrementally after each accepted move, and iterate sweeps until the local objective is exhausted.

What would settle it

Run ReQuant with the same starting model and calibration data but recompute $\mathbf{H}$ and $\mathbf{B}$ from the actual quantized activations after each sweep instead of reusing the initial statistics. If refreshing the statistics yields lower perplexity or a different set of accepted moves than the paper's one-pass version, the one-pass surrogate is not tracking the true deployment objective.

Watch

Extended reading notes

Core claim

ReQuant establishes that completed PTQ outputs are feasible starting points rather than terminal solutions. Treating the layer-wise objective $L(W^q) = \|WX - W^q \widetilde{X}\|_F^2$, it performs discrete coordinate descent over integer codes on the inherited grid; since each output row depends only on the corresponding weight row, the loss decomposes row-wise, and each candidate move is scored in O(1) using a cached Hessian and gradient. A move is accepted exactly when it lowers the row reconstruction loss, so every accepted update strictly decreases the full layer objective, and finite termination follows from finiteness of the grid. The experiments show consistent improvements in perplexity, KL divergence, and ten-task average accuracy across RTN, AWQ, GPTQ, and GPTAQ initializers on Llama-3 and Qwen3 models, with RTN+ReQuant approaching or beating GPTAQ under W4A4.

Load-bearing premise

The refinement scores are computed from activation statistics collected once from the initial quantized model, and the paper assumes those statistics stay accurate as the integer codes change; if accepted updates shift downstream activations, the optimized solution may be optimal for outdated statistics rather than for the actual deployment.

Editorial extensions

If this is right

  • Any existing PTQ pipeline can be extended by a refinement stage that touches only integer codes, so the deployed bit-width, scales, zero-points, and kernels stay exactly as they were.
  • A simple round-to-nearest start, after enough sweeps, approaches or exceeds a strong activation-aware initializer under the same format, meaning much of the PTQ performance gap is correctable discrete assignment error.
  • The improvement is largest where quantization is hardest: lower bit-widths and weaker initializers, with gains still present on GPTQ and GPTAQ.
  • Offline refinement cost is controlled by the sweep count $T$, so practitioners can trade offline time for accuracy and pay nothing at serving time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because $\mathbf{H}$ and $\mathbf{B}$ are collected once from the initial quantized prefix, the monotone guarantee covers the surrogate objective, not necessarily the true deployment objective; a testable extension is to refresh these statistics every sweep and compare the resulting assignments.
  • The paper's own no-QuaRot GPTQ result, where perplexity improves while average downstream accuracy edges down, shows that the layer-wise objective is a proxy; practitioners should validate refinement on the specific downstream metric they care about.
  • The same row-wise discrete coordinate machinery transfers to any layer with linear structure, such as convolutional layers or sparse expert projections in mixture-of-experts models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ReQuant, a post-training quantization refinement stage that treats an existing quantized model as a feasible starting point and optimizes its integer codes on the fixed quantization grid. The layer-wise objective is the GPTAQ-style activation-aware reconstruction error ||WX - W_q \tilde X||^2, where \tilde X denotes activations collected once from the initial quantized prefix (Eq. 3). Candidate coordinate moves are scored in closed form via Eq. 7, and updates that strictly decrease the row loss are accepted (Algorithm 1). The paper proves finite termination under fixed calibration statistics (Appendix A.2), analyzes per-layer complexity (Appendix A.3), and reports experiments across Llama-3 8B/70B, Qwen3-14B, and Qwen3-235B, several bit widths, and RTN/AWQ/GPTQ/GPTAQ initializers, with perplexity, KL, and ten-task zero-shot accuracy as metrics.

Significance. If the claims hold, ReQuant provides a format-preserving, initializer-agnostic post-processing stage that can be composed with existing PTQ pipelines, and the results suggest that a substantial share of the performance gap between simple and advanced PTQ initializers is correctable fixed-grid assignment error. The derivations and the finite-termination proof in Appendix A are clear and, as far as I checked, correct; the experimental coverage is unusually broad, including a 235B MoE model. The paper is also honest about several small reversals and about seed variability in Appendix B.9. The main weakness is a mismatch between the theoretical guarantee, which is stated for a frozen-statistics surrogate, and the empirical claim about the deployed model; this gap, together with the absence of repeated-run statistics for the main tables, needs to be addressed before the 'consistently improves' claim is fully supported.

major comments (3)
  1. [§3.2, §3.3, Appendix A.2] The layer traversal order is unspecified, and the activation statistics \tilde H = \tilde X \tilde X^T and B are collected once from the initial quantized prefix and never refreshed in Algorithm 1. If layers are refined front-to-back or in parallel, the input activations \tilde X of downstream layers change after upstream layers are refined, so the monotone-decrease guarantee of Appendix A.2 applies only to the stale surrogate L(W_q) = ||WX - W_q \tilde X_initial||^2, not to the actual reconstruction error of the deployed model ||WX - W_q \tilde X_new||^2. The no-QuaRot W4A16 GPTQ result in Table 13 (average accuracy 64.63 to 64.41 while WikiText PPL/KL improve) is consistent with this mismatch. The authors should specify a traversal order that keeps \tilde X valid during refinement (e.g., reverse layer order), or refresh the statistics between sweeps, and should state explicitly whether the theoretical guarantee concerns the surrogate or the deployed objective; if it concerns only the surrogate, the abstract's claim that accepted updates 'strictly reduce the mean squared reconstruction error' needs qualification.
  2. [§4, Appendix B.9] The main tables (Tables 1, 2, 3, 4, 5, 6, and the appendix tables) report single runs without error bars. The only repeated-run estimate in the paper is the QuaRot seed-stability study in Appendix B.9, which reports average accuracy 65.27 ± 0.55 for one configuration. Several headline improvements on strong initializers are smaller than this spread: for example, Table 1 reports GPTQ+ReQuant W4A16 on Llama-3 8B as +0.33 and GPTAQ+ReQuant W4A16 on Qwen3-14B as +0.04, and Table 3 reports GPTAQ+ReQuant W3A4 as +0.06. To support the central claim of consistent improvement across all four initializers, the authors should report repeated runs or seed-averaged results with confidence intervals for at least the main comparisons, or restrict the claim to settings where the gains exceed the measured variability.
  3. [§4.1] The paper states that WikiText-2 is used as the calibration dataset and that perplexity is also reported on WikiText-2, but it does not state whether the calibration and evaluation splits are disjoint. If the same sequences are used for both, the WikiText-2 PPL/KL improvements are in-domain and may be optimistic. Please specify the exact split (e.g., train/validation/test) and, ideally, calibrate on a different corpus or report held-out perplexity so that the reported WikiText-2 numbers are not confounded with calibration-set overlap.
minor comments (4)
  1. [§4.5] The text claims that ReQuant can be composed with AdaRound/BRECQ/FlexRound-style construction-time optimization, but the experiments only compare with FlexRound; consider adding a sentence noting that the AdaRound/BRECQ composition is not experimentally evaluated, or softening the claim.
  2. [§4.6, Table 6] The main text refers to 'Top-20 KL' for the Qwen3-235B experiment, but the table header simply says 'KL'; please define Top-20 KL and make the table header consistent with the text.
  3. [Appendix B.8, Table 14] The note 'GPTQ rows reuse Table 4' is confusing because Table 4 reports RTN and GPTQ under different headings; please make the reuse explicit by repeating the rows or by clearly labeling the source table.
  4. [§3.3] The sentence 'if the remaining network is locally Lipschitz, smaller layer residuals tighten a bound on later hidden-state perturbations' is used to connect the layer-wise objective to downstream metrics; a formal statement or a reference for this bound would help the reader assess how tight that connection is expected to be.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the monotone-decrease guarantee is an algorithm invariant and all downstream improvements are empirical measurements.

full rationale

The paper's derivation chain is self-contained. Section 3.2 defines the layer-wise objective L(Wq) = ||WX - Wq eX||_F^2 (Eq. 3), rewrites the row loss as a quadratic in the quantization error e (Eq. 5), derives the gradient g and loss-change formula (Eqs. 6-7), and Algorithm 1 accepts a candidate only if Delta L < 0. Monotone decrease of L is therefore an invariant of the acceptance rule, and finite termination (Appendix A.2) follows from finiteness of the fixed grid combined with strict decrease. These are mathematical properties of the defined objective, not predictions that could be forced by fitting. The only definitional component is that the guarantee concerns L computed with precomputed activation statistics eX and eH/B; the paper is explicit that this is a layer-wise calibration proxy: 'Like other reconstruction-based PTQ methods, ReQuant optimizes a layer-wise calibration objective, and held-out metrics are reported empirically' (Limitations). No downstream gain is claimed by proof; PPL, KL, and zero-shot accuracy are reported as measurements against external benchmarks. No parameter is fitted to the reported outcomes and then reused as a prediction: K=2 and T=4 are fixed hyperparameters with ablations, and the method is evaluated across multiple models, bit-widths, initializers, and held-out tasks. Reusing GPTAQ's activation-aware objective is an explicit design choice, not an imported uniqueness theorem or self-citation chain; the cited PTQ methods are external prior work. The honest reporting of a reversal (Table 13: GPTQ+ReQuant average accuracy drops from 64.63 to 64.41 while WikiText PPL improves) further indicates that the results are measured rather than constructed to fit the central claim. The stale-statistics concern raised about eH/B is a validity gap between surrogate and deployment objective, appropriately a correctness risk, not a form of circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two assumptions not established by the paper: that layer-wise reconstruction error is a good surrogate for task performance, and that freezing activation statistics during refinement is valid. The only user-chosen numbers are the neighborhood size K and the sweep count T, both ablated. No new entities are introduced.

free parameters (2)
  • K = 2 (default; ablated over {1,2,3})
    Neighborhood size for candidate integer offsets. Chosen by hand, with small effect on quality as shown in Table 15.
  • T = 4 (default; swept over 0 to 8)
    Number of coordinate sweeps. Controls the compute-accuracy trade-off, with diminishing returns at larger T as shown in Table 4 and Figure 2.
assumptions (3)
  • domain assumption Layer-wise reconstruction error on calibration activations is a valid proxy for final model quality on downstream tasks.
    Used to score every candidate move (Eq. 3). Section 3.3 acknowledges the proxy and reports downstream metrics empirically, but the optimization itself only sees the proxy.
  • ad hoc to paper Activation statistics eH and B collected once from the initial quantized prefix remain accurate after weight-code updates.
    ReQuant computes eH and B once before sweeps (Eq. 6) and never re-collects activations. Stale statistics could decouple scores from the true deployment objective.
  • standard math Standard results for convex quadratics and finite grids justify coordinate-descent updates and termination.
    Used in Sections 3.2 and 3.3 and Appendix A.2. This is unproblematic background mathematics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization." pith.science (2026). https://pith.science/paper/4P3RRPC4

@misc{pith2026260807019,
  author       = {Pith},
  title        = {Pith review of: ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4P3RRPC4}},
  note         = {Machine review of arXiv:2608.07019}
}
read the original abstract

Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models. Existing PTQ methods typically obtain an initial quantized model through heuristic rules or greedy optimization, and once quantization is completed the resulting integer assignments are usually treated as final. This observation motivates a complementary optimization stage within PTQ that keeps quantized weights improvable after an executable quantized model has been produced, while preserving the quantized format. We introduce ReQuant, a backpropagation-free fixed-grid refinement procedure for this stage. Agnostic to the PTQ initializer, ReQuant takes an existing quantized model as a feasible starting point and iteratively revisits its discrete weight assignments on the fixed quantization grid. Accepted updates strictly reduce the mean squared reconstruction error and remain on the original grid. In this way, ReQuant turns the initially fixed PTQ output into an iteratively optimizable discrete solution and serves as a plug-and-play post-processing stage for existing PTQ pipelines. Experiments across diverse model families, bit-widths, and downstream tasks show that ReQuant consistently improves quantized models from heterogeneous PTQ initializers, with especially large gains on simple initializers and lower bit-widths. Notably, ReQuant can refine a simple round-to-nearest initialization across multiple sweeps until it approaches or surpasses GPTAQ under the same quantization format. These results establish ReQuant as a practical complementary stage for further improving existing PTQ pipelines.

Figures

Figures reproduced from arXiv: 2608.07019 by the authors.

Figure 1
Figure 1. Overview of ReQuant. ReQuant refines an initial PTQ model by iteratively updating discrete weight assignments on the fixed quantization grid, improving reconstruction error while preserving the quantized format. AdaRound [33] learns continuous rounding variables through gradient-based optimization; BRECQ [34] extends reconstruction-based PTQ to block-level settings; and FlexRound [29] jointly learns element-wise div… view at source ↗
Figure 2
Figure 2. Effect of the number of ReQuant refinement sweeps [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. W4A4: sensitivity to calibration sample size for GPTAQ + QuaRot + ReQuant on Llama-3 8B. (left) [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 27 canonical work pages

  1. [1]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in Neural Information Processing Systems, 30, 2017. 10 ReQuant: Fixed-Grid Discrete Refinement for Post-Training QuantizationA PREPRINT

    work page 2017

  2. [2]

    Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901, 2020

    work page 1901

  3. [3]

    GPT-4 Technical Report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 Technical Report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  4. [4]

    DeepSeek-V2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Daya Guo, et al. DeepSeek-V2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    arXiv 2024

  5. [5]

    Qwen Technical Report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen Technical Report.arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  6. [6]

    LLaMA: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  7. [7]

    GPT3.int8(): 8-bit matrix multiplication for transformers at scale.Advances in Neural Information Processing Systems, 35:30318–30332, 2022

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. GPT3.int8(): 8-bit matrix multiplication for transformers at scale.Advances in Neural Information Processing Systems, 35:30318–30332, 2022

    work page 2022

  8. [8]

    PaLM: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1–113, 2023

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240):1–113, 2023

    work page 2023

Show all 51 references
  1. [9]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    2022

  2. [10]

    Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P

    Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P. Bosma, Zongwei Zhou, Tao Wang, Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hell...

    2022

  3. [11]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. InProceedings of the IEEE Conference on Computer Vision and...

    2018

  4. [12]

    Pact: Parameterized clipping activation for quantized neural networks

    Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakr- ishnan. Pact: Parameterized clipping activation for quantized neural networks. InInternational Conference on Learning Representations (ICLR), 2018

    2018

  5. [13]

    AWQ: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of Machine Learning and Systems (MLSys), 6:87–100, 2024

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware weight quantization for on-device llm compression and acceleration.Proceedings of Machine Learning and Systems (MLSys), 6:...

    2024

  6. [14]

    SmoothQuant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. InInternational Conference on Machine Learning (ICML), pages 38087–38099, 2023

    2023

  7. [15]

    Mahoney, and Kurt Keutzer

    Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. InLow-Power Computer Vision, pages 291–326. Chapman and Hall/CRC, 2022

    2022

  8. [16]

    Binarized neural networks.Advances in neural information processing systems, 29, 2016

    Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks.Advances in neural information processing systems, 29, 2016

    2016

  9. [17]

    Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432, 2013

    2013 arXiv

  10. [18]

    Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems, 36:10088–10115, 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems, 36:10088–10115, 2023

    2023

  11. [19]

    LLM-QAT: Data-free quantization-aware training for large language models

    Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. LLM-QAT: Data-free quantization-aware training for large language models. InFindings of the Association for Computational Linguistics...

    2024

  12. [20]

    EfficientQAT: Efficient quantization-aware training for large language models

    Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. EfficientQAT: Efficient quantization-aware training for large language models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), pages 10081–1...

    2025

  13. [21]

    Dl-qat: Weight-decomposed low-rank quantization-aware training for large language models

    Wenjing Ke, Zhe Li, Dong Li, Lu Tian, and Emad Barsoum. Dl-qat: Weight-decomposed low-rank quantization-aware training for large language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 113–119, 2024

    2024

  14. [22]

    PV-Tuning: Beyond straight-through estimation for extreme llm compression.Advances in Neural Information Processing Systems, 37:5074–5121, 2024

    Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtárik. PV-Tuning: Beyond straight-through estimation for extreme llm compression.Advances in Neural Information Processing Systems, 37:5074–5121, 2024

    2024

  15. [23]

    A survey of low-bit large language models: Basics, systems, and algorithms.Neural Networks, page 107856, 2025

    Ruihao Gong, Yifu Ding, Zining Wang, Chengtao Lv, Xingyu Zheng, Jinyang Du, Yang Yong, Shiqiao Gu, Haotong Qin, Jinyang Guo, et al. A survey of low-bit large language models: Basics, systems, and algorithms.Neural Networks, page 107856, 2025

    2025

  16. [24]

    GPTQ: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers.arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  17. [25]

    GPTAQ: Efficient finetuning-free quantization for asymmetric calibration.arXiv preprint arXiv:2504.02692, 2025

    Yuhang Li, Ruokai Yin, Donghyun Lee, Shiting Xiao, and Priyadarshini Panda. GPTAQ: Efficient finetuning-free quantization for asymmetric calibration.arXiv preprint arXiv:2504.02692, 2025

    2025 arXiv

  18. [26]

    Accurate post training quantization with small calibration sets

    Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets. InProceedings of the 38th International Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Research, pages 4466–447...

    2021

  19. [27]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier-free 4-bit inference in rotated llms.Advances in Neural Information Processing Systems (NeurIPS), 37, 2024

    2024

  20. [28]

    Spinquant: Llm quantization with learned rotations.arXiv preprint arXiv:2405.16406, 2024

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. Spinquant: Llm quantization with learned rotations.arXiv preprint arXiv:2405.16406, 2024

    2024 arXiv

  21. [29]

    FlexRound: Learnable rounding based on element-wise division for post-training quantization

    Jung Hyun Lee, Jeonghoon Kim, Se Jung Kwon, and Dongsoo Lee. FlexRound: Learnable rounding based on element-wise division for post-training quantization. InProceedings of the 40th International Conference on Machine Learning, volume 202 ofProceedings of Machine Learning Resear...

    2023

  22. [30]

    Post-training 4-bit quantization of convolutional networks for rapid deployment

    Ron Banner, Yury Nahshan, and Daniel Soudry. Post-training 4-bit quantization of convolutional networks for rapid deployment. InAdvances in Neural Information Processing Systems, volume 32, 2019

    2019

  23. [31]

    Omniquant: Omnidirectionally calibrated quantization for large language models.arXiv preprint arXiv:2308.13137, 2023

    Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Omnidirectionally calibrated quantization for large language models.arXiv preprint arXiv:2308.13137, 2023

    2023 arXiv

  24. [32]

    Optimal brain compression: A framework for accurate post-training quantization and pruning

    Elias Frantar and Dan Alistarh. Optimal brain compression: A framework for accurate post-training quantization and pruning. Advances in Neural Information Processing Systems, 35:4475–4488, 2022

    2022

  25. [33]

    Up or down? adaptive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. InProceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research...

    2020

  26. [34]

    BRECQ: Pushing the limit of post-training quantization by block reconstruction

    Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. BRECQ: Pushing the limit of post-training quantization by block reconstruction. InInternational Conference on Learning Representations, 2021

    2021

  27. [35]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  28. [36]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  29. [37]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  30. [38]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  31. [39]

    Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843, 2016

    2016 arXiv

  32. [40]

    Enhancing chat language models by scaling high-quality instructional conversations.arXiv preprint arXiv:2305.14233, 2023

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations.arXiv preprint arXiv:2305.14233, 2023

    2023 arXiv

  33. [41]

    NuminaMath: A large-scale math reasoning dataset.https://huggingface.co/datasets/AI-MO/NuminaMath-CoT, 2024

    Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. NuminaMath: A large-scale math reasoning dataset.htt...

    2024

  34. [42]

    Think you have solved question answering? try ARC, the AI2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try ARC, the AI2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  35. [43]

    BoolQ: Exploring the surprising difficulty of natural Yes/No questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the surprising difficulty of natural Yes/No questions. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computatio...

    2019

  36. [44]

    C-Eval: A multi-level multi-discipline chinese evaluation suite for foundation models

    Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. C-Eval: A multi-level multi-discipline chinese evaluation suite for foundation models. InAdvances in Neural I...

    2023

  37. [45]

    HellaSwag: Can a machine really finish your sentence? InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, 2019

    2019

  38. [46]

    The LAMBADA dataset: Word prediction requiring a broad discourse context

    Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc-Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring a broad discourse context. InProceedings of the 54th Annual Meeting of th...

    2016

  39. [47]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391, 2018

    2018

  40. [48]

    PIQA: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. PIQA: Reasoning about physical commonsense in natural language. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7432–7439, 2020

    2020

  41. [49]

    SocialIQA: Commonsense reasoning about social interactions

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. SocialIQA: Commonsense reasoning about social interactions. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural La...

    2019

  42. [50]

    WinoGrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. WinoGrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    2021

  43. [51]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023. 13 ReQuant: Fixed-Grid ...

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.