REVIEW 4 major objections 4 minor 26 references
StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models
T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read A layer-by-layer calibration method lets 8-bit speech foundation models keep ASR accuracy within 0.3% word error rate while cutting model size to about a quarter.
desk verdict Useful empirical recipe for PTQ of speech foundation models with a believable W8A8 result; the isolated per-layer sensitivity selection needs a joint-selection ablation before I'd trust it beyond W8A8. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the two-stage calibration in Algorithms 1 and 2. Stage 1 selects a clipping set S by temporarily quantizing each layer alone with percentile scaling and keeping any layer whose WER increase exceeds gamma = 0.25%. Stage 2, for layers in S, clips the activation histogram at a candidate cut-off percentile p between 0 and 0.5 (grid step 0.01) and computes the scale factor as the bin-center value that minimizes the mean squared error between original clipped bins and their quantized-dequantized reconstructions; layers outside S use the same MSE search without clipping. This separates outlier removal from scale selection, so the quantization range is driven by reconstruction error over the dominant distribution rather than by the full dynamic range.
What would settle it
On the same LibriSpeech dev-clean set, apply the full percentile-clipping and MSE calibration of Stage 2 to every layer, bypassing the Stage-1 selection; if the resulting W8A8 or W6A6 WER is no worse than StableQuant's selected-layer result, then the isolated-sensitivity selection is not responsible for the method's gains.
Extended reading notes
Core claim
The central claim is that the failure of PTQ on speech foundation models comes from the CNN front-end, not from the Transformer stack: the paper measures CNN activation ranges up to about 100 times wider than Transformer activations, with strong layer-to-layer differences in both HuBERT and wav2vec2.0. StableQuant's recipe is to treat layers differently: quantize each layer alone with a naive percentile scale, flag every layer whose WER increase exceeds a 0.25% threshold, and then restrict outlier clipping to those flagged layers, searching the cut-off percentile p in [0, 0.5] at 0.01 steps and selecting the scale that minimizes MSE on the clipped histogram. On the ASR task, this yields 2.35% WER for HuBERT-Large at W8A8 (baseline 2.16%) and 3.07% for wav2vec2.0 (baseline 2.78%), with graceful degradation at W6A6 where conventional percentile, MSE, and entropy calibrations collapse to WER above 42%.
Load-bearing premise
The method rests on the assumption that a layer's sensitivity measured when only that layer is quantized predicts which layers are harmful when all layers are quantized together, even though quantized errors from different layers interact.
Editorial extensions
If this is right
- At W8A8, HuBERT-Large with StableQuant needs 353 MB versus 1205 MB full precision and still beats full-precision HuBERT-Base, which stands at 6.33% WER, with 2.35% WER.
- At W6A6, the same method keeps HuBERT at 3.98% and wav2vec2.0 at 6.10% WER, where percentile, MSE, and entropy calibration each exceed 42%, so 6-bit deployment becomes usable.
- Because the calibration is post-training and uses only the dev set, no student network, distillation, or fine-tuning is needed to compress the models.
- Inference time roughly doubles when both weights and activations are quantized to 8 bits and run through TensorRT on an RTX 3090, across audio lengths from 1 to 600 seconds.
- The same two-stage recipe works for both HuBERT and wav2vec2.0, which use different self-supervised objectives and slightly different front-ends, supporting the claim that the method is not architecture-specific.
Reading between the lines
- An implied next test is to run StableQuant on a third speech foundation model with a different front-end, such as WavLM; if the under-0.3% WER drop does not survive, the 'regardless of architecture' claim would need to be narrowed.
- The paper's Stage-1 sensitivity test quantizes one layer at a time; a natural extension is a joint or greedy subset selection that quantizes several layers together, since quantized errors from different layers may interact and change which layers most need clipping.
- The cut-off percentile and the gamma threshold are tuned on LibriSpeech dev-clean; a robustness check would be whether the same settings transfer to noisy or out-of-domain speech without recalibration.
- Because StableQuant only changes the scale-calibration step, it is complementary to weight-rounding optimizers such as AdaRound; the paper compares against AdaRound but does not test the combination, which may push W6A6 accuracy higher.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes StableQuant, a post-training quantization method for speech foundation models. The method first identifies 'sensitive' layers by quantizing each layer's activations individually at 8 bits and measuring the WER increase relative to an FP32 baseline (Stage 1, Algorithm 1); it then searches a percentile clipping threshold for the selected layers and an MSE-based scale for all layers (Stage 2, Algorithm 2). Experiments on HuBERT-Large and wav2vec2.0 fine-tuned for ASR on LibriSpeech report W8A8 WER of 2.35% and 3.07% versus baselines of 2.16% and 2.78%, model size reduction from 1205 MB to 353 MB for HuBERT-Large, and roughly a 2x inference speedup under TensorRT. The paper concludes that StableQuant achieves less than 0.3% WER degradation at 8-bit quantization for both models.
Significance. If the results are reproducible, the paper addresses an underexplored and practically important problem: post-training quantization for speech foundation models with CNN feature extractors. The empirical demonstration that all-layer quantization is much harder than Transformer-only quantization (Table I) is a useful observation, and the proposed combination of layer selection and percentile clipping is simple and plausible. The authors provide a public code repository link, which aids reproducibility. The main limitations are the lack of validation of the layer-selection assumption, the absence of error bars or sensitivity analyses, and insufficient detail on comparison baselines; these prevent the 'optimal' and '<0.3% drop' claims from being fully established.
major comments (4)
- [III-B, Algorithm 1 (Stage 1)] The selection of the clipping-layer set S is made by quantizing one layer at a time while leaving all other layers in full precision (line 4 sets only Nq(al) to the quantized version). This treats per-layer quantization error as separable and assumes that the layers most sensitive in isolation are also the layers that matter after all weights and activations are quantized to W8A8. In a Transformer-CNN stack with residual connections, quantization errors can interact, so this assumption is not self-evident. The paper does not report an ablation against any joint selection procedure (e.g., greedy forward selection at W8A8, clipping all CNN layers, or leave-one-out at W8A8). Because S directly determines which layers receive percentile clipping in Stage 2, this is load-bearing for the central W8A8 claim; please add such an ablation.
- [III-B, IV-B] The threshold gamma=0.25% and the percentile grid P in [0,0.5] with step 0.01 are selected on the dev-clean set, and all reported WER numbers come from a single run. The headline result is close to the threshold: for wav2vec2.0 the W8A8 WER increase is 0.29%, just under the 0.3% bound, and for HuBERT it is 0.19%, comparable to gamma. A sensitivity analysis over gamma, the calibration set, and the histogram bin count B, or repeated runs with different dev/calibration splits, is needed to establish that the '<0.3%' claim is robust rather than a consequence of hyperparameter selection.
- [IV-B, Table II] The columns 'Percentile', 'MSE', and 'Entropy' are presented as conventional calibration baselines, but the caption says the table applies 'the proposed layer-adaptive calibration strategies.' It is not clear whether these baselines use the same selected layer set S as StableQuant, whether they also perform a cut-off ratio search, or whether they represent standard single-scale calibration applied uniformly to all layers. As written, the comparison conflates the calibration objective with the layer-selection mechanism, which is exactly the proposed contribution. Please specify the exact protocol for each baseline.
- [IV-B, Table III] The AdaRound comparison reports a single WER of 2.66% with no indication of the bit width, whether activation quantization is included, what calibration data were used, or whether the same TensorRT pipeline was applied. The claim that StableQuant 'surpasses the quantized results with AdaRound' therefore cannot be fully evaluated. Please detail the AdaRound setup and, ideally, report both methods under identical W8A8 conditions.
minor comments (4)
- [III-B, Algorithm 2] The histogram H_{D,clip}(X) is not defined as normalized, and the MSE expression in line 10 weights squared errors by histogram counts; please clarify the normalization and state how the bin count B was chosen.
- [Abstract and Conclusion] The abstract and conclusion claim 'optimal quantization performance regardless of the network architecture type,' but the experiments include only two SFMs sharing a CNN+Transformer structure; please temper 'regardless' or add evidence for more architectures.
- [IV-C, Fig. 2] The numeric inference times are not given in the caption or the text; reporting exact milliseconds values would make the 'roughly twofold' speedup claim easier to verify.
- [II-A and Table I] Section II-A begins without a clear lead-in sentence immediately after the heading, and Table I uses the abbreviation 'TR-Layer' without definition; minor editing would improve readability.
Circularity Check
No significant circularity: StableQuant is an empirical PTQ method evaluated on held-out test-clean.
full rationale
StableQuant is an empirical post-training quantization scheme rather than a first-principles derivation. Its adaptive components are Stage 1 layer selection (based on per-layer WER increase under naive quantization, compared to a threshold gamma) and Stage 2 percentile cutoff search over P on the dev-clean set. The final WER results are reported on test-clean after validation on dev-clean, so the held-out evaluation prevents the dev-set tuning from being presented as a prediction. No fitted parameter is renamed as a prediction: popt and the clipping set S are selected on dev-clean, and the paper explicitly states that final ASR evaluation is on test-clean. There are no load-bearing self-citations; the references are external prior works used for comparison rather than to justify the proposed algorithm. The weakest assumption, that isolated per-layer sensitivity predicts joint quantization behavior, is a correctness or robustness concern, not a circularity concern. Therefore no circular steps are present.
Assumptions & free parameters
free parameters (5)
- cut-off percentile p =
0.20 (HuBERT W8A8), 0.16 (W6A6), 0.0 (W4A4), 0.11 (wav2vec2.0 W8A8), 0.21 (W6A6), 0.05 (W4A4)
- sensitivity threshold gamma =
0.25% WER
- histogram bin count B =
Not stated
- candidate grid for p =
51 values from 0.0 to 0.5
- calibration and dev dataset =
LibriSpeech dev-clean
assumptions (4)
- domain assumption Symmetric quantization (z=0) is sufficient for SFMs.
- domain assumption CNN activations have wider dynamic range and more outliers than Transformer activations.
- domain assumption MSE between clipped histograms is a good proxy for task-level WER.
- ad hoc to paper Single-layer quantization degradation with percentile calibration predicts which layers need clipping in the jointly quantized network.
Cite this review
Pith. "Pith review of StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models." pith.science (2026). https://pith.science/paper/UNTW7LYM
@misc{pith2026250414915,
author = {Pith},
title = {Pith review of: StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/UNTW7LYM}},
note = {Machine review of arXiv:2504.14915}
}
read the original abstract
In this paper, we propose StableQuant, a novel adaptive post-training quantization (PTQ) algorithm for widely used speech foundation models (SFMs). While PTQ has been successfully employed for compressing large language models (LLMs) due to its ability to bypass additional fine-tuning, directly applying these techniques to SFMs may not yield optimal results, as SFMs utilize distinct network architecture for feature extraction. StableQuant demonstrates optimal quantization performance regardless of the network architecture type, as it adaptively determines the quantization range for each layer by analyzing both the scale distributions and overall performance. We evaluate our algorithm on two SFMs, HuBERT and wav2vec2.0, for an automatic speech recognition (ASR) task, and achieve superior performance compared to traditional PTQ methods. StableQuant successfully reduces the sizes of SFM models to a quarter and doubles the inference speed while limiting the word error rate (WER) performance drop to less than 0.3% with 8-bit quantization.
Figures
Reference graph
Works this paper leans on
-
[1]
Self-supervised speech representation learning: A review,
Abdelrahman Mohamed, Hung-yi Lee, Lasse Borgholt, Jakob D. Hav- torn, Joakim Edin, Christian Igel, Katrin Kirchhoff, Shang-Wen Li, Karen Livescu, Lars Maaløe, Tara N. Sainath, and Shinji Watanabe, “Self-supervised speech representation learning: A review,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1179–1210, 2022
work page 2022
-
[2]
Superb: Speech processing universal performance benchmark,
Shu wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y . Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da- Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung yi Lee, “Superb: Speech processing universal performance bench...
work page 2021
-
[3]
wav2vec: Unsupervised pre-training for speech recognition,
Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli, “wav2vec: Unsupervised pre-training for speech recognition,” in Inter- speech 2019, 2019, pp. 3465–3469
work page 2019
-
[4]
Investigating self-supervised learning for speech enhance- ment and separation,
Zili Huang, Shinji Watanabe, Shu-wen Yang, Paola Garc ´ıa, and Sanjeev Khudanpur, “Investigating self-supervised learning for speech enhance- ment and separation,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 6837–6841
work page 2022
-
[5]
Speak, read and prompt: High-fidelity text-to-speech with minimal supervision,
Eugene Kharitonov, Damien Vincent, Zal ´an Borsos, Rapha ¨el Marinier, Sertan Girgin, Olivier Pietquin, Matt Sharifi, Marco Tagliasacchi, and Neil Zeghidour, “Speak, read and prompt: High-fidelity text-to-speech with minimal supervision,” Transactions of the Association for Compu- tational Linguistics, vol. 11, pp. 1703–1718, 2023
work page 2023
-
[6]
Soundstream: An end-to-end neural audio codec,
Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 30, pp. 495–507, 2022
work page 2022
-
[7]
Attention is all you need,
A Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017
2017
-
[8]
A survey of quantization methods for efficient neural network inference,
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer, “A survey of quantization methods for efficient neural network inference,” in Low-Power Computer Vision, pp. 291–326. Chapman and Hall/CRC, 2022
work page 2022
Show all 26 references
-
[9]
Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert,
Heng-Jui Chang, Shu-wen Yang, and Hung-yi Lee, “Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022, pp. 7087–7091
2022
-
[10]
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT,
Rui Wang, Qibing Bai, Junyi Ao, Long Zhou, Zhixiang Xiong, Zhihua Wei, Yu Zhang, Tom Ko, and Haizhou Li, “LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT,” in Proc. Interspeech 2022 , 2022, pp. 1686–1690
2022
-
[11]
Structured pruning of self-supervised pre-trained models for speech recognition and understanding,
Yifan Peng, Kwangyoun Kim, Felix Wu, Prashant Sridhar, and Shinji Watanabe, “Structured pruning of self-supervised pre-trained models for speech recognition and understanding,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICAS...
2023
-
[12]
BERT: Pre-training of deep bidirectional transformers for language understanding,
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan...
2019
-
[13]
Post training 4-bit quantization of convolutional networks for rapid-deployment,
Ron Banner, Yury Nahshan, and Daniel Soudry, “Post training 4-bit quantization of convolutional networks for rapid-deployment,” Advances in Neural Information Processing Systems , vol. 32, 2019
2019
-
[14]
Zeroq: A novel zero shot quantization framework,
Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W Mahoney, and Kurt Keutzer, “Zeroq: A novel zero shot quantization framework,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 13169–13178
2020
-
[15]
A white paper on neural network quantization,
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bon- darenko, Mart Van Baalen, and Tijmen Blankevoort, “A white paper on neural network quantization,” arXiv preprint arXiv:2106.08295 , 2021
2021 arXiv
-
[16]
The era of 1-bit llms: All large language models are in 1.58 bits,
Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei, “The era of 1-bit llms: All large language models are in 1.58 bits,” arXiv preprint arXiv:2402.17764, 2024
2024 arXiv
-
[17]
SmoothQuant: Accurate and efficient post-training quantization for large language models,
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han, “SmoothQuant: Accurate and efficient post-training quantization for large language models,” in Proceedings of the 40th International Conference on Machine Learning, Andreas Krause, Emma Brunskill, Ky...
2023
-
[18]
Wavlm: Large-scale self-supervised pre-training for full stack speech processing,
Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Process- ing, v...
2022
-
[19]
wav2vec 2.0: A framework for self-supervised learning of speech representations,
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems , vol. 33, pp. 12449–12460, 2020
2020
-
[20]
Hubert: Self- supervised speech representation learning by masked prediction of hidden units,
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakho- tia, Ruslan Salakhutdinov, and Abdelrahman Mohamed, “Hubert: Self- supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing...
2021
-
[21]
Integer quantization for deep learning inference: Principles and empirical evaluation,
Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Mi- cikevicius, “Integer quantization for deep learning inference: Principles and empirical evaluation,” arXiv preprint arXiv:2004.09602 , 2020
2004 arXiv
-
[22]
Quantizing deep convolutional networks for efficient inference: A whitepaper,
Raghuraman Krishnamoorthi, “Quantizing deep convolutional networks for efficient inference: A whitepaper,” arXiv preprint arXiv:1806.08342, 2018
2018 arXiv
-
[23]
Libri-light: A benchmark for asr with limited or no supervision,
J. Kahn, M. Rivi `ere, W. Zheng, E. Kharitonov, Q. Xu, P. E. Mazar ´e, J. Karadayi, V . Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux, “Libri-light: A benchmark for asr with limited or no supervision,” in ICASSP 2020 -...
2020
-
[24]
Librispeech: an asr corpus based on public domain audio books,
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210
2015
-
[25]
8-bit inference with tensorrt,
Szymon Migacz, “8-bit inference with tensorrt,” in GPU technology conference, 2017, vol. 2, p. 5
2017
-
[26]
Up or down? adaptive rounding for post- training quantization,
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort, “Up or down? adaptive rounding for post- training quantization,” in International Conference on Machine Learn- ing. PMLR, 2020, pp. 7197–7206
2020
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.