Pith. sign in

REVIEW 4 major objections 5 minor 57 references

The paper claims that activation scale factors for quantized speech models can be chosen by minimizing the task loss with an evolution strategy, achieving lossless INT8 and near-lossless INT4 quantization across multiple speech tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:32 UTC pith:RSB5477T

load-bearing objection The ESC method is a genuine addition to speech-model quantization, but the paper's central 'near-lossless INT4' claim is contradicted by its own Table 1 and needs a major rewrite before publication. the 4 major comments →

arxiv 2603.08173 v2 pith:RSB5477T submitted 2026-03-09 cs.SD cs.AI

Evolution Strategy-Based Calibration for Low-Bit Quantization of Speech Models

Lucas Rakotoarivony This is my paper
classification cs.SD cs.AI
keywords quantizationcalibrationevolution strategyCMA-ESspeech modelsINT4activation scalingpost-training quantization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Speech models, unlike vision or text models, produce activations spread over huge dynamic ranges, so the usual ways of picking quantization clipping ranges waste most of the 4-bit or 8-bit levels. The paper's answer is to stop inferring activation scales from the activation distribution and instead choose them by directly minimizing the task loss on a hundred training samples. The proposed method, ESC, first fits each layer's scale locally by matching its FP32 and quantized outputs, then refines all scales jointly with the CMA-ES evolution strategy. The paper reports that this keeps five speech models—recognition, speaker, enhancement, text-to-speech, and classification—at full-precision accuracy under INT8 and near full-precision accuracy under INT4, claiming the first near-lossless full-INT4 result across multiple speech tasks. Combined with existing post-training quantization techniques, ESC cuts the accuracy gap further, e.g., to about 1% relative accuracy loss on a classification model.

Core claim

On its own terms, the paper establishes that activation calibration, not weight quantization, is the bottleneck for low-bit speech models, and that this bottleneck is a solvable optimization problem. ESC defines the quantization scales as decision variables S and minimizes the task-level error E(f_q(x;S), y) using a two-step local-global procedure. After an MSE-based per-layer initialization, CMA-ES searches the scale space with a budget of 100 evaluations; the mean of the final search distribution is used as the deployed scaling. The reported consequence is lossless INT8 and near-lossless INT4 quantization across five speech tasks, with an average 2.31× inference speedup at INT8.

What carries the argument

The central object is the vector of per-layer activation scale factors S={s_1,...,s_N}; the identity at the heart of the method is S* = argmin_S E(f_q(x;S), y), treating calibration as a continuous black-box optimization rather than a histogram-clipping rule. The mechanism carrying the argument is the two-step scheme: (1) local MSE minimization of each integer layer's output error against its FP32 counterpart gives a stable start; (2) global CMA-ES refinement accounts for cross-layer effects that local fits miss. Using the mean of CMA-ES's final sampling distribution, rather than the best candidate, acts as a variance-reduction step on a noisy objective.

Load-bearing premise

Everything hinges on the idea that 100 training samples and 100 CMA-ES evaluations are enough to fix the activation scales so they generalize to the official test sets across all five architectures; the paper reports no sensitivity analysis, no repeated runs, and no loss curves comparing calibration and test behavior.

What would settle it

Take one architecture, e.g., the speech recognizer, and re-run ESC with n = 25, 50, 200, 500 calibration samples and Γ = 50, 200, 500, recording WER each time. If the near-lossless INT4 results drift by more than the reported margins across these settings—or if independent CMA-ES seeds give materially different scales—then the reported numbers reflect a specific calibration-set draw rather than a generally working method.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Fully integer INT8 speech models can be deployed without accuracy loss, giving 1.34×–5.07× latency improvements and roughly 2–4× smaller footprints.
  • Full INT4 speech models become usable: WER, EER, and accuracy move only a few percent, which opens the door to on-device ASR and speaker verification.
  • ESC is complementary to PTQ methods: pairing it with NoisyQuant, BC, SmoothQuant, or HyQ improves INT4 results further, so the method does not compete with weight-optimization schemes.
  • Quantization can improve particular speech tasks—PESQ for enhancement rises 18% relative to FP32 in INT4—so calibration choices can act as an implicit regularizer.
  • Because only 100 calibration samples are needed, high-quality low-bit quantization no longer requires access to large training sets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct transfer test: if the mechanism is task-loss minimization rather than activation statistics, ESC should also protect vision and NLP models with heavy-tailed activations (e.g., transformer outliers), a setting the paper does not test.
  • The n=100, Γ=100 budget implies a scaling law the paper leaves open: results may either improve markedly with more samples or already sit at a plateau; measuring that curve would separate a robust effect from a lucky calibration set.
  • The same black-box formulation could be extended beyond per-layer scales to per-channel or per-token ranges, or to co-searching weight and activation scales, since the objective and search algorithm make no distributional assumptions.
  • The MP-SENet PESQ improvement raises a testable possibility: aggressive INT4 quantization may filter out minor learned noise and stabilize enhancement outputs; checking across noise conditions could make INT4 the preferred deployment format for such tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ESC (Evolution Strategy-Based Calibration), a post-training calibration method for low-bit quantization of speech models. Activation scale factors are first initialized layer-wisely by minimizing the MSE between FP32 and quantized layer outputs, then jointly refined with CMA-ES on a task-loss objective evaluated on 100 calibration samples. The method is evaluated on five speech tasks (ASR, speaker verification, speech enhancement, TTS, audio classification) under full INT8 and INT4 weight/activation quantization, and is also combined with several existing PTQ methods. The authors claim unaltered performance under INT8 quantization and 'near-lossless' INT4 quantization across multiple speech tasks.

Significance. If the claims were fully supported, the paper would provide a practical, architecture-agnostic calibration scheme for a genuinely underexplored setting — low-bit activation quantization for speech models. The scope of the empirical study is a clear strength: five standard tasks, official test splits, and integration with six PTQ methods. The INT8 results do support the 'lossless' claim: ESC tracks full precision closely on all five models. The method also appears to be consistently better than the calibration baselines in the INT4 setting. However, the advertised headline — 'near-lossless performance for full INT4 quantization across multiple speech tasks' — is not supported by the reported numbers, and the absence of repeated runs, sensitivity analyses, or convergence evidence makes the robustness of the improvement uncertain. The paper is a solid candidate after substantial revision and careful re-framing of its claims.

major comments (4)
  1. [Abstract; Table 1; Section 5] The central claim of near-lossless INT4 quantization is contradicted by Table 1. For Conformer, ESC INT4 gives WER 38.49 vs. FP32 15.94 (141% relative degradation); for ECAPA, EER 11.28 vs. 0.97 (≈10.6× increase); for FastSpeech 2, Mel loss 98.34 vs. 40.18 (145% relative increase). Only MP-SENet and AST are close to FP32. Table 2 shows the same pattern when ESC is combined with PTQ methods. The abstract, Section 1, and Section 5 should either restrict the near-lossless claim to INT8 and to the two tasks where INT4 is indeed near-lossless, or explicitly define 'near-lossless' relative to a quantized baseline rather than to full precision.
  2. [§4.1; Eq. (3)] All results are single optimization runs with no error bars, no seeds, and no re-sampling of the calibration set. Eq. (3) minimizes the task loss on n=100 calibration samples with a CMA-ES budget of Γ=100, so the optimized scales are selected for those 100 samples. The paper needs to show that this generalizes to the official test splits: report mean/std over at least 3 seeds and a sensitivity analysis with respect to n and Γ. Without this, the near-lossless INT4 numbers may be overfit to the calibration set, and the INT8 improvements over baselines (e.g., Conformer WER 16.01 vs. 16.09) are within what could be noise.
  3. [§4.2] The Percentile baseline is evaluated with three thresholds (99.99, 99.999, 99.9999) and 'the best result' is reported for simplicity. If the best threshold is selected using the official test metrics, this is an oracle selection that favors the baseline; at minimum it makes the comparison opaque. Please report all three thresholds or specify that the choice is made on a held-out validation subset, and include the selected percentile per task. The same concern applies to any hyperparameter choices for the PTQ baselines in Table 2.
  4. [§3.3; §4.1] The core algorithmic choices — CMA-ES step size σ=0.1, evaluation budget Γ=100, and use of the final distribution mean instead of the best candidate — are not justified or ablated. Since the paper's contribution is the evolution-strategy optimization, the reader needs at least one convergence/ablation plot per model (task loss vs. number of evaluations) and the wall-clock calibration cost. As written, it is unclear whether the reported gains come from the global optimization stage or essentially from the local MSE initialization described in §3.2.
minor comments (5)
  1. [Figure 2] The figure caption contains an apparent placeholder '{OPEN}' before 'Calibration data'; this should be removed.
  2. [Abstract; Table 2] The abstract states 'achieving a 1% relative accuracy degradation on the AST model,' but Table 2 reports 96.76% vs. 98.13% FP32 accuracy for HyQ+ESC (1.40% relative) and 96.41% for ESC alone (1.75% relative). Please state the exact setting and calculation.
  3. [References] Reference [44] points to a generic TensorRT GitHub page, not to the specific pytorch_quantization implementation used. Please cite the exact repository and version.
  4. [Table 2] The caption does not state which values are bold or how best results are determined across methods; add a note about statistical significance or at least multiple seeds.
  5. [Eq. (1)] The definition of the Int function is typographically unclear; specify whether it is round-to-nearest or floor, and define the zero-point Z consistently with the symmetric case α=−β.

Circularity Check

0 steps flagged

No significant circularity: ESC is an empirical calibration scheme evaluated on held-out test splits; the paper contains no equation-level reduction and no load-bearing self-citations.

full rationale

The paper's derivation chain is empirical rather than definitional. ESC optimizes per-layer activation scale factors S via Eq. (3), S* = arg min_S E(f_q(x;S), y), using n=100 training samples, then evaluates the resulting quantized models on the official test splits of LibriSpeech, VoxCeleb, VoiceBank-DEMAND, LJSpeech, and Speech Commands V2, as stated in Section 4.1. The calibration set and the evaluation set are disjoint, so the reported test metrics are held out and not forced by the optimization objective. The fact that the calibration objective is task-loss-like is normal for calibration and does not constitute a reduction of the reported results to the fitted inputs. There are no self-citations by the sole author in the reference list, and the cited prior methods (MSE initialization, CMA-ES, PTQ baselines) are used as tools or baselines rather than as unverified load-bearing premises. The abstract's 'near-lossless' claim is difficult to reconcile with the large degradations in Table 1 for Conformer, ECAPA, and FastSpeech 2, but that is an internal consistency/correctness concern, not circularity. No Eq. X = Eq. Y by construction, fitted parameter renamed as prediction, or self-citation chain was found.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

ESC does not postulate new physical or architectural entities; it optimizes existing per-layer activation scale factors. The main extra assumptions are about the representativeness of the 100-sample calibration set, the sufficiency of a 100-evaluation CMA-ES budget, and the choice to leave non-target operators under Max calibration. These are empirical modeling choices rather than new entities.

free parameters (5)
  • CMA-ES initial step size σ = 0.1
    Set in §4.1; controls the exploration radius; no sensitivity analysis or tuning procedure is reported.
  • CMA-ES evaluation budget Γ = 100
    Set in §4.1; the paper does not show that 100 evaluations is sufficient for high-dimensional scale vectors.
  • Calibration sample count n = 100
    Set in §4.1; no evidence that 100 training samples are representative across tasks.
  • Activation scale vector S (per-layer) = not reported per layer
    The central fitted quantities, optimized via Eq. (3); per-layer values and statistics are not shipped.
  • Percentile level for Percentile baseline = best of 99.99, 99.999, 99.9999
    §4.2 reports the best result for the baseline; this is favorable to the baseline and not counted in its cost.
axioms (6)
  • domain assumption Uniform symmetric quantization (Eqs. 1-2) is an adequate model for activations and weights.
    Section 3.1 adopts the scheme from [23]; if the uniform symmetric model is unrepresentative for some operators, the results would not hold.
  • domain assumption Audio activations have unusually large dynamic ranges compared with vision/NLP, making standard calibration lossy.
    Figure 1 illustrates this for Conformer/ResNet/BERT, but it is not formally established beyond these examples.
  • domain assumption Task loss on 100 calibration samples is a faithful proxy for test performance.
    Eq. (3) and §4.1; no overfitting check or calibration-vs-test loss curves are provided.
  • ad hoc to paper CMA-ES with Γ=100 evaluations can optimize the per-layer scale vector well enough.
    §3.3-§4.1; no convergence analysis or evidence for high-dimensional robustness is given.
  • domain assumption Applying quantization only to Conv, Linear, and LayerNorm activations with Max calibration elsewhere is sufficient.
    §4.1; other operators are not optimized by ESC.
  • domain assumption MSE-based local initialization [21] provides a good starting basin for the global optimization.
    §3.2; no ablation replacing the initialization is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 9257 in / 12682 out tokens · 114254 ms · 2026-08-02T18:32:51.035310+00:00 · methodology

0 comments
read the original abstract

Quantization has become essential for the efficient deployment of speech processing systems. Although widely studied, most existing quantization methods were developed for vision and NLP architectures, while the specific challenges of audio signals remain largely overlooked. In particular, we show that audio activations can exhibit large calibration ranges, leading to significant information loss when standard calibration techniques are applied. To address this, we propose ESC, an Evolution Strategy-based Calibration method that formulates activation scaling as an optimization problem and solves it using a two-step local-global scheme driven by an evolution strategy. ESC enables unaltered performance under full INT8 quantization and is the first calibration method to achieve near-lossless performance for full INT4 quantization across multiple speech tasks. Integrating ESC with PTQ methods further reduces performance loss, achieving a 1% relative accuracy degradation on the AST model.

Figures

Figures reproduced from arXiv: 2603.08173 by Lucas Rakotoarivony.

Figure 1
Figure 1. Figure 1: Illustration of quantization behavior across audio (Conformer [3]), vision (ResNet [16]), and NLP (BERT [17]) models. Left: Cumulative distribution of normalized activa￾tion values, showing an approximately uniform distribution for ResNet, a rapidly saturating distribution for BERT, and a highly compressed distribution for Conformer. Right: Relative per￾formance under weight and activation quantization usi… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed ESC method. First, each layer-wise activation scaling factor is locally optimized by min￾imizing the MSE between the FP32 and quantized layer outputs. Then, all scaling factors are jointly refined using the CMA￾ES algorithm to minimizes the task-specific error between the quantized model output yˆ and the target y. Consequently, defining the scaling factor requires first deter￾mini… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 20 linked inside Pith

  1. [1]

    However, deploying these models in real-world scenarios with limited memory and computational resources typically requires quantization into hardware-friendly integer formats

    Introduction Modern speech models have achieved near human-level per- formance on many tasks thanks to large-scale pretraining on massive datasets [1, 2] and advanced architectures like transformers-based models [3, 4]. However, deploying these models in real-world scenarios with limited memory and computational resources typically requires quantization i...

    Pith/arXiv arXiv 2026

  2. [2]

    Quantization Quantization enables deployment of neural networks in resource-constrained settings by reducing memory and compu- tational requirements [5]

    Related Work 2.1. Quantization Quantization enables deployment of neural networks in resource-constrained settings by reducing memory and compu- tational requirements [5]. It maps floating-point values to dis- crete integers while aiming to preserve model accuracy. A key challenge, called calibration, is the selection of the scaling fac- tor, which is det...

  3. [3]

    Methods 3.1. Quantization Formulation As proposed in [23], we employ the widely used quantization scheme defined as follows: Q(r) =Int r/s −Z,(1) Here,Qdenotes the quantization operator,ris a real-valued input (either an activation or a weight),sis a real-valued scal- ing factor, andZis an integer zero point. The Int function maps a real value to an integ...

  4. [4]

    MSE(sN) MSE(s2) MSE(s1) Output: ŷ s1, s2,…, sN Parameters Objective function 𝓔(y,ŷ) CMA-ES

    Local optimization FP32 Model INT Model Layer N Layer 2 Layer 1 . . . MSE(sN) MSE(s2) MSE(s1) Output: ŷ s1, s2,…, sN Parameters Objective function 𝓔(y,ŷ) CMA-ES

  5. [5]

    First, each layer-wise activation scaling factor is locally optimized by min- imizing the MSE between the FP32 and quantized layer outputs

    Global optimization Figure 2:Overview of the proposed ESC method. First, each layer-wise activation scaling factor is locally optimized by min- imizing the MSE between the FP32 and quantized layer outputs. Then, all scaling factors are jointly refined using the CMA- ES algorithm to minimizes the task-specific error between the quantized model outputˆyand ...

    arXiv

  6. [6]

    Experiments 4.1. Experimental Setup To ensure a representative evaluation across speech processing domains, we conduct experiments on five widely used speech- based tasks: speech recognition, speaker recognition, speech enhancement, text-to-speech, and audio classification. For each task, the corresponding model is evaluated on the official test split of ...

    arXiv

  7. [7]

    Brecq: Pushing the limit of post-training quantization by block reconstruction,

    Y . Li, R. Gong, X. Tan, Y . Yang, P. Hu, Q. Zhang, F. Yu, W. Wang, and S. Gu, “Brecq: Pushing the limit of post-training quantization by block reconstruction,”arXiv preprint arXiv:2102.05426, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    Our study highlights that, unlike in vision or NLP, audio models are particularly sensitive to activation quan- tization, providing strong motivation for our approach

    Conclusion In this paper, we proposed a novel two-stage calibration scheme that combines local MSE-based optimization with a global evolutionary strategy to optimize activation scaling factors in speech models. Our study highlights that, unlike in vision or NLP, audio models are particularly sensitive to activation quan- tization, providing strong motivat...

  9. [9]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  10. [10]

    wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,”Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020

    2020

  11. [11]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wuet al., “Conformer: Convolution- augmented transformer for speech recognition,”arXiv preprint arXiv:2005.08100, 2020

    Pith/arXiv arXiv 2005

  12. [12]

    Ast: Audio spectrogram transformer,

    Y . Gong, Y .-A. Chung, and J. Glass, “Ast: Audio spectrogram transformer,”arXiv preprint arXiv:2104.01778, 2021

    Pith/arXiv arXiv 2021

  13. [13]

    Model compression and efficient infer- ence for large language models: A survey,

    W. Wang, W. Chen, Y . Luo, Y . Long, Z. Lin, L. Zhang, B. Lin, D. Cai, and X. He, “Model compression and efficient infer- ence for large language models: A survey,”arXiv preprint arXiv:2402.09748, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Up or down? adaptive rounding for post-training quantization,

    M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort, “Up or down? adaptive rounding for post-training quantization,” inInternational conference on machine learning. PMLR, 2020, pp. 7197–7206

    2020

  15. [15]

    Whisper-kdq: A lightweight whisper via guided knowledge dis- tillation and quantization for efficient asr,

    H. Shao, W. Wang, B. Liu, X. Gong, H. Wang, and Y . Qian, “Whisper-kdq: A lightweight whisper via guided knowledge dis- tillation and quantization for efficient asr,”CoRR, 2023

    2023

  16. [16]

    Smoothquant: Accurate and efficient post-training quantization for large language models,

    G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han, “Smoothquant: Accurate and efficient post-training quantization for large language models,” inInternational conference on ma- chine learning. PMLR, 2023, pp. 38 087–38 099

    2023

  17. [17]

    Gptq: Ac- curate post-training quantization for generative pre-trained trans- formers,

    E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “Gptq: Ac- curate post-training quantization for generative pre-trained trans- formers,”arXiv preprint arXiv:2210.17323, 2022

    Pith/arXiv arXiv 2022

  18. [18]

    Towards one-bit asr: Extremely low-bit con- former quantization using co-training and stochastic precision,

    Z. Li, H. Xu, Z. Jin, L. Meng, T. Wang, H. Wang, Y . Chen, M. Cui, S. Hu, and X. Liu, “Towards one-bit asr: Extremely low-bit con- former quantization using co-training and stochastic precision,” arXiv preprint arXiv:2505.21245, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Bittts: Highly compact text-to-speech using 1.58-bit quanti- zation and weight indexing,

    M. Kawamura, T. Hasumi, Y . Shirahata, and R. Yamamoto, “Bittts: Highly compact text-to-speech using 1.58-bit quanti- zation and weight indexing,”arXiv preprint arXiv:2506.03515, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2704–2713

    2018

  21. [21]

    Outlier reduction with gated attention for improved post-training quanti- zation in large sequence-to-sequence speech foundation models,

    D. Wagner, I. Baumann, K. Riedhammer, and T. Bocklet, “Outlier reduction with gated attention for improved post-training quanti- zation in large sequence-to-sequence speech foundation models,” arXiv preprint arXiv:2406.11022, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Ultra-low bit post-training quantization of large speech models via k-means clustering and mixed precision allocation,

    T. Gu, B. Liu, H. Wang, and Y . Qian, “Ultra-low bit post-training quantization of large speech models via k-means clustering and mixed precision allocation,” inProc. Interspeech 2025, 2025, pp. 1988–1992

    2025

  23. [23]

    A survey of quantization methods for efficient neural network inference,

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neural network inference,” inLow-power computer vision. Chapman and Hall/CRC, 2022, pp. 291–326

    2022

  24. [24]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  25. [25]

    Bert: Pre- training of deep bidirectional transformers for language under- standing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language under- standing,” inProceedings of the 2019 conference of the North American chapter of the association for computational linguis- tics: human language technologies, volume 1 (long and short pa- pers), 2019, pp. 4171–4186

    2019

  26. [26]

    Improving the speed of neural networks on cpus,

    V . Vanhoucke, A. Senior, M. Z. Maoet al., “Improving the speed of neural networks on cpus,” inProc. deep learning and unsu- pervised feature learning NIPS workshop, vol. 1, no. 2011, 2011, p. 4

    2011

  27. [27]

    Discovering low-precision networks close to full-precision networks for efficient embedded inference,

    J. L. McKinstry, S. K. Esser, R. Appuswamy, D. Bablani, J. V . Arthur, I. B. Yildiz, and D. S. Modha, “Discovering low-precision networks close to full-precision networks for efficient embedded inference,”arXiv preprint arXiv:1809.04191, 2018

    Pith/arXiv arXiv 2018

  28. [28]

    Nvidia 8-bit inference with tensorrt,

    S. Migacz, “Nvidia 8-bit inference with tensorrt,”GPU Technol- ogy Conference, 2017

    2017

  29. [29]

    Low-bit quantization of neural networks for efficient inference,

    Y . Choukroun, E. Kravchik, F. Yang, and P. Kisilev, “Low-bit quantization of neural networks for efficient inference,” in2019 IEEE/CVF International Conference on Computer Vision Work- shop (ICCVW). IEEE, 2019, pp. 3009–3018

    2019

  30. [30]

    Evolutionsstrategien,

    I. Rechenberg, “Evolutionsstrategien,” inSimulationsmethoden in der Medizin und Biologie: Workshop, Hannover, 29. Sept.–1. Okt

  31. [31]

    Natural evolution strategies,

    D. Wierstra, T. Schaul, T. Glasmachers, Y . Sun, J. Peters, and J. Schmidhuber, “Natural evolution strategies,”The Journal of Machine Learning Research, vol. 15, no. 1, pp. 949–980, 2014

    2014

  32. [32]

    Evolu- tion strategies as a scalable alternative to reinforcement learning,

    T. Salimans, J. Ho, X. Chen, S. Sidor, and I. Sutskever, “Evolu- tion strategies as a scalable alternative to reinforcement learning,” arXiv preprint arXiv:1703.03864, 2017

    Pith/arXiv arXiv 2017

  33. [33]

    Noisyquant: Noisy bias-enhanced post-training activation quan- tization for vision transformers,

    Y . Liu, H. Yang, Z. Dong, K. Keutzer, L. Du, and S. Zhang, “Noisyquant: Noisy bias-enhanced post-training activation quan- tization for vision transformers,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 20 321–20 330

    2023

  34. [34]

    Hyq: Hardware-friendly post- training quantization for cnn-transformer hybrid networks,

    N. J. Kim, J. Lee, and H. Kim, “Hyq: Hardware-friendly post- training quantization for cnn-transformer hybrid networks,” in Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24. International Joint Conferences on Artificial Intelligence Organization, vol. 8, 2024, pp. 4291– 4299

    2024

  35. [35]

    Ditas: Quantizing diffusion trans- formers via enhanced activation smoothing,

    Z. Dong and S. Q. Zhang, “Ditas: Quantizing diffusion trans- formers via enhanced activation smoothing,” in2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025, pp. 4606–4615

    2025

  36. [36]

    Minimize quantization output error with bias compensa- tion,

    C. Gong, H. Zheng, M. Hu, Z. Lin, D.-P. Fan, Y . Zhang, and T. Li, “Minimize quantization output error with bias compensa- tion,”arXiv preprint arXiv:2404.01892, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    2019 evolutionary algorithms re- view,

    A. N. Sloss and S. Gustafson, “2019 evolutionary algorithms re- view,”arXiv preprint arXiv:1906.08870, 2019

    Pith/arXiv arXiv 2019

  38. [38]

    The cma evolution strategy: a comparing review,

    N. Hansen, “The cma evolution strategy: a comparing review,” Towards a new evolutionary computation: Advances in the esti- mation of distribution algorithms, pp. 75–102, 2006

    2006

  39. [39]

    High dimensions and heavy tails for natural evolution strategies,

    T. Schaul, T. Glasmachers, and J. Schmidhuber, “High dimensions and heavy tails for natural evolution strategies,” inProceedings of the 13th annual conference on Genetic and evolutionary compu- tation, 2011, pp. 845–852

    2011

  40. [40]

    Inves- tigating rnn-based speech enhancement methods for noise-robust text-to-speech,

    C. V . Botinhao, X. Wang, S. Takaki, and J. Yamagishi, “Inves- tigating rnn-based speech enhancement methods for noise-robust text-to-speech,” in9th ISCA speech synthesis workshop, 2016, pp. 159–165

    2016

  41. [41]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020

    Pith/arXiv arXiv 2006

  42. [42]

    Gdrq: Group-based distribution reshaping for quantization,

    H. Yu, T. Wen, G. Cheng, J. Sun, Q. Han, and J. Shi, “Gdrq: Group-based distribution reshaping for quantization,” arXiv preprint arXiv:1908.01477, 2019

    Pith/arXiv arXiv 1908

  43. [43]

    Squashed weight distribution for low bit quantization of deep models,

    N. Str ¨om, H. Khan, and W. Hamza, “Squashed weight distribution for low bit quantization of deep models,” 2022

    2022

  44. [44]

    Cma-es for hyperparame- ter optimization of deep neural networks,

    I. Loshchilov and F. Hutter, “Cma-es for hyperparame- ter optimization of deep neural networks,”arXiv preprint arXiv:1604.07269, 2016

    Pith/arXiv arXiv 2016

  45. [45]

    tracing and deployed with TensorRT [46]. Although this GPU is chosen for its mature software ecosystem [45, 46] and reliable deployment tools, our quantization and export strategy is not limited to GPUs and can be extended to other hardware platforms, including embedded AI processors [47, 48]. Since Tensor Cores accelerate INT8 operations for both weights...

  46. [46]

    Lib- rispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an asr corpus based on public domain audio books,” in2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210

    2015

  47. [47]

    Ecapa- tdnn: Emphasized channel attention, propagation and ag- gregation in tdnn based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa- tdnn: Emphasized channel attention, propagation and ag- gregation in tdnn based speaker verification,”arXiv preprint arXiv:2005.07143, 2020

    Pith/arXiv arXiv 2005

  48. [48]

    V oxceleb2: Deep speaker recognition,

    J. S. Chung, A. Nagrani, and A. Zisserman, “V oxceleb2: Deep speaker recognition,”arXiv preprint arXiv:1806.05622, 2018

    Pith/arXiv arXiv 2018

  49. [49]

    Mp-senet: A speech enhance- ment model with parallel denoising of magnitude and phase spec- tra,

    Y .-X. Lu, Y . Ai, and Z.-H. Ling, “Mp-senet: A speech enhance- ment model with parallel denoising of magnitude and phase spec- tra,”arXiv preprint arXiv:2305.13686, 2023

    Pith/arXiv arXiv 2023

  50. [51]

    The lj speech dataset,

    K. Ito and L. Johnson, “The lj speech dataset,” https://keithito. com/LJ-Speech-Dataset/, 2017

    2017

  51. [52]

    Speech commands: A dataset for limited-vocabulary speech recognition,

    P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,”arXiv preprint arXiv:1804.03209, 2018

    Pith/arXiv arXiv 2018

  52. [53]

    pytorch quantization: http://github.com/nvidia/tensorrt

    NVIDIA. pytorch quantization: http://github.com/nvidia/tensorrt

  53. [54]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antigaet al., “Pytorch: An imperative style, high-performance deep learning library,”Ad- vances in neural information processing systems, vol. 32, 2019

    2019

  54. [55]

    TensorRT: https://developer.nvidia.com/tensorrt

    NVIDIA. TensorRT: https://developer.nvidia.com/tensorrt

  55. [56]

    11.3 metis aipu: A 12nm 15tops/w 209.6 tops soc for cost-and energy-efficient inference at the edge,

    P. A. Hager, B. Moons, S. Cosemans, I. A. Papistas, B. Roose- leer, J. Van Loon, R. Uytterhoeven, F. Zaruba, S. Koumousi, M. Stanisavljevicet al., “11.3 metis aipu: A 12nm 15tops/w 209.6 tops soc for cost-and energy-efficient inference at the edge,” in2024 IEEE International Solid-State Circuits Confer- ence (ISSCC), vol. 67. IEEE, 2024, pp. 212–214

    2024

  56. [57]

    Survey of machine learning accelerators,

    A. Reuther, P. Michaleas, M. Jones, V . Gadepally, S. Samsi, and J. Kepner, “Survey of machine learning accelerators,” in2020 IEEE high performance extreme computing conference (HPEC). IEEE, 2020, pp. 1–12

    2020

  57. [1977]

    Springer, 1978, pp. 83–114

    1978