Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that co-training 2-bit and 1-bit Conformer speech models with stochastic precision yields word-error rates statistically indistinguishable from full precision at up to 16.6x compression.

desk verdict A well-ablated recipe that genuinely reaches lossless 1-bit Switchboard, but the 'lossless' headline outruns the starred cells on LibriSpeech and the 1.5-bit majority rule. read the letter →

arxiv 2505.21245 v1 pith:QPL2JMGQ submitted 2025-05-27 cs.SD eess.AS

Zhaoqing Li , Haoning Xu , Zengrui Jin , Lingwei Meng , Tianzi Wang , Huimeng Wang , Youjun Chen , Mingyu Cui
show 2 more authors
Shujie Hu Xunying Liu
This is my paper
classification cs.SDeess.AS
keywords low-bitquantizationConformerASRquantization-awaretrainingco-trainingstochasticprecisionlearnablescalingfactormodelcompressionworderrorrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that Conformer automatic speech recognition systems, the standard encoder-decoder architecture with feed-forward, attention, and convolution modules, can be compressed to 2-bit and 1-bit weights without a statistically significant rise in word error rate (WER). That goal matters because earlier attempts at 2-bit or 1-bit ASR quantization reported large WER increases, and because lower bit widths directly shrink memory and enable simpler arithmetic at deployment. The proposed recipe trains a 2-bit model and a 1-bit model at the same time with shared weights, adds a KL-divergence term so the 2-bit model guides the 1-bit one, and inserts a stochastic-precision sub-model that randomly binarizes layers during training. On the 300-hour Switchboard and 960-hour LibriSpeech corpora, the paper reports weight compression of up to 16.2x and 16.6x while the WER increase stays under the paper's significance threshold.

What carries the argument

The load-bearing object is the quantization co-training framework: one set of shared weights simultaneously realizes a 2-bit model, a 1-bit model obtained by dropping to the binary quantization table $Q_1=\alpha\{-1,+1\}$, and a stochastic-precision model formed each iteration by randomly binarizing a subset of encoder layers. The 2-bit model acts as a teacher whose logits constrain the 1-bit and stochastic-precision students via a stop-gradient KL-divergence term, keeping the teacher-student gap small enough that knowledge transfer works at 1 bit. A tensor-wise learnable scaling factor $\alpha$ in $\hat{W}=\alpha\,\Pi_{Q_n}(\mathrm{clip}(W/\alpha,\cdot))$ absorbs much of the quantization error with negligible extra parameters, and the Straight-Through Estimator provides the gradient path through the rounding operation.

What would settle it

Run the 1-bit LibriSpeech system on the full test-clean and dev-clean sets and apply the same significance test to all four LibriSpeech sets jointly; if WER on any set rises significantly, the claimed 16.6x lossless compression is not supported. Likewise, evaluating all three sampled 1.5-bit Switchboard systems and requiring all to pass would test the 16.2x claim.

Watch

Extended reading notes

Core claim

The central claim is that performance-lossless extremely low-bit quantization of Conformer ASR is attainable by training a family of precisions together instead of compressing a single model. In the authors' framework, a 2-bit Conformer, a 1-bit Conformer, and a stochastically sampled partially binarized model share the same underlying weights; the 2-bit model's output logits regularize the lower-precision students through a stop-gradient KL-divergence term. Tensor-wise learnable scaling factors set the quantization range, and the Straight-Through Estimator routes gradients through the non-differentiable quantization step. With these components, the paper reports a lossless 2-bit encoder and a lossless 1-bit encoder on Switchboard, a 16.2x compression from a 1.5-bit mixed-precision encoder, and a 16.6x compression from a 1-bit encoder with 4-bit decoder and 8-bit CNN on LibriSpeech, at the cost of only 204 extra scaling parameters.

Load-bearing premise

The losslessness claim rests on how 'no statistically significant WER increase' is counted: the test is applied separately to each test set, some sets are left unstarred, and a 1.5-bit Switchboard system counts as lossless if at least two of three sampled systems pass.

Editorial extensions

If this is right

  • On Switchboard, a 2-bit encoder with 8-bit CNN and 4-bit decoder is reported lossless at 13.1x compression, and a 2-bit encoder with 4-bit CNN at 14.9x.
  • On Switchboard, a 1.5-bit encoder with 8-bit CNN reaches 16.2x compression and is counted lossless when at least two of three sampled systems pass the significance test.
  • A 1-bit encoder with 4-bit CNN reaches 26.2x compression on Switchboard with a 2.6% relative WER increase, and its WER is comparable to earlier fully 2-bit Conformer systems.
  • On LibriSpeech, the 2-bit system is lossless on test-clean, test-other, and dev-other at 12.2x compression, while the 1-bit system is lossless on test-other and dev-other at 16.6x.
  • Because 1-bit and 2-bit systems share weights, a deployed 2-bit model contains the 1-bit model for free, and the framework can emit multiple bit-width configurations from one training run.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the losslessness results survive a multiple-testing correction across all held-out sets, extremely low-bit quantization becomes a practical default for on-device ASR rather than a research benchmark.
  • Beyond the paper: the stochastic-precision schedule, which binarizes deeper layers with higher probability, resembles a curriculum; varying that schedule is a natural lever to trade a few percent of WER for additional compression beyond the fixed schedule tested.
  • Beyond the paper: the co-training idea of treating a degraded model as a weight-sharing sub-model should transfer to other compression axes, such as width or depth pruning, and to quantizing activations rather than only weights.
  • Beyond the paper: a direct testable extension is to apply the same framework to much larger pre-trained speech encoders, where the 2-bit teacher guidance could be replaced by a full-precision teacher during fine-tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes a quantization-aware co-training framework for extremely low-bit (2-bit and 1-bit) Conformer ASR weight quantization, combining learnable tensor-wise scaling factors, KL-divergence regularization from a 2-bit model, and stochastic precision that randomly binarizes layers during training. Experiments on the 300-hour Switchboard and 960-hour LibriSpeech corpora report WERs for a progressive set of systems, claiming performance-lossless compression at 2-bit and 1-bit and maximum compression ratios of 16.2x and 16.6x, with statistical significance assessed via MAPSSWE at alpha=0.05. The paper also compares favorably against prior low-bit ASR quantization methods and reports negligible extra quantization parameters.

Significance. If the headline claims held under a clearly pre-specified losslessness criterion, this would be a meaningful advance for deploying ASR on memory-constrained devices: 1-bit and 2-bit Conformer encoders with 4-bit decoders and 12-17x weight compression, plus only 204 extra scale parameters, would be a substantial practical result. The paper's strengths include the careful progressive ablation (Tables 1 and 2, Fig. 2), the use of an external paired significance test (MAPSSWE), and systematic comparison with prior methods on LibriSpeech. The main weakness is that the 'lossless' claim is stated more strongly than the evidence: the significance criterion is applied per test set in Table 3 but only to a subset of sets, and the 1.5-bit Switchboard claim uses a majority-of-three sampling rule with per-sample results not reported.

major comments (4)
  1. [§4.4, Table 3] The abstract's unqualified 'performance-lossless 2-bit and 1-bit quantization' is not supported by the paper's own significance marks. In Table 3, the 1-bit system F1 (16.6x) carries no star on test-clean (2.79 vs 2.55) or dev-clean (2.57 vs 2.33), and the 2-bit system F2 carries no star on dev-clean (2.45 vs 2.33). Section 4.4 bullet 3 implicitly concedes this by listing only test-clean/test-other/dev-other for F2 and only test-other/dev-other for F1. Please either restrict the losslessness claim to the test sets where the criterion is met, or pre-specify a pooled criterion over all held-out sets and apply it uniformly.
  2. [§4.3, Table 2, footnote 2] The 16.2x Switchboard claim for the 1.5-bit system E3 rests on a majority rule: footnote 2 states that losslessness requires only 2 of 3 sampled systems to be lossless, and Table 2 reports only the average WER across the three samples. This means a failing sample can be hidden, and the single star on the average column does not reveal whether individual test sets (Hub5'00, RT02, RT03) or individual sampled masks pass. Please report per-sample and per-test-set results, and state whether the abstract's 'maximum performance-lossless compression' refers to a majority-of-samples criterion or to a guarantee on all samples.
  3. [§3.3, Eqs. (6)-(7)] The method is presented as using a 2-bit teacher to guide a 1-bit student, but the two systems share all weights and are optimized jointly. The stop-gradient SG(p_int2) in Eq. (5) only stops gradient flow through the KL term; the student losses L_int1 and L_SP in Eqs. (6)-(7) still update the same shared weights that produce the teacher's logits. The teacher is therefore not an independent model, and the mechanism is better described as self-regularization than as distillation. Please state this explicitly and provide evidence that the KL term behaves as intended (e.g., ablate whether using a detached or pretrained teacher changes the results).
  4. [§4.2-4.4, all tables] All reported systems appear to be single training runs, and the 1.5-bit systems involve random layer binarization. MAPSSWE is a paired test for a fixed model on a fixed test set; it does not capture variance due to training seeds or sampled stochastic-precision masks. Consequently, the unqualified wording 'performance-lossless' across systems may not be reproducible. At a minimum, report multiple seeds or masks for the headline systems (S2, S1, E3, F2/F1), or explicitly state that the claim is about the particular trained systems.
minor comments (5)
  1. [Abstract] The sentence 'Maximum overall performance-lossless compression ratios of 16.2 and 16.6 times are achieved ... respectively' is ambiguous. The 16.2x result is a mixed 1.5-bit Switchboard system (E3) while the 16.6x result is a 1-bit LibriSpeech system (F1); please state the datasets and bit configurations in the abstract.
  2. [§4.1] The stochastic-precision schedule, described as 'binarized at probabilities increasing from 0.2 to 0.9 for layers from 1 to 12 with a log-linear schedule,' should define the probability event (binarizing the layer) and whether the sampling is independent per iteration and per utterance.
  3. [Table 2 caption] The caption says a star in the last column denotes no significant WER increase, but the paper does not explain whether the MAPSSWE test is run on the average across the three Switchboard test sets or on each test set individually. This should be stated explicitly.
  4. [Table 3 caption] The symbol 'Δ' (meaning 'our implementation') is used in rows 4 and 6 but is not defined in the table caption.
  5. [Conclusion] The final paragraph repeats 'performance-lossless 2-bit and 1-bit quantization' without the qualifications used in Section 4.4; please align the conclusion with the evidence presented in Tables 2 and 3.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the co-training/KL self-distillation is an empirical recipe, and losslessness is measured against external baselines.

full rationale

The paper's apparent self-reference is that the 2-bit 'teacher' and 1-bit 'student' share weights and are trained jointly (Fig. 1, Eq. 7), so the KL term in Eq. 5 is a form of self-distillation. But self-distillation is not circular here: the evaluated claim—1-bit or 2-bit quantized Conformer WER—is measured on held-out test sets against a separately trained full-precision baseline (ID 0 in Tables 1-3), and Table 1 shows the method can fail badly (ID 4: 19.60 avg WER vs. baseline 12.86), so the positive results are not forced by construction. The loss functions in Eqs. (4), (6), and (7) are weighted sums of ordinary training objectives, not restatements of the WER or compression-ratio outcomes, and no fitted parameter is renamed as a prediction. The statistical fragility of the 'performance-lossless' wording—the per-cell star convention in Tables 2-3 and footnote 2's 'no less than 2 out of 3' rule for 1.5-bit systems—is a multiple-testing/reporting concern, not a circular derivation. Self-citations [22] and [25] are used only as comparison systems; the load-bearing baseline and significance test [37] are external. Accordingly, the central claim has independent empirical content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on standard QAT practice plus several hand-set hyperparameters; the central empirical claim is grounded in external WER benchmarks, not on a mathematical derivation. No new physical or architectural entities are postulated.

free parameters (4)
  • lambda1 (co-training student loss weight) = 0.5
    Eq. (7): weights the 1-bit and stochastic-precision losses; chosen by hand to balance loss magnitudes, not systematically tuned.
  • lambda2 (KL regularization weight) = 1.0
    Eq. (7): weights the KL distillation terms; chosen by hand.
  • Layer binarization probability schedule for stochastic precision = 0.2 (layer 1) to 0.9 (layer 12), log-linear
    Section 4.1: schedule chosen similarly to layer drop; controls how often each encoder layer is binarized and is central to the reported gains.
  • gamma in Conformer multitask loss = 0.2
    Eq. (1): CTC versus attention interpolation coefficient, stated as empirically set; inherited from baseline configuration and used in all systems.
assumptions (5)
  • domain assumption Straight-Through Estimator gives useful gradients through the non-differentiable quantizer
    Section 3.1 relies on STE [31] for backpropagation; no convergence guarantee, standard QAT practice.
  • domain assumption A single set of full-precision weights can simultaneously serve 2-bit, 1-bit, and stochastic-precision projections
    Eqs. (4)-(7) train all submodels on one shared weight matrix; if the optimization landscape prevents this, co-training fails.
  • domain assumption 2-bit logits are a beneficial teacher for the 1-bit student
    Eq. (5) uses KL divergence between 2-bit and 1-bit outputs; requires the teacher-student gap to be small enough for distillation to help.
  • domain assumption MAPSSWE one-sided significance test with alpha=0.05 is the correct criterion for no WER increase
    Sections 4.2-4.4 use Gillick and Cox [37]; assumes paired test-set comparisons and no multiple-testing correction.
  • domain assumption ESPnet baseline recipes give full-precision systems that are valid references for compression ratio and WER
    Section 4.1 configures baselines using ESPnet; no explicit recipe or commit hash is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision." pith.science (2026). https://pith.science/paper/QPL2JMGQ

@misc{pith2026250521245,
  author       = {Pith},
  title        = {Pith review of: Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QPL2JMGQ}},
  note         = {Machine review of arXiv:2505.21245}
}
read the original abstract

Model compression has become an emerging need as the sizes of modern speech systems rapidly increase. In this paper, we study model weight quantization, which directly reduces the memory footprint to accommodate computationally resource-constrained applications. We propose novel approaches to perform extremely low-bit (i.e., 2-bit and 1-bit) quantization of Conformer automatic speech recognition systems using multiple precision model co-training, stochastic precision, and tensor-wise learnable scaling factors to alleviate quantization incurred performance loss. The proposed methods can achieve performance-lossless 2-bit and 1-bit quantization of Conformer ASR systems trained with the 300-hr Switchboard and 960-hr LibriSpeech corpus. Maximum overall performance-lossless compression ratios of 16.2 and 16.6 times are achieved without a statistically significant increase in the word error rate (WER) over the full precision baseline systems, respectively.

Figures

Figures reproduced from arXiv: 2505.21245 by the authors.

Figure 2
Figure 2. Accuracy (↑) of different 1-bit (a) and 2-bit (b) con￾former systems on the validation set as a function of training epochs. “int1” and “int2” represent naively quantized 1-bit and 2-bit conformer systems, respectively. “CNNf ” denotes keeping the convolution modules with full-precision. “Sc.” de￾notes using learnable scaling factors. “Co-T.”, “KL”, and “SP” are for techniques of co-training, KL-regularization, and … view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evolution Strategy-Based Calibration for Low-Bit Quantization of Speech Models

    cs.SD 2026-03 conditional novelty 5.0 of 10

    Using CMA-ES to jointly optimize activation quantization scales keeps speech-model accuracy near full precision under full INT8 and INT4 quantization.

Reference graph

Works this paper leans on

45 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    However, performance progress tends to accompany an increasing number of model parameters and the need for computation and storage resources [7]

    Introduction Modern automatic speech recognition (ASR) models such as Conformer have achieved significant progress on various speech recognition scenarios [1–6]. However, performance progress tends to accompany an increasing number of model parameters and the need for computation and storage resources [7]. As a result, model compression with a neutral or ...

    work page

  2. [2]

    I love speech

    significant performance degradation[22, 28–30]. For ex- ample, relative increases in WER of more than 10% and 15% are reported in [30] for 2-bit and 1-bit quantization, respec- tively. [28] obtained a 2-bit Conformer model at the cost of 17% relative WER degradation compared to the 4-bit version with large-scale training data. In addition, the absence of ...

    arXiv 2025

  3. [3]

    Conformer) is a popular E2E ASR architecture that achieves state-of-the-art performance on many speech recognition tasks [1]

    Conformer ASR Systems The convolution-augmented Transformer (a.k.a. Conformer) is a popular E2E ASR architecture that achieves state-of-the-art performance on many speech recognition tasks [1]. Similar to Transformers, a Conformer encoder consists of multiple blocks stacked together, where each block is further composed of the following modules in sequenc...

    work page

  4. [4]

    Quantization Aware Co-Training In this paper, we focus specifically on extremely low-bit quan- tization of Conformer ASR systems. In particular, our goal is to develop Conformer models with 2-bit or 1-bit weight param- eters, which can reduce the memory footprint and also provide speed-up due to lower memory usage [28]. To do so, we pro- pose a new quanti...

    work page

  5. [5]

    Experimental Setup We conduct experiments on two commonly used ASR datasets:

    Experiments 4.1. Experimental Setup We conduct experiments on two commonly used ASR datasets:

    work page

  6. [6]

    espnet/egs2

    training on the 300-hr Switchboard corpus [34] and testing on NIST Hub5’00, RT02, and RT03 test sets. 2) training on the LibriSpeech 960-hr datasets [35] and testing on test-clean, test- other, dev-clean, and dev-other testsets. The full-precision base- line Conformer systems are configured using the ESPnet [36] recipes in “espnet/egs2”. All the quantized...

    work page

  7. [7]

    We achieved performance-lossless 2-bit and 1-bit quantization of Conformer systems

    Conclusion We proposed a novel method of low-bit quantization based on the co-training framework with KL-divergence regularization and stochastic precision. We achieved performance-lossless 2-bit and 1-bit quantization of Conformer systems. We ob- tained a 16.2x compressed Conformer with no statistically sig- nificant WER increase and a 26.2x compressed C...

    work page

  8. [8]

    14200220, 14200021, 14200324 and Innovation Technology Fund grant No

    Acknowledgements This research is supported by Hong Kong RGC GRF grant No. 14200220, 14200021, 14200324 and Innovation Technology Fund grant No. ITS/218/21

Show all 45 references
  1. [9]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wuet al., “Conformer: Convolution- augmented transformer for speech recognition,” inInterspeech, 2020

    2020

  2. [10]

    Branchformer: Parallel MLP-attention architectures to capture local and global context for speech recognition and understanding,

    Y . Peng, S. Dalmia, I. Lane, and S. Watanabe, “Branchformer: Parallel MLP-attention architectures to capture local and global context for speech recognition and understanding,” inICML, 2022

    2022

  3. [11]

    wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” inNeurIPS, 2020

    2020

  4. [12]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM T-ASLP, vol. 29, pp. 3451–3460, 2021

    2021

  5. [13]

    Zipformer: A faster and better encoder for automatic speech recognition,

    Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y . Yang, Z. Jin, L. Lin, and D. Povey, “Zipformer: A faster and better encoder for automatic speech recognition,” inICLR, 2023

    2023

  6. [14]

    Hybrid CTC/attention architecture for end-to-end speech recog- nition,

    S. Watanabe, T. Hori, S. Kim, J. R. Hershey, and T. Hayashi, “Hybrid CTC/attention architecture for end-to-end speech recog- nition,”IEEE Journal of Selected Topics in Signal Processing, 2017

    2017

  7. [15]

    Efficient speech representation learning with low-bit quantization,

    C.-F. Yeh, W.-N. Hsu, P. Tomasello, and A. Mohamed, “Efficient speech representation learning with low-bit quantization,”arXiv preprint arXiv:2301.00652, 2022

    2022 arXiv

  8. [16]

    A survey of quantization methods for efficient neu- ral network inference,

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neu- ral network inference,” inLow-Power Computer Vision, 2022, pp. 291–326

    2022

  9. [17]

    Binarized neural networks,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Ben- gio, “Binarized neural networks,” inNIPS, 2016

    2016

  10. [18]

    XNOR- Net: Imagenet classification using binary convolutional neural networks,

    M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi, “XNOR- Net: Imagenet classification using binary convolutional neural networks,” inECCV, 2016

    2016

  11. [19]

    Towards accurate binary convolu- tional neural network,

    X. Lin, C. Zhao, and W. Pan, “Towards accurate binary convolu- tional neural network,” inNIPS, 2017

    2017

  12. [20]

    Bi-Real Net: Enhancing the performance of 1-bit CNNs with improved representational capability and advanced training algo- rithm,

    Z. Liu, B. Wu, W. Luo, X. Yang, W. Liu, and K.-T. Cheng, “Bi-Real Net: Enhancing the performance of 1-bit CNNs with improved representational capability and advanced training algo- rithm,” inECCV, 2018

    2018

  13. [21]

    Accurate and efficient 2-bit quantized neural networks,

    J. Choi, S. Venkataramani, V . V . Srinivasan, K. Gopalakrishnan, Z. Wang, and P. Chuang, “Accurate and efficient 2-bit quantized neural networks,”Proceedings of Machine Learning and Systems, vol. 1, pp. 348–359, 2019

    2019

  14. [22]

    BitNet: Scaling 1-bit transform- ers for large language models,

    H. Wang, S. Ma, L. Dong, S. Huang, H. Wang, L. Ma, F. Yang, R. Wang, Y . Wu, and F. Wei, “BitNet: Scaling 1-bit transform- ers for large language models,”arXiv preprint arXiv:2310.11453, 2023

    2023 arXiv

  15. [23]

    The era of 1-bit LLMs: All large language models are in 1.58 bits,

    S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, and F. Wei, “The era of 1-bit LLMs: All large language models are in 1.58 bits,”arXiv preprint arXiv:2402.17764, 2024

    2024 arXiv

  16. [24]

    BiT: Robustly binarized multi- distilled transformer,

    Z. Liu, B. Oguz, A. Pappu, L. Xiao, S. Yih, M. Li, R. Kr- ishnamoorthi, and Y . Mehdad, “BiT: Robustly binarized multi- distilled transformer,” inNeurIPS, 2022

    2022

  17. [25]

    BinaryBERT: Pushing the limit of bert quantization,

    H. Bai, W. Zhang, L. Hou, L. Shang, J. Jin, X. Jiang, Q. Liu, M. Lyu, and I. King, “BinaryBERT: Pushing the limit of bert quantization,” inACL, 2021

    2021

  18. [26]

    Binary deep neural networks for speech recognition

    X. Xiang, Y . Qian, and K. Yu, “Binary deep neural networks for speech recognition.” inInterspeech, 2017

    2017

  19. [27]

    Binary neural networks for speech recognition,

    Y .-m. Qian and X. Xiang, “Binary neural networks for speech recognition,”Frontiers of Information Technology & Electronic Engineering, vol. 20, no. 5, pp. 701–715, 2019

    2019

  20. [28]

    4-bit conformer with native quantization aware training for speech recognition,

    S. Ding, P. Meadowlark, Y . He, L. Lew, S. Agrawal, and O. Ry- bakov, “4-bit conformer with native quantization aware training for speech recognition,” inInterspeech, 2022

    2022

  21. [29]

    Integer- only zero-shot quantization for efficient speech recognition,

    S. Kim, A. Gholami, Z. Yao, N. Lee, P. Wang, A. Nrusimha, B. Zhai, T. Gao, M. W. Mahoney, and K. Keutzer, “Integer- only zero-shot quantization for efficient speech recognition,” in ICASSP, 2022

    2022

  22. [30]

    Lossless 4-bit quantization of architecture compressed conformer asr systems on the 300-hr switchboard corpus,

    Z. Li, T. Wang, J. Deng, J. Xu, S. Hu, and X. Liu, “Lossless 4-bit quantization of architecture compressed conformer asr systems on the 300-hr switchboard corpus,” inInterspeech, 2023

    2023

  23. [31]

    4-bit quantization of LSTM-based speech recognition models,

    A. Fasoli, C.-Y . Chen, M. Serrano, X. Sun, N. Wang, S. Venkataramani, G. Saon, X. Cui, B. Kingsbury, W. Zhanget al., “4-bit quantization of LSTM-based speech recognition models,” inInterspeech, 2021

    2021

  24. [32]

    Mixed precision quan- tization of transformer language models for speech recognition,

    J. Xu, S. Hu, J. Yu, X. Liu, and H. Meng, “Mixed precision quan- tization of transformer language models for speech recognition,” inICASSP, 2021

    2021

  25. [33]

    One-pass multiple conformer and founda- tion speech systems compression and quantization using an all-in- one neural model,

    Z. Li, H. Xu, T. Wang, S. Hu, Z. Jin, S. Hu, J. Deng, M. Cui, M. Geng, and X. Liu, “One-pass multiple conformer and founda- tion speech systems compression and quantization using an all-in- one neural model,” inInterspeech, 2024

    2024

  26. [34]

    Mixed precision low- bit quantization of neural network language models for speech recognition,

    J. Xu, J. Yu, S. Hu, X. Liu, and H. Meng, “Mixed precision low- bit quantization of neural network language models for speech recognition,”IEEE/ACM T-ASLP, vol. 29, pp. 3679–3693, 2021

    2021

  27. [35]

    USM-Lite: Quantization and sparsity aware fine-tuning for speech recogni- tion with universal speech models,

    S. Ding, D. Qiu, D. Rim, Y . He, O. Rybakov, B. Li, R. Prab- havalkar, W. Wang, T. N. Sainath, Z. Hanet al., “USM-Lite: Quantization and sparsity aware fine-tuning for speech recogni- tion with universal speech models,” inICASSP, 2024

    2024

  28. [36]

    2-bit conformer quantization for automatic speech recog- nition,

    O. Rybakov, P. Meadowlark, S. Ding, D. Qiu, J. Li, D. Rim, and Y . He, “2-bit conformer quantization for automatic speech recog- nition,” inInterspeech, 2023

    2023

  29. [37]

    USM RNN-T model weights binarization,

    O. Rybakov, D. Serdyuk, and C. Zheng, “USM RNN-T model weights binarization,” inInterspeech, 2024

    2024

  30. [38]

    Compressed MoE asr model based on knowledge distillation and quantization,

    Y . Yuan, Z. You, S. Feng, D. Su, Y . Liang, X. Shi, and D. Yu, “Compressed MoE asr model based on knowledge distillation and quantization,” inInterspeech, 2023

    2023

  31. [39]

    Estimating or propa- gating gradients through stochastic neurons for conditional com- putation,

    Y . Bengio, N. L ´eonard, and A. Courville, “Estimating or propa- gating gradients through stochastic neurons for conditional com- putation,”arXiv preprint arXiv:1308.3432, 2013

    2013 arXiv

  32. [40]

    Co-training 2L submodels for visual recognition,

    H. Touvron, M. Cord, M. Oquab, P. Bojanowski, J. Verbeek, and H. J ´egou, “Co-training 2L submodels for visual recognition,” in CVPR, 2023

    2023

  33. [41]

    QKD: Quantization-aware knowledge distillation,

    J. Kim, Y . Bhalgat, J. Lee, C. Patel, and N. Kwak, “QKD: Quantization-aware knowledge distillation,”arXiv preprint arXiv:1911.12491, 2019

    1911 arXiv

  34. [42]

    SWITCH- BOARD: Telephone speech corpus for research and develop- ment,

    J. J. Godfrey, E. C. Holliman, and J. McDaniel, “SWITCH- BOARD: Telephone speech corpus for research and develop- ment,” inICASSP, 1992

    1992

  35. [43]

    Lib- riSpeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- riSpeech: An ASR corpus based on public domain audio books,” inICASSP, 2015

    2015

  36. [44]

    ESPnet: End-to-end speech processing toolkit,

    S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y . Unno, N. E. Y . Soplin, J. Heymann, M. Wiesner, N. Chenet al., “ESPnet: End-to-end speech processing toolkit,”arXiv preprint arXiv:1804.00015, 2018

    2018 arXiv

  37. [45]

    Some statistical issues in the comparison of speech recognition algorithms,

    L. Gillick and S. J. Cox, “Some statistical issues in the comparison of speech recognition algorithms,” inICASSP, 1989

    1989

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.