Pith. sign in

REVIEW 2 major objections 3 minor 36 references

Masked Self-distilled Transducer-based Keyword Spotting with Semi-autoregressive Decoding

T0 review · 2 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Randomly masking the RNN-T predictor in training, plus a fused AR/NAR decoding score, gives keyword spotting that keeps clean-condition accuracy and suppresses false alarms in noise.

desk verdict Plausible fix for predictor overfitting in RNN-T KWS, with real gains on noisy sets, but the headline SAR claim rests on test-set-tuned hyperparameters and a table inconsistency. read the letter →

arxiv 2505.24820 v1 pith:75ADAVTP submitted 2025-05-30 cs.SD eess.AS

Yu Xi , Xiaoyu Gu , Haoyu Li , Jun Song , Bo Zheng , Kai Yu This is my paper
classification cs.SDeess.AS
keywords RNN-Tkeywordspottingmaskedself-distillationsemi-autoregressivedecodingnon-autoregressiveoverfittingsuppressionstreamingtransducerfalsealarms
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a specific failure mode of RNN-T keyword spotting: the prediction network, fed always the same keyword sequence during training, overfits to the keyword and fires false alarms in noise. The proposed solution is to randomly zero out individual predictor output vectors during training and to force the masked network's logits to match the unmasked ones through a KL-divergence self-distillation loss. This makes the model usable in a non-autoregressive mode with the predictor fully masked, and a semi-autoregressive decoding score then blends the autoregressive and non-autoregressive scores per frame. The paper reports that SAR decoding achieves the highest recall at four false alarms per hour on all five English and Mandarin test sets, with the largest wins where the autoregressive baseline collapses. If correct, this gives a single transducer model that can switch between clean and noisy deployment conditions without retraining or multi-stage pipelines.

What carries the argument

The mechanism has three parts. First, token-level random masking of the predictor output, $h^{\text{mask}}_u = \text{RandomMask}(h^{\text{text}}_u, \gamma_{\text{mask}})$, which lets the encoder and joiner learn to classify the keyword from acoustics alone. Second, a self-distillation term $L_{\text{MSD}} = \sum_{t,u} D_{\text{KL}}(p^{\text{token}}_{t,u} \| p^{m,\text{token}}_{t,u})$ that stabilizes training by making masked logits imitate unmasked logits, with total loss $L = L_{\text{RNN-T}} + \lambda_{\text{mask}} L^{\text{mask}}_{\text{RNN-T}} + \lambda_{\text{MSD}} L_{\text{MSD}}$. Third, the semi-autoregressive decoding score $\text{Score}^{\text{SAR}}_t = \alpha \cdot \text{Score}^{\text{AR}}_t \oplus (1-\alpha) \cdot \text{Score}^{\text{NAR}}_t$, computed on the keyword-specific lattice of the TDT-KWS search, with path-length normalization and timeout pruning as detailed in Algorithm 1.

What would settle it

Take the same models and datasets, choose $\alpha$ and $\gamma_{\text{mask}}$ on a separate development set (for example a held-out slice of MobvoiHotwords) instead of the test sets, and compare recall at #FA=4 across all five test sets; if SAR does not beat the better of AR and NAR on every set, the claim of consistent superiority is refuted.

Watch

Extended reading notes

Core claim

The paper claims that masked self-distillation (MSD) training lets an RNN-T keyword spotter decode without its predictor, and that fusing the two modes beats either mode alone. Concretely, for each training utterance the predictor output $h^{\text{text}}$ is forwarded once normally and once with each token vector masked with probability $\gamma_{\text{mask}} = 0.35$; the joint network produces logits from both, the RNN-T loss is applied to both, and an additional KL term pulls the masked logits toward the unmasked ones. At inference the model can run the standard autoregressive streaming search from [19] or a non-autoregressive variant in which the predictor output is fully masked, and the semi-autoregressive score combines the two per-frame activation scores with a coefficient $\alpha$ (0.5 for English, 0.3 for noisy Mandarin). On the reported test sets SAR beats both AR and NAR everywhere, and on the noisy Xiaowen set it lifts recall at #FA=4 from 26.30 (AR) and 95.33 (NAR) to 95.68, while keeping the clean-condition gains of AR. The paper interprets this as evidence that the prediction network is useful in clean conditions but harmful when it has memorized the keyword, and that masking redirects learning toward acoustic evidence.

Load-bearing premise

The reported superiority of SAR (and the choice of the best mask ratio) rests on picking $\alpha$ and $\gamma_{\text{mask}}$ by looking at the test sets themselves; the paper reports no held-out validation split, so a fair re-tuning could shrink or erase the advantage.

Editorial extensions

If this is right

  • MSD training makes the transducer's encoder and joiner learn to decode keywords without the predictor, so NAR decoding becomes a stateless acoustic model that resists false alarms in noise.
  • Since AR and NAR share one model, switching between them (or fusing them) costs nothing at deployment; no retraining is needed to get the robust mode.
  • The same AR/NAR/SAR framework transfers across fixed English keywords, open-vocabulary English keywords, and noisy Mandarin keywords, with only $\alpha$ changed between clean and noisy conditions.
  • MSD's gains concentrate where the plain model fails: the paper reports average absolute improvements of 18.03 recall points for NAR and 14.47 for AR across the five test sets, with the largest gains on noisy data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mask-and-distill recipe might transfer to other transducer applications with degenerate input sequences, such as keyword-boosted ASR or TTS with a fixed prompt, where the predictor likewise sees constant text.
  • The choice of $\alpha$ could become data-driven per utterance or per signal-to-noise ratio: a model that predicts when acoustic evidence is compromised could weight NAR more heavily, potentially doing better than the fixed 0.3/0.5 split used here.
  • Because MSD only changes training and decoding of a single transducer, it could be combined with multi-stage detectors rather than replacing them; the fused SAR score could feed, not fight, a second-stage verifier.
  • A direct testable extension is to apply the same masking recipe to the joiner's audio branch and measure whether self-distillation then also protects against acoustic overfitting, not just predictor overfitting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper proposes a masked self-distillation (MSD) training strategy for RNN-T-based keyword spotting, where the predictor output is randomly masked during training and a KL-divergence loss enforces consistency between masked and unmasked transducer logits. This enables non-autoregressive (NAR) decoding that fully masks the predictor output at inference. The paper also introduces semi-autoregressive (SAR) decoding, which linearly combines AR and NAR activation scores with a coefficient α. Experiments on three datasets (Snips, LibriKWS-20, MobvoiHotwords) report that MSD alleviates predictor overfitting, especially on noisy Mandarin data, and that SAR consistently achieves the best recall across all test sets.

Significance. If the claims hold, the paper offers a simple and effective way to mitigate the prediction-network overfitting problem in RNN-T KWS, supported by extensive ablations across multiple datasets and decoding strategies. The MSD training idea (random masking plus self-distillation) is intuitive and the reported NAR gains on noisy conditions are substantial. The paper also clearly compares against AR, NAR, greedy, and beam-search baselines. However, the significance is tempered by evaluation-protocol weaknesses (no validation split for hyperparameter selection) and an unresolved numeric inconsistency between the two main result tables.

major comments (2)
  1. [Section IV.B and Section V.A] The evaluation protocol does not include a validation set for selecting the SAR fusion coefficient α and the mask ratio γ_mask. Section IV.B states that α is set to 0.5 for English and 0.3 for Mandarin, and Section V.A (Table II) chooses γ_mask=0.35 as the best value on the LibriKWS-20 test-clean/test-other sets. Because these choices are made on the same test sets used to report the headline claim that 'SAR decoding consistently achieves the best results across all datasets' (Section V.C), the reported SAR advantage over AR and NAR may be an in-sample selection artifact. The authors should either fix α and γ_mask a priori, tune them on held-out development data, or report performance across a grid of values with statistical significance (e.g., bootstrap confidence intervals). Without such evidence, the claim that SAR is consistently best is not demonstrated out-of-sample.
  2. [Table III vs. Table IV] There is an inconsistency for the Wenwen dataset in the AR decoding condition with L_MSD enabled. Table III reports a recall of 90.44 for AR [19], while Table IV reports 60.44 for the same condition (row AR, L_MSD ✓). Both tables appear to describe the same model trained with the proposed MSD loss. If 60.44 is correct, the comparison in Section V.C ('96.72 vs. 90.44') and the SAR-vs-AR margin on Wenwen change substantially, and the average improvement for AR in Table IV (+14.47) is computed from 60.44. Conversely, if 90.44 is correct, Table IV is wrong. The authors must reconcile these numbers, verify which value is correct, and ensure all tables and text are consistent before the manuscript can be accepted.
minor comments (3)
  1. [Section V.C] There is a typo in the phrase 'useful contextual sementic information'; 'sementic' should be 'semantic'.
  2. [Table II] The row for γ_mask=0.45 reads '98.7492.02' with a missing space; it should be '98.74 92.02'.
  3. [Section IV.B] In the training details, 'a mini-batch size constrained by the stricter of 12,288 frames or 64 samples' is awkward; consider rephrasing to 'the minimum of the two limits'.

Circularity Check

2 steps flagged · score 4.0 of 10

SAR's 'consistently best' claim is partly in-sample: α and γ_mask are selected on the same test sets used for the headline comparison.

  1. fitted input called prediction [Section IV.B (Evaluation); used in Section V.C (Table III)]
    "For the three English test sets (EN Fixed and EN Arbitrary), we set the SAR decoding coefficient to 0.5 as described in Equation (9) and Algorithm 1. For the challenging Mandarin Wenwen and Xiaowen datasets (ZH Noisy), we set α=0.3 to give dominance to the NAR score in the SAR decoding process."

    The SAR fusion weight α is selected per test scenario, with no validation split or cross-validation described, and the same test sets are then used in Section V.C to conclude that 'SAR decoding consistently achieves the best results across all datasets.' The α values encode the prior observation that AR degrades on Mandarin noisy data and NAR dominates there, while English favors AR. The claimed consistency is therefore not an out-of-sample prediction: the fusion coefficient is effectively fit to the evaluation points. The fused scores are not purely forced because SAR can exceed both components, but the headline comparison is partially determined by the in-sample choice of α.

  2. fitted input called prediction [Section V.A (Optimal mask ratio), Table II; used in Table III]
    "Table II presents the KWS performances with different mask probabilities γmask. ... the best results achieved when the mask ratio is around 0.35. ... For all subsequent experiments, we use the optimal mask ratio γmask = 0.35, unless otherwise specified."

    The mask ratio γ_mask is selected by comparing macro-recall on the LibriKWS-20 test-clean and test-other sets in Table II, and those same two test sets are then used in Table III to support the claim that SAR 'consistently achieves the best results across all datasets.' Because the selection criterion is the test-set recall and no held-out validation set is reported, the optimality of γ_mask=0.35 and the subsequent SAR comparisons on these sets are in-sample. This is a fitted hyperparameter presented as a discovered optimum rather than a fresh prediction.

full rationale

The MSD training derivation itself is not circular: Equations (2)-(8) define random masking and KL self-distillation from the same model's unmasked logits, and Algorithm 1 is a concrete decoding procedure. The motivation from HAT [21] and the AR lattice from TDT-KWS [19] are external prior work; [19] is a self-citation but it is an independently published decoding algorithm used as a component and baseline, so it does not make the present claim load-bearing on an unverified self-citation. The circularity-like issue is confined to the evaluation: both the SAR fusion coefficient α and the mask ratio γ_mask are selected on the same test sets used to assert that SAR is consistently best, with no validation split or cross-validation described. This makes the headline experimental claim partly in-sample, but the MSD ablations in Table IV provide independent support for the training contribution, so the paper is not wholly circular. A separate table inconsistency (Wenwen AR with L_MSD: 60.44 in Table IV vs 90.44 in Table III) is a correctness concern, not a circularity, and is not scored here.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central method rests on the assumption that prediction-network overfitting is the cause of false alarms and that masking the predictor output can fix it. The remaining 'free' choices are hyperparameters (alpha, gamma, loss weights, decoding heuristics) that are either fixed arbitrarily or tuned on test sets. No new physical entities are introduced.

free parameters (6)
  • alpha = 0.5 (English), 0.3 (Mandarin)
    SAR fusion coefficient in Equation (9) and Algorithm 1, set per test scenario with no validation set.
  • gamma_mask = 0.35
    Mask probability on predictor output, chosen from Table II on LibriKWS-20 test sets.
  • lambda_mask = 1
    Loss weight in Equation (8), chosen without reported ablation.
  • lambda_MSD = 0.003
    Loss weight for self-distillation in Equation (8), chosen without reported ablation.
  • S_Bonus = Not stated
    Bonus score in Algorithm 1, value not specified in the paper.
  • T_out = Not stated
    Timeout for path-length pruning in Algorithm 1, value not specified.
assumptions (5)
  • domain assumption RNN-T prediction network overfits to fixed keyword transcripts when trained on keyword-specific data.
    Stated in Section II, attributed to [16]; not directly measured in this paper.
  • domain assumption Randomly masking the predictor output during training reduces reliance on the predictor and alleviates overfitting.
    Core to MSD training, Section III.A, inspired by HAT [21].
  • domain assumption Self-distillation between unmasked and masked logits helps the model converge when predictor information is removed.
    Claimed in Section III.A based on preliminary experiments that are not shown.
  • domain assumption The keyword-specific lattice search algorithm from [19] is a suitable decoder for both AR and NAR modes.
    Used throughout Section III.B; not independently validated here.
  • standard math RNN-T loss and KL divergence are standard and correctly applied.
    Equations (5)-(8).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Masked Self-distilled Transducer-based Keyword Spotting with Semi-autoregressive Decoding." pith.science (2026). https://pith.science/paper/75ADAVTP

@misc{pith2026250524820,
  author       = {Pith},
  title        = {Pith review of: Masked Self-distilled Transducer-based Keyword Spotting with Semi-autoregressive Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/75ADAVTP}},
  note         = {Machine review of arXiv:2505.24820}
}
read the original abstract

RNN-T-based keyword spotting (KWS) with autoregressive decoding~(AR) has gained attention due to its streaming architecture and superior performance. However, the simplicity of the prediction network in RNN-T poses an overfitting issue, especially under challenging scenarios, resulting in degraded performance. In this paper, we propose a masked self-distillation (MSD) training strategy that avoids RNN-Ts overly relying on prediction networks to alleviate overfitting. Such training enables masked non-autoregressive (NAR) decoding, which fully masks the RNN-T predictor output during KWS decoding. In addition, we propose a semi-autoregressive (SAR) decoding approach to integrate the advantages of AR and NAR decoding. Our experiments across multiple KWS datasets demonstrate that MSD training effectively alleviates overfitting. The SAR decoding method preserves the superior performance of AR decoding while benefits from the overfitting suppression of NAR decoding, achieving excellent results.

Figures

Figures reproduced from arXiv: 2505.24820 by the authors.

Figure 1
Figure 1. The training and inference overview of the proposed system. Both conventional and masked RNN-T training share the same architecture and parameters. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 26 canonical work pages

  1. [19]

    TDT-KWS: fast and accurate keyword spotting using token-and-duration transducer,

    Y . Xi, H. Li, B. Yang, H. Li, H. Xu, and K. Yu, “TDT-KWS: fast and accurate keyword spotting using token-and-duration transducer,” in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024, pp. 11 351–11 355

    work page 2024

  2. [1]

    Small-footprint keyword spotting using deep neural networks,

    G. Chen, C. Parada, and G. Heigold, “Small-footprint keyword spotting using deep neural networks,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 4087–4091

    work page

  3. [2]

    Multi-task learning and weighted cross-entropy for dnn-based keyword spotting,

    S. Panchapagesan, M. Sun, A. Khare, S. Matsoukas, A. Mandal, B. Hoffmeister, and S. Vitaladevuni, “Multi-task learning and weighted cross-entropy for dnn-based keyword spotting,” inProc. Interspeech, 2016, pp. 760–764

    2016

  4. [3]

    Compressed time delay neural network for small-footprint keyword spotting,

    M. Sun, D. Snyder, Y . Gao, V . K. Nagaraja, M. Rodehorst, S. Pan- chapagesan, N. Strom, S. Matsoukas, and S. Vitaladevuni, “Compressed time delay neural network for small-footprint keyword spotting,” inProc. Interspeech, 2017, pp. 3607–3611

    work page 2017

  5. [4]

    Text adaptive detection for customizable keyword spotting,

    Y . Xi, T. Tan, W. Zhang, B. Yang, and K. Yu, “Text adaptive detection for customizable keyword spotting,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2022, pp. 6652– 6656

    work page 2022

  6. [5]

    Text-aware speech separation for multi-talker keyword spotting,

    H. Li, B. Yang, Y . Xi, L. Yu, T. Tan, H. Li, and K. Yu, “Text-aware speech separation for multi-talker keyword spotting,” inInterspeech 2024, 2024, pp. 337–341

    work page 2024

  7. [6]

    MFA-KWS: Effective Keyword Spotting with Multi-head Frame-asynchronous Decoding

    Y . Xi, H. Li, X. Gu, Y . Jiang, and K. Yu, “MFA-KWS: Effective keyword spotting with multi-head frame-asynchronous decoding,” 2025. [Online]. Available: https://arxiv.org/abs/2505.19577

    work page Pith review arXiv 2025

  8. [7]

    Self-attention transducers for end-to-end speech recognition,

    Z. Tian, J. Yi, J. Tao, Y . Bai, and Z. Wen, “Self-attention transducers for end-to-end speech recognition,” inProc. Interspeech, G. Kubin and Z. Kacic, Eds., 2019, pp. 4395–4399

    work page 2019

Show all 36 references
  1. [8]

    Transformer-transducer: End-to-end speech recognition with self-attention,

    C. Yeh, J. Mahadeokar, K. Kalgaonkar, Y . Wang, D. Le, M. Jain, K. Schubert, C. Fuegen, and M. L. Seltzer, “Transformer-transducer: End-to-end speech recognition with self-attention,”CoRR, vol. abs/1910.12977, 2019. [Online]. Available: http://arxiv.org/abs/1910. 12977

    1910 arXiv

  2. [9]

    Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss,

    Q. Zhang, H. Lu, H. Sak, A. Tripathi, E. McDermott, S. Koo, and S. Kumar, “Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss,” inProc. IEEE In- ternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 202...

    2020

  3. [10]

    ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context,

    W. Han, Z. Zhang, Y . Zhang, J. Yu, C.-C. Chiu, J. Qin, A. Gulati, R. Pang, and Y . Wu, “ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context,” in Proc. Interspeech, 2020, pp. 3610–3614

    2020

  4. [11]

    Conformer: Convolution- augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented Transformer for Speech Recognition,” inProc. Interspeech, 2020, pp. 5036–5040

    2020

  5. [12]

    Large-scale streaming end-to-end speech translation with neural transducers,

    J. Xue, P. Wang, J. Li, M. Post, and Y . Gaur, “Large-scale streaming end-to-end speech translation with neural transducers,”arXiv preprint arXiv:2204.05352, 2022

    2022 arXiv

  6. [13]

    V ALL-T: decoder-only generative transducer for robust and decoding-controllable text-to-speech,

    C. Du, Y . Guo, H. Wang, Y . Yang, Z. Niu, S. Wang, H. Zhang, X. Chen, and K. Yu, “V ALL-T: decoder-only generative transducer for robust and decoding-controllable text-to-speech,”CoRR, vol. abs/2401.14321,

    arXiv

  7. [14]

    Streaming small-footprint keyword spotting using sequence-to- sequence models,

    Y . He, R. Prabhavalkar, K. Rao, W. Li, A. Bakhtin, and I. Mc- Graw, “Streaming small-footprint keyword spotting using sequence-to- sequence models,”Proc. IEEE ASRU, pp. 474–481, 2017

    2017

  8. [15]

    Adaptation of rnn transducer with text-to-speech technology for keyword spotting,

    E. Sharma, G. Ye, W. Wei, R. Zhao, Y . Tian, J. Wu, L. He, E. Lin, and Y . Gong, “Adaptation of rnn transducer with text-to-speech technology for keyword spotting,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2020, pp. 7484– 7488

    2020

  9. [16]

    Improving rnn trans- ducer modeling for small-footprint keyword spotting,

    Y . Tian, H. Yao, M. Cai, Y . Liu, and Z. Ma, “Improving rnn trans- ducer modeling for small-footprint keyword spotting,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2021, pp. 5624–5628

    2021

  10. [17]

    Rnn-t based open-vocabulary keyword spot- ting in mandarin with multi-level detection,

    Z. Liu, T. Li, and P. Zhang, “Rnn-t based open-vocabulary keyword spot- ting in mandarin with multi-level detection,”Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 5649–5653, 2021

    2021

  11. [18]

    CaTT- KWS: A multi-stage customized keyword spotting framework based on cascaded transducer-transformer,

    Z. Yang, S. Sun, J. Li, X. Zhang, X. Wang, L. Ma, and L. Xie, “CaTT- KWS: A multi-stage customized keyword spotting framework based on cascaded transducer-transformer,” inProc. Interspeech, 2022, pp. 1681– 1685

    2022

  12. [20]

    U2-KWS: unified two-pass open-vocabulary keyword spotting with keyword bias,

    A. Zhang, P. Zhou, K. Huang, Y . Zou, M. Liu, and L. Xie, “U2-KWS: unified two-pass open-vocabulary keyword spotting with keyword bias,” inProc. IEEE ASRU, 2023, pp. 1–8

    2023

  13. [21]

    Three-in-one: Fast and accurate transducer for hybrid-autoregressive speech recognition,

    H. Xu, T. M. Bartley, V . Bataev, and B. Ginsburg, “Three-in-one: Fast and accurate transducer for hybrid-autoregressive speech recognition,” in The Thirteenth International Conference on Learning Representations, 2025

    2025

  14. [22]

    Nemo: a toolkit for building ai applications using neural modules,

    O. Kuchaiev, J. Li, H. Nguyen, O. Hrinchuk, R. Leary, B. Ginsburg, S. Kriman, S. Beliaev, V . Lavrukhin, J. Cook, P. Castonguay, M. Popova, J. Huang, and J. M. Cohen, “Nemo: a toolkit for building ai applications using neural modules,” inNeurIPS Workshop on Systems for ML, 2019

    2019

  15. [23]

    Pruned rnn-t for fast, memory-efficient asr training,

    F. Kuang, L. Guo, W. Kang, L. Lin, M. Luo, Z. Yao, and D. Povey, “Pruned rnn-t for fast, memory-efficient asr training,” inInterspeech 2022, 2022, pp. 2068–2072

    2022

  16. [24]

    Rnn- transducer with stateless prediction network,

    M. Ghodsi, X. Liu, J. Apfel, R. Cabrera, and E. Weinstein, “Rnn- transducer with stateless prediction network,” inICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2020, pp. 7049–7053

    2020

  17. [25]

    Dropout: A simple way to prevent neural networks from overfitting,

    N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,”Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929–1958, 2014. [Online]. Available: http://jmlr.org/papers/v15/srivasta...

    1929

  18. [26]

    Efficient keyword spotting using dilated convolutions and gating,

    A. Coucke, M. Chlieh, T. Gisselbrecht, D. Leroy, M. Poumeyrol, and T. Lavril, “Efficient keyword spotting using dilated convolutions and gating,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019, pp. 6351–6355

    2019

  19. [27]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotovet al., “Librispeech: an asr corpus based on public domain audio books,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  20. [28]

    AISHELL-2: transforming mandarin ASR research into industrial scale,

    J. Du, X. Na, X. Liu, and H. Bu, “AISHELL-2: transforming mandarin ASR research into industrial scale,”CoRR, 2018. [Online]. Available: http://arxiv.org/abs/1808.10583

    2018 arXiv

  21. [29]

    Region proposal network based small-footprint keyword spotting,

    J. Hou, Y . Shi, M. Ostendorf, M. Hwang, and L. Xie, “Region proposal network based small-footprint keyword spotting,”IEEE Signal Process. Lett., vol. 26, no. 10, pp. 1471–1475, 2019. [Online]. Available: https://doi.org/10.1109/LSP.2019.2936282

    2019

  22. [30]

    Audio augmentation for speech recognition

    T. Ko, V . Peddinti, D. Povey, and S. Khudanpur, “Audio augmentation for speech recognition.” inProc. Interspeech, 2015, p. 3586

    2015

  23. [31]

    Specaugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” inProc. Interspeech, 2019, pp. 2613– 2617

    2019

  24. [32]

    Deep-FSMN for large vocabulary continuous speech recognition,

    S. Zhang, M. Lei, Z. Yan, and L. Dai, “Deep-FSMN for large vocabulary continuous speech recognition,” inProc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2018, pp. 5869– 5873

    2018

  25. [33]

    Learning pronunciation from a foreign language in speech synthesis networks,

    Y . Lee and T. Kim, “Learning pronunciation from a foreign language in speech synthesis networks,”CoRR, vol. abs/1811.09364, 2018. [Online]. Available: http://arxiv.org/abs/1811.09364

    2018 arXiv

  26. [34]

    Adam: A method for stochastic optimiza- tion,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimiza- tion,” in3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y . Bengio and Y . LeCun, Eds., 2015

    2015

  27. [35]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019

    2019

  28. [2024]

    Available: https://doi.org/10.48550/arXiv.2401.14321

    [Online]. Available: https://doi.org/10.48550/arXiv.2401.14321

    doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.