Pith. sign in

REVIEW 4 major objections 7 minor 39 references

BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BLR-MoE extends language-routing MoE to self-attention and adds router augmentation and expert pruning, achieving a 16.09% relative WER reduction over LR-MoE on a 10,000-hour multilingual ASR corpus.

desk verdict Applied ASR paper with a real confound in the headline out-of-domain number; expert pruning is the most solid piece and the rest is a fixable evaluation issue. read the letter →

arxiv 2501.12602 v1 pith:WYQRWY46 submitted 2025-01-22 cs.CL cs.SDeess.AS

Guodong Ma , Wenxuan Wang , Lifeng Zhou , Yuting Yang , Yuke Li , Binbin Du This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords language-routingmixtureofexpertsmultilingualASRattentionlanguageconfusionrouteraugmentationexpertpruningend-to-endout-of-domainrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that language confusion in the LR-MoE multilingual ASR architecture has two separable sources: confusion inside self-attention and confusion in the language-identification router that assigns speech to expert subnetworks. To address both, it proposes BLR-MoE, which applies mixture-of-experts to the value and output matrices of self-attention, augments the router with a stronger TDNN-based language classifier that can be fine-tuned on cheap audio-language pairs, and prunes experts at inference when the set of possible languages is known. On a 10,000-hour four-language corpus, BLR-MoE reduces word error rate by 16.09% relative to LR-MoE, with a 19.09% relative gain on the Commonvoice out-of-domain test set and a 9.41% absolute gain in router accuracy. The paper's core claim is that decoupling the two confusion sources and attacking each one directly makes multilingual end-to-end ASR more accurate and more adaptable to mismatched domains.

What carries the argument

The central mechanism is the shared language router combined with language-specific expert subnetworks that now cover both the feed-forward network and the value/output projections of multi-head self-attention. In BLR-MoE, a gated network (upgraded to a TDNN-based LID adapter) produces a soft assignment over language experts; the same assignment selects FFN experts and attention experts. Expert pruning is an inference-time operation: given prior knowledge that certain languages are absent, the router's output distribution is masked so data cannot be routed to pruned experts. This mechanism is what lets a single trained model be reconfigured per scenario without retraining.

What would settle it

Run BLR-MoE on an out-of-domain test set whose audio-language pairs were never used for router fine-tuning, and compare WER with and without attention-MoE. If the out-of-domain gain collapses to roughly the in-domain gain, then the reported 19.09% OOD improvement comes from router adaptation, not from the architecture.

Watch

Extended reading notes

Core claim

BLR-MoE is a language-routing mixture-of-experts speech recognizer that treats the multilingual modeling problem as the product of a language-identification term $P(L|X)$ and a language-conditioned acoustic-modeling term $P(Y|L,X)$. The paper claims that in the earlier LR-MoE both terms still suffer from language confusion: the shared self-attention layers cannot separate languages, and the simple linear LID router misroutes under domain mismatch. Its remedy is threefold: (1) attention-MoE on the $V$ and $O$ projection matrices of each attention head, so each language expert owns a larger private subnetwork; (2) router augmentation, replacing the linear gated network with a TDNN-based LID module and decoupling it from the frozen ASR stack so it can be fine-tuned with audio-language pairs; (3) expert pruning, which removes paths to languages known to be absent at inference. The reported experiments show a 16.09% relative WER drop over LR-MoE overall, 3.98% relative in-domain, and 19.09% relative on Commonvoice; ablations attribute 6.51% relative improvement to attention-MoE plus expert pruning without router augmentation, and 7.36% without LID fine-tuning.

Load-bearing premise

The headline out-of-domain result assumes that fine-tuning the LID router on 323.7 hours of Commonvoice audio-language pairs is part of a fair deployment scenario rather than target-domain adaptation; if that tuning is the actual driver, the architecture-only robustness gain is much smaller, near 6.51% relative.

Editorial extensions

If this is right

  • If BLR-MoE works as claimed, mixture-of-experts for multilingual ASR should be applied to attention as well as FFN, because the two are complementary rather than redundant.
  • Router augmentation implies that domain robustness can be bought with audio-language pairs (no transcripts) by fine-tuning only the LID router while the ASR stack stays frozen.
  • Expert pruning turns the trained model into a configuration that can be specialized at inference to any known subset of languages, including single-language deployment.
  • The 9.41% absolute router-accuracy gain should reduce cross-language misrecognition in production systems where the router previously sent data to the wrong expert.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline 19.09% out-of-domain gain likely overstates what the architecture alone contributes, because the router was fine-tuned on 323.7 hours of Commonvoice audio-language pairs; the architecture-only effect, from Table II, is closer to the 6.51% relative improvement seen when router augmentation is removed.
  • If audio-language data is genuinely cheaper than transcribed audio, the decoupled router design suggests a practical domain-adaptation recipe: keep the frozen ASR model and swap or fine-tune only a small LID module per target domain; this could be tested across more languages and domains.
  • The expert-pruning results expose a confusion graph among languages (Chinese and Japanese closer, English and Arabic more confusable), which could inform language clustering or hierarchical routing in future multilingual ASR designs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes BLR-MoE, an extension of the LR-MoE multilingual end-to-end ASR architecture. The authors decompose language confusion in LR-MoE into confusion in self-attention and in the LID-based router, and address the two sources with three modifications: (i) applying MoE to the value/output projections of self-attention in addition to the FFN, (ii) replacing the router with a TDNN-based gated network that can be fine-tuned on audio-language pairs, and (iii) expert pruning at inference when a subset of languages is known a priori. Experiments on a 10,000-hour four-language (ZH/EN/JA/AR) dataset with Commonvoice used as an out-of-domain test report a 16.09% relative WER reduction over LR-MoE, including 19.09% on Commonvoice, 3.98% in-domain, and a 9.41% absolute improvement in router accuracy.

Significance. If the reported gains are attributable to the architecture, the paper makes a useful contribution: attention-MoE for multilingual ASR is relatively underexplored, the decoupled LID-router adaptation is practical, and expert pruning offers a training-free way to specialize a deployed model. The strengths of the paper are the large-scale controlled comparison against the self-cited LR-MoE baseline, the component-wise ablations, explicit parameter counts, and a consistent improvement pattern across ablations. However, the headline out-of-domain gain is confounded by target-domain router fine-tuning, the attention-MoE ablation is not fully isolated, and the absence of variance information leaves several small gains unsubstantiated.

major comments (4)
  1. [§III-B, Table I] The Commonvoice condition is described as out-of-domain, but the LID router is fine-tuned on 323.7 hours of Commonvoice audio-language pairs (Table I, 'Train set with LID label only'; §III-B). The 19.09% Commonvoice improvement of ID-2 over ID-1 therefore mixes architecture changes with explicit target-domain adaptation, and the 16.09% overall headline inherits this confound. The paper should either present ID-3 versus ID-1 (8.14% relative on Commonvoice, 7.36% overall) as the architecture-only out-of-domain result, or justify router adaptation as part of the intended deployment scenario and state that the out-of-domain claim is conditional on it.
  2. [§III-B, Table II] The ablation meant to isolate attention-MoE does not actually isolate it. §III-B states that LR-MoE uses a simple linear layer for the gated network, while ID-4 in Table II retains a stronger 'FFN' router. Thus the 6.51% relative improvement of ID-4 over ID-1 could be due in part to router-capacity increase rather than to attention-MoE. If ID-1's router is also an FFN, then §III-B's description of LR-MoE is inconsistent with Table II. Please confirm that the 'FFN' router in ID-1 and ID-4 is identical, or add an ablation that combines attention-MoE with exactly the same gated network as LR-MoE; otherwise the statement that attention-MoE alone gives 6.51% is unsupported.
  3. [§IV-B, Table IV] The expert-pruning result is misreported. The text says that 'if we only let the model recognize Arabic audio (ID-6), compared with ID-0, a relative improvement of 34.28% is obtained in the Commonvoice Arabic test.' In Table IV, ID-6 corresponds to pruning ZH and EN, and its Arabic WER is 33.73 versus ID-0's 45.04, a 25.1% relative improvement; ID-7 (pruning ZH, EN, JA) gives 38.5%. The 34.28% figure does not match any entry in Table IV. Please correct the row reference or the calculation.
  4. [§IV-B, Tables II and IV] No error bars or significance tests are reported. Several claims depend on small differences: the in-domain comparisons in Table II (e.g., JA 9.77 vs 10.05, AR 8.42 vs 8.57) and many expert-pruning comparisons in Table IV differ by less than 1% absolute WER. The central architecture-only gain of 6.51% (ID-4 vs ID-1) is a single-run comparison. Please report multiple-seed variance or a significance test for the key pairwise comparisons, or temper the corresponding claims.
minor comments (7)
  1. [Table II] The model name 'BLE-MoE' in the ID-2 row should be 'BLR-MoE'; 'Vallina' in ID-0 should be 'Vanilla'.
  2. [Table II] The label '- only LID tune' for ID-3 is confusing; based on the text it means removal of the LID-tuning component, not 'only LID tune'. Please rename it (e.g., 'w/o LID tune').
  3. [§IV-A] The sentence 'we only use the part of English Commonvoice data and use the speed perturbation for other data' is unclear and appears inconsistent with Table I, which lists Commonvoice training and test durations for all four languages. Please rewrite to specify exactly which Commonvoice subsets are used and for which languages.
  4. [Eqs. (2)-(8)] The notation W^{h,e}_V and W^{h,e}_O uses the expert index e before the set E is defined in Eq. (8); please reorder or define e earlier. Also, the paper does not state whether all attention heads in a layer share the same expert chosen by the router, or whether heads can select different experts.
  5. [Tables II and III] The baseline numbering is inconsistent across tables: in Table II, ID-0 is the Vanilla CTC model and ID-1 is LR-MoE, while in Table III ID-0 is LR-MoE. Please align the identifiers to avoid confusion.
  6. [Table II] The 'Avg_all' column appears to be the unweighted average of the in-domain and Commonvoice averages. Please state the averaging scheme, since the 16.09% headline reduction is computed on this summary value and depends on the equal weighting of the two test conditions.
  7. [§III-B] Please specify how the TDNN-based router is trained with the ASR branch (e.g., the frame-level or utterance-level LID loss, the gradient flow into the shared encoder), and how expert pruning is implemented at inference (e.g., whether router probabilities are renormalized over the remaining experts).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's empirical claims are measured on held-out test sets, and Eq. (1) is a definitional identity rather than a derivation of results.

full rationale

No circular derivation is present. The only equation in the paper, Eq. (1), is a definitional factorization of P(Y|X) into a LID term and an ASR term; it is used to motivate the architecture, but none of the empirical WER or router-accuracy results are derived from it algebraically. The attention-MoE equations (2)-(8) are definitions of K, Q, V, and O with language experts, not a derivation that reduces to its own conclusion. Router augmentation fine-tunes only the LID router on 323.7 hours of Commonvoice audio-language pairs and evaluates on a separate Commonvoice test set; this is an adaptation protocol with a possible external-validity confound, but it is not a fitted parameter renamed as a prediction and does not make the headline result true by construction. The LR-MoE baseline is prior work by overlapping authors, but it is used as a comparison system rather than as a load-bearing theorem or uniqueness argument, so the self-citation does not create circularity. The paper also provides ablations (ID-3 and ID-4) that remove router augmentation or LID tuning, showing that attention-MoE alone yields a 6.51% relative improvement over LR-MoE; even if those ablations do not perfectly isolate every variable, the central claim retains independent empirical content. The skeptic's concern about the 19.09% out-of-domain gain being confounded by target-domain router adaptation is best framed as an experimental-design and correctness risk, not as a circularity in the derivation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on standard supervised learning assumptions and several domain-specific modeling choices. No new theoretical entity is introduced. The main hand-chosen hyperparameters are the LID loss weight, the shared/MLE layer split, the number of language experts, and the TDNN router size.

free parameters (4)
  • LID loss weight lambda_lid = 0.3
    Hand-chosen weight for the auxiliary language identification loss in the multi-task training objective (Section IV-A).
  • Shared/MLE layer split = 6 shared + 6 MLE layers
    Architecture choice following LR-MoE [9]; not tuned in this paper (Section IV-A).
  • Number of language experts = 4
    One expert per supported language (ZH, EN, JA, AR); set by the language inventory rather than by data fitting (Section III-A).
  • TDNN router channel size = 500
    Hand-chosen capacity for the time-delay neural network used in router augmentation (Section IV-A).
assumptions (4)
  • domain assumption Language confusion is the dominant error source in multilingual MoE ASR and can be decomposed into attention-level and router-level confusion.
    Section II motivates the paper's design from this decomposition, but no direct analysis of attention confusion is provided.
  • domain assumption Making self-attention V and O projections language-specific reduces attention-level language confusion.
    Section III-A assumes attention-MoE will help based on prior work in language modeling ([25,26]) and ASR pathways ([23,24]).
  • domain assumption Pruning experts according to prior language knowledge reduces router errors and improves WER.
    Section III-C assumes that removing experts for impossible languages prevents misrouting; the paper evaluates this only with oracle language-set information.
  • domain assumption Fine-tuning the LID router on audio-language pairs from the target domain improves multilingual ASR without degrading the ASR branch.
    Section III-B assumes that lower-cost audio-language data is sufficient to adapt the router and that frozen ASR weights preserve ASR quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR." pith.science (2026). https://pith.science/paper/WYQRWY46

@misc{pith2026250112602,
  author       = {Pith},
  title        = {Pith review of: BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WYQRWY46}},
  note         = {Machine review of arXiv:2501.12602}
}
read the original abstract

Recently, the Mixture of Expert (MoE) architecture, such as LR-MoE, is often used to alleviate the impact of language confusion on the multilingual ASR (MASR) task. However, it still faces language confusion issues, especially in mismatched domain scenarios. In this paper, we decouple language confusion in LR-MoE into confusion in self-attention and router. To alleviate the language confusion in self-attention, based on LR-MoE, we propose to apply attention-MoE architecture for MASR. In our new architecture, MoE is utilized not only on feed-forward network (FFN) but also on self-attention. In addition, to improve the robustness of the LID-based router on language confusion, we propose expert pruning and router augmentation methods. Combining the above, we get the boosted language-routing MoE (BLR-MoE) architecture. We verify the effectiveness of the proposed BLR-MoE in a 10,000-hour MASR dataset.

Figures

Figures reproduced from arXiv: 2501.12602 by the authors.

Figure 1
Figure 1. The processing of the MoE router router, based on LR-MoE, we propose attention-MoE, expert pruning, and router-augmentation for MASR, re￾spectively, thus forming the BLR-MoE architecture. • The BLR-MoE can enable the model to quickly adapt to different language and domain needs with the help of expert pruning and router augmentation, respectively. • Our proposed BLR-MoE model has undergone rigorous evaluation on a 1… view at source ↗
Figure 3
Figure 3. Schematic diagram of MoE modules. (a) The raw MLE, (b) The MLE [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 31 canonical work pages

  1. [1]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmid- huber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in ICML, 2006, pp. 369–376

    work page 2006

  2. [2]

    Sequence transduction with recurrent neural networks,

    Alex Graves, “Sequence transduction with recurrent neural networks,” arXiv preprint arXiv:1211.3711 , 2012

    arXiv 2012

  3. [3]

    Joint ctc-attention based end-to-end speech recognition using multi-task learning,

    Suyoun Kim, Takaaki Hori, and Shinji Watanabe, “Joint ctc-attention based end-to-end speech recognition using multi-task learning,” in ICASSP 2017 . IEEE, 2017, pp. 4835–4839

    work page 2017

  4. [4]

    Speech-transformer: A no- recurrence sequence-to-sequence model for speech recognition,

    Linhao Dong, Shuang Xu, and Bo Xu, “Speech-transformer: A no- recurrence sequence-to-sequence model for speech recognition,” in ICASSP 2018 , 2018, pp. 5884–5888

    work page 2018

  5. [5]

    Leveraging Phone Mask Training for Phonetic-Reduction-Robust E2E Uyghur Speech Recognition,

    Guodong Ma, Pengfei Hu, Jian Kang, Shen Huang, and Hao Huang, “Leveraging Phone Mask Training for Phonetic-Reduction-Robust E2E Uyghur Speech Recognition,” in Proc. Interspeech 2021, 2021, pp. 306– 310

    work page 2021

  6. [6]

    Speech-text based multi-modal training with bidirectional attention for improved speech recognition,

    Yuhang Yang, Haihua Xu, Hao Huang, Eng Siong Chng, and Sheng Li, “Speech-text based multi-modal training with bidirectional attention for improved speech recognition,” in ICASSP 2023 , 2023, pp. 1–5

    work page 2023

  7. [7]

    Enhancing multilingual speech recognition through language prompt tuning and frame-level language adapter,

    Song Li, Yongbin You, Xuezhi Wang, Ke Ding, and Guanglu Wan, “Enhancing multilingual speech recognition through language prompt tuning and frame-level language adapter,” in ICASSP 2024 , 2024, pp. 10941–10945

    work page 2024

  8. [8]

    Robust speech recognition via large- scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large- scale weak supervision,” in ICML. PMLR, 2023, pp. 28492–28518

    work page 2023

Show all 39 references
  1. [9]

    Language- Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition,

    Wenxuan Wang, Guodong Ma, Yuke Li, and Binbin Du, “Language- Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition,” in Proc. INTERSPEECH 2023 , 2023, pp. 1389–1393

    2023

  2. [10]

    Multilingual distilwhisper: Efficient distillation of multi-task speech models via language-specific experts,

    Thomas Palmeira Ferraz, Marcely Zanon Boito, Caroline Brun, and Vassilina Nikoulina, “Multilingual distilwhisper: Efficient distillation of multi-task speech models via language-specific experts,” in ICASSP 2024, 2024, pp. 10716–10720

    2024

  3. [11]

    Mole : Mixture of language experts for multi-lingual automatic speech recognition,

    Yoohwan Kwon and Soo-Whan Chung, “Mole : Mixture of language experts for multi-lingual automatic speech recognition,” in ICASSP 2023, 2023, pp. 1–5

    2023

  4. [12]

    A parameter-efficient language extension framework for multilingual asr,

    Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, and Tan Lee, “A parameter-efficient language extension framework for multilingual asr,” in Interspeech 2024, 2024, pp. 3929–3933

    2024

  5. [13]

    Lae-st-moe: Boosted language-aware encoder using speech translation auxiliary task for e2e code-switching asr,

    Guodong Ma, Wenxuan Wang, Yuke Li, Yuting Yang, Binbin Du, and Haoran Fu, “Lae-st-moe: Boosted language-aware encoder using speech translation auxiliary task for e2e code-switching asr,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) , 2023, pp. 1–8

    2023

  6. [14]

    Hierarchical softmax for end-to-end low- resource multilingual speech recognition,

    Qianying Liu, Zhuo Gong, Zhengdong Yang, Yuhang Yang, Sheng Li, Chenchen Ding, Nobuaki Minematsu, Hao Huang, Fei Cheng, Chenhui Chu, and Sadao Kurohashi, “Hierarchical softmax for end-to-end low- resource multilingual speech recognition,” in ICASSP 2023 , 2023, pp. 1–5

    2023

  7. [15]

    Investiga- tion into phone-based subword units for multilingual end-to-end speech recognition,

    Saierdaer Yusuyin, Hao Huang, Junhua Liu, and Cong Liu, “Investiga- tion into phone-based subword units for multilingual end-to-end speech recognition,” in ICASSP 2023 , 2023, pp. 1–5

    2023

  8. [16]

    U2++ moe: Scaling 4.7 x pa- rameters with minimal impact on rtf,

    Xingchen Song, Di Wu, Binbin Zhang, Dinghao Zhou, Zhendong Peng, Bo Dang, Fuping Pan, and Chao Yang, “U2++ moe: Scaling 4.7 x pa- rameters with minimal impact on rtf,” arXiv preprint arXiv:2404.16407 , 2024

    2024 arXiv

  9. [17]

    Lupet: Incorporating hierarchical information path into multilingual asr,

    Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, and Tan Lee, “Lupet: Incorporating hierarchical information path into multilingual asr,” in Interspeech 2024 , 2024, pp. 3979–3983

    2024

  10. [18]

    Rapid language adaptation for multilingual e2e speech recognition using encoder prompting,

    Yosuke Kashiwagi, Hayato Futami, Emiru Tsunoo, Siddhant Arora, and Shinji Watanabe, “Rapid language adaptation for multilingual e2e speech recognition using encoder prompting,” in Interspeech 2024, 2024, pp. 2900–2904

    2024

  11. [19]

    Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision,

    Saierdaer Yusuyin, Te Ma, Hao Huang, Wenbo Zhao, and Zhijian Ou, “Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision,” arXiv e-prints , p. arXiv:2406.02166, June 2024

    2024 arXiv

  12. [20]

    Extending multilingual asr to new languages using supplementary encoder and decoder components,

    Yerbolat Khassanov, Zhipeng Chen, Tianfeng Chen, Tze Yuang Chong, Wei Li, Lu Lu, and Zejun Ma, “Extending multilingual asr to new languages using supplementary encoder and decoder components,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal ...

    2024

  13. [21]

    Lora-whisper: Parameter-efficient and extensible mul- tilingual asr,

    Zheshu Song, Jianheng Zhuo, Yifan Yang, Ziyang Ma, Shixiong Zhang, and Xie Chen, “Lora-whisper: Parameter-efficient and extensible mul- tilingual asr,” in Interspeech 2024, 2024, pp. 3934–3938

    2024

  14. [22]

    Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages,

    Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Andrew Rosenberg, Rohit Prabhavalkar, Daniel S. Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor Strohman, Bhuvana Rama...

    2023 arXiv

  15. [23]

    Dynamic asr pathways: An adaptive masking approach towards efficient pruning of a multilingual asr model,

    Jiamin Xie, Ke Li, Jinxi Guo, Andros Tjandra, Yuan Shangguan, Leda Sari, Chunyang Wu, Junteng Jia, Jay Mahadeokar, and Ozlem Kalinli, “Dynamic asr pathways: An adaptive masking approach towards efficient pruning of a multilingual asr model,” in ICASSP 2024, 2024, pp. 12201– 12205

    2024

  16. [24]

    Learning asr pathways: A sparse multilingual asr model,

    Mu Yang, Andros Tjandra, Chunxi Liu, David Zhang, Duc Le, and Ozlem Kalinli, “Learning asr pathways: A sparse multilingual asr model,” in ICASSP 2023 , 2023, pp. 1–5

    2023

  17. [25]

    SwitchHead: Accelerating Transformers with Mixture-of-Experts At- tention,

    Róbert Csordás, Piotr Pi˛ ekos, Kazuki Irie, and Jürgen Schmidhuber, “SwitchHead: Accelerating Transformers with Mixture-of-Experts At- tention,” arXiv e-prints , p. arXiv:2312.07987, Dec. 2023

    2023 arXiv

  18. [26]

    Multi-Head Mixture-of-Experts,

    Xun Wu, Shaohan Huang, Wenhui Wang, and Furu Wei, “Multi-Head Mixture-of-Experts,” arXiv e-prints , p. arXiv:2404.15045, Apr. 2024

    2024 arXiv

  19. [27]

    Attention is all you need,

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishw...

    2017

  20. [28]

    Phoneme recognition using time-delay neural networks,

    A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K.J. Lang, “Phoneme recognition using time-delay neural networks,” IEEE Trans- actions on Acoustics, Speech, and Signal Processing , vol. 37, no. 3, pp. 328–339, 1989

    1989

  21. [29]

    Aishell- 1: An open-source mandarin speech corpus and a speech recognition baseline,

    Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng, “Aishell- 1: An open-source mandarin speech corpus and a speech recognition baseline,” in 2017 O-COCOSDA. IEEE, 2017, pp. 1–5

    2017

  22. [30]

    Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition,

    Binbin Zhang, Hang Lv, Pengcheng Guo, Qijie Shao, Chao Yang, Lei Xie, Xin Xu, Hui Bu, Xiaoyu Chen, Chenchen Zeng, Di Wu, and Zhendong Peng, “Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition,” in ICASSP 2022, 2022, pp. 6182–6186

    2022

  23. [31]

    Librispeech: An asr corpus based on public domain audio books,

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: An asr corpus based on public domain audio books,” in ICASSP 2015 , 2015, pp. 5206–5210

    2015

  24. [32]

    GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio,

    Guoguo Chen, Shuzhou Chai, Guan-Bo Wang, Jiayu Du, Wei-Qiang Zhang, Chao Weng, Dan Su, Daniel Povey, Jan Trmal, Junbo Zhang, Mingjie Jin, Sanjeev Khudanpur, Shinji Watanabe, Shuaijiang Zhao, Wei Zou, Xiangang Li, Xuchen Yao, Yongqing Wang, Zhao You, and Zhiyong Yan, “GigaSpeec...

    2021

  25. [33]

    Reazonspeech: A free and massive corpus for japanese asr,

    Yue Yin, Daijiro Mori, and Seiji Fujimoto, “Reazonspeech: A free and massive corpus for japanese asr,” in NLP 2023 , 2023

    2023

  26. [34]

    The mgb-2 challenge: Arabic multi- dialect broadcast media recognition,

    Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang, “The mgb-2 challenge: Arabic multi- dialect broadcast media recognition,” in SLT 2016 , 2016, pp. 279–284

    2016

  27. [35]

    Common voice: A massively-multilingual speech corpus,

    Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Co...

    2020

  28. [36]

    Audio augmentation for speech recognition,

    T. Ko, Vijayaditya Peddinti, D. Povey, and S. Khudanpur, “Audio augmentation for speech recognition,” in INTERSPEECH, 2015

    2015

  29. [37]

    SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing,

    Taku Kudo and John Richardson, “SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing,” arXiv e-prints , p. arXiv:1808.06226, Aug. 2018

    2018 arXiv

  30. [38]

    Adam: A method for stochastic optimization,

    Diederik P Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  31. [39]

    Dropout: a simple way to prevent neural networks from overfitting,

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The journal of machine learning research , vol. 15, no. 1, pp. 1929–1958, 2014

    1929

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.