Pith. sign in

REVIEW 4 major objections 5 minor 42 references

UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognition

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read UME upcycles a pretrained 200M ASR model into a 1B mixture-of-experts model, cutting character error rate by up to 11.9% while keeping latency nearly unchanged and reducing training time by 86.7% versus training from scratch.

desk verdict Solid ASR-specific upcycling paper with a clean initialization identity, but the headline 86.7% training-time saving is not yet supported for lack of convergence curves. read the letter →

arxiv 2412.17507 v1 pith:Q6STB2GF submitted 2024-12-23 eess.AS

Li Fu , Shanyong Yu , Siqi Li , Lu Fan , Youzheng Wu , Xiaodong He This is my paper · ORCID
classification eess.AS
keywords automaticspeechrecognitionmixtureofexpertsmodelupcyclingscalingefficienttraininglayerfreezingexpertbalancing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a pretrained dense automatic speech recognition model can be scaled into a much larger mixture-of-experts (MoE) model cheaply by reusing its weights instead of training from scratch. The authors replace each feed-forward network with several expert copies, route inputs through the top two experts, and continue training only the MoE layers with a load-balancing loss. On a 170k-hour Mandarin and English corpus, the upcycled Conformer-1B-UME beats the 200M pretrained baseline by up to 11.9% relative character error rate while keeping real-time factor comparable, and uses 86.7% less training time than a 1B model trained from scratch. The same recipe applied to the public Paraformer model on a 10k-hour dataset yields up to 15.6% relative error reduction. If correct, this makes large ASR models substantially more accessible by turning an existing small checkpoint into a strong starting point for a bigger model.

What carries the argument

The load-bearing mechanism is the renormalized top-k router: instead of softmax over all N experts, UME applies softmax only to the logits of the k selected experts, so the routing weights sum to 1. Because each expert FFN is an exact copy of the pretrained FFN, the MoE layer's output equals the dense layer's output at initialization, guaranteeing the upcycled model starts with the same recognition behavior as the smaller model. Two auxiliary mechanisms carry the continued-training phase: layer freezing (only MoE layers are trainable), which preserves the pretrained feature extractors and reduces training cost, and the load-balancing loss from Switch Transformers, which encourages even expert utilization and improves accuracy. This combination makes the expanded model trainable in about two days on eight A100 GPUs.

What would settle it

Train the same 1B MoE architecture from scratch with compute matched to UME's 2-day budget and compare final CER on the same test sets; if the from-scratch model matches or beats Conformer-1B-UME, the claimed efficiency advantage collapses. A second check: verify directly that the upcycled model's outputs equal the dense model's outputs at initialization on a held-out batch, since the renormalized top-k identity is what the whole argument rests on.

Watch

Extended reading notes

Core claim

UME's central claim is that upcycling works because the expanded MoE model can be initialized to exactly reproduce the pretrained dense model's output, providing an optimal starting point for continued training. Concretely, each FFN is replaced by N copies of itself and the router uses a renormalized softmax over the top-k experts, so the weighted sum of expert outputs equals the original FFN output at initialization. Continued training then updates only the expert FFNs and router while freezing attention, down-sampling, and normalization layers, with a differentiable load-balancing loss to spread tokens across experts. In experiments, the resulting Conformer-1B-UME achieves 11.9% relative CER reduction over the Conformer-200M baseline on Mandarin and English test sets, maintains an RTF of 0.0042 versus 0.0032 for the dense baseline, and needs 2 days rather than 15 days of training compared with from-scratch 1B models. The same procedure applied to Paraformer-200M on a public 10k-hour dataset gives a 15.6% relative CER reduction without substantially increasing RTF.

Load-bearing premise

The headline efficiency comparison assumes the from-scratch 1B baselines were trained to the same convergence as the 2-day UME continued training; if those baselines were undertrained, the 86.7% training-time saving would be overstated.

Editorial extensions

If this is right

  • Speech teams with an existing dense ASR checkpoint can scale to a 1B-parameter MoE model in days rather than weeks, with lower error rates and near-identical inference latency.
  • The upcycling recipe transfers across architectures: it works for both Conformer (autoregressive) and Paraformer (non-autoregressive) models, and across 170k-hour and 10k-hour training data.
  • Layer freezing alone cuts continued-training time by 33% relative to full fine-tuning while improving accuracy, and the load-balancing loss consistently lowers CER across all tested Mandarin and English sets.
  • Increasing the number of experts from 4 to 8 and top-k from 1 to 2 yields further error reductions, indicating the method's gains are not saturated at the reported configuration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The renormalized top-k initialization trick should transfer to any upcycling setting, including LLMs and vision models, where exact output parity at initialization may stabilize sparse upcycling; this is a testable extension beyond ASR.
  • A fairer comparison would report compute-normalized learning curves for UME versus from-scratch training; without those, the 86.7% saving is an upper bound rather than a measured advantage.
  • The layer-freezing result hints that attention features from the smaller model are already close to sufficient, so capacity added through experts is the main lever; this suggests a scaling recipe of keep-the-backbone, grow-the-experts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes UME, a method to upcycle a pretrained dense ASR model into a larger Mixture-of-Experts (MoE) model by copying each feed-forward network into N expert copies, renormalizing the top-k router weights, and then continuing training with frozen non-MoE layers and an expert-balancing loss. Experiments on Conformer (170k-hour mixed Mandarin/English data) and Paraformer (public 10k-hour data) show consistent CER improvements over the dense baselines on several test sets, with only a modest RTF increase, and a large claimed training-time saving versus training 1B models from scratch. The structural identity in Eq. (1) — that the MoE output equals the dense output at initialization when all experts are identical copies and the selected weights are renormalized — is correct and is a clean conceptual contribution. However, the central efficiency claim rests on incomplete convergence evidence, and the paper selectively omits a metric on which UME underperforms its baseline, so the current presentation overstates the strength of the results.

Significance. If the empirical claims hold, UME is a practically useful scaling recipe for ASR: it converts a 200M dense checkpoint into a 1B MoE model while keeping inference latency close to the dense model, and it reports consistent gains on Mandarin and English test suites. The weight-reuse identity (Eq. (1)) is a simple and correct structural observation that makes the upcycling initialization principled. The paper also includes ablations (layer freezing, expert balancing, expert count) on two architectures, which strengthens the empirical contribution. The main value is in demonstrating that sparse upcycling, previously studied for language models, transfers to ASR at scale. However, the significance is currently weakened by the lack of compute-normalized or convergence-matched efficiency comparisons and by a selective metric report that hides a degradation on one benchmark.

major comments (4)
  1. [Section IV-A / Table I] The headline 86.7% training-time saving is not supported by the evidence as presented. The only stated control for the from-scratch baselines C2/C3 is early stopping after 6k steps without a validation-loss decrease, but the paper does not report validation-loss curves, final step counts, FLOPs, or compute-normalized CER. Because C4 starts from a fully converged C0 and only trains MoE layers, it reaches good accuracy in 2 days by construction, while C2/C3 must train all parameters from random initialization. If C2/C3 were stopped before their validation plateau, the saving would be overstated. Please report learning curves, step counts, and compute-normalized comparisons for all models, and describe the validation split used for early stopping.
  2. [Section IV-D / Table III] The paper claims "consistently" superior performance and "significantly outperforms" baselines, but Table III reports that Paraformer-200M achieves 2.74% CER on SpeechIO while Paraformer-1B-UME achieves 3.09% on the same benchmark — i.e., UME is worse than the pretrained baseline on this metric. This result is omitted from Table II, which reports only Wenetspeech and Aishell-4 for Paraformer. The text acknowledges forgetting for FMFT and w/o layer freezing but does not state that even with layer freezing, the upcycled Paraformer underperforms the pretrained model on SpeechIO. Please report all metrics consistently and discuss this trade-off, or revise the claim of consistent improvement.
  3. [Section III-C2 / Eq. (2)] The expert-balancing loss in Eq. (2) uses f_i = (1/T) sum_t 1{argmax(W_N) = i}, which counts only the highest-probability expert per token. The model, however, uses top-k routing with k=2, so a token is dispatched to two experts and the load on an expert should be measured by whether that expert is in the selected top-k set. As written, the auxiliary loss penalizes imbalance only among first-choice experts, which is inconsistent with the stated routing scheme. Please clarify the definition of W_N relative to the top-k renormalized W_k used in Eq. (1), and correct the dispatch fraction to count all selected experts (summing to k), or explain why the argmax-based definition is appropriate for top-k training.
  4. [Section III-C1 / Table I] The row "w/o layer freezing" in Table I is not defined precisely. It should state whether this variant trains all parameters (attention, downsampling, layernorm, etc.) or only the MoE layers and some other subset. Since layer freezing is a claimed contributor to both performance and training-time savings, the exact trainable-parameter set must be specified for reproducibility. Currently the reader cannot tell whether the 3-day time difference is due to training all parameters or to a different optimizer/learning-rate schedule.
minor comments (5)
  1. [Section II] The related work cites Sparse Upcycling [15] but does not discuss how UME differs from or extends it beyond the domain change from language to speech; a brief comparison would help position the contribution.
  2. [Table II] The RTF increases from 0.0061 to 0.0079 for the Paraformer UME model (about 30%); calling this "without substantially increasing RTF" is defensible but deserves a one-sentence justification, especially since the Conformer UME increase is smaller.
  3. [Section IV-D / Fig. 2] The expert-usage analysis in Fig. 2 shows distributions for only two SpeechIO scenarios; reporting the standard deviation or a summary statistic across all 26 scenarios would make the balancing effect easier to assess.
  4. [Section III-B1] The notation W_N and W_k is used before being fully defined in Eq. (2); please define both explicitly in Section III-B1 to avoid confusion between the full router distribution and the renormalized top-k distribution.
  5. [Section IV-A] The paper does not report the random seed or the exact number of training steps for each model; adding these details would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: UME's gains are empirical measurements, and Eq. (1) is a construction identity, not a fitted prediction.

full rationale

The paper's central claims rest on measured CER/WER values on public test sets (SpeechIO, Wenetspeech, Aishell-4, Gigaspeech) and on reported training days in Table I. The 11.9% relative CER reduction and 86.7% training-time saving are arithmetic comparisons of these empirical numbers, not quantities derived from fitted parameters or from the method's definitions. Eq. (1) is a structural identity showing that, because each expert is initialized as a copy of the pretrained FFN and the router renormalizes the selected top-k weights, the MoE layer's initial output equals the dense layer's output; this is an initialization guarantee by construction, not a prediction whose output is forced by an input fit. The continued-training gains (layer freezing, expert balancing, N/k choices) are supported by ablations in Tables III and IV and are externally falsifiable. The only self-citation ([29], used to contrast knowledge-distillation regularization) appears in related-work positioning and is not load-bearing. The possible concern that the from-scratch 1B baselines (C2/C3) may not be fully converged is an experimental-fairness question about baseline optimization, not a circularity in the derivation. No step in the paper reduces to its own inputs by definition or by a self-citation chain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its claims rest on three hand-chosen hyperparameters and on four background assumptions, the most consequential of which is the layer-freezing design choice.

free parameters (3)
  • N (number of experts per MoE layer) = 8
    Chosen after an ablation in Table IV on the same test sets used for the headline results; the paper says N=8 and k=2 are 'a balanced choice'.
  • k (top-k expert selection) = 2
    Chosen after the same ablation; top-k selection affects the renormalization in Eq (1) and controls inference cost.
  • alpha (expert balancing loss coefficient) = 0.01
    Taken from Switch Transformer [18] without ASR-specific tuning, per Section III-C2; it controls the balance term in Eq (3).
assumptions (4)
  • standard math Softmax over the selected top-k logits sums to 1, so copied identical experts reproduce the dense FFN output exactly.
    Used in Eq (1) to guarantee the upcycled model starts with identical outputs to the pretrained model.
  • ad hoc to paper The pretrained dense checkpoint provides a strong enough feature representation that freezing attention, downsampling, and layer normalization does not cap achievable performance.
    This is the central design choice in Section III-C1; the paper validates it empirically but does not systematically study which modules must be frozen.
  • domain assumption The 170k-hour mixed Mandarin/English dataset is representative and the evaluation benchmarks are unbiased measures of ASR quality.
    All conclusions about error reduction rely on these benchmarks; the dataset composition and curation are not described in detail.
  • ad hoc to paper The expert balancing loss coefficient alpha=0.01, borrowed from language modeling, behaves appropriately for ASR training.
    Section III-C2 sets alpha following [18] without ablating it on ASR data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognition." pith.science (2026). https://pith.science/paper/Q6STB2GF

@misc{pith2026241217507,
  author       = {Pith},
  title        = {Pith review of: UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q6STB2GF}},
  note         = {Machine review of arXiv:2412.17507}
}
read the original abstract

Recent advancements in scaling up models have significantly improved performance in Automatic Speech Recognition (ASR) tasks. However, training large ASR models from scratch remains costly. To address this issue, we introduce UME, a novel method that efficiently Upcycles pretrained dense ASR checkpoints into larger Mixture-of-Experts (MoE) architectures. Initially, feed-forward networks are converted into MoE layers. By reusing the pretrained weights, we establish a robust foundation for the expanded model, significantly reducing optimization time. Then, layer freezing and expert balancing strategies are employed to continue training the model, further enhancing performance. Experiments on a mixture of 170k-hour Mandarin and English datasets show that UME: 1) surpasses the pretrained baseline by a margin of 11.9% relative error rate reduction while maintaining comparable latency; 2) reduces training time by up to 86.7% and achieves superior accuracy compared to training models of the same size from scratch.

Figures

Figures reproduced from arXiv: 2412.17507 by the authors.

Figure 1
Figure 1. Illustration of the proposed UME. To efficiently scale up, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Distribution of activated experts across Conformer-1B-UME [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 36 canonical work pages

  1. [1]

    Attention is all you need,

    Ashish Vaswani, Noam Shazeer, Niki Parmar, et al., “Attention is all you need,” in Conference on Neural Information Processing Systems (NeurIPS), 2017

    work page 2017

  2. [2]

    Recent advances in end-to-end automatic speech recognition,

    Jinyu Li, “Recent advances in end-to-end automatic speech recognition,” APSIPA Transactions on Signal and Information Processing , 2022

    work page 2022

  3. [3]

    End-to- end speech recognition: A survey,

    Rohit Prabhavalkar, Takaaki Hori, Tara N Sainath, et al., “End-to- end speech recognition: A survey,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2023

    work page 2023

  4. [4]

    Bigssl: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition,

    Yu Zhang, Daniel Park, Wei Han, et al., “Bigssl: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition,” IEEE Journal of Selected Topics in Signal Processing , 2022

    work page 2022

  5. [5]

    Wavlm: Large- scale self-supervised pre-training for full stack speech processing,

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, et al., “Wavlm: Large- scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , 2022

    work page 2022

  6. [6]

    Pushing the limits of semi- supervised learning for automatic speech recognition,

    Yu Zhang, James Qin, Daniel S Park, et al., “Pushing the limits of semi- supervised learning for automatic speech recognition,” in Conference on Neural Information Processing Systems (NeurIPS) , 2020

    work page 2020

  7. [7]

    Robust speech recogni- tion via large-scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, et al., “Robust speech recogni- tion via large-scale weak supervision,” in International Conference on Machine Learning (ICML) , 2023

    work page 2023

  8. [8]

    Google usm: Scaling au- tomatic speech recognition beyond 100 languages,

    Yu Zhang, Wei Han, James Qin, et al., “Google usm: Scaling au- tomatic speech recognition beyond 100 languages,” arXiv preprint arXiv:2303.01037, 2023

    arXiv 2023

Show all 42 references
  1. [9]

    On efficient training of large-scale deep learning models: A literature review,

    Li Shen, Yan Sun, Zhiyuan Yu, et al., “On efficient training of large-scale deep learning models: A literature review,” arXiv preprint arXiv:2304.03589, 2023

    2023 arXiv

  2. [10]

    Net2net: Acceler- ating learning via knowledge transfer,

    Tianqi Chen, Ian Goodfellow, and Jonathon Shlens, “Net2net: Acceler- ating learning via knowledge transfer,” in International Conference on Learning Representations (ICLR) , 2016

    2016

  3. [11]

    Data efficient neural scaling law via model reusing,

    Peihao Wang, Rameswar Panda, and Zhangyang Wang, “Data efficient neural scaling law via model reusing,” in International Conference on Machine Learning (ICML) , 2023

    2023

  4. [12]

    Efficient large scale language modeling with mixtures of experts,

    Mikel Artetxe, Shruti Bhosale, Naman Goyal, et al., “Efficient large scale language modeling with mixtures of experts,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022

    2022

  5. [13]

    Mixtral of experts,

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, et al., “Mixtral of experts,” arXiv preprint arXiv:2401.04088 , 2024

    2024 arXiv

  6. [14]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    Machel Reid, Nikolay Savinov, Denis Teplyashin, et al., “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” arXiv preprint arXiv:2403.05530 , 2024

    2024 arXiv

  7. [15]

    Sparse upcycling: Training mixture-of-experts from dense checkpoints,

    Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, et al., “Sparse upcycling: Training mixture-of-experts from dense checkpoints,” in International Conference on Learning Representations (ICLR) , 2023

    2023

  8. [16]

    Moe jetpack: From dense checkpoints to adaptive mixture of experts for vision tasks,

    Xingkui Zhu, Yiran Guan, Dingkang Liang, Yuchao Chen, Yuliang Liu, and Xiang Bai, “Moe jetpack: From dense checkpoints to adaptive mixture of experts for vision tasks,” arXiv preprint arXiv:2406.04801 , 2024

    2024 arXiv

  9. [17]

    Outra- geously large neural networks: The sparsely-gated mixture-of-experts layer,

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, et al., “Outra- geously large neural networks: The sparsely-gated mixture-of-experts layer,” in International Conference on Learning Representations (ICLR), 2017

    2017

  10. [18]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

    William Fedus, Barret Zoph, and Noam Shazeer, “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,” Journal of Machine Learning Research , 2022

    2022

  11. [19]

    Speechmoe: Scaling to large acoustic models with dynamic routing mixture of experts,

    Zhao You, Shulin Feng, Dan Su, et al., “Speechmoe: Scaling to large acoustic models with dynamic routing mixture of experts,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH) , 2021

    2021

  12. [20]

    Speechmoe2: Mixture-of-experts model with improved routing,

    Zhao You, Shulin Feng, Dan Su, et al., “Speechmoe2: Mixture-of-experts model with improved routing,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022

    2022

  13. [21]

    3m: Multi-loss, multi-path and multi-level neural networks for speech recognition,

    Zhao You, Shulin Feng, Dan Su, et al., “3m: Multi-loss, multi-path and multi-level neural networks for speech recognition,” in International Symposium on Chinese Spoken Language Processing (ISCSLP) , 2022

    2022

  14. [22]

    Language-routing mix- ture of experts for multilingual and code-switching speech recognition,

    Wenxuan Wang, Guodong Ma, Yuke Li, et al., “Language-routing mix- ture of experts for multilingual and code-switching speech recognition,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH) , 2023

    2023

  15. [23]

    Ba-moe: Boundary-aware mixture-of-experts adapter for code-switching speech recognition,

    Peikun Chen, Fan Yu, Yuhao Liang, et al., “Ba-moe: Boundary-aware mixture-of-experts adapter for code-switching speech recognition,” in IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2023

    2023

  16. [24]

    Mole: Mixture of language experts for multi-lingual automatic speech recognition,

    Yoohwan Kwon and Soo-Whan Chung, “Mole: Mixture of language experts for multi-lingual automatic speech recognition,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023

    2023

  17. [25]

    Mixture-of-expert conformer for streaming multilingual asr,

    Ke Hu, Bo Li, Tara N Sainath, et al., “Mixture-of-expert conformer for streaming multilingual asr,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH) , 2023

    2023

  18. [26]

    U2++ moe: Scal- ing 4.7 x parameters with minimal impact on rtf,

    Xingchen Song, Di Wu, Binbin Zhang, et al., “U2++ moe: Scal- ing 4.7 x parameters with minimal impact on rtf,” arXiv preprint arXiv:2404.16407, 2024

    2024 arXiv

  19. [27]

    Conformer: Convolution-augmented transformer for speech recognition,

    Anmol Gulati, James Qin, Chung-Cheng Chiu, et al., “Conformer: Convolution-augmented transformer for speech recognition,” in Pro- ceedings of the Annual Conference of the International Speech Commu- nication Association (INTERSPEECH) , 2020

    2020

  20. [28]

    Massively multilingual asr: A lifelong learning solution,

    Bo Li, Ruoming Pang, Yu Zhang, et al., “Massively multilingual asr: A lifelong learning solution,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022

    2022

  21. [29]

    Incremental learning for end-to-end automatic speech recognition,

    Li Fu, Xiaoxiao Li, Libo Zi, et al., “Incremental learning for end-to-end automatic speech recognition,” in IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) , 2021

    2021

  22. [30]

    A comparison of parameter-efficient asr domain adaptation methods for universal speech and language models,

    Khe Chai Sim, Zhouyuan Huo, Tsendsuren Munkhdalai, et al., “A comparison of parameter-efficient asr domain adaptation methods for universal speech and language models,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024

    2024

  23. [31]

    Lora: Low-rank adaptation of large language models,

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, et al., “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations (ICLR) , 2022

    2022

  24. [32]

    Sparsely shared lora on whisper for child speech recognition,

    Wei Liu, Ying Qin, Zhiyuan Peng, et al., “Sparsely shared lora on whisper for child speech recognition,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024

    2024

  25. [33]

    Residual adapters for parameter-efficient asr adaptation to atypical and accented speech,

    Katrin Tomanek, Vicky Zayats, Dirk Padfield, et al., “Residual adapters for parameter-efficient asr adaptation to atypical and accented speech,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2021

    2021

  26. [34]

    Exploiting adapters for cross-lingual low-resource speech recognition,

    Wenxin Hou, Han Zhu, Yidong Wang, et al., “Exploiting adapters for cross-lingual low-resource speech recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2021

    2021

  27. [35]

    Funasr: A fundamental end-to-end speech recognition toolkit,

    Zhifu Gao, Zerui Li, Jiaming Wang, et al., “Funasr: A fundamental end-to-end speech recognition toolkit,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), 2023

    2023

  28. [36]

    Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to- end speech recognition,

    Zhifu Gao, Shiliang Zhang, Ian McLoughlin, et al., “Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to- end speech recognition,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH) , 2022

    2022

  29. [37]

    Llama pro: Progressive llama with block expansion,

    Chengyue Wu, Yukang Gan, Yixiao Ge, et al., “Llama pro: Progressive llama with block expansion,” arXiv preprint arXiv:2401.02415 , 2024

    2024 arXiv

  30. [38]

    Smartfrz: An efficient train- ing framework using attention-based layer freezing,

    Sheng Li, Geng Yuan, Yue Dai, et al., “Smartfrz: An efficient train- ing framework using attention-based layer freezing,” in International Conference on Learning Representations (ICLR) , 2023

    2023

  31. [39]

    Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition,

    Binbin Zhang, Hang Lv, Pengcheng Guo, et al., “Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2022

    2022

  32. [40]

    Aishell-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario,

    Yihui Fu, Luyao Cheng, Shubo Lv, et al., “Aishell-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEE...

    2021

  33. [41]

    Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,

    Guoguo Chen, Shuzhou Chai, Guanbo Wang, et al., “Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” in Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH) , 2021

    2021

  34. [42]

    Connection- ist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,

    Alex Graves, Santiago Fern ´andez, Faustino Gomez, et al., “Connection- ist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,” in International Conference on Machine Learning (ICML), 2006

    2006

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.