SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

Dongxing Xu; Yanhua Long; Yuang Zheng; Yuxiang Mei

REVIEW 3 major objections 6 minor 31 references

SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A parallel pair of speech encoders feeding a LoRA-tuned language model achieves 11.76% CER/WER on the blind evaluation set, beating the official baseline by 8.41 points without additional training data.

desk verdict Competitive challenge system result with consistent arithmetic, but the central encoder choice rests on an unreported full-data ablation. read the letter →

arxiv 2507.03343 v2 pith:F254GRH5 submitted 2025-07-04 cs.CL eess.AS

Yuxiang Mei , Yuang Zheng , Dongxing Xu , Yanhua Long This is my paper

classification cs.CLeess.AS

keywords multilingualautomaticspeechrecognitionconversationallargelanguagemodelparallelencoderlow-rankadaptationWhispermHuBERTtri-stagetraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a strong multilingual conversational speech recognizer can be assembled from existing components: two pretrained speech encoders, Whisper-large-v3 and mHuBERT-147, whose hidden states are concatenated and projected into the Qwen2.5-7B language model, fine-tuned with low-rank adaptation. The training recipe matters as much as the architecture: the authors use a tri-stage schedule that first trains only the projector, then adapts the encoders, and only then adapts the LLM, because jointly updating everything failed to converge. With a language-aware prompt prepended to the LLM input and 1,500 hours of the challenge's training data, the system reports 14.37% CER/WER on the development set and 11.76% on the blind evaluation set, an 8.41-point absolute gain over the official baseline. The practical point is that complementary supervision—Whisper's semantic knowledge plus mHuBERT's self-supervised acoustic coverage—can be fused cheaply and yield a large error reduction without new data.

What carries the argument

The mechanism that carries the argument is the parallel-speech-encoder plus tri-stage training. Two encoders run on the same waveform: Whisper-large-v3 with LoRA injected into its attention query and value matrices, and mHuBERT-147 fully fine-tuned. Their outputs are concatenated along the feature dimension, downsampled 4x by a projector built from a 1D convolution, a strided convolution, a two-layer MLP, and LayerNorm, and then fed into Qwen2.5-7B, whose own LoRA modules are activated only in the final stage. A natural-language prompt naming the target language is prepended to every LLM input. The staged schedule—projector pretraining, encoder adaptation, then LLM adaptation—is what prevents the optimization collapse the paper says occurred when all parts were updated together.

What would settle it

Run the identical tri-stage training on all 1,500 hours with Qwen2-Audio as the second encoder instead of mHuBERT and evaluate on the same blind set; if the Qwen2-Audio system matches or beats 11.76% CER/WER, the claimed encoder choice and the attributed source of the gain would be undercut.

Watch

Extended reading notes

Core claim

The central discovery is that concatenating the hidden states of a supervised encoder (Whisper-large-v3) and a self-supervised encoder (mHuBERT-147), projecting the 2048-dimensional joint representation down to the LLM's input dimension, and fine-tuning the whole chain in three stages reaches 11.76% CER/WER on the blind evaluation set, against 20.17% for the baseline system that uses Whisper alone with inference-time text normalization and language-id prompting. On the development set the gain is smaller but still clear: 14.37% versus 18.03%. The authors attribute the improvement to the complementary acoustic and linguistic representations, to LoRA-based fine-tuning that limits overfitting, and to the language-aware instruction prompt that tells the decoder which language to transcribe. They also report that starting with all modules unfrozen collapses training, so the staged order is a functional part of the recipe, not a convenience.

Load-bearing premise

The load-bearing premise is that mHuBERT is the better second encoder when trained on all 1,500 hours; the paper asserts this full-data comparison but shows only a one-quarter-data ablation with frozen encoders, which favors Qwen2-Audio.

Editorial extensions

If this is right

A competitive multilingual conversational ASR system can be built from publicly available encoders and an LLM, with no additional training data beyond the challenge's 1,500 hours.
A self-supervised encoder (mHuBERT) and a supervised encoder (Whisper) contribute complementary information, so concatenating them beats using either alone.
Freezing and staging the training—projector first, then encoders, then LLM—is required for stable convergence; simultaneous end-to-end fine-tuning fails.
Language-aware prompting at the LLM input is an effective, low-cost way to steer transcription toward the correct language.
The much larger gain on the blind evaluation set than on the development set suggests the parallel-encoder architecture generalizes beyond the training distribution rather than merely memorizing it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the reported choice of mHuBERT over Qwen2-Audio rests on full-data experiments that are not shown; the visible quarter-data ablation actually favors Qwen2-Audio, so the encoder-selection claim is open until the full comparison is published.
The architecture suggests a reusable recipe for other speech-to-text tasks: take one supervised and one self-supervised encoder, concatenate their features, and wrap them around an LLM with a staged LoRA schedule; speech translation and code-switched recognition are natural testbeds.
If the generalization gap is real, the approach may be especially valuable for accented or out-of-domain audio; a per-language or per-accent breakdown of the 8.41-point gain would make that concrete and is not reported in the paper.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, quick read of arXiv:2507.03343. This is a challenge system paper: parallel Whisper-large-v3 + mHuBERT-147 encoders concatenated into a projector, then into Qwen2.5-7B with LoRA, plus a language prompt and a tri-stage training schedule. The headline, 11.76% CER/WER on the MLC-SLM blind eval set, is internally consistent and the eval uses MeetEval, so I don't doubt the arithmetic. The gain over the baseline is large. What's new is the specific combination and the benchmark result; the components are all known.

What the paper does well: the system description is clear, the tri-stage training rationale (joint training fails to converge) is honestly reported, and they properly hold out the official dev set. The 1/4-data ablation shows both dual-encoder variants beat the Whisper-only baseline, so the parallel-encoder idea has some support.

The soft spot is exactly where the stress-test points. In Table 2, Whisper+Qwen2Audio gets 18.63% vs Whisper+mHuBERT 19.32% on 1/4 data with frozen added encoders. They adopt mHuBERT anyway, citing "better generalization performance observed in full-data experiments" that never appear in the paper. The final system differs in two ways: full 1500h training and full fine-tuning of mHuBERT. So the reader can't tell whether mHuBERT is actually better under the final protocol, or whether Qwen2Audio would match or beat it. Since the paper attributes the eval improvement to the mHuBERT integration, this is load-bearing. It doesn't sink the 11.76% claim, but it does mean the architecture's justification is incomplete.

Other gaps: no per-language results, no confidence intervals, no code. Minor: the Qwen2.5-7B reference points to the Qwen3 report, and there are typos. The absence of code is common for challenge papers but worth noting.

Bottom line: a serious ASR researcher working on LLM-based multilingual systems gets a useful recipe and a solid benchmark data point, but should not treat the mHuBERT choice as validated. As a paper, it deserves a serious referee, but the referee should send it back for the full-data ablation (or a clear statement that the choice is heuristic) plus per-language numbers. I'd read it for the result, not for the architecture conclusion.

Referee Report

3 major / 6 minor

Summary. The paper describes SHNU-mASR, the authors' submission to Track 1 of the INTERSPEECH 2025 MLC-SLM Challenge. The system uses a parallel speech encoder composed of the Whisper-large-v3 encoder (adapted with LoRA) and the mHuBERT-147 encoder (fully fine-tuned), concatenates their output representations, projects them through a convolutional/MLP projector, and feeds the result together with a language-aware prompt into Qwen2.5-7B, which is adapted with LoRA. Training proceeds in three stages: projector pretraining, encoder adaptation, and LLM adaptation. On the blind evaluation set, the system reports an average CER/WER of 11.76%, compared with 20.17% for the enhanced challenge baseline, an 8.41 absolute (41.7% relative) improvement, using the same 1500-hour training data. The authors attribute the gain primarily to the integration of the mHuBERT auxiliary encoder.

Significance. If the reported numbers are accurate, SHNU-mASR is a strong challenge result and a useful empirical data point: it shows that a parallel-encoder architecture with a LoRA-tuned LLM and a staged training schedule can beat the official baseline on a blind multilingual conversational ASR test set without adding target-domain training data. The evaluation uses the standard MeetEval toolkit on a blind set, and the arithmetic in the tables is internally consistent. The main scientific limitation is that the central architecture choice, mHuBERT over Qwen2Audio, is justified by full-data experiments that are never reported, and the only reported ablation actually favors Qwen2Audio in a setting that differs from the final system. The paper would be acceptable as a challenge system description only after that gap is closed and per-language results are provided.

major comments (3)

[Section 4.2, Table 2] The only reported ablation comparing auxiliary encoders uses one quarter of the training data with the added speech encoders frozen, and in that setting Whisper+Qwen2Audio (18.63%) beats Whisper+mHuBERT (19.32%). The final system instead uses the full 1500-hour training set and full fine-tuning of mHuBERT (Sections 2.1.2 and 2.4), yet the choice of mHuBERT is justified only by 'better generalization performance observed in full-data experiments,' which are not shown. This is load-bearing because Section 4.3 attributes the 11.76% blind-eval result to 'the integration of an additional speech encoder (mHuBERT).' Please include the full-data comparison under the final tri-stage protocol, with per-language results; if that comparison is not available, the causal attribution of the gain to mHuBERT should be withdrawn or explicitly reworded as a design choice based on unpublished experiments.
[Section 3.3, Table 3] Only a pooled average CER/WER is reported across the 11 languages. Because Japanese, Korean, and Thai are scored with CER while the remaining languages are scored with WER, and because the training data is heavily imbalanced (English contributes roughly 500 hours while each other language contributes about 100 hours), the average can hide large per-language failures. The paper should report per-language error rates on the dev set, and on the eval set if the challenge rules permit, along with the number of utterances and confidence intervals or significance tests, so that the 'multilingual' claim can actually be verified.
[Section 4.3, Table 3] The headline comparison is against Baseline-Qwen + Norm + LID, where the LID-aware prompt is applied only at inference time (Section 4.1), whereas SHNU-mASR is trained with the language-aware prompt (Section 2.3). The reported 8.41-point improvement therefore conflates the parallel-encoder architecture with the effect of training-time prompt conditioning. Please add an ablation that trains the baseline with the same language-aware prompt, or explicitly state that the gain is relative to an inference-only-prompt baseline; otherwise the attribution of the improvement to the architecture is not isolated.

minor comments (6)

[Section 4.3, first bullet] The sentence 'Fine-tuning only the projector and LLM via LoRA' contradicts Sections 2.1.2 and 2.4, where Whisper is adapted with LoRA and mHuBERT is fully fine-tuned; please correct this misdescription of the final system.
[Introduction and Section 3, headings] There are several typos: 'aoucstic' should be 'acoustic', 'Experment Setup' should be 'Experiment Setup', and 'evaulation set' should be 'evaluation set'.
[References] Reference [25] is cited for Qwen2.5-7B, but the entry is titled 'Qwen3 technical report'; please cite the correct Qwen2.5 model report or correct the model name in the text.
[Section 2.3] The paper does not specify how the language label in the language-aware prompt is obtained at inference time, i.e., whether it comes from ground-truth metadata or from a separate language identification module; please clarify, since this prompt is part of the final system.
[Abstract and Section 5] The claim 'without increasing the baseline training data' is true for the challenge corpus, but the system uses additional pretrained components such as mHuBERT and Qwen2.5-7B; consider clarifying that the comparison holds the target-domain training data fixed rather than the total parameter budget or pretraining data.
[Tables 1 and 3] The baseline is named inconsistently as 'Baseline-Qwen + Norm + LID' and 'Baseline-Qwen + LID + Norm'; please use a single name throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports external benchmark results from a challenge system; no claimed prediction reduces to a fitted parameter or self-citation chain.

full rationale

This is an empirical challenge-system paper. The central claim, 11.76% CER/WER on the blind evaluation set, is an externally scored outcome, not a quantity derived from the model's own fitted parameters by construction. No equation in the paper defines a predicted value in terms of the training targets or test-set observations; the language-aware prompt (Eq. 7-9) is applied consistently at training and inference, so its effect is an ablation-style comparison rather than a fit-as-prediction. The choice of mHuBERT over Qwen2-Audio is justified in Section 4.2 by 'the better generalization performance observed in full-data experiments,' but those experiments are not shown, and the reported 1/4-data ablation actually favors Qwen2-Audio (18.63% vs. 19.32%). This is a real evidence gap for the architecture decision, but it is not circular: the shown ablation does not define the final 11.76% result, and the missing full-data experiment is not equivalent by construction to any reported number. Citations to prior work, including [24], are used as contextual support for mHuBERT's adaptability and are not load-bearing self-citations by the present authors. Overall, the paper's results stand or fall on external benchmark evaluation and reproducibility, not on a circular derivation chain.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

This is an empirical systems paper: all model weights are learned from data, and the free parameters listed are the hand-chosen hyperparameters and design choices that materially affect the reported result. The axioms are the external pre-trained models and evaluation assumptions that the paper does not verify.

free parameters (5)

LoRA rank and alpha for LLM = rank=16, alpha=8
Hand-chosen hyperparameters for Qwen2.5-7B adaptation that affect how strongly the LLM is fine-tuned.
LoRA rank and alpha for Whisper encoder = rank=8, alpha=16
Hand-chosen hyperparameters for parameter-efficient adaptation of Whisper-large-v3.
Projector temporal downsampling factor = 4x (strided convolution)
Determines the sequence length fed to the LLM; chosen to reduce computational cost.
Learning rate = 1.0e-04
Standard Adam optimizer setting; affects convergence and final performance.
Training epochs = 6
Number of passes over the 1500-hour training set; early stopping is implied via internal validation but not described.

assumptions (3)

domain assumption Whisper-large-v3, mHuBERT-147, and Qwen2.5-7B are valid pre-trained models whose features are suitable for concatenation and projection into a shared space.
The entire architecture assumes these off-the-shelf components work together without special alignment.
domain assumption The challenge-provided 1500-hour training set and ~2-hour-per-language development set are used without leakage; the official dev set is held out.
Section 3.1 describes this split; the reliability of the reported eval numbers depends on this assumption.
domain assumption MeetEval computes CER/WER consistently with the challenge evaluation protocol.
Section 3.3 specifies the toolkit; the correctness of the reported metrics is taken on faith.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge." pith.science (2026). https://pith.science/paper/F254GRH5

@misc{pith2026250703343,
  author       = {Pith},
  title        = {Pith review of: SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F254GRH5}},
  note         = {Machine review of arXiv:2507.03343}
}

read the original abstract

This paper describes SHNU multilingual conversational speech recognition system (SHNU-mASR, team name-"maybe"), submitted to Track 1 of the INTERSPEECH 2025 MLC-SLM Challenge. Our system integrates a parallel-speech-encoder architecture with a large language model (LLM) to form a unified multilingual ASR framework. The parallel-speech-encoder consists of two pre-trained encoders, the Whisper-large-v3 encoder and mHuBERT-147 encoder. Their output embeddings are concatenated and fed into the LLM, enabling the model to leverage complementary acoustic and linguistic knowledge and achieve competitive performance. Moreover, we adopt a tri-stage training strategy to jointly update the low-rank adaptation modules and projector parameters of both the speech encoders and the LLM. In addition, we incorporate an additional language-aware prompt at the LLM input to enhance language-specific text generation. The SHNU-mASR system achieves an overall character/word error rate (CER/WER) of 11.76% on the blind evaluation set of the challenge, outperforming the official MLC-SLM baseline by 8.41 absolute CER/WER, without increasing the baseline training data.

Figures

Figures reproduced from arXiv: 2507.03343 by the authors.

**Figure 1.** Overall model structure. (a)The baseline structure of MLC-SLM. (b)SHNU-mASR adopts a parallel-speech-encoder design, followed by a projection module before integration with the LLM.(c) Projector block architecture details. 2.2. Projector To leverage the complementary strengths of Whisper and mHuBERT, we concatenate their output hidden states to form a unified representation for the downstream large language model … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 24 canonical work pages

[1]

Multilin- gual ASR (mASR) extends this capability to multiple languages within a single system, enhancing global communication and supporting low-resource languages

Introduction Automatic Speech Recognition (ASR) has advanced signifi- cantly, enabling accurate conversion of speech to text. Multilin- gual ASR (mASR) extends this capability to multiple languages within a single system, enhancing global communication and supporting low-resource languages. By learning shared repre- sentations, it improves both recognitio...

work page 2025
[2]

SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

Proposed SHNU-mASR System Our proposed SHNU-mASR system mainly composed of three modules: a parallel-speech-encoder, a projector and a language- aware prompted LLM. The whole architecture of SHNU-mASR is illustrated in Fig. 1. arXiv:2507.03343v2 [cs.CL] 8 Jul 2025 2.1. Parallel-speech-encoder The parallel speech encoder consists of two complementary en- c...

work page Pith review arXiv 2025
[3]

Experment Setup 3.1. Datasets The training set consists of multilingual conversational speech data across 11 languages: English (en), French (fr), German (de), Italian (it), Portuguese (pt), Spanish (es), Japanese (ja), Korean (ko), Russian (ru), Thai (th), and Vietnamese (vi). Each audio clip features a natural dialogue between two speakers dis- cussing ...

work page
[4]

Results and Discussions 4.1. Baseline with Inference-Time Text Normalization and LID-Aware Prompting on the MLC-SLM dev-set The inference-time text normalization (Norm) process removes repetitive tokens from the LLM output during inference, which helps to reduce spurious repetitions and improves text quality. Table 1: Average CER/WER (%) on the MLC-SLM de...

work page
[5]

Conclusions This paper introduces our submission to the INTER- SPEECH 2025 MLC-SLM Challenge-Multilingual Conversa- tional Speech Recognition track. Our SHNU-mASR system integrates a parallel-speech-encoder architecture with a LoRA- tuned large language model, enhanced by a tri-stage training strategy and language-specific prompting. Without increasing th...

work page 2025
[6]

Robust speech recognition via large-scale weak su- pervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” in Proceedings of International Conference on Ma- chine Learning (ICML), 2023, pp. 28 492–28 518

work page 2023
[7]

Attention Is All You Need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” Advances in neural information processing systems, vol. 30, 2017

work page 2017
[8]

Lora- Whisper: Parameter-Efficient and Extensible Multilingual ASR,

Z. Song, J. Zhuo, Y . Yang, Z. Ma, S. Zhang, and X. Chen, “Lora- Whisper: Parameter-Efficient and Extensible Multilingual ASR,” in Proceedings of Interspeech, 2024, pp. 3934–3938

work page 2024

Show all 31 references

[9]

Multilin- gual distilwhisper: Efficient distillation of multi-task speech mod- els via language-specific experts,

T. P. Ferraz, M. Z. Boito, C. Brun, and V . Nikoulina, “Multilin- gual distilwhisper: Efficient distillation of multi-task speech mod- els via language-specific experts,” in Proceedings of IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP), 202...

2024
[10]

Whisperx: Time- accurate speech transcription of long-form audio,

M. Bain, J. Huh, T. Han, and A. Zisserman, “Whisperx: Time- accurate speech transcription of long-form audio,” inProceedings of Interspeech, 2023, pp. 4489–4493

2023
[11]

Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling,

S. Gandhi, P. von Platen, and A. M. Rush, “Distil-whisper: Robust knowledge distillation via large-scale pseudo labelling,” arXiv preprint arXiv:2311.00430, 2023

2023 arXiv
[12]

Mixture-of- Expert Conformer for Streaming Multilingual ASR,

K. Hu, B. Li, T. Sainath, Y . Zhang, and F. Beaufays, “Mixture-of- Expert Conformer for Streaming Multilingual ASR,” in Proceed- ings of Interspeech, 2023, pp. 3327–3331

2023
[13]

Language-routing mixture of experts for multilingual and code-switching speech recognition,

W. Wang, G. Ma, Y . Li, and B. Du, “Language-routing mixture of experts for multilingual and code-switching speech recognition,” in Proceedings of Interspeech, 2023, pp. 1389–1393

2023
[14]

BLR- MoE: Boosted language-routing mixture of experts for domain- robust multilingual E2E ASR,

G. Ma, W. Wang, L. Zhou, Y . Yang, Y . Li, and B. Du, “BLR- MoE: Boosted language-routing mixture of experts for domain- robust multilingual E2E ASR,” in Proceedings of IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025

2025
[15]

SC-MoE: Switch conformer mixture of experts for unified streaming and non-streaming code- switching ASR,

S. Ye, S. Chen, X. Hu, and X. Xu, “SC-MoE: Switch conformer mixture of experts for unified streaming and non-streaming code- switching ASR,” in Proceedings of Interspeech, 2024, pp. 3999– 4003

2024
[16]

Saml: Speaker adaptive mixture of lora experts for end-to-end ASR,

Q. Zhao, G. Sun, C. Zhang, M. Xu, and T. F. Zheng, “Saml: Speaker adaptive mixture of lora experts for end-to-end ASR,” in Proceedings of Interspeech, 2024, pp. 777–781

2024
[17]

mHuBERT-147: A compact multilingual HuBERT model,

M. Zanon Boito, V . Iyer, N. Lagos, L. Besacier, and I. Calapode- scu, “mHuBERT-147: A compact multilingual HuBERT model,” in Proceedings of Interspeech, 2024, pp. 3939–3943

2024
[18]

MSR- 86K: An evolving, multilingual corpus with 86,300 hours of tran- scribed audio for speech recognition research,

S. Li, Y . You, X. Wang, Z. Tian, K. Ding, and G. Wan, “MSR- 86K: An evolving, multilingual corpus with 86,300 hours of tran- scribed audio for speech recognition research,” in Proceedings of Interspeech, 2024, pp. 1245–1249

2024
[19]

ML-SUPERB 2.0: Benchmarking multilin- gual speech models across modeling constraints, languages, and datasets,

J. Shi, S.-H. Wang, W. Chen, M. Bartelds, V . Bannihatti Ku- mar, J. Tian, X. Chang, D. Jurafsky, K. Livescu, H. yi Lee, and S. Watanabe, “ML-SUPERB 2.0: Benchmarking multilin- gual speech models across modeling constraints, languages, and datasets,” in Proceedings of Interspe...

2024
[20]

MLS: A large-scale multilingual dataset for speech research,

V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “MLS: A large-scale multilingual dataset for speech research,” in Proceedings of Interspeech, 2020, pp. 2757–2761

2020
[21]

Anatomy of industrial scale multilingual ASR,

F. M. Ramirez, L. Chkhetiani, A. Ehrenberg, R. McHardy, R. Botros, Y . Khare, A. Vanzo, T. Peyash, G. Oexle, M. Liang, I. Sklyar, E. Fakhan, A. Etefy, D. McCrystal, S. Flamini, D. Do- nato, and T. Yoshioka, “Anatomy of industrial scale multilingual ASR,” CoRR, vol. abs/2404.09...

2024 arXiv
[22]

LLaMA: Open and efficient foundation language models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “LLaMA: Open and efficient foundation language models,” CoRR, vol. abs/2302.13971, 2023

2023 arXiv
[23]

BERT: Pre- training of deep bidirectional transformers for language under- standing,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language under- standing,” in Proceedings of North American Chapter of the As- sociation for Computational Linguistics, 2019

2019
[24]

Ideal-LLM: Integrating dual encoders and language-adapted LLM for multilingual speech-to-text,

H. Xue, W. Ren, X. Geng, K. Wei, L. Li, Q. Shao, L. Yang, K. Diao, and L. Xie, “Ideal-LLM: Integrating dual encoders and language-adapted LLM for multilingual speech-to-text,” arXiv preprint arXiv:2409.11214, 2024

2024 arXiv
[25]

SALMONN: Towards generic hearing abilities for large language models,

C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. MA, and C. Zhang, “SALMONN: Towards generic hearing abilities for large language models,” in Proceedings of The Twelfth Inter- national Conference on Learning Representations, 2024

2024
[26]

MinMo: A multimodal large language model for seamless voice interaction,

Q. Chen, Y . Chen, Y . Chen, M. Chen, Y . Chen, C. Deng, Z. Du, R. Gao, C. Gao, Z. Gao et al. , “MinMo: A multimodal large language model for seamless voice interaction,” arXiv preprint arXiv:2501.06282, 2025

2025 arXiv
[27]

Qwen2-audio technical report,

Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

2024 arXiv
[28]

LoRA: Low-rank adaptation of large language models

E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “LoRA: Low-rank adaptation of large language models.” Proceedings of ICLR, vol. 1, no. 2, p. 3, 2022

2022
[29]

Unveiling the potential of LLM-based asr on chinese open-source datasets,

X. Geng, T. Xu, K. Wei, B. Mu, H. Xue, H. Wang, Y . Li, P. Guo, Y . Dai, L. Li et al., “Unveiling the potential of LLM-based asr on chinese open-source datasets,” in Proceedings of International IEEE 14th Symposium on Chinese Spoken Language Processing (ISCSLP), 2024, pp. 26–30

2024
[30]

Qwen3 technical report,

A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv et al., “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025

2025 arXiv
[31]

MeetEval: A toolkit for computation of word error rates for meeting transcription systems,

T. von Neumann, C. Boeddeker, M. Delcroix, and R. Haeb- Umbach, “MeetEval: A toolkit for computation of word error rates for meeting transcription systems,” in Proceedings of 7th International Workshop on Speech Processing in Everyday En- vironments (CHiME 2023), 2023, pp. 27–32

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Multilin- gual ASR (mASR) extends this capability to multiple languages within a single system, enhancing global communication and supporting low-resource languages

Introduction Automatic Speech Recognition (ASR) has advanced signifi- cantly, enabling accurate conversion of speech to text. Multilin- gual ASR (mASR) extends this capability to multiple languages within a single system, enhancing global communication and supporting low-resource languages. By learning shared repre- sentations, it improves both recognitio...

work page 2025

[2] [2]

SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

Proposed SHNU-mASR System Our proposed SHNU-mASR system mainly composed of three modules: a parallel-speech-encoder, a projector and a language- aware prompted LLM. The whole architecture of SHNU-mASR is illustrated in Fig. 1. arXiv:2507.03343v2 [cs.CL] 8 Jul 2025 2.1. Parallel-speech-encoder The parallel speech encoder consists of two complementary en- c...

work page Pith review arXiv 2025

[3] [3]

Experment Setup 3.1. Datasets The training set consists of multilingual conversational speech data across 11 languages: English (en), French (fr), German (de), Italian (it), Portuguese (pt), Spanish (es), Japanese (ja), Korean (ko), Russian (ru), Thai (th), and Vietnamese (vi). Each audio clip features a natural dialogue between two speakers dis- cussing ...

work page

[4] [4]

Results and Discussions 4.1. Baseline with Inference-Time Text Normalization and LID-Aware Prompting on the MLC-SLM dev-set The inference-time text normalization (Norm) process removes repetitive tokens from the LLM output during inference, which helps to reduce spurious repetitions and improves text quality. Table 1: Average CER/WER (%) on the MLC-SLM de...

work page

[5] [5]

Conclusions This paper introduces our submission to the INTER- SPEECH 2025 MLC-SLM Challenge-Multilingual Conversa- tional Speech Recognition track. Our SHNU-mASR system integrates a parallel-speech-encoder architecture with a LoRA- tuned large language model, enhanced by a tri-stage training strategy and language-specific prompting. Without increasing th...

work page 2025

[6] [6]

Robust speech recognition via large-scale weak su- pervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” in Proceedings of International Conference on Ma- chine Learning (ICML), 2023, pp. 28 492–28 518

work page 2023

[7] [7]

Attention Is All You Need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” Advances in neural information processing systems, vol. 30, 2017

work page 2017

[8] [8]

Lora- Whisper: Parameter-Efficient and Extensible Multilingual ASR,

Z. Song, J. Zhuo, Y . Yang, Z. Ma, S. Zhang, and X. Chen, “Lora- Whisper: Parameter-Efficient and Extensible Multilingual ASR,” in Proceedings of Interspeech, 2024, pp. 3934–3938

work page 2024