The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge

Ming Cheng; Ming Li; Yuke Lin; Ze Li

REVIEW 3 major objections 6 minor 27 references

The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A diarization-aware LLM pipeline reaches 18.08% tcpWER on multilingual multi-speaker ASR.

desk verdict A competent challenge-system report whose headline gain over its own prior model is likely confounded with in-domain fine-tuning, but whose official benchmark scores are externally grounded. read the letter →

arxiv 2507.09499 v1 pith:RHMMCDIK submitted 2025-07-13 eess.AS cs.SD

Yuke Lin , Ming Cheng , Ze Li , Ming Li This is my paper · ORCID

classification eess.AScs.SD

keywords multi-speakerASRspeakerdiarizationlargelanguagemodellanguage-specificadaptersLoRAfine-tuningtcpWERMLC-SLMchallengeembeddings

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports the authors' entry to Task 2 of the MLC-SLM Challenge, where the goal is to transcribe raw, unsegmented audio with multiple speakers and attribute each utterance to the right speaker without oracle labels or time boundaries. The system chains a neural diarization model, a speaker-embedding extractor, and a Qwen2.5-based language model decoder that consumes speaker-enrollment triplets made of embedding, start time, and end time. The central claim is that this pipeline achieves 23.56% tcpWER on the development set and 18.08% on the test set, against 76.12% and 60.39% for the official baseline. It further claims that fine-tuning separate language-specific adapters and LoRA modules improves the test score from 20.44% to 18.08% relative to the base diarization-aware model. A sympathetic reader would take this as evidence that a frozen LLM with lightweight per-language adaptation can handle multilingual multi-speaker transcription when given diarization-derived speaker context.

What carries the argument

The load-bearing mechanism is the speaker-enrollment triplet: a per-utterance tuple of speaker embedding, start time, and end time, built from the output of the S2SND diarization model and a frozen ResNet34 speaker-verification model. Each triplet acts as a reference instruction that tells the LLM decoder which voice to attend to over which time window, and multiple triplets are processed in a single decoding pass. The audio is encoded by a frozen Whisper-large-v3 semantic encoder and a frozen ResNet34 speaker encoder whose outputs are fused by a gated cross-attention mechanism, after which only the adapters and LoRA modules of the Qwen2.5 decoder are trainable. Language-specific adaptation works by training a separate adapter and LoRA branch for each target language and selecting the branch according to the provided language label.

What would settle it

Remove or corrupt the language labels at test time, forcing each utterance through a randomly chosen adapter/LoRA branch, and compare the resulting tcpWER to the 18.08% test score. If the error does not rise meaningfully, the language-specific component is not what drives the reported gain.

Watch

Extended reading notes

Core claim

The paper's discovery is a working system rather than a new theory: on the MLC-SLM dataset, the pipeline obtains a time-constrained permutation word error rate of 23.56% on the development set and 18.08% on the test set, outperforming the official baseline by more than 52 and 42 percentage points respectively. Replacing the generic model's adapters with language-specific adapters and LoRA branches lowers the test tcpWER from 20.44% to 18.08%, with the largest gains in French, Japanese, Russian, and Spanish, while English, Thai, and Italian stay flat or regress slightly. The paper attributes the improvement to the diarization-aware triplet decoding strategy, in which each speaker's embedding and time interval are fed to the LLM as an instruction to transcribe that speaker's utterance. Language identity is provided by the task and used at inference to select the matching adapter branch.

Load-bearing premise

The system assumes that every test utterance comes with a correct language label, because the language-specific adapter and LoRA branch must be selected per utterance before decoding.

Editorial extensions

If this is right

The system can transcribe and attribute speech without oracle speaker labels or time boundaries, since the S2SND diarization model supplies the boundaries and the embedding extractor supplies the speaker identity.
Language-specific adapter and LoRA fine-tuning improves the overall test tcpWER from 20.44% to 18.08% while leaving the backbone encoders and LLM frozen, so per-language specialization costs only a small number of trainable parameters.
The largest gains appear in French, Japanese, Russian, and Spanish, while high-resource English and some languages that already performed well show marginal or slightly negative changes.
Because tcpWER jointly measures word accuracy and temporal alignment of speaker turns, the reported numbers imply that the final system improves both recognition content and speaker-time attribution relative to the baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the per-language gains are robust, the same adapter-and-LoRA partitioning could be applied to dialects or acoustic domains, not just languages, whenever the test-time category is known.
The reliance on provided language identity means the system's robustness to missing or mislabeled language metadata is untested; an automatic language classifier would be a natural extension and could change the observed gains.
The 14.27% diarization error rate on the development set suggests that improving the upstream diarization could yield further tcpWER reductions, since transcription is conditioned on the diarized time boundaries.
The reported comparison is a system-level contest result rather than a component-level ablation, so the relative contribution of diarization, embeddings, gated fusion, and language-specific adapters is not separately established.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a concise system description for the MLC-SLM Challenge Task 2. What’s actually new is modest: the authors take their prior diarization-aware MS-ASR framework [13] and add per-language adapter/LoRA branches in the LLM decoder. The benchmark numbers come from the official evaluation server, so the bottom-line tcpWER improvement over the provided baseline is real, externally grounded evidence. The paper is clearly written and honest enough to note that some languages regress or show negligible gains.

The soft spots are in proportion. First, the comparison that supports the claimed benefit of language-specific fine-tuning — MS-ASR* at 18.08% test tcpWER versus MS-ASR at 20.44% — is not a controlled one as reported. Section 2.4 says implementation details are “entirely consistent with” [13] and that the authors “additionally fine-tune” adapters/LoRA, but it never says the MS-ASR row was first adapted to the MLC-SLM training set. If the 20.44% row is the originally published model applied off-the-shelf, then the 2.36-point gap may reflect in-domain fine-tuning plus adapter specialization, not specialization alone. That concern does survive a close reading. It doesn’t invalidate the final system’s scores, but it does weaken the specific novelty claim.

Second, the paper provides no code, no training configuration, no hyperparameters, no error bars, and no per-language test results (only dev per-language, average test). For a challenge report this is common, but it means the results are not independently reproducible from the text. Third, the system assumes language identity is known per utterance to select the right branch; the paper acknowledges this implicitly but doesn’t discuss the failure mode if the label is wrong or absent. That’s a real limitation for deployment.

The architecture itself is not new, the incremental contribution is not conceptually deep, and the self-citation density is high — but that is normal for a system built on the group’s own prior work. The writing is competent, and the external evaluation gives the main numbers credibility.

Who is this for? Someone tracking MLC-SLM or building multi-speaker ASR from this framework will find it useful as a status report. It deserves a serious referee, but the referee’s main ask should be for a matched baseline: retrain or at least fine-tune the non-language-specific MS-ASR model on the same MLC-SLM data before attributing the gain to per-language specialization. Also request code or detailed configs. On balance, conditional acceptance for the challenge track is right; if this were a general ASR paper, I’d want the control condition fixed first.

Referee Report

3 major / 6 minor

Summary. The manuscript reports the DKU system for Task 2 of the MLC-SLM Challenge, which requires multi-speaker automatic speech recognition from raw audio without oracle speaker labels or time boundaries. The system cascades a sequence-to-sequence neural diarization model (S2SND-Small), a frozen ResNet34 speaker-embedding extractor, and a Qwen2.5-based diarization-aware ASR model that is conditioned on speaker-enrollment triplets via gated cross-attention. For the challenge, the authors augment the LLM decoder with language-specific adapters and LoRA modules, selecting the branch on the basis of the input language label. The system reports tcpWER of 23.56% on the development set and 18.08% on the test set, compared with 24.95%/20.44% for the base MS-ASR model and 76.12%/60.39% for the official baseline.

Significance. The headline result is credible because the test numbers were produced by the official evaluation server and are not under the authors' control. If the comparison between MS-ASR and MS-ASR* is properly controlled, the paper demonstrates a practical benefit of per-language adapter/LoRA specialization in an LLM-based MS-ASR pipeline and a strong challenge outcome. However, the novelty over the authors' prior work [13] is incremental and the paper provides no code, hyperparameters, or error bars, so reproducibility and statistical robustness rest on the challenge report alone. The per-language development results show several regressions, so the aggregate gain is not uniformly distributed.

major comments (3)

[Section 2.4, Table 1] The control condition for the language-specific gain is underspecified. The text says the MS-ASR implementation details are "entirely consistent with" [13] and that the authors "additionally fine-tune" adapters/LoRA, but it never states whether the MS-ASR row in Table 1 was first fine-tuned on the MLC-SLM training data. If the MS-ASR row is the original published model applied without in-domain fine-tuning, then the improvement from 20.44% to 18.08% test tcpWER conflates in-domain adaptation with the effect of the language-specific branches. The authors should either state explicitly that both rows were trained on the same MLC-SLM data, with the only difference being the adapters/LoRA, or provide an additional control row.
[Section 2.4, last paragraph] The system requires the language identity of each test utterance to select the correct adapter/LoRA branch, but the abstract and introduction describe the task as multi-speaker ASR "directly from raw audio" without mentioning this requirement. If the language label is absent, noisy, or incorrect, the reported gains would not materialize. The authors should state the language-label assumption explicitly in the abstract or introduction and discuss robustness to language misclassification or code-switching.
[Table 1, Section 3] Several languages regress on the development set under the language-specific model (e.g., English-American 23.01 to 23.52, English-Australian 14.40 to 15.37, English-British 18.69 to 19.15, Italian 19.90 to 21.10), yet Section 3 concludes that language-specific adaptation "can lead to further improvements generally." The aggregate test gain is 2.36 points, but without per-language test scores or variance estimates it is unclear whether the regressions are noise or systematic domain mismatch. The authors should provide per-language test results if available, or temper the general-improvement claim.

minor comments (6)

[Section 2.2] The phrase "without requiring Oracle V AD" appears to contain a typo and should read "oracle VAD."
[Section 2.2] The names "V oxCeleb2" and "V oxBlink2" contain spacing artifacts and should be rendered as "VoxCeleb2" and "VoxBlink2."
[Section 4] The sentence "we build a ASR system" should read "we build an ASR system."
[Table 1] The column header "Baseline 2" is ambiguous; the authors should clarify the official baseline designation and state whether test-set per-language scores are unavailable from the server or simply omitted.
[Section 2.4] The term "adapter" is used both for the projection module in the gated fusion path and for the language-specific adapter parameters; the authors should clarify whether these are the same or distinct components.
[General] Because Section 2.4 delegates all implementation details to the original paper [13], the report is not self-contained; providing a link to released code or a brief training configuration would improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: benchmark scores are external; self-citations describe components with independent validation.

full rationale

The paper's main quantitative claims are challenge-set tcpWER numbers obtained through the official evaluation server, so they are externally grounded and not derived from the paper's own assumptions. The system architecture is inherited from the authors' prior work [13], but the paper does not use that citation to justify any performance property; it reports new results on the MLC-SLM benchmark. The diarizer [18] and embedding extractor [22] are also self-cited, yet their effectiveness is supported by independent metrics (DER 14.27% on the MLC-SLM development set and EER 1.52% on Vox-O), not by assumption. The only comparison that could raise concern is the attribution of the MS-ASR* gain over MS-ASR to language-specific adapter/LoRA branches, because Section 2.4 states that implementation details are 'entirely consistent with its original paper [13]' and does not explicitly say whether the base MS-ASR row was fine-tuned on the MLC-SLM training set. This is a potential confound between in-domain adaptation and language specialization, but it is not circularity: the numbers are measured, not derived from the claim. Similarly, the requirement that language identity be known at inference is an explicit assumption, not a circular one. No equation, fitted parameter, or predicted quantity reduces to its own input by construction. The self-citations are present but not load-bearing in a circular sense, so the paper receives a low score.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities or postulates. It relies on prior self-cited models and standard training procedures. The only fitted parameters are the language-specific adapter and LoRA weights, whose values are not disclosed.

free parameters (1)

Language-specific adapter and LoRA weights = not reported
The reported gains depend on fine-tuning these parameters on per-language data, but the paper gives no details on the fitted values or training sizes.

assumptions (4)

domain assumption The Diarization-aware Multi-speaker ASR framework described in [13] works as stated.
The paper states implementation details are entirely consistent with its original paper [13], relying on that work without re-deriving it.
domain assumption The S2SND diarization outputs are accurate enough to provide useful triplets.
The system's input triplets depend on diarization; a DER of 14.27% is reported on the development set, but downstream sensitivity is not analyzed.
domain assumption The frozen Whisper-large-v3 and ResNet34 encoders provide sufficient semantic and speaker features.
No fine-tuning of these encoders is performed, so their quality is assumed from prior work.
domain assumption Language identity is available for each test utterance.
Adapter selection depends on this assumption, as stated in Section 2.4: 'the language identity is provided as part of the task specification'.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge." pith.science (2026). https://pith.science/paper/RHMMCDIK

@misc{pith2026250709499,
  author       = {Pith},
  title        = {Pith review of: The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RHMMCDIK}},
  note         = {Machine review of arXiv:2507.09499}
}

read the original abstract

We present the DKU system for Task 2 of the MLC-SLM Challenge, which aims to perform multi-speaker automatic speech recognition directly from raw audio without Oracle speaker labels or time boundaries. Our approach builds upon a diarization-aware framework integrating speaker embeddings and temporal utterance boundaries into a Qwen2.5-based large language model (LLM). Then, we enhance the system's multilingual performance by fine-tuning language-specific adapters and LoRA modules within the LLM decoder. Finally, our system achieves the tcpWER of 23.56\% and 18.08\% on the development and test sets of the MLC-SLM dataset, substantially outperforming the official baseline.

Figures

Figures reproduced from arXiv: 2507.09499 by the authors.

**Figure 1.** Overview of our developed system. 2. System Description 2.1. Overall Architecture Our system adopts a diarization-aware pipeline architecture for multi-speaker automatic speech recognition, shown in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 17 canonical work pages

[13]

Recognizing multi-talker speech with permutation invariant training,

D. Yu, X. Chang, and Y . Qian, “Recognizing multi-talker speech with permutation invariant training,” in Proc. Interspeech, 2017, pp. 2456–2460

work page 2017
[1]

who spoke what and when

Introduction Automatic speech recognition (ASR) has achieved remarkable success in single-speaker scenarios [1–3], but real-world con- versations often involve multiple speakers, overlapping speech, and code-switching, posing significant challenges to current ap- proaches. Multi-speaker ASR (MS-ASR) [4–11] aims to ad- dress the “who spoke what and when” p...

work page
[2]

The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge

System Description 2.1. Overall Architecture Our system adopts a diarization-aware pipeline architecture for multi-speaker automatic speech recognition, shown in Fig. 1. The system comprises three main components: the speaker di- arization model, the embedding extractor, and the diarization- aware multi-speaker ASR system. The input audio is firstly proce...

work page Pith review arXiv 2025
[3]

All systems are evaluated on the development set, while only average scores are available on the test set via the official evaluation server

Experimental Results Table 1 presents the tcpWER results of three systems evalu- ated on the MLC-SLM dataset: the official baseline, the origi- nal version of MS-ASR model [13], and the language-specific MS-ASR* model developed for this challenge. All systems are evaluated on the development set, while only average scores are available on the test set via...

work page
[4]

Conclusions In this work, we build a ASR system for the MLC-SLM Challenge based on the Diarization-aware Multi-speaker ASR framework. The system first performs speaker diarization and embedding extraction to obtain speaker-enrollment triplets, which are then used to guide a large language model for speaker-attributed transcription. For this challenge, we ...

work page
[5]

Conformer: Convolution-augmented transformer for speech recognition,

A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” in Proc. Interspeech, 2020, pp. 5036–5040

2020
[6]

Recent advances in end-to-end automatic speech recognition,

J. Li et al. , “Recent advances in end-to-end automatic speech recognition,” APSIPA Transactions on Signal and Information Processing, vol. 11, no. 1, 2022

work page 2022
[7]

Zipformer: A faster and better encoder for automatic speech recognition,

Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y . Yang, Z. Jin, L. Lin, and D. Povey, “Zipformer: A faster and better encoder for automatic speech recognition,” in Proc. ICLR, 2024

work page 2024

Show all 27 references

[8]

Survey of end-to-end multi-speaker au- tomatic speech recognition for monaural audio,

X. He and J. Whitehill, “Survey of end-to-end multi-speaker au- tomatic speech recognition for monaural audio,” arXiv preprint arXiv:2505.10975, 2025

2025 arXiv
[9]

The chime-7 dasr challenge: Distant meeting transcrip- tion with multiple devices in diverse scenarios,

S. Cornell, M. Wiesner, S. Watanabe, D. Raj, X. Chang, P. Gar- cia, M. Maciejewski, Y . Masuyama, Z.-Q. Wang, S. Squartini et al., “The chime-7 dasr challenge: Distant meeting transcrip- tion with multiple devices in diverse scenarios,” arXiv preprint arXiv:2306.13734, 2023

2023 arXiv
[10]

The chime-7 challenge: System description and performance of nemo team’s dasr system,

T. J. Park, H. Huang, A. Jukic, K. Dhawan, K. C. Puvvada, N. Koluguri, N. Karpov, A. Laptev, J. Balam, and B. Ginsburg, “The chime-7 challenge: System description and performance of nemo team’s dasr system,” arXiv preprint arXiv:2310.12378 , 2023

arXiv
[11]

Lexical speaker error cor- rection: Leveraging language models for speaker diarization error correction,

R. Paturi, S. Srinivasan, and X. Li, “Lexical speaker error cor- rection: Leveraging language models for speaker diarization error correction,” in Proc. Interspeech, 2023, pp. 3567–3571

2023
[12]

Di- arizationlm: Speaker diarization post-processing with large lan- guage models,

Q. Wang, Y . Huang, G. Zhao, E. Clark, W. Xia, and H. Liao, “Di- arizationlm: Speaker diarization post-processing with large lan- guage models,” in Proc. Interspeech, 2024, pp. 3754–3758

2024
[14]

Seri- alized output training for end-to-end overlapped speech recogni- tion,

N. Kanda, Y . Gaur, X. Wang, Z. Meng, and T. Yoshioka, “Seri- alized output training for end-to-end overlapped speech recogni- tion,” in Proc. Interspeech, 2020, pp. 2797–2801

2020
[15]

Streaming speaker-attributed asr with token-level speaker embeddings,

N. Kanda, J. Wu, Y . Wu, X. Xiao, Z. Meng, X. Wang, Y . Gaur, Z. Chen, J. Li, and T. Yoshioka, “Streaming speaker-attributed asr with token-level speaker embeddings,” in Proc. Interspeech, 2022, pp. 521–525

2022
[16]

Meeteval: A toolkit for computation of word error rates for meeting transcription systems,

T. v. Neumann, C. B. Boeddeker, M. Delcroix, and R. Haeb- Umbach, “Meeteval: A toolkit for computation of word error rates for meeting transcription systems,” in Proc. CHiME, 2023, pp. 27–32

2023
[17]

Diarization-aware multi-speaker automatic speech recognition via large language models,

Y . Lin, M. Cheng, Z. Li, B. Tang, and M. Li, “Diarization-aware multi-speaker automatic speech recognition via large language models,” arXiv preprint arXiv:2506.05796, 2025

2025 arXiv
[18]

Llama: Open and efficient foundation language models,

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

2023 arXiv
[19]

Qwen2.5: A party of foundation models,

Q. Team, “Qwen2.5: A party of foundation models,” September
[20]

Robust speech recognition via large-scale weak su- pervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” in Proc. ICML. PMLR, 2023, pp. 28 492–28 518

2023
[21]

Deepseek-v3 technical re- port,

A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseek-v3 technical re- port,” arXiv preprint arXiv:2412.19437, 2024

2024 arXiv
[22]

Lora: Low-rank adaptation of large language models

E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al. , “Lora: Low-rank adaptation of large language models.” Proc. ICLR, vol. 1, no. 2, p. 3, 2022

2022
[23]

Sequence-to-sequence neural di- arization with automatic speaker detection and representation,

M. Cheng, Y . Lin, and M. Li, “Sequence-to-sequence neural di- arization with automatic speaker detection and representation,” arXiv preprint arXiv:2411.13849, 2024

2024 arXiv
[24]

Deep residual learning for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. CVPR, 2016, pp. 770–778

2016
[26]

V oxceleb2: Deep speaker recognition,

J. S. Chung, A. Nagrani, and A. Zisserman, “V oxceleb2: Deep speaker recognition,” in Proc. Interspeech, 2018, pp. 1086–1090

2018
[27]

V oxblink2: A 100k+ speaker recognition corpus and the open- set speaker-identification benchmark,

Y . Lin, M. Cheng, F. Zhang, Y . Gao, S. Zhang, and M. Li, “V oxblink2: A 100k+ speaker recognition corpus and the open- set speaker-identification benchmark,” inProc. Interspeech, 2024, pp. 4263–4267

2024
[2024]

Available: https://qwenlm.github.io/blog/qwen2

[Online]. Available: https://qwenlm.github.io/blog/qwen2. 5/

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [13]

Recognizing multi-talker speech with permutation invariant training,

D. Yu, X. Chang, and Y . Qian, “Recognizing multi-talker speech with permutation invariant training,” in Proc. Interspeech, 2017, pp. 2456–2460

work page 2017

[2] [1]

who spoke what and when

Introduction Automatic speech recognition (ASR) has achieved remarkable success in single-speaker scenarios [1–3], but real-world con- versations often involve multiple speakers, overlapping speech, and code-switching, posing significant challenges to current ap- proaches. Multi-speaker ASR (MS-ASR) [4–11] aims to ad- dress the “who spoke what and when” p...

work page

[3] [2]

The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge

System Description 2.1. Overall Architecture Our system adopts a diarization-aware pipeline architecture for multi-speaker automatic speech recognition, shown in Fig. 1. The system comprises three main components: the speaker di- arization model, the embedding extractor, and the diarization- aware multi-speaker ASR system. The input audio is firstly proce...

work page Pith review arXiv 2025

[4] [3]

All systems are evaluated on the development set, while only average scores are available on the test set via the official evaluation server

Experimental Results Table 1 presents the tcpWER results of three systems evalu- ated on the MLC-SLM dataset: the official baseline, the origi- nal version of MS-ASR model [13], and the language-specific MS-ASR* model developed for this challenge. All systems are evaluated on the development set, while only average scores are available on the test set via...

work page

[5] [4]

Conclusions In this work, we build a ASR system for the MLC-SLM Challenge based on the Diarization-aware Multi-speaker ASR framework. The system first performs speaker diarization and embedding extraction to obtain speaker-enrollment triplets, which are then used to guide a large language model for speaker-attributed transcription. For this challenge, we ...

work page

[6] [5]

Conformer: Convolution-augmented transformer for speech recognition,

A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” in Proc. Interspeech, 2020, pp. 5036–5040

2020

[7] [6]

Recent advances in end-to-end automatic speech recognition,

J. Li et al. , “Recent advances in end-to-end automatic speech recognition,” APSIPA Transactions on Signal and Information Processing, vol. 11, no. 1, 2022

work page 2022

[8] [7]

Zipformer: A faster and better encoder for automatic speech recognition,

Z. Yao, L. Guo, X. Yang, W. Kang, F. Kuang, Y . Yang, Z. Jin, L. Lin, and D. Povey, “Zipformer: A faster and better encoder for automatic speech recognition,” in Proc. ICLR, 2024

work page 2024