Pith. sign in

REVIEW 3 major objections 5 minor 16 references

The paper reports a multilingual conversational ASR system that freezes both a Whisper-large-v3 encoder and a Qwen2.5-7B-Instruct LLM, training only a small Linear-ReLU-Linear adaptor and rank-64 LoRA, and claims a 9.83% average word/charac

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A frozen Whisper-large-v3 encoder plus a trainable adaptor plus LoRA-adapted Qwen2.5-7B achieves 9.83% WER/CER on 11-language conversational ASR and third place in MLC-SLM 2025 Track 1.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A competent challenge system report with a useful ablation table; the 9.83% figure is a single external measurement, not a reproducible claim. the 3 major comments →

arxiv 2508.14916 v1 pith:N5LYIEKP submitted 2025-08-15 eess.AS cs.AIcs.CL

Transsion Multilingual Speech Recognition System for MLC-SLM 2025 Challenge

Xiaoxiao Li , An Zhu , Youhai Jiang , Fengjie Zhu This is my paper
classification eess.AS cs.AIcs.CL
keywords multilingual automatic speech recognitionEncoder-Adaptor-LLMWhisper-large-v3Qwen2.5LoRAMLC-SLM 2025conversational speechparameter-efficient fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a multilingual conversational speech recognition system built for Track 1 of the MLC-SLM 2025 challenge. The central claim is that a frozen Whisper-large-v3 encoder, a trainable Linear-ReLU-Linear adaptor, and a frozen Qwen2.5-7B-Instruct LLM adapted with rank-64 LoRA reach a 9.83% average word/character error rate across 11 languages and rank third among global participants. A sympathetic reader should care because the result suggests that strong LLM-based ASR can be obtained without updating the large pretrained backbones: only the adaptor and LoRA weights are trained. The paper also shows through ablations that adding a subset of the external MSR-86K corpus and scaling the LLM from 1.5B to 7B parameters each reduce error on the development set.

Core claim

The system follows the Encoder-Adaptor-LLM design. A frozen Whisper-large-v3 encoder processes 128-dimensional Mel filterbank features into 20ms-frame representations; a trainable adaptor splices frames to 40ms and maps the resulting 2560-dimensional input through a Linear-ReLU-Linear block into the 3584-dimensional embedding space of a frozen Qwen2.5-7B-Instruct LLM. The LLM is adapted only through LoRA with rank 64 and alpha 16. Training uses the competition corpus plus a random 11-language subset of MSR-86K, with punctuation removed and text lowercased, formatted with the Qwen chat template. After four epochs with an effective batch size of 64 and a learning rate of 1e-4, followed by a tw

What carries the argument

The load-bearing mechanism is the Encoder-Adaptor-LLM pipeline with both large modules frozen. Whisper-large-v3 supplies acoustic representations at 20ms per frame; a trainable adaptor halves the frame rate and projects through Linear-ReLU-Linear to the LLM's 3584-dimensional embedding space; and the frozen Qwen2.5-7B-Instruct decoder is steered toward transcription only by rank-64 LoRA. The design concentrates all learned task-specific behavior in the small adaptor and LoRA parameters, which is what makes the system parameter-efficient and modular.

Load-bearing premise

The held-out evaluation set is similar to the training distribution: the 1,500-hour competition corpus plus the selected MSR-86K subset, with dev-driven checkpoint averaging and a final dev-inclusive fine-tune, is representative enough for the reported 9.83% to generalize.

What would settle it

Obtain the MLC-SLM 2025 Track 1 evaluation set, run the submitted checkpoints, and recompute per-language WER/CER. If the mean does not reproduce 9.83%, or if a few languages carry the average while most are far worse, the central claim fails. A sharper test is to retrain the identical pipeline without the final two-epoch fine-tune on the dev set and measure the evaluation-set difference.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A 7B-parameter LLM can act as a competitive multilingual ASR decoder without unfreezing its weights, pointing toward parameter-efficient LLM adaptation as a sufficient recipe.
  • External data diversity, even a random subset of MSR-86K, gives consistent error reductions on top of the competition corpus, supporting the value of broad multilingual data.
  • Because the encoder and LLM are frozen, the architecture is modular: upgrading either pretrained backbone would require retraining only the adaptor and LoRA.
  • Adding the development set to the final two epochs improves from the dev-best 10.01% to the submitted 9.83%, meaning the reported evaluation number includes dev-set fine-tuning as part of the pipeline.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the evaluation distribution closely resembles the development data, the dev-inclusive fine-tuning likely contributes to the reported 9.83%, so readers should expect larger errors on genuinely out-of-domain conversational speech.
  • The adaptor's frame-splicing halves the number of decoder steps; on longer conversational turns this efficiency gain could be decisive, and a latency comparison between 20ms and 40ms frame rates would test it.
  • Since the fine-tuned Whisper encoder did not beat the frozen one in initial tests, the frozen encoder's pretrained representations may already be well matched to this task; a more thorough encoder-tuning schedule is left open by the paper.
  • The system's modularity implies a low-cost upgrade path: newer speech encoders or stronger instruction-tuned LLMs could be dropped in and re-evaluated with the same adaptor/LoRA recipe.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes the Transsion Speech Team's system for Track 1 of the MLC-SLM 2025 multilingual conversational ASR challenge. The architecture is an Encoder-Adaptor-LLM pipeline: a frozen Whisper-large-v3 encoder, a trainable frame-splicing Linear-ReLU-Linear adaptor, and a frozen Qwen2.5-7B-Instruct LLM adapted with rank-64 LoRA. The system is trained on the 1,500-hour competition corpus plus a randomly selected, 11-language subset of the MSR-86K dataset, with checkpoint averaging and a final additional 2-epoch fine-tune that includes the development set. The paper reports a final evaluation-set WER/CER of 9.83% across 11 languages and third place in the challenge, with ablations showing progressive gains from data augmentation and model scaling.

Significance. If the reported result holds, the paper demonstrates that a parameter-efficient combination of a frozen Whisper encoder, a small adaptor, and a frozen 7B LLM with LoRA can be highly competitive for multilingual conversational ASR. The ablation study is useful: it quantifies the marginal benefit of the MSR-86K subset and of scaling from Qwen2-1.5B to Qwen2.5-7B, and it compares against fine-tuned Whisper baselines. Concrete training details (batch size, learning rate, LoRA rank/alpha, checkpoint averaging) are given. The main weakness is that the final submission's only evaluation is the external challenge result; no held-out or development-set result is reported for the final Dev-inclusive fine-tuned model, and no uncertainty estimates are provided.

major comments (3)
  1. [Section 3.3] The final system is obtained by incorporating the Dev set into the training data and fine-tuning for another 2 epochs, but no Dev or held-out WER/CER is reported for this final configuration. Table 2 ends at 10.01% for the best pre-final model, and the claimed 9.83% is an external evaluation-set number. As written, the reader cannot separate the effect of adding Dev to training from the effect of the additional epochs, and there is no internal check against overfitting to Dev. Please report a held-out evaluation for the final training procedure (e.g., a split of Dev held out before the final fine-tune) and per-language evaluation numbers, and state the size of Dev and eval sets.
  2. [Table 1 / Section 2] The MSR-86K subset is described only as 'randomly selected' for the 11 languages, with no seed, selection code, membership list, or duration-filtering criteria. Since the MSR-86K subset contributes a 0.78% absolute improvement in the ablation (10.79 vs. 11.57 for Qwen2-1.5B, and 10.01 vs. 11.57 for Qwen2.5-7B), the central result depends on this undocumented choice. Provide the selection protocol and seed, or release the subset metadata; if that is not possible, show stability across multiple subset draws.
  3. [Table 2 / Section 3.1] All WER/CER numbers in Table 2 are point estimates without confidence intervals, per-language breakdowns, or the size of the Dev set. The difference between the best ablation (10.01% Dev) and the final submission (9.83% eval) is 0.18 percentage points, measured on different sets, so without variance information or significance testing the claim that the final Dev-inclusive fine-tune improved performance is not supported. Add Dev/eval utterance and hour counts, report per-language results, and provide bootstrap confidence intervals or an equivalent uncertainty estimate.
minor comments (5)
  1. [Section 1 / Section 2] Typographical issues: 'we presents' should be 'we present', and the Section 2 heading appears as 'DA TASET' instead of 'DATASET'.
  2. [Figure 2] Figure 2 appears to contain corrupted placeholder characters and non-rendered Korean text. Replace with a clean, vector-formatted diagram.
  3. [Section 3.1] The learning-rate schedule is described as linear decay to zero over 1,000,000 steps, while training is performed for 4 epochs plus 2 additional epochs. Clarify whether the schedule restarts for the final 2-epoch fine-tune and how many total steps are actually used; otherwise the effective learning rate at the end of training is ambiguous.
  4. [Table 2] State whether the WER/CER in Table 2 is pooled across languages or macro-averaged, and add the number of Dev utterances/hours used for evaluation.
  5. [Section 3.1] Use consistent notation for LoRA hyperparameters (e.g., LoRA rank and LoRA alpha), and consider a small table for the hyperparameter settings used in all experiments.

Circularity Check

0 steps flagged

No significant circularity: the paper's central claim is a held-out challenge evaluation result, not a derivation from fitted inputs.

full rationale

The paper is a challenge system description whose central claim is an externally measured WER/CER of 9.83% on the MLC-SLM 2025 Track 1 evaluation set. There is no derivation chain whose conclusion is equivalent to its premises: the system architecture (frozen Whisper encoder + Linear-ReLU-Linear adaptor + frozen Qwen2.5-7B-Instruct with LoRA) is an empirical configuration, and the ablation numbers in Table 2 are measurements on the Dev set, not fit constants. The only potentially questionable step is Section 3.3, where the Dev set is incorporated into the final training data and the model is fine-tuned for two more epochs, with the final number reported only on the external evaluation set. This is a standard challenge practice and a reproducibility/generalization caveat rather than a circularity: the evaluation set is held out by the challenge organizers, the paper does not claim to predict this number from any fitted parameter, and no equation in the paper reduces the final result to a training input. The external citations (Whisper, Qwen, MSR-86K, Encoder-Adaptor-LLM baseline) are independent prior work, not self-citations by the present authors, and no uniqueness theorem or ansatz is imported from the authors' own prior publications. Accordingly, the derivation chain, such as it is, is self-contained as an empirical engineering report, and no circular step can be exhibited from the paper's text.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper contributes no new theoretical entities; the central claim rests on standard pretrained models and conventional hyperparameter choices. All numbers affecting the result are hand-selected or inherited from pretraining; no sensitivity analysis is provided, so the ledger is small but unverified.

free parameters (5)
  • LoRA rank = 64
    Chosen by hand without ablation; affects capacity of task adaptation.
  • LoRA alpha = 16
    Scaling factor chosen by hand; interacts with rank.
  • Effective total batch size = 64
    4 per GPU x 8 GPUs times gradient accumulation; chosen for optimization.
  • Initial learning rate = 1e-4
    Chosen by hand, linearly decayed; no sensitivity analysis.
  • Decoding beam size = 6
    Used for all ablations and final inference; no beam sweep reported.
axioms (4)
  • domain assumption The Whisper-large-v3 encoder pretrained weights provide robust acoustic features when frozen.
    The system relies on this in Section 3 and ablation results; no analysis of failure cases.
  • domain assumption The Qwen2.5-7B-Instruct pretrained LLM can be adapted to ASR via LoRA without catastrophic forgetting.
    Assumed from related work [1-5] and used in Section 3; not independently tested.
  • domain assumption Competition speech corpus and MSR-86K subset are correctly transcribed and representative.
    Dataset section Table 1 and training use; transcription quality not verified in the paper.
  • domain assumption Randomly selected MSR-86K subset is unbiased and sufficient.
    Section 2 states random selection due to time constraints; no seed or reproducibility details.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Transsion Multilingual Speech Recognition System for MLC-SLM 2025 Challenge." pith.science (2026). https://pith.science/paper/N5LYIEKP

@misc{pith2026250814916,
  author       = {Pith},
  title        = {Pith review of: Transsion Multilingual Speech Recognition System for MLC-SLM 2025 Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N5LYIEKP}},
  note         = {Machine review of arXiv:2508.14916}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents the architecture and performance of a novel Multilingual Automatic Speech Recognition (ASR) system developed by the Transsion Speech Team for Track 1 of the MLC-SLM 2025 Challenge. The proposed system comprises three key components: 1) a frozen Whisper-large-v3 based speech encoder, leveraging large-scale pretraining to ensure robust acoustic feature extraction; 2) a trainable adaptor module using Linear-ReLU-Linear transformation mechanisms to effectively align speech and text representations; and 3) a frozen Qwen2.5-7B-Instruct large language model (LLM) integrated with trainable LoRA for optimized contextual linguistic decoding. By systematically combining pretrained models with task specific fine-tuning, the system achieved a word/character error rate (WER/CER) of 9.83% across 11 languages in the evaluation set and ranked third place among global participants.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 5 canonical work pages · 1 internal anchor

  1. [1]

    However, the scarcity of real-world conversational speech data , especially in multilingual contexts, poses a significant challenge to advancing the field[6][7][8][9]

    Introduction Large Language Models (LLMs) have emerged as transfor- mative foundation models across language processing tasks, with growing applications in speech and audio domains includ- ing automatic speech recognition (ASR) and spoken dialogue systems[1][2][3][4][5]. However, the scarcity of real-world conversational speech data , especially in multil...

    work page 2025

  2. [2]

    In addition, to enhance the generalization capability of the models, we utilized the open-source MSR-86K[9] dataset as an external resource

    DA TASET The competition provides a multilingual conversational speech corpus consisting of 1,500 hours of real-world conversational speech recordings across 11 diverse languages: English (en), French (fr), German (de), Italian (it), Portuguese (pt), Spanish (es), Japanese (ja), Korean (ko), Russian (ru), Thai (th), and Vietnamese (vi), representing a wid...

    work page

  3. [3]

    System Description The overall multilingual ASR architecture is illustrated in Fig- ure 1. The system is designed based on the standard Encoder- Adaptor-LLM architecture[5], comprising the following com- ponents: Frozen Speech Encoder: The system utilizes the Whisper- large-v3[10] encoder, a pre-trained model capable of processing 128-dimensional Mel filt...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    Conclusions This paper presents a Multilingual Automatic Speech Recog- nition (ASR) framework, developed for the MLC-SLM 2025 Challenge, based on an Encoder-Adaptor-LLM architecture. The system incorporates a frozen Whisper-large-v3 encoder for robust acoustic feature extraction, a trainable adaptor to en- sure efficient alignment between speech and text ...

    work page 2025

  5. [5]

    Qwen-audio: Advancing universal audio understand- ing via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understand- ing via unified large-scale audio-language models,”arXiv preprint arXiv:2311.07919, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Salmonn: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large language models,” arXiv preprint arXiv:2310.13289, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities,” arXiv preprint arXiv:2305.11000, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    An embarrassingly simple approach for llm with strong asr capacity,

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wang, Z. Du, F. Yu, Q. Chen, S. Zheng, S. Zhanget al., “An embarrassingly simple approach for llm with strong asr capacity,” arXiv preprint arXiv:2402.08846 , 2024

    Pith/arXiv arXiv 2024

  10. [10]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,”arXiv preprint arXiv:1912.06670, 2019

    Pith/arXiv arXiv 1912

  11. [11]

    Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhang et al. , “Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” arXiv preprint arXiv:2106.06909, 2021

    Pith/arXiv arXiv 2021

  12. [12]

    Gigaspeech 2: An evolving, large-scale and multi-domain asr corpus for low-resource languages with au- tomated crawling, transcription and refinement,

    Y . Yang, Z. Song, J. Zhuo, M. Cui, J. Li, B. Yang, Y . Du, Z. Ma, X. Liu, Z. Wang et al., “Gigaspeech 2: An evolving, large-scale and multi-domain asr corpus for low-resource languages with au- tomated crawling, transcription and refinement,” arXiv preprint arXiv:2406.11546, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Msr- 86k: An evolving, multilingual corpus with 86,300 hours of tran- scribed audio for speech recognition research,

    S. Li, Y . You, X. Wang, Z. Tian, K. Ding, and G. Wan, “Msr- 86k: An evolving, multilingual corpus with 86,300 hours of tran- scribed audio for speech recognition research,” arXiv preprint arXiv:2406.18301, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Robust speech recognition via large-scale weak su- pervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” in International Conference on Machine Learning . PMLR, 2023, pp. 28 492–28 518

    work page 2023

  15. [15]

    Qwen2 technical report,

    A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Yang, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. Peng, R. Men, R. Gao, R. Lin, S. Wang, S. Bai, S. Tan, T. Zhu, T. Li, T...

    Pith/arXiv arXiv 2024

  16. [16]

    Qwen2.5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y . Fan, Y . Su, Y . Zhang, Y . Wan, Y . Liu, Z. Cui, Z. Zhang, and Z. Qi...

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.