TalTech Systems for the Interspeech 2025 ML-SUPERB 2.0 Challenge

Artem Fedorchenko; Tanel Alum\"ae

REVIEW 2 major objections 6 minor 28 references

TalTech Systems for the Interspeech 2025 ML-SUPERB 2.0 Challenge

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper describes the top-scoring system in the ML-SUPERB 2.0 challenge, a two-stage pipeline that first identifies the language and then routes the audio to one of three pretrained speech-recognition models.

desk verdict A useful, honest challenge report: the hybrid LID with uroman bigram LMs is a real contribution, and the top-score claim is credible but needs the official rubric to be fully verified. read the letter →

arxiv 2506.01458 v1 pith:P6KKJNNE submitted 2025-06-02 cs.CL eess.AS

Tanel Alumäe , Artem Fedorchenko This is my paper

classification cs.CLeess.AS

keywords multilingualspeechrecognitionlanguageidentificationML-SUPERB2.0low-resourcelanguageshybridIDmodelroutinguromanpretrainedmodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper describes the system that scored highest overall in the ML-SUPERB 2.0 challenge, a benchmark covering 154 languages and more than 200 language varieties. The authors' claim is that a rule-based pipeline—first identify the language, then route the audio to one of three pretrained speech-recognition models—is enough to beat all other challenge entries across this language set. The evidence for the claim is a hybrid language identifier that scores each utterance twice: once with acoustic embeddings and once with a generative classifier that decodes speech into romanized characters and picks the language whose bigram language model fits best. This matters because the system's most surprising errors point to low-resource language data quality, not model architecture, as the remaining bottleneck.

What carries the argument

The load-bearing mechanism is the hybrid language-ID model, because every ASR decision downstream depends on its predicted language label. Its acoustic half pools dimension-wise weighted averages of the frozen SeamlessM4T encoder's 24 layers with multi-resolution multi-head attention, then classifies the resulting embeddings with length normalization, linear discriminant analysis, and logistic regression. Its generative half runs the MMS-zeroshot CTC model once to get posteriors over uroman (romanized) characters, then decodes those posteriors under language-specific lexicons and bigram language models, choosing the language with the highest total score; this half needs only text, not audio, for each target language. On the development set the two halves score 85.3% and 70.7% language-ID accuracy alone, and their uniform interpolation reaches 89.9%, cutting errors by 31%.

What would settle it

Re-run the blind evaluation after replacing the web-scraped training audio for Highland Puebla Nahuatl and Northeastern Thai with manually verified native-speaker recordings, keeping everything else fixed. If the Greek-to-Highland Puebla Nahuatl and Arabic-to-Northeastern Thai confusions disappear, the reported language-ID numbers depended on data quality; if they persist, the data-quality explanation is wrong and the model itself lacks a robust representation for these languages.

Watch

Extended reading notes

Core claim

The paper's central claim is that the top-scoring ML-SUPERB 2.0 system works as a two-stage combination rather than one giant model. Stage one is a hybrid language-ID model: a language-embedding classifier on the frozen SeamlessM4T speech encoder is linearly interpolated with a generative classifier built on MMS-zeroshot, which turns speech into uroman (romanized) character posteriors and scores them under per-language bigram language models. Stage two maps each predicted language to one of three ASR models: a fine-tuned SeamlessM4Tv2, MMS-1B-all with newly trained adapters for eight low-resource languages, or MMS-zeroshot with a 4-gram language model. On the blind evaluation set the combined system reaches 86.8% language-ID accuracy, a mean ASR character error rate of 27.4%, and 56.6% dialect language-ID accuracy, taking the top overall rank and beating both organizer baselines by wide margins.

Load-bearing premise

The load-bearing premise is that the web-scraped and custom-collected audio and text used for the low-resource languages are good enough and representative enough of the evaluation speech; if that fails, the reported language-ID and ASR gains for those languages will not carry over.

Editorial extensions

If this is right

Routing among three pretrained ASR models beats any single model in the set: with oracle language labels, the optimized language-to-model mapping lowers mean character error rate to 7.9% on the regular development set, and the final system with predicted labels reaches 11.7%.
Combining the acoustic embedding classifier with the generative classifier improves language identification substantially: uniform interpolation cuts development-set errors by 31% (and dialect-set errors by 23%) compared with the embedding model alone.
The generative language-ID component extends to new languages with text only, since it needs no audio data to build its language-specific bigram language models.
On the blind evaluation, the routing system keeps the worst-language character error rate at 82.5%, below the 92.8% and 118.8% posted by the two organizer baselines, while also scoring highest on dialect language ID.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The design suggests a cheap path to broader coverage: because the generative language-ID identifier is trained only on text, adding a new language or dialect could be done by supplying a text corpus and building a bigram language model, leaving the audio models unchanged.
The two confusions the authors flag (Greek predicted as Highland Puebla Nahuatl; Arabic as Northeastern Thai) offer a direct probe of the data-quality explanation: replacing the web-scraped audio for those two languages with verified native recordings would be a clean A/B test.
The 28-gigabyte uncompressed system size and the strict 24-gigabyte inference memory limit hint that the challenge constrains architecture; distilling the routing decisions into a smaller model is a natural next step the paper does not pursue.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a genuinely useful challenge systems report, and the LID combination is the actual contribution. The authors interpolate two very different signals—a Seamless-encoder embedding classifier and a generative 'phonotactic' scorer built by running MMS-zeroshot's uroman CTC output through per-language bigram LMs—and get dev LID from 85.3/70.7 to 89.9. That's a 31% error reduction, and it's a simple, cheap idea others can reuse. The ASR part is more conventional: pick among three pretrained models per language, with some custom adapters and finetuning.

The paper is honest about data sources and model choices, and the blind evaluation shows ranks 1/1/1 for LID, CER, and dialect LID, with internally consistent numbers. I think the system really did well.

Soft spots, in order of importance.

First, the 'top overall score' claim is not checkable from the paper. Table 6 gives per-category ranks and two baselines, but no aggregate scoring formula, no composite totals, and no other participants' scores. First in three categories doesn't logically entail first overall if the composite weights categories differently. This is likely a real win, but the manuscript should give the official rubric or a link to the leaderboard. The stress-test note is right about that.

Second, there are no error bars or significance statements anywhere. For a challenge paper that's typical, but with a single evaluation set and no variation, those numbers are point estimates and should be read that way.

Third, the per-language ASR mapping was chosen on dev performance, so the dev-set CER rows are partly post-selection. The prose discloses this, but Table 4's last two rows look like clean held-out numbers.

Fourth, dialect LID accuracy drops from 84.9% on dev dialects to 56.6% on evaluation dialects, and the paper doesn't mention it. For a challenge whose theme is not leaving varieties behind, that's a surprising hole. The authors do flag web-scraped data quality as the likely cause of the two weird confusions, which is fair, but the scale of the drop deserves a sentence or two.

None of this is load-bearing. The generative LID idea is new, the evaluation is external and consistent, and the failure modes are disclosed. The right audience is anyone building practical multilingual ASR/LID, especially for low-resource languages. I'd send it to peer review; it deserves a serious referee, with a request to add the scoring formula, say something about uncertainty, and address the dialect gap.

Referee Report

2 major / 6 minor

Summary. The paper describes the TalTech system for the Interspeech 2025 ML-SUPERB 2.0 Challenge. The system combines a hybrid language identification (LID) component, which interpolates a SeamlessM4T-based language embedding classifier with a generative LID model built on MMS-zeroshot and language-specific bigram language models, with a speech recognition component that routes each language to one of three models (fine-tuned SeamlessM4Tv2, MMS-1B-all with custom adapters, or MMS-zeroshot) based on development-set performance. The authors report the best LID accuracy (86.8%), lowest mean CER (27.4%), and highest dialectal LID accuracy (56.6%) on the blind evaluation set, and claim the top overall score in the challenge. The paper also documents additional training data sources for low-resource languages and discusses common LID confusion errors.

Significance. If the reported results are taken at face value, the paper demonstrates that a relatively lightweight hybrid LID pipeline with per-language ASR model routing can outperform two strong organizer baselines by substantial margins on a 154-language benchmark, including for dialectal varieties. The work is a useful engineering contribution to multilingual speech processing, with a clear system description and a third-party blind evaluation on a Dynabench server. Strengths include the explicit comparison with two baselines in Table 6, the release of per-language CER results in supplementary material, and the honest acknowledgment of limitations in web-scraped training data for some low-resource languages (Section 4.1). However, the headline claim of 'top overall score' is not self-contained in the manuscript, because the aggregate scoring formula and the full leaderboard are not provided.

major comments (2)

[Abstract and Section 4.3] The central claim that the system 'obtained the top overall score in the challenge' (abstract) and was 'ranked as the highest in the evaluation leaderboard' (Section 4.3) is not verifiable from the evidence presented. Table 6 reports only this system's per-category ranks (1, 1, 5, 4, 1, 2 for LID, CER, CER-STD, CER-15-worst, Dialect LID, Dialect CER) and scores for two organizer baselines. It does not provide the challenge's aggregate scoring formula, the total composite score for any system, or the scores and ranks of other participants. Per-category ranks do not by themselves entail an overall first place; another system could rank second in all categories and still win under a weighted composite or other aggregation. To make the claim supported, the authors should either include the official aggregate scoring formula and total scores for all participants, cite the public leaderboard, or explicitly restrict the claim to the documented per-category results.
[Section 4.3 and Table 6] The paper states that the system obtained 'the lowest mean ASR CER' and 'the best overall LID accuracy' among participants. The table only demonstrates superiority over the two listed baselines; without other participants' scores, 'among participants' is not established. If the challenge rules prohibit disclosing other teams' results, this should be stated explicitly and a reference to the official results page should be provided. Otherwise, the abstract and introduction should be revised to say 'top among the systems compared' or similar, or the missing data should be added.

minor comments (6)

[Section 1] The phrase 'follows an rule-based pipeline' should read 'follows a rule-based pipeline'.
[Section 2.1.1] The description of the layer aggregation weights is confusing: 'the weights specific to each dimension sum to one' could be clarified, for example by stating that for each of the 1024 output dimensions, the 24 layer weights form a softmax-like normalized vector. This would improve reproducibility.
[Section 2.1.2] The sentence 'The model trains on labeled data from 1,078 languages' should be 'The model is trained on labeled data...' for grammatical correctness.
[Section 2.2.1] The fine-tuning details for SeamlessM4Tv2 (learning rate, number of epochs, data filtering criteria, compute budget) are missing. Since this is a key component, adding at least the main hyperparameters would aid reproducibility.
[Section 4.2] In Table 4, the column header 'CER 15 worst↓' is somewhat ambiguous; it would be clearer as 'CER (15 worst)↓' or 'CER-15-worst↓' to match the text in Section 1. Also, indicate explicitly that all CER values are percentages.
[Section 4.3] The sentence 'The evaluation scores of the system, together with those of two organizer baseline systems, are given in Table 6' is slightly misleading because the table does not show scores for other participants; consider rephrasing to '...together with those of the two organizer baseline systems...'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the headline results are externally measured blind-test scores, not derived from the paper's inputs.

full rationale

The paper's central claims (LID accuracy 86.8%, mean CER 27.4%, dialect LID 56.6%, and 'top overall score') come from a blind Dynabench evaluation, not from a derivation that re-imports its own assumptions. The hybrid LID model is trained on external data (VoxLingua107, ML-SUPERB public set, GlotLID text) and its parameters are not fitted to the evaluation metrics; the evaluation set is blind and independent. The ASR routing among SeamlessM4Tv2, MMS-1B-all, and MMS-zeroshot is selected on development-set CER, which can optimism-bias the dev rows of Table 4, but Table 6 reports blind-test results unaffected by that selection. Self-citations ([4] VoxLingua107, [8] accent-robust LID) are data sources or related work and are not load-bearing justifications. The only substantive concern is that the 'top overall score' assertion is not fully reconstructible from Table 6 because the aggregate scoring formula and other teams' totals are omitted; that is a reporting/verifiability gap, not circular reasoning.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical systems report, so its ledger is mostly domain assumptions about data quality and the evaluation server, plus hand-chosen configuration constants. There is no derivation, so there are no parameters hidden inside a mathematical argument. The most consequential choice is the per-language ASR model mapping, which was tuned on the development set.

free parameters (3)

Language-to-ASR model mapping = SeamlessM4T for 56 languages, MMS-1B-all for about 84, MMS-zeroshot for 5 (Table 5)
Chosen based on development set CER per language, so the dev-set 'optimized combination' numbers in Table 4 are post-selection; the mapping defines what was sent to the blind evaluation.
LM vocabulary size and text sample cap = 10,000 subword tokens; up to 100,000 lines per language
Hand-chosen constants for both the LID bigram LMs and the 4-gram MMS-zeroshot decoders (Sections 2.1.2 and 2.2.3); no sweep or sensitivity analysis is reported.
Embedding classifier configuration = 4 attention heads, 256-dim hidden, 8192-dim pooled vector, LDA dim 100, 512-dim layers
Hand-chosen hyperparameters for the Seamless-encoder LID model (Section 2.1.1); standard values, but no ablations are given for these choices.

assumptions (4)

domain assumption The Dynabench blind evaluation correctly measures the six metrics and determines the final ranking.
The entire claim of winning rests on this third-party server evaluation (Section 4.3, Table 6); if the evaluation or its scoring were flawed, the central claim falls.
domain assumption Web-scraped and custom-collected training data for the low-resource languages are of adequate quality and representativeness.
LID data for 12 languages and ASR data for 9 languages come from blogs, dialect corpora, LDC collections, and other external sources (Table 1); the paper itself blames poor web-sourced data quality for its two most surprising confusions (Section 4.1).
domain assumption Uroman heuristic romanization is an adequate intermediate representation for both LID scoring and ASR decoding across the target languages.
The generative LID and the MMS-zeroshot ASR path rely entirely on uroman characters (Sections 2.1.2 and 2.2.3); romanization errors would distort likelihood scores and transcriptions for complex-script languages.
domain assumption The pretrained SeamlessM4T, MMS-1B-all, and MMS-zeroshot models behave as documented in their original papers.
The system is assembled from these models with no independent verification of their capabilities in this paper (Sections 2.2.1 to 2.2.3); the per-language routing assumes their documented language support is accurate.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TalTech Systems for the Interspeech 2025 ML-SUPERB 2.0 Challenge." pith.science (2026). https://pith.science/paper/P6KKJNNE

@misc{pith2026250601458,
  author       = {Pith},
  title        = {Pith review of: TalTech Systems for the Interspeech 2025 ML-SUPERB 2.0 Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P6KKJNNE}},
  note         = {Machine review of arXiv:2506.01458}
}

read the original abstract

This paper describes the language identification and multilingual speech recognition system developed at Tallinn University of Technology for the Interspeech 2025 ML-SUPERB 2.0 Challenge. A hybrid language identification system is used, consisting of a pretrained language embedding model and a light-weight speech recognition model with a shared encoder across languages and language-specific bigram language models. For speech recognition, three models are used, where only a single model is applied for each language, depending on the training data availability and performance on held-out data. The model set consists of a finetuned version of SeamlessM4T, MMS-1B-all with custom language adapters and MMS-zeroshot. The system obtained the top overall score in the challenge.

Figures

Figures reproduced from arXiv: 2506.01458 by the authors.

**Figure 1.** Schematic diagram of the generative LID model based on the MMS-zeroshot model. first hidden layer after the pooling layer (before applying the ReLU activation). In order to train the final language embedding based language classifier, we extracted embeddings for a subset of MLSUPERB 2.0 training data, augmented with our custom webscraped data for languages that were not present in the provided data (see Section 3… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

[1]

Introduction The ML-SUPERB 2.0 Interspeech 2025 Challenge was held with the aim to advance the state-of-the-art in multilingual au- tomatic speech recognition (ASR) and language identification (LID) systems, with a particular focus on improving perfor- mance across a diverse range of languages and language vari- eties. The challenge evaluated systems on 1...

work page 2025
[2]

Methods 2.1. Language identification We use a hybrid spoken LID model, consisting of a language embedding model with a logistic regression classifier and a gen- erative multilingual speech recognition model. 2.1.1. Language embeddings model The backbone of the language embedding model is the speech encoder module from the SeamlessM4T speech translation an...

work page Pith review arXiv 2025
[3]

This dataset combines various multilingual speech corpora and covers 141 of the 153 target languages

Data The challenge organizers provided the ML-SUPERB 2.0 pub- lic set as a baseline dataset for both training and development. This dataset combines various multilingual speech corpora and covers 141 of the 153 target languages. They also provided a Table 2: LID accuracies (%) of the two models and their uni- form interpolation on two development sets. Em...

work page 1941
[4]

hierarchy

Experimental results 4.1. Language identification Table 2 shows the LID accuracies for both individual models and the uniform linear interpolation of the predicted likelihoods 4https://github.com/FormoSpeech/klokah_ crawler Table 3: Most common confusion errors (reference → pre- dicted) of the combined LID model. Dev Devdialects Tamil → Telugu German → Lu...

work page
[5]

Conclusion We presented TalTech’s system for the ML-SUPERB 2.0 Chal- lenge, which combines a hybrid LID approach with a hierarchi- cal ASR model selection strategy. The LID system uses both acoustic and linguistic information through the combination of a language embedding model and a generative classifier, achiev- ing 86.8% accuracy on the evaluation set...

work page
[6]

Acknowledgments This work was supported by the Estonian Centre of Excellence in Artificial Intelligence (EXAI)

work page
[7]

SUPERB: speech processing universal performance benchmark,

S. Yang, P. Chi, Y . Chuang, C. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G. Lin, T. Huang, W. Tseng, K. Lee, D. Liu, Z. Huang, S. Dong, S. Li, S. Watanabe, A. Mohamed, and H. Lee, “SUPERB: speech processing universal performance benchmark,” in Proc. of Interspeech, 2021

work page 2021
[8]

Seamless: Multilingual expressive and streaming speech translation,

Seamless Communication, L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Duppenthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haaheim, J. Hoffman, M.-J. Hwang, H. In- aguma, C. Klaiber, I. Kulikov, P. Li, D. Licht, J. Maillard, R. Mav- lyutov, A. Rakotoarison, K. R. Sadagopan, A. Ramakrishnan, T. Tran, G. Wenzek, Y . Yang, E. Ye, I. Evtim...

work page 2023

Show all 28 references

[9]

Multi-resolution multi-head attention in deep speaker embedding,

Z. Wang, K. Yao, X. Li, and S. Fang, “Multi-resolution multi-head attention in deep speaker embedding,” in Proc. of ICASSP, 2020

2020
[10]

V oxLingua107: a dataset for spoken lan- guage recognition,

J. Valk and T. Alum ¨ae, “V oxLingua107: a dataset for spoken lan- guage recognition,” in Proc. IEEE SLT Workshop, 2021

2021
[11]

A study on data augmentation of reverberant speech for robust speech recognition,

T. Ko, V . Peddinti, D. Povey, M. L. Seltzer, and S. Khudanpur, “A study on data augmentation of reverberant speech for robust speech recognition,” in Proc. of ICASSP, 2017

2017
[12]

Language identification using phone- based acoustic likelihoods,

L. Lamel and J. Gauvain, “Language identification using phone- based acoustic likelihoods,” in Proc. of ICASSP, 1994

1994
[13]

Language identification using phoneme recog- nition and phonotactic language modeling,

M. A. Zissman, “Language identification using phoneme recog- nition and phonotactic language modeling,” in Proc. of ICASSP. IEEE, 1995

1995
[14]

Improving language identification of accented speech,

K. Kukk and T. Alum ¨ae, “Improving language identification of accented speech,” in Proc. of Interspeech, 2022

2022
[15]

Scaling a simple approach to zero-shot speech recognition,

J. Zhao, V . Pratap, and M. Auli, “Scaling a simple approach to zero-shot speech recognition,” ArXiv preprint, 2024

2024
[16]

Out-of-the-box universal Romanization tool uroman,

U. Hermjakob, J. May, and K. Knight, “Out-of-the-box universal Romanization tool uroman,” in Proceedings of ACL 2018, System Demonstrations, 2018

2018
[17]

GlotLID: Language identification for low-resource languages,

A. H. Kargaran, A. Imani, F. Yvon, and H. Schuetze, “GlotLID: Language identification for low-resource languages,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023

2023
[18]

Subword regularization: Improving neural network translation models with multiple subword candidates,

T. Kudo, “Subword regularization: Improving neural network translation models with multiple subword candidates,” in Proc. of ACL, 2018

2018
[19]

Leveraging the multilingual Indone- sian ethnic languages dataset in self-supervised models for low- resource ASR task,

S. Sakti and B. A. Titalim, “Leveraging the multilingual Indone- sian ethnic languages dataset in self-supervised models for low- resource ASR task,” Proc. of ASRU, 2023

2023
[20]

Thai dialect corpus and transfer-based curriculum learning investigation for dialect automatic speech recognition,

A. Suwanbandit, B. Naowarat, O. Sangpetch, and E. Chuang- suwanich, “Thai dialect corpus and transfer-based curriculum learning investigation for dialect automatic speech recognition,” in Proc. of Interspeech, 2023

2023
[21]

Sim- inchik: A speech corpus for preservation of Southern Quechua,

R. Cardenas, R. Zevallos, R. Baquerizo, and L. Camacho, “Sim- inchik: A speech corpus for preservation of Southern Quechua,” ISI-NLP 2, 2018

2018
[22]

Building large mono- lingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages,

D. Goldhahn, T. Eckart, and U. Quasthoff, “Building large mono- lingual dictionaries at the Leipzig corpora collection: From 100 to 200 languages,” in Proc. of LREC, 2012

2012
[23]

Leveraging end-to-end ASR for endangered language documentation: An empirical study on Yol´oxochitl Mixtec,

J. Shi, J. D. Amith, R. Castillo Garc ´ıa, E. Guadalupe Sierra, K. Duh, and S. Watanabe, “Leveraging end-to-end ASR for endangered language documentation: An empirical study on Yol´oxochitl Mixtec,” in Proc. of EACL, 2021

2021
[24]

Crowd-sourced speech corpora for Javanese, Sundanese, Sinhala, Nepali, and Bangladeshi Bengali,

O. Kjartansson, S. Sarin, K. Pipatsrisawat, M. Jansche, and L. Ha, “Crowd-sourced speech corpora for Javanese, Sundanese, Sinhala, Nepali, and Bangladeshi Bengali,” in Proc. The 6th Intl. Workshop on Spoken Language Technologies for Under- Resourced Languages (SLTU), 2018

2018
[25]

Combining spectral and self-supervised features for low resource speech recognition and translation,

D. Berrebbi, J. Shi, B. Yan, O. L ´opez-Francisco, J. D. Amith, and S. Watanabe, “Combining spectral and self-supervised features for low resource speech recognition and translation,” in Proc. of Interspeech, 2022

2022
[26]

Scaling speech technology to 1,000+ languages,

V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi, A. Baevski, Y . Adi, X. Zhang, W.-N. Hsu, A. Conneau, and M. Auli, “Scaling speech technology to 1,000+ languages,” ArXiv preprint, 2023

2023
[27]

Espnet: End-to-end speech processing toolkit,

S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y . Unno, N. E. Y . Soplin, J. Heymann, M. Wiesner, N. Chen, A. Ren- duchintala, and T. Ochiai, “Espnet: End-to-end speech processing toolkit,” in Proc. of Interspeech, 2018

2018
[28]

Dynabench: Rethinking benchmarking in NLP,

D. Kiela, M. Bartolo, Y . Nie, D. Kaushik, A. Geiger, Z. Wu, B. Vidgen, G. Prasad, A. Singh, P. Ringshia, Z. Ma, T. Thrush, S. Riedel, Z. Waseem, P. Stenetorp, R. Jia, M. Bansal, C. Potts, and A. Williams, “Dynabench: Rethinking benchmarking in NLP,” 2021

2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Introduction The ML-SUPERB 2.0 Interspeech 2025 Challenge was held with the aim to advance the state-of-the-art in multilingual au- tomatic speech recognition (ASR) and language identification (LID) systems, with a particular focus on improving perfor- mance across a diverse range of languages and language vari- eties. The challenge evaluated systems on 1...

work page 2025

[2] [2]

Methods 2.1. Language identification We use a hybrid spoken LID model, consisting of a language embedding model with a logistic regression classifier and a gen- erative multilingual speech recognition model. 2.1.1. Language embeddings model The backbone of the language embedding model is the speech encoder module from the SeamlessM4T speech translation an...

work page Pith review arXiv 2025

[3] [3]

This dataset combines various multilingual speech corpora and covers 141 of the 153 target languages

Data The challenge organizers provided the ML-SUPERB 2.0 pub- lic set as a baseline dataset for both training and development. This dataset combines various multilingual speech corpora and covers 141 of the 153 target languages. They also provided a Table 2: LID accuracies (%) of the two models and their uni- form interpolation on two development sets. Em...

work page 1941

[4] [4]

hierarchy

Experimental results 4.1. Language identification Table 2 shows the LID accuracies for both individual models and the uniform linear interpolation of the predicted likelihoods 4https://github.com/FormoSpeech/klokah_ crawler Table 3: Most common confusion errors (reference → pre- dicted) of the combined LID model. Dev Devdialects Tamil → Telugu German → Lu...

work page

[5] [5]

Conclusion We presented TalTech’s system for the ML-SUPERB 2.0 Chal- lenge, which combines a hybrid LID approach with a hierarchi- cal ASR model selection strategy. The LID system uses both acoustic and linguistic information through the combination of a language embedding model and a generative classifier, achiev- ing 86.8% accuracy on the evaluation set...

work page

[6] [6]

Acknowledgments This work was supported by the Estonian Centre of Excellence in Artificial Intelligence (EXAI)

work page

[7] [7]

SUPERB: speech processing universal performance benchmark,

S. Yang, P. Chi, Y . Chuang, C. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shi, X. Chang, G. Lin, T. Huang, W. Tseng, K. Lee, D. Liu, Z. Huang, S. Dong, S. Li, S. Watanabe, A. Mohamed, and H. Lee, “SUPERB: speech processing universal performance benchmark,” in Proc. of Interspeech, 2021

work page 2021

[8] [8]

Seamless: Multilingual expressive and streaming speech translation,

Seamless Communication, L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Duppenthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haaheim, J. Hoffman, M.-J. Hwang, H. In- aguma, C. Klaiber, I. Kulikov, P. Li, D. Licht, J. Maillard, R. Mav- lyutov, A. Rakotoarison, K. R. Sadagopan, A. Ramakrishnan, T. Tran, G. Wenzek, Y . Yang, E. Ye, I. Evtim...

work page 2023