Pith. sign in

REVIEW 3 major objections 5 minor 58 references

MEUSLI claims that a single lightweight projector can pair a frozen speech encoder with a frozen multilingual LLM to deliver end-to-end ASR in 28 European languages, and that the same projector can bootstrap new languages and other speech t

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 05:45 UTC pith:AVQXCIDX

load-bearing objection A useful, credible open multilingual projector family for 28 EU languages; the bootstrapping claim is overstated because the test languages are in Whisper's training set, but the core results and release hold up. the 3 major comments →

arxiv 2607.22100 v1 pith:AVQXCIDX submitted 2026-07-24 cs.CL cs.AIeess.AS

MEUSLI: a Multilingual Projector for LLM-based ASR and Beyond

Lorenzo Concina , Seraphina Fong , Marco Matassoni , Alessio Brutti This is my paper
classification cs.CL cs.AIeess.AS
keywords speech recognitionlarge language modelmultilinguallinear projectorlow-resource languagescontinual learningspeech translationtopic identification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that one small trainable component — a linear projector that maps speech features into the token-embedding space of a language model — can be trained once on many languages and then reused both for direct multilingual ASR and as a starting point for languages and tasks with very little data. It argues that the projector's multilingual pretraining transfers to low-resource and even supposedly unseen languages, and that this transfer extends beyond transcription to speech translation and topic identification. The concrete evidence is a set of word-error-rate, BLEU, and accuracy tables comparing MEUSLI against both a frozen encoder alone and projectors trained from scratch, plus a public release of the trained projectors. For a reader, the payoff would be a fully open-source, reproducible route to speech-enabled LLMs for the long tail of languages.

Core claim

On the paper's own terms, the central discovery is that a single 17-million-parameter linear projector, trained on about 7,600 hours of open speech data from 28 European languages, links a frozen speech encoder to a frozen multilingual LLM and produces an end-to-end speech recognizer that beats the encoder alone on most of those languages, with especially large gains on low-resource languages. The same projector initialization then improves low-resource fine-tuning, and, using a replay buffer of roughly 1,000 samples per original language, lets new languages be added without catastrophic forgetting. The paper further reports that fine-tuning MEUSLI on under five hours of labeled data per lan

What carries the argument

The central object is the linear projector: a single hidden layer with ReLU followed by a regression layer (17.31M trainable parameters) that maps temporally downsampled output of a frozen pre-trained speech encoder into the embedding space of a frozen multilingual LLM. A small low-rank adaptation (rank 8, scaling factor 32, 1.38M parameters) is applied on the LLM. Training uses cross-entropy loss on the LLM's token predictions with a text prompt appended to the projected speech embeddings; the same machinery is reused unchanged for fine-tuning, continual learning with data replay, and multitask extensions.

Load-bearing premise

The load-bearing premise is that Ukrainian and Albanian were excluded from both the speech encoder's and the LLM's pretraining corpora, so that their improvement proves bootstrapping of truly unseen languages; in fact the speech encoder used was trained on both languages.

What would settle it

Run the same bootstrapping experiment on a language that is verifiably absent from both the speech encoder's and the LLM's pretraining corpora (a low-resource language with a few hours of publicly available audio). If fine-tuning MEUSLI on that language, with ~30 hours of data, fails to produce a large WER improvement over training a projector from scratch, then the paper's claim that multilingual projector pretraining bootstraps languages unseen by the encoder collapses. A minimal check: verify the encoder's training data list for Ukrainian and Albanian; their presence alone falsifies the pap

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A fully open-source, reproducible speech-to-text path for 28 European languages becomes available, removing dependence on proprietary or English-only speech-enabled LLMs.
  • Low-resource languages gain a practical recipe: initialize from the multilingual projector rather than training a projector from scratch, which in the reported experiments roughly halves WER for Breton and Maltese.
  • New languages can be bootstrapped with tens of hours of data or less, and data replay with about 1,000 samples per original language prevents the catastrophic forgetting seen in naive fine-tuning.
  • The same projector weights transfer to speech translation and topic identification with under five hours of labeled data per language-task, so the ASR-pretrained projector acts as a general spoken-language-understanding initialization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's 'unseen language' claim rests on Ukrainian and Albanian being absent from both the speech encoder and the LLM; since the chosen encoder was actually trained on those languages, the experiment demonstrates adaptation to languages the encoder already understands. A truly held-out language test remains an open experiment.
  • The reported gains over the frozen encoder alone suggest the LLM's linguistic priors are being used, but the paper does not isolate how much of the improvement comes from the encoder's multilingual representation versus the LLM's priors; an ablation removing the LLM (replacing it with a text-only decoder) would help.
  • A testable extension is to apply the same bootstrapping recipe to a language with no presence in either the encoder or the LLM pretraining (e.g., an endangered language with a few hours of recordings). If the MEUSLI advantage over from-scratch persists, the continual-learning bootstrapping claim survives; if it collapses, the reported gains depend on the encoder's prior knowledge.
  • Because the projector is only about 17 million parameters while the language model is orders of magnitude larger, the paper implies the alignment bottleneck is small; a natural next experiment is sweeping projector capacity downward to see whether an even simpler linear map suffices at this scale.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MEUSLI, a multilingual linear projector for SLAM-style LLM-based ASR that connects a frozen Whisper-large-v3-turbo encoder to frozen open-source multilingual LLMs (EuroLLM-1.7B-Instruct, EuroLLM-9B, Apertus-8B). The projector is trained on 7,622 hours of speech from Common Voice, FLEURS, and VoxPopuli across 28 European languages, and the authors report WERs on in-domain (CV, FL) and out-of-domain (MLC-SLM) test sets. They further study low-resource fine-tuning for Breton and Maltese, bootstrapping for Ukrainian and Albanian, continual learning via data replay, and transfer to speech translation and topic identification on SIB-Fleurs. The models are released on HuggingFace.

Significance. If the central claims hold, MEUSLI is a valuable open resource: it is the first fully open multilingual SLAM-style projector family covering 28 European languages, with public model weights and a systematic comparison across three LLMs and multiple evaluation sets. The paper also provides practical training details and shows that multilingual pretraining helps low-resource adaptation, which is a useful contribution to the open SpeechLLM ecosystem. The release of the projectors on HuggingFace supports reproducibility and downstream use. However, the extent to which the method 'bootstraps languages not seen in training' is overstated, and the multitask transfer results are partially confounded by training/evaluation data overlap, both of which temper the significance unless corrected.

major comments (3)
  1. [§3.2, Table 4, Abstract, Contribution (3)] The bootstrapping experiment claims that Ukrainian and Albanian were 'excluded from both the speech encoder and EuroLLM pretraining corpora.' This is factually incorrect for the encoder: Whisper-large-v3-turbo was trained on both Ukrainian and Albanian. The reported gains (16.34% vs. 20.43% WER for Ukrainian, and 75.61% for Albanian) therefore demonstrate adaptation to languages the frozen encoder already understands, not extension to 'languages not seen in training' as claimed in the abstract and contribution (3). This weakens the 'unseen language' claim substantially. Please either re-run the experiment with truly encoder-unseen languages or reframe the claim as adaptation to languages unseen by the LLM only.
  2. [§4, Table 6, §5] The multitask transfer results are partially circular in an empirical sense: the MEUSLI projector was pretrained on FLEURS audio, and SIB-Fleurs shares the same audio segments, differing only in the added ST and TID annotations. The authors disclose this overlap in Section 5, but the Section 6 conclusion that 'MEUSLI transfers effectively beyond ASR' is stronger than the evidence supports. The current design cannot separate the benefit of multilingual projector pretraining from the benefit of having previously seen the exact audio during ASR pretraining. Please temper the claim or conduct an evaluation on a truly unseen speech corpus for ST/TID.
  3. [§3.1, Table 3] The text states that 'MEUSLI remains beneficial even when paired with a specialized monolingual encoder.' This is contradicted by the Maltese row: with the adapted encoder, MEUSLI fine-tuning yields 23.7% WER versus 18.0% for the monolingual from-scratch system. The benefit is observed only for Breton (24.5% vs. 27.7%). Please either restrict the claim to Breton, or report variance/error bars and statistical significance before making a general statement.
minor comments (5)
  1. [Abstract] Typo: 'continual leaning' should be 'continual learning'.
  2. [Table 4] 'Ukranian' should be 'Ukrainian' in the table header.
  3. [§4.2] The multitask experiments defer detailed methodology to an 'Interspeech 2026 paper' that is not available. For the present manuscript to be self-contained, please include at least the prompt templates, optimization details, and the exact train/eval split sizes.
  4. [§3.2] The Albanian training set is only 46 minutes; the from-scratch baseline fails to converge (389% WER), which may be a degenerate output rather than a meaningful comparison. Report the number of utterances, decoding settings, and any filtering used.
  5. [Figure 1] The figure shows the SLAM-ASR pipeline but does not indicate where LoRA is applied to the LLM. Adding a label would improve clarity.

Circularity Check

0 steps flagged

No circular derivation: core ASR results rest on held-out benchmarks; the 'unseen' and multitask issues are evidentiary weaknesses, not circular reductions.

full rationale

The core derivation is self-contained: MEUSLI is a linear projector trained on Common Voice, FLEURS, and VoxPopuli and evaluated on held-out splits of those benchmarks plus the external MLC-SLM test set. The WER tables (Tables 2-5) report genuine experiments, not fits renamed as predictions. The bootstrapping section (3.2) rests on a false factual premise: 'Ukrainian and Albanian, two low-resource European languages excluded from both the speech encoder and EuroLLM pretraining corpora' is incorrect for Whisper-large-v3-turbo, which was trained on Ukrainian and Albanian. This weakens the 'unseen language' claim as an empirical/correctness matter, but it is not circular: the reported WERs are not equal to any fitted input or self-citation. The multitask 'beyond ASR' section is partly confounded by the disclosed FLEURS/SIB-Fleurs audio overlap, as the authors state in Section 5: 'the MEUSLI projector was originally pre-trained on multilingual ASR data from FLEURS ..., which overlaps with the SIB-Fleurs dataset used in this multitask extension.' This is an evaluation-contamination issue rather than a definitional or fitted-input circularity, since the new ST/TID labels are genuinely task-specific. Self-citations [8] and [34] provide prior empirical context and continual-learning methodology, but they are not used as a uniqueness theorem and do not force the central ASR result. No equation sets an output equal to an input, and no fitted parameter is renamed as a prediction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The reported model relies on open corpora and frozen pretrained encoders/LLMs; the paper's own design choices (downsampling, LoRA rank, data caps, replay buffer) are hand-set and not derived. No new physical or architectural entities are postulated; the key unstated assumption is that Whisper features already represent the target languages, including those claimed to be unseen.

free parameters (5)
  • LoRA rank r=8, scaling alpha=32 = r=8, alpha=32
    Reported as best LoRA configuration in Section 2.1; tunable hyperparameters chosen by the authors, not derived.
  • Temporal downsampling factor k=5 = 5
    Chosen to mitigate length mismatch between speech and text representations (Section 2.2); affects ASR performance.
  • Per-language sample cap = 100,000 samples per language per dataset
    Data-balancing choice to reduce skew (Section 2.3); implicitly determines the language mix.
  • Replay buffer size = 1,000 samples per language
    Rehearsal buffer size for continual learning (Section 3.3); chosen with no ablation reported.
  • Training hyperparameters = lr=1e-4, 3 epochs, batch=8; 10 epochs for fine-tuning
    Training configuration in Sections 2.2 and 3.1; standard hand-set hyperparameters.
axioms (4)
  • domain assumption Frozen Whisper-large-v3-turbo embeddings contain sufficient audio information for 28 EU languages and for bootstrapping Ukrainian and Albanian.
    Central to the projector approach; however, Section 3.2's claim that Ukrainian and Albanian are excluded from the encoder is factually wrong for Whisper, which supports both languages.
  • domain assumption A single multilingual linear projector can align acoustic features to LLM token space across 28 languages.
    Assumed from SLAM-ASR and prior work; the paper does not compare to nonlinear projectors in this multilingual setting.
  • domain assumption WER on Common Voice, FLEURS, and MLC test sets is a valid measure of ASR capability.
    Standard ASR evaluation, but the results are single-run with no error bars.
  • domain assumption SIB-Fleurs can measure transfer beyond ASR without a significant overlap confound.
    The paper acknowledges in Section 5 that MEUSLI was pretrained on FLEURS, which overlaps with SIB-Fleurs, so the multitask transfer numbers are partly confounded.

pith-pipeline@v1.3.0-alltime-deepseek · 12808 in / 14619 out tokens · 141173 ms · 2026-08-01T05:45:58.839251+00:00 · methodology

0 comments
read the original abstract

Lightweight projectors are an established way to connect pre-trained speech encoders with large language models (LLMs), mapping acoustic features into token-level embeddings for tasks like ASR and spoken question answering. Existing systems, however, typically only support a few languages and are often limited to English. We introduce MEUSLI, the first open-science multilingual projector family that links a Whisper encoder with open-source multilingual LLMs, enabling fully open-source end-to-end ASR in 28 European languages. MEUSLI extends prior monolingual pipelines, delivering strong results across high- and low-resource languages. Using proper continual leaning techniques, MEUSLI can be easily extended to other languages not seen in training. We further demonstrate that the MEUSLI projector can be leveraged beyond ASR, enabling multilingual speech translation and topic identification with only a few hours of task specific supervision per language. Overall, MEUSLI provides a solid foundation for multilingual speech understanding tasks, supporting scalable and inclu- sive open-source SpeechLLM

Figures

Figures reproduced from arXiv: 2607.22100 by Alessio Brutti, Lorenzo Concina, Marco Matassoni, Seraphina Fong.

Figure 1
Figure 1. Figure 1: The proposed MEUSLI (Multilingual EU Speech LIn￾ear projector) training pipeline, following the SLAM-ASR ar￾chitecture from [7, 8]. to reduce LLM costs. During training, the projector learns to align acoustic and textual representations within a unified generative framework, enabling end-to-end speech tasks such as ASR, speech translation, and spoken language understand￾ing [11, 12, 13, 14] Despite growing… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the MEUSLI fine-tuning and bootstrap￾ping pipeline for expanding language coverage. tic conditions. These results motivate further experiments on fine-tuning for low-resource languages and bootstrapping un￾seen ones, as discussed in Sec. 3.1 and 3.2. 3.1. Further Finetuning on Low-resource Languages Although the proposed pipeline demonstrates broad linguistic coverage, its performance is st… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 11 linked inside Pith

  1. [1]

    Introduction Large Language Models (LLMs) have transformed natural lan- guage processing, enabling systems that generalize across di- verse text-based tasks. Extending these capabilities to speech is essential for natural multimodal interaction, motivating the rise of Speech Language Models (SLMs), which allow LLMs to process audio directly [1]. Tradition...

    Pith/arXiv arXiv 1900

  2. [2]

    MEUSLI: a Multilingual Projector for 28 EU Languages (and Beyond) In this work, we address these gaps by introducing MEUSLI, the first fully open multilingual projector for LLM-based ASR supporting 28 European languages. MEUSLI extends the SLAM-ASR architecture to a large multilingual setting by con- necting a pre-trained Whisper encoder to open-source pr...

  3. [3]

    Moreover, we show that new languages can be added using state-of-the-art continual learning methodologies

    Unlike recent work on SpeechLLMs, we consider low- resource languages and demonstrate that MEUSLI can be ef- fectively fine-tuned to support them, enabling new languages to be bootstrapped from only limited amounts of speech data. Moreover, we show that new languages can be added using state-of-the-art continual learning methodologies

  4. [4]

    We provide details of model configurations to facilitate suc- cessful fine-tuning within the SLAM-ASR framework

  5. [5]

    Limitations We note that the MEUSLI projector was originally pre-trained on multilingual ASR data from FLEURS (see Section 2.3), which overlaps with the SIB-Fleurs dataset used in this mul- titask extension. Although we adapt the projector to new down- stream tasks rather than ASR alone, this pretraining data may partially contribute to the strong transfe...

  6. [6]

    Transcribe speech to text

    We show experimentally that, starting from MEUSLI, multi- lingual multitask projectors can be obtained with only a small amount of adaptation data, extending support beyond ASR to speech translation and topic identification. To conclude, our work lays the foundation for scalable, open and inclusive speech-to-text systems powered by LLMs. 2.1. The MEUSLI M...

    2025

  7. [7]

    Mono” refers to a monolingual projector trained on the aforementioned data, “MEUSLI

    Results and Discussion We first evaluated the performance of the multilingual projector across the 28 European languages described in Section 2.3. Ta- ble 2 reports the WER obtained on the test splits of Common V oice (CV), FLEURS (FL), and the 2025 MLC-SLM Chal- lenge (MLC). We compare the results obtained with the 3 LLMs againstWhisper-large-v3-turbo. W...

    2025

  8. [8]

    Towards Multiple Tasks While MEUSLI demonstrates effective multilingual adaptation for low-resource ASR (Sections 3.1 and 3.2), speech systems often need to support multiple downstream tasks beyond tran- scription. Recent work has explored multilingual or multitask SpeechLLMs [36, 37, 38, 39, 40, 41, 42], but multilingual mul- titask adaptation under data...

    2026

  9. [9]

    Connecting Speech Encoder and Large Language Model for ASR,

    W. Yu, C. Tang, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Connecting Speech Encoder and Large Language Model for ASR,”ICASSP 2024 - 2024 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), pp. 12 637–12 641, 2023

    2024

  10. [10]

    Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets,

    X. Geng, T. Xu, K. Wei, B. Mu, H. Xue, H. Wang, Y . Li, P. Guo, Y . Dai, L. Li, M. Shao, and L. Xie, “Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets,”2024 IEEE 14th International Symposium on Chinese Spoken Language Pro- cessing (ISCSLP), pp. 26–30, 2024

    2024

  11. [11]

    Experiments show that this approach extends prior English- only pipelines to multilingual settings with strong performance across high- and low-resource languages

    Conclusions In this work, we presented MEUSLI, a multilingual linear projector that bridges a pretrained speech encoder (Whisper) with pretrained multilingual LLMs, enabling end-to-end ASR across 28 European languages using fully open-source mod- els. Experiments show that this approach extends prior English- only pipelines to multilingual settings with s...

  12. [12]

    101135916)

    Acknowledgments This work has received funding from the European Union’s Horizon Europe research and innovation program under the project ELOQUENCE (Grant Agreement No. 101135916). Views and opinions expressed are, however, those of the au- thor(s) only and do not necessarily reflect those of the Euro- pean Union or Research Executive Agency. Neither the ...

  13. [13]

    A survey on speech large language models,

    P. Jing, W. Yucheng, F. Yangui, X. Yu, L. Xu, Z. Xizhuo, and Y . Kai, “A survey on speech large language models,” arXiv:2410.18908, 2024

    arXiv 2024

  14. [14]

    Large language models are strong audio-visual speech recognition learners,

    U. Cappellazzo, M. Kim, H. Chen, P. Ma, S. Petridis, D. Falavi- gna, A. Brutti, and M. Pantic, “Large language models are strong audio-visual speech recognition learners,” inICASSP, 2025, pp. 1–5

    2025

  15. [15]

    Recent advances in speech language models: A survey,

    C. Wenqian, Y . Dianzhi, J. Xiaoqi, M. Ziqiao, Z. Guangyan, W. Qichao, G. Yiwen, and K. Irwin, “Recent advances in speech language models: A survey,”arXiv:2410.03751, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Qwen-audio: Advancing universal audio understand- ing via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understand- ing via unified large-scale audio-language models,”arXiv preprint arXiv:2311.07919, 2023

    Pith/arXiv arXiv 2023

  17. [17]

    Qwen2-audio technical report,

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Linet al., “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Salmonn: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “Salmonn: Towards generic hearing abilities for large language models,”arXiv preprint arXiv:2310.13289, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    An embarrassingly sim- ple approach for LLM with strong ASR capacity,

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wang, Z. Du, F. Yu, Q. Chen, S. Zheng, S. Zhanget al., “An embarrassingly sim- ple approach for LLM with strong ASR capacity,”arXiv preprint arXiv:2402.08846, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Speech LLMs in Low- Resource Scenarios: Data V olume Requirements and the Im- pact of Pretraining on High-Resource Languages,

    S. Fong, M. Matassoni, and A. Brutti, “Speech LLMs in Low- Resource Scenarios: Data V olume Requirements and the Im- pact of Pretraining on High-Resource Languages,” inInterspeech 2025, 2025, pp. 2003–2007

    2025

  21. [21]

    Towards atypical speech transcription using LLM-based ASR,

    J. Zhang and A. Mohan, “Towards atypical speech transcription using LLM-based ASR,” inInterspeech, 2025

    2025

  22. [22]

    8 ✗ ASR open-weights BLOOMZMMS [23] 102 ✓ multi open MEUSLI 28+ ✓ ASR open

  23. [23]

    CTC-Assisted LLM-Based Contextual ASR,

    G. Yang, Z. Ma, Z. Gao, S. Zhang, and X. Chen, “CTC-Assisted LLM-Based Contextual ASR,”2024 IEEE Spoken Language Technology Workshop (SLT), pp. 126–131, 2024

    2024

  24. [24]

    Advancing Multi-Talker ASR Performance With Large Language Models,

    M. Shi, Z. Jin, Y . Xu, Y . Xu, S.-X. Zhang, K. Wei, Y . Shao, C. Zhang, and D. Yu, “Advancing Multi-Talker ASR Performance With Large Language Models,”2024 IEEE Spoken Language Technology Workshop (SLT), pp. 14–21, 2024

    2024

  25. [25]

    Speech Recogni- tion Meets Large Language Model: Benchmarking, Models, and Exploration,

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wang, Z. Du, F. Yu, Q. Chen, S. Zheng, S. Zhang, and X. Chen, “Speech Recogni- tion Meets Large Language Model: Benchmarking, Models, and Exploration,” inAAAI Conference on Artificial Intelligence, 2025

    2025

  26. [26]

    OSUM: Advancing open speech understand- ing models with limited resources in academia,

    X. Geng, K. Wei, Q. Shao, S. Liu, Z. Lin, Z. Zhao, G. Li, W. Tian, P. Chen, Y . Liet al., “OSUM: Advancing open speech understand- ing models with limited resources in academia,”arXiv preprint arXiv:2501.13306, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not,

    F. Verdini, P. Melucci, S. Perna, F. Cariaggi, M. Gaido, S. Papi, S. Mazurek, M. Kasztelnik, L. Bentivogli, S. Brati `eres, P. Meri- aldo, and S. Scardapane, “How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not,” inInterspeech, 2025

    2025

  28. [28]

    Performance evaluation of slam-asr: The good, the bad, the ugly, and the way forward,

    S. Kumar, I. Thorbecke, S. Burdisso, E. Villatoro-Tello, M. KE, K. Hacio ˘glu, P. Rangappa, P. Motlicek, A. Ganapathiraju, and A. Stolcke, “Performance evaluation of slam-asr: The good, the bad, the ugly, and the way forward,” in2025 IEEE International Conference on Acoustics, Speech, and Signal Processing Work- shops (ICASSPW). IEEE, 2025, pp. 1–5

    2025

  29. [29]

    Effective Text Adaptation For LLM-Based ASR Through Soft Prompt Fine-Tuning,

    Y . Ma, Z. Liu, and O. Kalinli, “Effective Text Adaptation For LLM-Based ASR Through Soft Prompt Fine-Tuning,”2024 IEEE Spoken Language Technology Workshop (SLT), pp. 64–69, 2024

    2024

  30. [30]

    Reducing prompt sensitivity in llm-based speech recognition through learnable projection,

    S. Burdisso, E. Villatoro-Tello, S. Kumar, S. Madikeri, A. Carofilis, P. Rangappa, M. K. E, K. Hacioglu, P. Motlicek, and A. Stolcke, “Reducing prompt sensitivity in llm-based speech recognition through learnable projection,” 2026. [Online]. Available: https://arxiv.org/abs/2601.20898

    Pith/arXiv arXiv 2026

  31. [31]

    SALSA: Speedy ASR-LLM Synchronous Aggregation,

    A. Mittal, D. Prabhu, S. Sarawagi, and P. Jyothi, “SALSA: Speedy ASR-LLM Synchronous Aggregation,” inInterspeech 2024, 2024, pp. 3485–3489

    2024

  32. [32]

    Skip-Salsa: Skip Synchronous Fusion of ASR LLM De- coders,

    ——, “Skip-Salsa: Skip Synchronous Fusion of ASR LLM De- coders,” inInterspeech, 2025

    2025

  33. [33]

    The Applicability of Wav2Vec2 and Whisper for Low-Resource Maltese ASR,

    A. Williams, A. Demarco, and C. Borg, “The Applicability of Wav2Vec2 and Whisper for Low-Resource Maltese ASR,”2nd Annual Meeting of the ELRA/ISCA SIG on Under-resourced Lan- guages (SIGUL 2023), 2023

    2023

  34. [34]

    Prompt- ing large language models with speech recognition abilities,

    Y . Fathullah, C. Wu, E. Lakomkin, J. Jia, Y . Shangguan, K. Li, J. Guo, W. Xiong, J. Mahadeokar, O. Kalinliet al., “Prompt- ing large language models with speech recognition abilities,” in ICASSP, 2024

    2024

  35. [35]

    Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training,

    P. Denisov and T. Vu, “Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training,” inFindings of the Association for Computational Lin- guistics: NAACL, 2024

    2024

  36. [36]

    Language family mat- ters: Evaluating llm-based asr across linguistic boundaries,

    Y . Zhang, R. Shekhar, and H. Mouratidis, “Language family mat- ters: Evaluating llm-based asr across linguistic boundaries,”arXiv preprint arXiv:2601.18899, 2026

    arXiv 2026

  37. [37]

    Eurollm: Multilingual language models for europe,

    P. H. Martinset al., “Eurollm: Multilingual language models for europe,”Procedia Computer Science, vol. 255, pp. 53–62, 2025, proceedings of the Second EuroHPC user day

    2025

  38. [38]

    Apertus: Democratizing Open and Compliant LLMs for Global Language Environments,

    A. Hern ´andez-Canoet al., “Apertus: Democratizing Open and Compliant LLMs for Global Language Environments,” https:// arxiv.org/abs/2509.14233, 2025

    arXiv 2025

  39. [39]

    Performance evaluation of slam-asr: The good, the bad, the ugly, and the way forward,

    S. Kumar, I. Thorbecke, S. Burdisso, E. Villatoro-Tello, M. K E, K. Hacio ˘glu, P. Rangappa, P. Motlicek, A. Ganapathiraju, and A. Stolcke, “Performance evaluation of slam-asr: The good, the bad, the ugly, and the way forward,” in2025 IEEE International Conference on Acoustics, Speech, and Signal Processing Work- shops (ICASSPW), 2025

    2025

  40. [40]

    The Eloquence team submission for task 1 of MLC-SLM chal- lenge,

    L. Concina, J. Luque, A. Brutti, M. Matassoni, and Y . Zhang, “The Eloquence team submission for task 1 of MLC-SLM chal- lenge,” inWorkshop on Multilingual Conversational Speech Lan- guage Model (MLC-SLM), 2025

    2025

  41. [41]

    Lora: Low-rank adaptation of large language mod- els

    E. J. Huet al., “Lora: Low-rank adaptation of large language mod- els.” inICLR, 2022

    2022

  42. [42]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in LREC 2020, 2020, pp. 4211–4215

    2020

  43. [43]

    Fleurs: Few-shot learning evaluation of universal representations of speech,

    A. Conneau, M. Ma, S. Khanuja, Y . Zhang, V . Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “Fleurs: Few-shot learning evaluation of universal representations of speech,” arXiv preprint arXiv:2205.12446, 2022. [Online]. Available: https://arxiv.org/abs/2205.12446

    Pith/arXiv arXiv 2022

  44. [44]

    V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,

    C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haz- iza, M. Williamson, J. Pino, and E. Dupoux, “V oxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” inACL, 2021

    2021

  45. [45]

    SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects,

    D. I. Adelani, H. Liu, X. Shen, N. Vassilyev, J. O. Alabi, Y . Mao, H. Gao, and A. E.-S. Lee, “SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects,” 2023

    2023

  46. [46]

    Scalable expansion of multilingual speech llms for asr: A continual learning ap- proach,

    L. Concina, M. Matassoni, and A. Brutti, “Scalable expansion of multilingual speech llms for asr: A continual learning ap- proach,” inProceedings of Speech Language Models in Low- Resource Settings: Performance, Evaluation, and Bias Analysis (SPEAKABLE) @ LREC 2026, N. Hosseini-Kivanani, A. Brutti, M. Matassoni, S. Dowerah, D. Liga, and C. Schommer, Eds. ...

    2026

  47. [47]

    Experience replay for continual learning,

    D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne, “Experience replay for continual learning,” inAdvances in Neural Information Processing Systems, 2019

    2019

  48. [48]

    SpeechMapper: Speech-to-text Embedding Projector for LLMs,

    B. Mohapatra, M. Z. Boito, and I. Calapodescu, “SpeechMapper: Speech-to-text Embedding Projector for LLMs,”arXiv preprint arXiv:2601.20417, 2026

    arXiv 2026

  49. [49]

    Contrastive learning for task- independent SpeechLLM-pretraining,

    M. Z ¨ufle and J. Niehues, “Contrastive learning for task- independent SpeechLLM-pretraining,” inFindings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, Eds. Vienna, Austria: Association for Computational Lin- guistics, Jul. 2025, pp. 8469–8490. [Online]. Available: https://aclanthology.org/...

    2025

  50. [50]

    NA VER LABS Europe Submission to the Instruction-following Track,

    B. Lee, M. Zanon-Boito, L. Besacier, and I. Calapodescu, “NA VER LABS Europe Submission to the Instruction-following Track,” inProceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), 2025, pp. 186–200

    2025

  51. [51]

    U-SAM: An Audio Lan- guage Model for Unified Speech, Audio, and Music Understand- ing,

    Z. Wang, X. Xia, X. Zhu, and L. Xie, “U-SAM: An Audio Lan- guage Model for Unified Speech, Audio, and Music Understand- ing,” inInterspeech 2025, 2025, pp. 2720–2724

    2025

  52. [52]

    TESU-LLM: Training Speech-LLMs Without Speech via Unified Encoder Alignment,

    T. Kim and J. H. Ko, “TESU-LLM: Training Speech-LLMs Without Speech via Unified Encoder Alignment,”arXiv preprint arXiv:2506.06343, 2025

    Pith/arXiv arXiv 2025

  53. [53]

    Zero-resource speech translation and recognition with LLMs,

    K. Mundnich, X. Niu, P. Mathur, S. Ronanki, B. Houston, V . R. Elluru, N. Das, Z. Hou, G. Huybrechts, A. Bhatiaet al., “Zero-resource speech translation and recognition with LLMs,” inICASSP 2025-2025 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  54. [54]

    Mixture of languages: Improved multilingual encoders through language grouping,

    J. M. Janeiro, B. Alastruey, F. Massa, M. Elbayad, B. Piwowarski, P. Gallinari, and L. Barrault, “Mixture of languages: Improved multilingual encoders through language grouping,” inProceed- ings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V . Peng, Eds. Suzhou, China: Asso...

    2025

  55. [55]

    A call for clarity in reporting BLEU scores,

    M. Post, “A call for clarity in reporting BLEU scores,” in Proceedings of the Third Conference on Machine Translation: Research Papers, O. Bojar, R. Chatterjee, C. Federmann, M. Fishel, Y . Graham, B. Haddow, M. Huck, A. J. Yepes, P. Koehn, C. Monz, M. Negri, A. N ´ev´eol, M. Neves, M. Post, L. Specia, M. Turchi, and K. Verspoor, Eds. Brussels, Belgium: A...

    2018

  56. [57]

    Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding,

    F. D. Schmidt, I. Vuli ´c, G. Glava ˇs, and D. I. Adelani, “Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding,” 2025. [Online]. Available: https: //arxiv.org/abs/2501.06117

    Pith/arXiv arXiv 2025

  57. [191]

    Available: https://aclanthology.org/W18-6319/

    [Online]. Available: https://aclanthology.org/W18-6319/

  58. [3000]

    Transcribe speech to text

    We optimize the projector using cross-entropy loss over the target token sequence for all tasks, training for up to 10 epochs with early stopping based on validation loss. Beam search with beam size of 4 is used during inference. The task-specific prompts used during training and infer- ence include: ASR (“Transcribe speech to text. ”), ST [41] (“Perform ...