Pith. sign in

REVIEW 3 major objections 5 minor 42 references

Gated query bank lifts multilingual speech LLMs 14% over matched baselines

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:37 UTC pith:FLYJGDFZ

load-bearing objection Real architectural idea, fair baselines, but headline numbers rest on an unvalidated GPT-4.1 judge and a cherry-picked comparison that ignores their own cascaded baseline. the 3 major comments →

arxiv 2603.07025 v2 pith:FLYJGDFZ submitted 2026-03-07 cs.CL

Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision

Shreyas Gopal , Donghang Wu , Ashutosh Anshul , Yeo Yue Heng , Yizhou Peng , Haoyang Li , Hexin Liu , Eng Siong Chng This is my paper
classification cs.CL
keywords speech language modelsmultilingual instruction followingknowledge distillationquery bankgating networkQ-Formerspoken question answeringASR-only supervision
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that a single shared speech-to-language adapter is too coarse for many languages: when one Q-Former-style projector is shared across six languages, high-resource languages interfere with lower-resource ones. It replaces the static query sequence with a bank of language-specific query tokens and a lightweight gating network that picks or mixes queries from the speech itself. Trained only on paired ASR audio and transcripts, 5.8K hours with the speech encoder and LLM frozen, the resulting model improves instruction-following by 14% over a matched multilingual distillation baseline and beats existing speech LLMs by 32% on a newly synthesized spoken question-answering benchmark. If right, this points to a cheap, scalable route to multilingual spoken assistants without task-specific supervision.

Core claim

On the paper's own terms: language interference in a shared Q-Former projector, not lack of data, is the main barrier to scaling ASR-only distillation to many languages. A per-language query bank plus a gating network that selects the most relevant query sequence for each utterance decouples language-specific phonetic and semantic information, and hard selection with a straight-through estimator works better than soft mixing. The authors also build Audio-MLQA, a five-language spoken question-answering set, and report that their best model reaches an average close-ended score of 3.96 versus 3.85 for the multilingual baseline and 4.14 for the text-only upper bound, while open-ended instruction

What carries the argument

A query bank: K learnable query-token sequences, one per language, replacing the single shared sequence in a Q-Former-style projector. A gating network maps speech embeddings to language logits; soft mixing averages queries by softmax weights, while hard selection chooses one query with a straight-through estimator so gradients still train the gate. Scheduled teacher forcing anneals from language-label supervision to model-predicted routing, and three losses train the system: cross-entropy on language identification, L2 matching of projected speech embeddings to transcript embeddings, and L2 matching of the frozen LLM's hidden states under speech versus text conditioning.

Load-bearing premise

The central empirical claim rests on treating an automated language-model judge's scores as ground truth for instruction-following and spoken QA, and the paper does not report human agreement on its new synthetic benchmarks; if the judge is biased by TTS quality or language-specific phrasing, the headline gains could be artifacts.

What would settle it

Take a random 100-item sample per language from the open-ended set and Audio-MLQA, score the same responses with human raters and with at least one alternative judge, and compare rankings: if the rankings diverge or agreement is weak, the 14% and 32% numbers are not established.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Multilingual speech assistants can be built from ASR transcripts alone: 5.8K hours of paired audio-transcript data, with the speech encoder and LLM frozen, yields instruction-following close to a text-only reference.
  • Language-aware routing is the active ingredient: the matched multilingual baseline without gating lags by 14% on average in open-ended tasks, with the largest gap in Indonesian, the least-represented language.
  • Hard query selection beats soft mixing, suggesting that forcing per-language specialization prevents dominant languages from contaminating a shared representation.
  • The synthesized Audio-MLQA benchmark offers a five-language spoken question-answering test on which the best model improves over existing speech LLMs by 32%.
  • Because the LLM remains frozen and only lightweight adapters are trained, the model inherits broad open-ended instruction-following ability without task-specific supervised fine-tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same query-bank-plus-gating recipe should transfer to other cross-modal adapters that suffer category or domain interference, such as accents, speaking styles, or noise conditions instead of languages; the paper does not test this.
  • A zero-shot language not in the bank would expose the method's boundary: hard selection would route it to the nearest known language, likely degrading to baseline; a continuous language embedding might handle unseen languages more gracefully.
  • The 32% close-ended gain over supervised-fine-tuning baselines may partly reflect those baselines' inability to handle the question-context retrieval task rather than a pure routing effect; an ablation with an oracle language label would isolate the routing contribution.
  • Because the gate is supervised by language labels, the method depends on reliable language IDs in training data; for code-switched speech, hard selection could fail by committing to a single language.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes language-aware distillation for multilingual instruction-following speech LLMs trained with ASR-only supervision. The architecture keeps a frozen Whisper-large-v3 encoder and a frozen Llama-SEA-LION-v3-8B-IT LLM, with a Q-Former-style projector whose query tokens are selected or mixed from a per-language query bank by a gating network. Training uses a language-ID loss plus input-embedding and output-hidden-state distillation against transcript-conditioned LLM signals. The authors compare against matched EN-DiVA and ML-DiVA baselines, existing end-to-end speech LLMs, and a cascaded ASR+LLM baseline, using GPT-4.1 model-as-judge scores on open-ended instruction-following benchmarks and on a newly synthesized Audio-MLQA spoken QA benchmark. They report a 14% open-ended improvement over ML-DiVA and a 32% improvement over existing Speech LLM baselines on Audio-MLQA.

Significance. If the empirical claims hold, the contribution is practically valuable: a resource-efficient extension of DiVA-style distillation to multilingual settings, with a simple and plausible gating/query-bank mechanism that addresses language interference. The proposed Audio-MLQA benchmark and the in-house open-ended TTS evaluation sets could also be useful to the community. The method is clearly specified and the ablations support the basic design choices. However, the headline results rest entirely on an unvalidated GPT-4.1 model-as-judge metric applied to self-built, TTS-synthesized benchmarks, and the comparison to the cascaded baseline is not stated in the abstract's 32% claim. The significance is therefore conditional until the evaluation is anchored to human judgment or objective metrics.

major comments (3)
  1. [§3.3, Table 2] The central empirical claims (14% open-ended gain over ML-DiVA, 32% Audio-MLQA gain over existing speech LLMs) are computed solely from GPT-4.1 0–5 model-as-judge scores on in-house TTS-synthesized benchmarks. No human-judge correlation, inter-annotator agreement, error bars, or multiple runs are reported. Citing PEDANTS [34] is not sufficient: that work addresses answer equivalence, not holistic subjective scoring of open-ended responses. This is load-bearing, because judge bias toward response style or TTS quality could manufacture the reported gains. Please add a human evaluation subset with agreement/correlation statistics, report variance/bootstrap intervals, and for close-ended Audio-MLQA also report objective metrics such as exact-match/F1.
  2. [§4.2, Table 2] The abstract and §4.2 state that the best model improves over existing Speech LLM baselines by 32% on Audio-MLQA, but Table 2's cascaded Whisper-v3 + SEA-LION-v3-8B-IT baseline scores 3.99, higher than the hard-gating model's 3.96. The 32% figure is computed only against end-to-end systems (SeaLLMs-Audio and Qwen2-Audio). As written, the claim is misleading. Please explicitly restrict the claim to end-to-end Speech LLMs, report the cascaded comparison, and discuss why the proposed end-to-end model underperforms the simple pipeline on close-ended QA.
  3. [§2.2.1, §4.3] The gating network is trained with language-ID labels and uses hard query selection at inference, so routing errors directly determine which query bank is used. Table 3 reports LID accuracy above 94.9%, but this is measured on ASR validation data, not on the TTS-synthesized evaluation sets, which use different voices and recording conditions. No evaluation-set LID accuracy or misrouting analysis is provided. Since a wrong language selection would degrade the model, please report gating accuracy on the evaluation data or otherwise show that routing is robust to domain shift.
minor comments (5)
  1. [Table 2] The column headers for the open-ended portion are garbled in the manuscript (e.g., repeated 'AE', unclear grouping under ZH/EN/ID). This makes the table very hard to read; please reformat with clear subheadings and a legend.
  2. [§2.1] Typo: 'The output Z acts as a for the frozen LLM' — missing noun (likely 'soft prefix').
  3. [§3.2.2] Duplicate wording: 'synthesize the spoken questions using using state-of-the-art TTS systems.'
  4. [Eq. (2)] The symbol T is reused for both speech time steps and the number of transcript positions in Y_head and Z_tail. This is confusing when L < T or N < T; please use distinct symbols and clarify the alignment/masking strategy.
  5. [§3.4, Eq. (4)] The values of λ_IN, λ_OUT, λ_LID are not reported, despite being free parameters in the objective. Please include them for reproducibility.

Circularity Check

0 steps flagged

No load-bearing circularity: ASR-only training objective is separate from held-out judged benchmarks; only a minor motivational self-citation appears.

full rationale

The claimed derivation chain is not circular. The training objective (Eqs. 1-4) consists of a language-identification loss on ASR language labels, an input distillation loss aligning projected speech embeddings with transcript LLM embeddings, and an output distillation loss matching frozen-LLM hidden states. None of these terms involve the evaluation benchmarks (UROBench/AlpacaEval subsets, Audio-OpenHermes, Audio-MLQA) or the GPT-4.1 judge scores. The query bank and gating network are trained with LID cross-entropy on ASR labels, but the reported instruction-following and QA gains are measured on held-out, TTS-synthesized tasks that are not part of the training loss. The self-created Indonesian open-ended set and Audio-MLQA are external to the training signal, and the matched baselines (EN-DiVA, ML-DiVA) are retrained on the same ASR data, so the comparison is not a fitted-input prediction. The only apparent self-citation, [21] in Section 1, supports the general motivation that language conditioning helps in multilingual ASR; it is not load-bearing for the architecture, the losses, or the headline numbers. The use of GPT-4.1 as judge is a validity concern (no human correlation, inter-annotator agreement, or error bars reported), but that is not a by-construction equivalence between prediction and input: the judge could in principle favor any model, and the paper does not optimize its training objective against that judge. Therefore no derivation step reduces to its own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

All parameters are standard training hyperparameters; there are no fitted constants tuning to the eval tasks. The main epistemic burden is the validity of the self-created TTS benchmarks and the judge, listed as axioms.

free parameters (4)
  • Loss weights λ_IN, λ_OUT, λ_LID
    Chosen by hand; not reported in the text, yet they determine the balance between input/output distillation and LID supervision.
  • Teacher forcing annealing schedule = cosine 1→0 by 50% steps
    Scheduled teacher forcing in §2.2.2 injects language labels into query selection for the first half of training; this is a design choice that could affect how independent the gate is.
  • Query length L = 256
    Ablated in Table 3; longer queries reduce distillation loss, but the final choice is a capacity decision.
  • WER filter threshold = 10%
    Training samples with WER>10% from expert ASR models are excluded (§3.1); the threshold affects the training distribution.
axioms (5)
  • domain assumption Hidden-state L2 regression is a close approximation to logit-level KL distillation for aligning speech and text behavior.
    Stated after Eq. (3), following DiVA [12]; if the approximation is poor, the output distillation signal may misalign.
  • domain assumption The last T projected speech embeddings align monotonically with the first T transcript embeddings ('audio tail').
    Eq. (2) assumes a fixed monotonic alignment between speech and text; long or misordered speech (background speech, disfluencies) could violate it.
  • ad hoc to paper GPT-4.1 model-as-judge scores are valid proxies for human judgment on multilingual open-ended instruction following and spoken QA.
    §3.3 relies on PEDANTS generally but provides no human correlation on the newly created ID/EN/ZH open-ended items or Audio-MLQA.
  • domain assumption Synthesized TTS audio is representative of natural spoken questions/instructions for evaluating speech understanding.
    All new evaluation audio is synthesized at 44.1kHz (§3.2.2); TTS artifacts could make the task easier or harder than real speech.
  • domain assumption The language label is available for training samples and the gating network generalizes to unseen utterances.
    LID loss uses language labels (§2.3) and scheduled teacher forcing uses labels during training; at inference the gate predicts language with ~95% accuracy (Table 3).

pith-pipeline@v1.3.0-alltime-deepseek · 8753 in / 14263 out tokens · 127058 ms · 2026-08-02T18:37:43.185205+00:00 · methodology

0 comments
read the original abstract

Speech Large Language Models (LLMs) that understand and follow instructions in many languages are useful for real-world interaction, but are difficult to train with supervised fine-tuning, requiring large, task-specific speech corpora. While recent distillation-based approaches train performant English-only Speech LLMs using only annotated ASR data by aligning text and speech using only a lightweight projector, these models under-perform when scaled to multilingual settings due to language interference in the shared projector. We address this by introducing language-aware distillation using a query bank and a gating network that selects or mixes query tokens using a Q-Former projector. Our approach shows gains of 14% over matched multilingual distillation baselines on instruction following. We further synthesize Audio-MLQA, a multilingual spoken QA benchmark built on MLQA with high-quality TTS questions. Our best model improves over existing Speech LLM baselines by 32% on Audio-MLQA.

Figures

Figures reproduced from arXiv: 2603.07025 by Ashutosh Anshul, Donghang Wu, Eng Siong Chng, Haoyang Li, Hexin Liu, Shreyas Gopal, Yeo Yue Heng, Yizhou Peng.

Figure 1
Figure 1. Figure 1: End-to-end model architecture. Trainable components are colored orange. 2. Methodology [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 9 linked inside Pith

  1. [1]

    Cascaded ASR-to- LLM pipelines convert speech signals to text early in the ASR stage, leading to information beyond semantics being discarded

    Introduction Speech Large Language Models (Speech LLMs) integrate a speech encoder with a text LLM to support speech-conditioned understanding, reasoning, and generation. Cascaded ASR-to- LLM pipelines convert speech signals to text early in the ASR stage, leading to information beyond semantics being discarded. In contrast, existing end-to-end approaches...

    Pith/arXiv arXiv 2026

  2. [2]

    audio tail

    Methodology Figure 1 illustrates our approach which consists of 4 compo- nents: (1) a frozen speech encoder, (2) a Q-Former projector, (3) a frozen LLM and (4) the query-selection module. 2.1. Model Architecture Building on prior Speech LLM models, we train a lightweight adapter using paired speech and transcript data. Given speechx and transcripty, the m...

  3. [3]

    Training Data Only annotated ASR data was used during the training of the model

    Experiments 3.1. Training Data Only annotated ASR data was used during the training of the model. The various sources and amounts of data are detailed in Table 1. We filter out audio samples with word error rate (WER%) greater than 10% from expert ASR models. 3.2. Evaluation Data Due to the lack of established evaluation benchmarks for mul- tilingual spee...

    2000

  4. [4]

    answer not found

    Results We establish the two matched baselines EN-DiV A (which is closest to the original DiVA model) and ML-DiV A trained on Table 2:GPT-4.1 Model-as-Judge scores for open-ended instruction following and close-ended spoken QA. Higher is better (↑). To keep comparison fair, if a model does not support a language, the score has been left blank. EN-DiVA and...

  5. [5]

    Conclusion In this study, we introduce a language-aware distillation frame- work to resolve the language interference bottleneck in distilled multilingual Speech LLMs. By replacing static query sequences with a dynamic query bank and gating mechanism, we effec- tively disentangle language-specific information while main- taining high data efficiency. Our ...

  6. [6]

    Acknowledgments The computational work for this article was partially performed on resources of the National Supercomputing Centre, Singapore (https://www.nscc.sg) and partially supported by the High Per- formance Computing Centre of Nanyang Technological Univer- sity, Singapore

  7. [7]

    Generative AI Use Disclosure Generative AI tools were used for limited assistance with manuscript editing and presentation (e.g., grammatical valida- tion, removal of redundant sentences and phrases, preparing La- TeX equations and LaTeX formatting suggestions). The liter- ature review, and all scientific contributions including but not limited to problem...

  8. [8]

    Blsp-emo: Towards empathetic large speech-language models,

    C. Wang, M. Liaoet al., “Blsp-emo: Towards empathetic large speech-language models,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 19 186–19 199

    2024

  9. [9]

    Benchmarking contextual and par- alinguistic reasoning in speech-llms: A case study with in-the- wild data,

    Q. Wang, H. B. Sailoret al., “Benchmarking contextual and par- alinguistic reasoning in speech-llms: A case study with in-the- wild data,”arXiv preprint arXiv:2509.16589, 2025

    arXiv 2025

  10. [10]

    Generative spoken dialogue language modeling,

    T. A. Nguyen, E. Kharitonov, J. Copet, Y . Adi, W.-N. Hsu, A. Elkahky, P. Tomasello, R. Algayres, B. Sagot, A. Mohamed et al., “Generative spoken dialogue language modeling,”Trans- actions of the Association for Computational Linguistics, vol. 11, pp. 250–266, 2023

    2023

  11. [11]

    Qwen-audio: Advancing universal audio un- derstanding via unified large-scale audio-language models,

    Y . Chu, J. Xuet al., “Qwen-audio: Advancing universal audio un- derstanding via unified large-scale audio-language models,”arXiv preprint arXiv:2311.07919, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    Qwen2-audio technical report,

    ——, “Qwen2-audio technical report,”arXiv preprint arXiv:2407.10759, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Salmonn: Towards generic hearing abilities for large language models,

    C. Tang, W. Yuet al., “Salmonn: Towards generic hearing abilities for large language models,” inProc. ICLR, 2024

    2024

  14. [14]

    Salm: Speech- augmented language model with in-context learning for speech recognition and translation,

    Z. Chen, H. Huang, A. Andrusenko, O. Hrinchuk, K. C. Puvvada, J. Li, S. Ghosh, J. Balam, and B. Ginsburg, “Salm: Speech- augmented language model with in-context learning for speech recognition and translation,” inProc. ICASSP. IEEE, 2024, pp. 13 521–13 525

    2024

  15. [15]

    Meralion-audiollm: Advancing speech and language understanding for singapore,

    Y . He, Z. Liuet al., “Meralion-audiollm: Advancing speech and language understanding for singapore,” inProc. ACL, 2025, pp. 22–30

    2025

  16. [16]

    Seallms-audio: Large audio- language models for southeast asia,

    C. Liu, M. Aljuniedet al., “Seallms-audio: Large audio- language models for southeast asia,” 2025. [Online]. Available: https://arxiv.org/abs/2511.01670

    arXiv 2025

  17. [17]

    Llama- omni: Seamless speech interaction with large language models,

    Q. Fang, S. Guo, Y . Zhou, Z. Ma, S. Zhang, and Y . Feng, “Llama- omni: Seamless speech interaction with large language models,” arXiv preprint arXiv:2409.06666, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Analyzing mitigation strategies for catastrophic forgetting in end-to-end training of spoken language models,

    C.-Y . Hsiao, K.-H. Lu, K.-W. Chang, C.-K. Yang, W.-C. Chen, and H.-y. Lee, “Analyzing mitigation strategies for catastrophic forgetting in end-to-end training of spoken language models,” Proc. Interspeech, 2025

    2025

  19. [19]

    Dis- tilling an end-to-end voice assistant without instruction training data,

    W. Held, Y . Zhang, M. Li, W. Shi, M. J. Ryan, and D. Yang, “Dis- tilling an end-to-end voice assistant without instruction training data,” inProc. ACL, 2025, pp. 7876–7891

    2025

  20. [20]

    Tango 2: Aligning diffusion-based text- to-audio generations through direct preference optimization,

    N. Majumder, C.-Y . Hung, D. Ghosal, W.-N. Hsu, R. Mihal- cea, and S. Poria, “Tango 2: Aligning diffusion-based text- to-audio generations through direct preference optimization,” in Proc. ACM MM, 2024, pp. 564–572

    2024

  21. [21]

    Instruction data generation and unsupervised adap- tation for speech language models,

    V . Noroozi, Z. Chen, S. Majumdar, S. Huang, J. Balam, and B. Ginsburg, “Instruction data generation and unsupervised adap- tation for speech language models,”Proc. Interspeech, 2024

    2024

  22. [22]

    Speechless: Speech Instruction Training Without Speech for Low Resource Languages,

    A. Dao, D. B. Vuet al., “Speechless: Speech Instruction Training Without Speech for Low Resource Languages,” inProc. Inter- speech, 2025, pp. 3239–3243

    2025

  23. [23]

    Blsp: Bootstrapping language-speech pre-training via behavior alignment of continuation writing,

    C. Wang, M. Liao, Z. Huang, J. Lu, J. Wu, Y . Liu, C. Zong, and J. Zhang, “Blsp: Bootstrapping language-speech pre-training via behavior alignment of continuation writing,”arXiv preprint arXiv:2309.00916, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Inte- grating speech self-supervised learning models and large language models for asr,

    L. Dong, Z. Yu, W. Wang, Y . Huang, S. Gao, and G. Zhou, “Inte- grating speech self-supervised learning models and large language models for asr,” inProc. Interspeech, 2024, pp. 3954–3958

    2024

  25. [25]

    An end-to-end speech summarization using large lan- guage model,

    H. Shang, Z. Li, J. Guo, S. Li, Z. Rao, Y . Luo, D. Wei, and H. Yang, “An end-to-end speech summarization using large lan- guage model,”Proc. Interspeech, 2024

    2024

  26. [26]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” inProc. ICML. PMLR, 2023, pp. 19 730–19 742

    2023

  27. [27]

    Robust speech recognition via large-scale weak su- pervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” inProc. ICML, ser. ICML’23. JMLR.org, 2023

    2023

  28. [28]

    A two-stage lora strategy for expanding language capabilities in multilingual asr models,

    C. Y . Kwok, H. Liu, J. Q. Yip, S. Li, and E. S. Chng, “A two-stage lora strategy for expanding language capabilities in multilingual asr models,”IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 2576–2590, 2025

    2025

  29. [29]

    Language-routing mixture of experts for multilingual and code-switching speech recognition,

    W. Wang, G. Ma, Y . Li, and B. Du, “Language-routing mixture of experts for multilingual and code-switching speech recognition,” Proc. Interspeech, 2023

    2023

  30. [30]

    Adap- tive gating in mixture-of-experts based language models,

    J. Li, Q. Su, Y . Yang, Y . Jiang, C. Wang, and H. Xu, “Adap- tive gating in mixture-of-experts based language models,” inProc. EMNLP, H. Bouamor, J. Pino, and K. Bali, Eds., 2023, pp. 3577– 3587

    2023

  31. [31]

    Sea-lion: Southeast asian languages in one network,

    R. Ng, T. N. Nguyen, H. Yuli, T. N. Chia, L. W. Yi, W. Q. Leong, X. Yong, J. G. Ngui, Y . Susanto, N. Chenget al., “Sea-lion: Southeast asian languages in one network,” inProc. IJCNLP and Asia-Pacific Chapter of ACL, 2025, pp. 512–526

    2025

  32. [32]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al- Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Common voice: A massively-multilingual speech corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proc. LREC, 2020, pp. 4211–4215

    2020

  34. [34]

    viV oice: Enabling vietnamese multi-speaker speech synthesis,

    Capleaf, “viV oice: Enabling vietnamese multi-speaker speech synthesis,” inHugging Face, 2024, license: CC BY-NC-SA 4.0 (gated access). [Online]. Available: https://huggingface.co/ datasets/capleaf/viV oice

    2024

  35. [35]

    Yodas: Youtube-oriented dataset for audio and speech,

    X. Li, S. Takamichi, T. Saeki, W. Chen, S. Shiota, and S. Watan- abe, “Yodas: Youtube-oriented dataset for audio and speech,” in2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–8

    2023

  36. [36]

    Magic data open source cor- pus (id: 101),

    Magic Data Technology Co., Ltd., “Magic data open source cor- pus (id: 101),” http://www.imagicdatatech.com/index.php/home/ dataopensource/data info/id/101, May 2019, accessed: 2026-02- 25

    2019

  37. [37]

    URO-bench: Towards comprehensive eval- uation for end-to-end spoken dialogue models,

    R. Yan, X. Liet al., “URO-bench: Towards comprehensive eval- uation for end-to-end spoken dialogue models,”Findings of the Association for Computational Linguistics: EMNLP 2025, 2025

    2025

  38. [38]

    V oicebench: Benchmarking llm-based voice assistants,

    Y . Chen, X. Yue, C. Zhang, X. Gao, R. T. Tan, and H. Li, “V oicebench: Benchmarking llm-based voice assistants,”arXiv preprint arXiv:2410.17196, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    AudioBench: A universal benchmark for audio large language models,

    B. Wang, X. Zou, G. Lin, S. Sun, Z. Liu, W. Zhang, Z. Liu, A. Aw, and N. F. Chen, “AudioBench: A universal benchmark for audio large language models,” inProc. NAACL, 2025, pp. 4297–4316

    2025

  40. [40]

    Mlqa: Evaluating cross-lingual extractive question answering,

    P. Lewis, B. O ˘guz, R. Rinott, S. Riedel, and H. Schwenk, “Mlqa: Evaluating cross-lingual extractive question answering,”arXiv preprint arXiv:1910.07475, 2019

    Pith/arXiv arXiv 1910

  41. [41]

    PEDANTS: Cheap but effective and interpretable answer equiv- alence,

    Z. Li, I. Mondal, H. Nghiem, Y . Liang, and J. L. Boyd-Graber, “PEDANTS: Cheap but effective and interpretable answer equiv- alence,” inFindings of EMNLP 2024, Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, Eds., 2024, pp. 9373–9398

    2024

  42. [42]

    Glm-4-voice: Towards intelligent and human-like end- to-end spoken chatbot,

    A. Zeng, Z. Du, M. Liu, K. Wang, S. Jiang, L. Zhao, Y . Dong, and J. Tang, “Glm-4-voice: Towards intelligent and human-like end- to-end spoken chatbot,”arXiv preprint arXiv:2412.02612, 2024

    Pith/arXiv arXiv 2024