pith. sign in

arxiv: 2606.25444 · v1 · pith:C3YLVFNAnew · submitted 2026-06-24 · 📡 eess.AS · cs.CL· cs.SD

Does Translation-Enhanced Speech Encoder Pre-training Affect Speech LLMs?

Tomoya Mizumoto , Yusuke Fujita This is my paper

Pith reviewed 2026-06-25 20:01 UTC · model grok-4.3

classification 📡 eess.AS cs.CLcs.SD
keywords speech translationspeech encoderpre-trainingSpeech LLMlanguage-agnosticcross-modal alignmentASR
0
0 comments X

The pith

Translation pre-training of speech encoders improves Speech LLM performance by aligning language-agnostic representations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that standard speech encoders from ASR tasks create language-specific spaces that misalign with the language-agnostic space of LLMs. Speech translation requires the encoder to map between languages, forcing it to learn shared representations. Experiments confirm that adding translation objectives during pre-training leads to better integration when connecting the encoder to an LLM. This matters because it offers a direct way to fix the structural mismatch in Speech LLM architectures without additional alignment steps.

Core claim

Speech translation provides a principled mechanism to align speech encoder representations with LLM spaces. Unlike ASR-based pre-training, translation objectives bridge different languages and produce language-agnostic representations, improving cross-modal integration and yielding superior results on downstream Speech LLM tasks.

What carries the argument

The translation objective during speech encoder pre-training, which enforces language-agnostic representations by requiring cross-language mapping.

If this is right

  • Improved cross-modal integration between the speech encoder and the LLM.
  • Superior performance on a range of downstream Speech LLM tasks.
  • Representations from the encoder that better match the unified space used by LLMs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Future models might skip separate alignment modules if translation pre-training is used.
  • Similar benefits could appear in other cross-lingual or multimodal setups.
  • The approach might generalize to other tasks requiring language-agnostic features.

Load-bearing premise

Observed performance improvements stem specifically from the translation objective fostering language-agnostic representations rather than from variations in data amount or training settings.

What would settle it

Running the pre-training experiments with exactly the same data volume and hyperparameters for both translation-enhanced and baseline setups, and finding equivalent performance on Speech LLM tasks.

read the original abstract

Connecting a pre-trained speech encoder to a Large Language Model (LLM) is the standard architecture for building Speech LLMs. However, a structural misalignment exists between the encoder and the LLM. Unlike encoders based on automatic speech recognition, which often produce representations in separate language-specific spaces, LLMs operate within a unified language-agnostic space. A mechanism is required to align the encoder's language-specific representations with the LLM's shared space. We argue that speech translation provides a principled way to achieve this. Unlike monolingual transcription, translation requires the model to bridge different languages and learn language-agnostic representations. We experimentally evaluate the impact of incorporating translation objectives into speech encoder pre-training. Our results demonstrate that translation-enhanced pre-training improves cross-modal integration and leads to superior performance across downstream Speech LLM tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper argues that speech encoders pre-trained with ASR objectives produce language-specific representations that are misaligned with the language-agnostic space of LLMs, and proposes that adding speech translation objectives during pre-training induces language-agnostic representations that improve cross-modal integration. It reports that this translation-enhanced pre-training yields superior performance on downstream Speech LLM tasks.

Significance. If the performance gains can be isolated to the translation objective after proper controls, the result would offer a concrete, principled pre-training strategy for improving encoder-LLM alignment in Speech LLMs. The work directly tests a hypothesis about representation spaces and supplies an empirical comparison that could guide future encoder design.

major comments (2)
  1. [Experiments] Experiments section: the manuscript provides no information on whether the translation-enhanced and baseline pre-training runs were matched for total data volume, number of languages, training steps, or optimization hyperparameters. This control is load-bearing for the central claim that observed gains arise from the translation objective creating language-agnostic representations rather than from differences in data scale or training regime.
  2. [Results] Results section: no baselines, statistical significance tests, or data-exclusion criteria are described, so it is not possible to assess whether the reported improvements are robust or attributable to the proposed mechanism.
minor comments (1)
  1. [Abstract] The abstract states the conclusion without any quantitative detail or reference to controls; a brief parenthetical on the scale of the comparison would improve clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback highlighting the need for explicit experimental controls and improved reporting. We address each major comment below and will revise the manuscript to incorporate the requested details.

read point-by-point responses

  1. Referee: [Experiments] Experiments section: the manuscript provides no information on whether the translation-enhanced and baseline pre-training runs were matched for total data volume, number of languages, training steps, or optimization hyperparameters. This control is load-bearing for the central claim that observed gains arise from the translation objective creating language-agnostic representations rather than from differences in data scale or training regime.

    Authors: We agree that matching these factors is essential to isolate the effect of the translation objective. The pre-training runs were matched on total data volume, number of languages, training steps, and optimization hyperparameters, differing only in the addition of the translation objective. We will revise the Experiments section to explicitly document these controls and confirm that the observed gains are attributable to the translation objective. revision: yes

  2. Referee: [Results] Results section: no baselines, statistical significance tests, or data-exclusion criteria are described, so it is not possible to assess whether the reported improvements are robust or attributable to the proposed mechanism.

    Authors: We acknowledge that these elements are necessary for assessing robustness. The revised manuscript will include comparisons against standard baselines, report statistical significance tests on the performance differences, and specify data-exclusion criteria. These additions will strengthen the attribution of improvements to the language-agnostic representations induced by translation-enhanced pre-training. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical comparison with no derivation chain

full rationale

The paper reports an experimental comparison of speech encoder pre-training with and without translation objectives, claiming improved downstream Speech LLM performance. No mathematical derivations, equations, or first-principles predictions are present that could reduce to inputs by construction. The central claim rests on observed empirical differences rather than self-definitional structures, fitted parameters renamed as predictions, or load-bearing self-citations. This is a standard empirical setup with no reduction of results to their own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so no specific free parameters, axioms, or invented entities can be identified from the text. The central claim rests on the validity of the reported experimental comparison.

pith-pipeline@v0.9.1-grok · 5664 in / 1091 out tokens · 31894 ms · 2026-06-25T20:01:06.153125+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 11 canonical work pages · 8 internal anchors

  1. [1]

    Within this framework, an adaptor maps continuous acoustic features into the embedding space of the LLM, facilitating seamless cross-modal understanding

    Introduction To build Speech Large Language Models (Speech LLMs) ca- pable of processing audio directly, a prevalent architecture in- tegrates a pre-trained speech encoder with an LLM via a train- able adaptor [1, 2, 3, 4]. Within this framework, an adaptor maps continuous acoustic features into the embedding space of the LLM, facilitating seamless cross-...

  2. [2]

    Does Translation-Enhanced Speech Encoder Pre-training Affect Speech LLMs?

    Related Works Ma et al. [16] demonstrated that Whisper inherently maps multilingual speech into a shared semantic space. By em- ploying Whisper as a standard encoder–decoder and simply fine-tuning its decoder, they achieved zero-shot cross-lingual transfer, confirming that translation objectives naturally in- duce language-agnostic representations. Althou...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  3. [3]

    To this end, we designed a controlled experimental framework

    Research Design The primary objective of this study is to investigate how incor- porating translation tasks during speech encoder pre-training af- fects the cross-modal integration of the encoder with an LLM. To this end, we designed a controlled experimental framework. Maintaining the same overall model architecture and adaptor training process, we syste...

  4. [4]

    Experiments In this section, we evaluate the impact of the different pre- training configurations by comparing the downstream genera- tive performance of the fully integrated Speech LLM. 4.1. Training Data and Experimental Setup Base Model Pre-training Data:We compiled a large-scale multilingual speech dataset comprising approximately 130k hours of audio ...

  5. [5]

    Intent” involves understanding user commands, evaluated on SLURP and Speech-massive. “Emo- tion

    The peak learning rate was set to1×10 −4 with a 500- step linear warmup, followed by a cosine decay schedule. This training stage was conducted on 8 NVIDIA H100 GPUs. 4.2. Model Configurations To implement our Speech LLM, we utilize the encoder portion of the Whisper medium architecture as our speech representa- tion extractor. For the core language model...

  6. [6]

    Compared to standard transcription or unidirectional baselines, this approach significantly improves downstream performance across both speech translation and classification tasks

    Conclusion In this work, we demonstrated that symmetric, bidirectional translation (X↔en) is a highly effective pre-training objec- tive for Speech LLMs. Compared to standard transcription or unidirectional baselines, this approach significantly improves downstream performance across both speech translation and classification tasks. Intent classification ...

  7. [7]

    Generative AI Use Disclosure Generative AI tools (Gemini and ChatGPT) were used for lan- guage editing and improving the phrasing of this manuscript

  8. [8]

    Prompting Large Language Models with Speech Recognition Abilities,

    Y . Fathullah, C. Wu, E. Lakomkin, J. Jia, Y . Shangguan, K. Li, J. Guo, W. Xiong, J. Mahadeokar, O. Kalinli, C. Fuegen, and M. Seltzer, “Prompting Large Language Models with Speech Recognition Abilities,” inProceedings of the 2024 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing, 2024, pp. 13 351–13 355

    2024

  9. [9]

    AudioPaLM: A Large Language Model That Can Speak and Listen

    P. K. Rubenstein, C. Asawaroengchai, D. D. Nguyen, A. Bapna, Z. Borsoset al., “AudioPaLM: A Large Language Model That Can Speak and Listen,” 2023, arXiv:2306.12925

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  10. [10]

    Integrating Pre-Trained Speech and Language Mod- els for End-to-End Speech Recognition,

    Y . Hono, K. Mitsuda, T. Zhao, K. Mitsui, T. Wakatsuki, and K. Sawada, “Integrating Pre-Trained Speech and Language Mod- els for End-to-End Speech Recognition,” inFindings of the Asso- ciation for Computational Linguistics, 2024, pp. 13 289–13 305

    2024

  11. [11]

    SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities,

    D. Zhang, S. Li, X. Zhang, J. Zhan, P. Wang, Y . Zhou, and X. Qiu, “SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities,” 2023, arXiv:2305.11000

    work page arXiv 2023

  12. [12]

    HuBERT: Self-Supervised Speech Rep- resentation Learning by Masked Prediction of Hidden Units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “HuBERT: Self-Supervised Speech Rep- resentation Learning by Masked Prediction of Hidden Units,” IEEE/ACM Transactions on Audio, Speech and Language Pro- cessing, pp. 3451–3460, 2021

    2021

  13. [13]

    Conformer: Convolution-augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Proceedings of the Interspeech 2020, 2020, pp. 5036–5040

    2020

  14. [14]

    wav2vec: Unsupervised Pre-Training for Speech Recognition,

    S. Schneider, A. Baevski, R. Collobert, and M. Auli, “wav2vec: Unsupervised Pre-Training for Speech Recognition,” inProceed- ings of Interspeech 2019, 2019, pp. 3465–3469

    2019

  15. [15]

    SLM: Bridge the Thin Gap Between Speech and Text Foundation Models,

    M. Wang, W. Han, I. Shafran, Z. Wu, C.-C. Chiu, Y . Cao, N. Chen, Y . Zhang, H. Soltau, P. K. Rubenstein, L. Zilka, D. Yu, G. Pundak, N. Siddhartha, J. Schalkwyk, and Y . Wu, “SLM: Bridge the Thin Gap Between Speech and Text Foundation Models,” inProceed- ings of the 2023 IEEE Automatic Speech Recognition and Under- standing Workshop, 2023

    2023

  16. [16]

    Self-Supervised Speech Representations are More Phonetic than Semantic,

    K. Choi, A. Pasad, T. Nakamura, S. Fukayama, K. Livescu, and S. Watanabe, “Self-Supervised Speech Representations are More Phonetic than Semantic,” inProceedings of Interspeech 2024, 2024, pp. 4578–4582

    2024

  17. [17]

    Robust speech recognition via large-scale weak su- pervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” inProceedings of the 40th International Conference on Machine Learning, 2023

    2023

  18. [18]

    Qwen2-Audio Technical Report

    Y . Chu, J. Xu, Q. Yang, H. Wei, X. Wei, Z. Guo, Y . Leng, Y . Lv, J. He, J. Lin, C. Zhou, and J. Zhou, “Qwen2-Audio Technical Report,” 2024, arXiv:2407.10759. [Online]. Available: https://arxiv.org/abs/2407.10759

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Qwen2.5-Omni Technical Report

    J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y . Fan, K. Dang, B. Zhang, X. Wang, Y . Chu, and J. Lin, “Qwen2.5-Omni Technical Report,” 2025, arXiv:2503.20215

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  20. [20]

    SALMONN: Towards Generic Hearing Abilities for Large Language Models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. MA, and C. Zhang, “SALMONN: Towards Generic Hearing Abilities for Large Language Models,” inProceedings of the Twelfth Inter- national Conference on Learning Representations, 2024

    2024

  21. [21]

    LLaMA-Omni: Seamless Speech Interaction with Large Lan- guage Models,

    Q. Fang, S. Guo, Y . Zhou, Z. Ma, S. Zhang, and Y . Feng, “LLaMA-Omni: Seamless Speech Interaction with Large Lan- guage Models,” inProceedings of the Thirteenth International Conference on Learning Representations, 2025

    2025

  22. [22]

    Sound- wave: Less is More for Speech-Text Alignment in LLMs,

    Y . Zhang, Z. Liu, F. Bu, R. Zhang, B. Wang, and H. Li, “Sound- wave: Less is More for Speech-Text Alignment in LLMs,” inPro- ceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics, 2025, pp. 18 718–18 738

    2025

  23. [23]

    Cross-Lingual Transfer Learning for Speech Translation,

    R. Ma, M. Qian, Y . Fathullah, S. Tang, M. Gales, and K. Knill, “Cross-Lingual Transfer Learning for Speech Translation,” in Proceedings of the 2025 Conference of the Nations of the Amer- icas Chapter of the Association for Computational Linguistics: Human Language Technologies, 2025, pp. 33–43

    2025

  24. [24]

    WavLLM: Towards Robust and Adaptive Speech Large Language Model,

    S. Hu, L. Zhou, S. Liu, S. Chen, L. Meng, H. Hao, J. Pan, X. Liu, J. Li, S. Sivasankaran, L. Liu, and F. Wei, “WavLLM: Towards Robust and Adaptive Speech Large Language Model,” inFindings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 4552–4572

    2024

  25. [25]

    Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-Audio: Advancing Universal Audio Un- derstanding via Unified Large-Scale Audio-Language Models,” 2023, arXiv:2311.07919

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    Llama-omni2: Llm-based real-time spoken chatbot with autoregressive streaming speech synthesis,

    Q. Fang, Y . Zhou, S. Guo, S. Zhang, and Y . Feng, “Llama-omni2: Llm-based real-time spoken chatbot with autoregressive streaming speech synthesis,” 2025. [Online]. Available: https://arxiv.org/abs/2505.02625

    work page arXiv 2025

  27. [27]

    Qwen3-Omni Technical Report

    J. Xu, Z. Guo, H. Hu, Y . Chu, X. Wang, J. He, Y . Wang, X. Shi, T. He, X. Zhu, Y . Lv, Y . Wang, D. Guo, H. Wang, L. Ma, P. Zhang, X. Zhang, H. Hao, Z. Guo, B. Yang, B. Zhang, Z. Ma, X. Wei, S. Bai, K. Chen, X. Liu, P. Wang, M. Yang, D. Liu, X. Ren, B. Zheng, R. Men, F. Zhou, B. Yu, J. Yang, L. Yu, J. Zhou, and J. Lin, “Qwen3-Omni Technical Report,” 2025...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  28. [28]

    TTA: Transcribe, Translate and Alignment for Cross-lingual Speech Representation,

    W. Liu, J. Li, Y . Shao, and D. Yu, “TTA: Transcribe, Translate and Alignment for Cross-lingual Speech Representation,” inPro- ceedings of the 2026 IEEE International Conference on Acoustics, Speech and Signal Processing, 2026

    2026

  29. [29]

    OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning,

    Y . Peng, S. Muhammad, Y . Sudo, W. Chen, J. Tian, C.-J. Lin, and S. Watanabe, “OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning,” inProceedings of the Interspeech 2025, 2025

    2025

  30. [30]

    End-to-End Speech Recognition Contextualization with Large Language Models,

    E. Lakomkin, C. Wu, Y . Fathullah, O. Kalinli, M. L. Seltzer, and C. Fuegen, “End-to-End Speech Recognition Contextualization with Large Language Models,” inProceedings of the 2024 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing, 2024, pp. 12 406–12 410

    2024

  31. [31]

    An Embar- rassingly Simple Approach for LLM with Strong ASR Capacity,

    Z. Ma, G. Yang, Y . Yang, Z. Gao, J. Wanget al., “An Embar- rassingly Simple Approach for LLM with Strong ASR Capacity,” arXiv:2402.08846, 2024

    work page arXiv 2024

  32. [32]

    Reproducing Whisper-Style Training Using An Open-Source Toolkit And Publicly Available Data,

    Y . Peng, J. Tian, B. Yan, D. Berrebbi, X. Chang, X. Li, J. Shi, S. Arora, W. Chen, R. Sharma, W. Zhang, Y . Sudo, M. Shakeel, J.- W. Jung, S. Maiti, and S. Watanabe, “Reproducing Whisper-Style Training Using An Open-Source Toolkit And Publicly Available Data,” inProceedings of the 2023 IEEE Automatic Speech Recog- nition and Understanding Workshop, 2023

    2023

  33. [33]

    Lib- rispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An ASR corpus based on public domain audio books,” inProceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing, 2015, pp. 5206–5210

    2015

  34. [34]

    ReazonSpeech: A free and massive corpus for Japanese ASR,

    Y . Yin, D. Mori, and S. Fujimoto, “ReazonSpeech: A free and massive corpus for Japanese ASR,” inProceedings of the 29th An- nual Meeting of the Association for Natural Language Processing (Domestic Conference), 2023, pp. 1134–1139

    2023

  35. [35]

    MLS: A Large-Scale Multilingual Dataset for Speech Research,

    V . Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “MLS: A Large-Scale Multilingual Dataset for Speech Research,” inProceedings of the Interspeech 2020, 2020

    2020

  36. [36]

    Wenetspeech: A 10000+ hours multi- domain mandarin corpus for speech recognition,

    B. Zhang, H. Lv, P. Guo, Q. Shao, C. Yang, L. Xie, X. Xu, H. Bu, X. Chen, C. Zenget al., “Wenetspeech: A 10000+ hours multi- domain mandarin corpus for speech recognition,” inProceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, 2022, pp. 6182–6186

    2022

  37. [37]

    Common V oice: A Massively-Multilingual Speech Corpus,

    R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common V oice: A Massively-Multilingual Speech Corpus,” in Proceedings of the 12th Conference on Language Resources and Evaluation, 2020, pp. 4211–4215

    2020

  38. [38]

    Qwen2.5 Technical Report

    Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2.5 Technical Re- port,” 2025, arXiv:2412.15115

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  39. [39]

    V oxPopuli: A Large- Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation,

    C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux, “V oxPopuli: A Large- Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confe...

    2021

  40. [40]

    FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech,

    A. Conneau, M. Ma, S. Khanuja, Y . Zhang, V . Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech,” inProceed- ings of the 2023 IEEE Spoken Language Technology Workshop, 2023, pp. 798–805

    2023

  41. [41]

    AISHELL-1: An open-source Mandarin speech corpus and a speech recognition baseline,

    H. Bu, J. Du, X. Na, B. Wu, and H. Zheng, “AISHELL-1: An open-source Mandarin speech corpus and a speech recognition baseline,” inProceedings of the 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment, 2017

    2017

  42. [42]

    JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis

    R. Sonobe, S. Takamichi, and H. Saruwatari, “JSUT corpus: free large-scale japanese speech corpus for end-to-end speech synthe- sis,”arXiv:1711.00354, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  43. [43]

    CoV oST 2: A Massively Multilin- gual Speech-to-Text Translation Corpus,

    C. Wang, A. Wu, and J. Pino, “CoV oST 2: A Massively Multilin- gual Speech-to-Text Translation Corpus,” inProceedings of the Interspeech 2021, 2021

    2021

  44. [44]

    Towards Speech Di- alogue Translation Mediating Speakers of Different Languages,

    S. Shimizu, C. Chu, S. Li, and S. Kurohashi, “Towards Speech Di- alogue Translation Mediating Speakers of Different Languages,” inFindings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 1122–1134

    2023

  45. [45]

    SLURP: A Spoken Language Understanding Resource Package,

    E. Bastianelli, A. Vanzo, P. Swietojanski, and V . Rieser, “SLURP: A Spoken Language Understanding Resource Package,” inPro- ceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020, pp. 7252–7262

    2020

  46. [46]

    Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond,

    B. Lee, I. Calapodescu, M. Gaido, M. Negri, and L. Besacier, “Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond,” inProceedings of the Interspeech 2024, 2024

    2024

  47. [47]

    MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations,

    S. Poria, D. Hazarika, N. Majumder, G. Naik, E. Cambria, and R. Mihalcea, “MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations,” inProceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, 2019, pp. 527–536

    2019

  48. [48]

    OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer,

    Y . Peng, J. Tian, W. Chen, S. Arora, B. Yan, Y . Sudo, M. Sha- keel, K. Choi, J. Shi, X. Chang, J. weon Jung, and S. Watan- abe, “OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer,” inProceedings of the Inter- speech 2024, 2024

    2024