Cascaded systems remain the most reliable for speech translation overall, but recent SpeechLLMs match or outperform them in many conditions while standalone speech models lag.
Cs-dialogue: A 104-hour dataset of spontaneous mandarin-english code-switching dialogues for speech recognition.arXiv preprint arXiv:2502.18913
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
abstract
Code-switching (CS), the alternation between two or more languages within a single conversation, presents significant challenges for automatic speech recognition (ASR) systems. Existing Mandarin-English code-switching datasets often suffer from limitations in size, spontaneity, and the lack of full-length dialogue recordings with transcriptions, hindering the development of robust ASR models for real-world conversational scenarios. This paper introduces CS-Dialogue, a novel large-scale Mandarin-English code-switching speech dataset comprising 104 hours of spontaneous conversations from 200 speakers. Unlike previous datasets, CS-Dialogue provides full-length dialogue recordings with complete transcriptions, capturing naturalistic code-switching patterns in continuous speech. We describe the data collection and annotation processes, present detailed statistics of the dataset, and establish benchmark ASR performance using state-of-the-art models. Our experiments, using Transformer, Conformer, and Branchformer, demonstrate the challenges of code-switching ASR, and show that existing pre-trained models such as Whisper still have the space to improve. The CS-Dialogue dataset will be made freely available for all academic purposes.
citation-role summary
dataset 1
citation-polarity summary
roles
dataset 1polarities
background 1representative citing papers
Agentic ASR adds closed-loop semantic correction to ASR and introduces S²ER, an LLM judge for meaning-level errors, showing larger gains on semantic than token metrics across multilingual benchmarks.
A multi-stage training method for LLM-based ASR uses new entropy allocation metrics to achieve competitive benchmark performance with 2.3B parameters while mitigating hallucinations via better encoder-LLM decoupling.
Merged bilingual CS-ASR models show only modest generalization to unseen language pairs, indicating limited transfer of code-switching capabilities.
SURE is a new standardized framework for evaluating and training speech foundation models and Speech LLMs to improve comparability and reproducibility under realistic conditions.
A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-level RAG for hotword customization.
DPO on three Audio LLMs using 100K preference pairs yields up to 89.6% in-distribution and 20.0% out-of-distribution MER reduction for code-switching transcription.
citing papers explorer
-
Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs
Cascaded systems remain the most reliable for speech translation overall, but recent SpeechLLMs match or outperform them in many conditions while standalone speech models lag.
-
Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation
Agentic ASR adds closed-loop semantic correction to ASR and introduces S²ER, an LLM judge for meaning-level errors, showing larger gains on semantic than token metrics across multilingual benchmarks.
-
Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
A multi-stage training method for LLM-based ASR uses new entropy allocation metrics to achieve competitive benchmark performance with 2.3B parameters while mitigating hallucinations via better encoder-LLM decoupling.
-
Towards Truly Multilingual ASR: Generalizing Code-Switching ASR to Unseen Language Pairs
Merged bilingual CS-ASR models show only modest generalization to unseen language pairs, indicating limited transfer of code-switching capabilities.
-
A Unified and Reproducible Experimentation Framework for Speech Understanding
SURE is a new standardized framework for evaluating and training speech foundation models and Speech LLMs to improve comparability and reproducibility under realistic conditions.
-
NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-level RAG for hotword customization.
-
Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
DPO on three Audio LLMs using 100K preference pairs yields up to 89.6% in-distribution and 20.0% out-of-distribution MER reduction for code-switching transcription.