Syllable-based rhythm and voice conversion of dysarthric speech, trained without labels, improves LF-MMI ASR word error rates on Torgo, especially for severe speakers, but not for fine-tuned Whisper.
Unsupervised Rhythm and Voice Conversion to Improve ASR on Dysarthric Speech
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Automatic speech recognition (ASR) systems struggle with dysarthric speech due to high inter-speaker variability and slow speaking rates. To address this, we explore dysarthric-to-healthy speech conversion for improved ASR performance. Our approach extends the Rhythm and Voice (RnV) conversion framework by introducing a syllable-based rhythm modeling method suited for dysarthric speech. We assess its impact on ASR by training LF-MMI models and fine-tuning Whisper on converted speech. Experiments on the Torgo corpus reveal that LF-MMI achieves significant word error rate reductions, especially for more severe cases of dysarthria, while fine-tuning Whisper on converted data has minimal effect on its performance. These results highlight the potential of unsupervised rhythm and voice conversion for dysarthric ASR. Code available at: https://github.com/idiap/RnV
citation-role summary
background 1
citation-polarity summary
fields
eess.AS 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Unsupervised Rhythm and Voice Conversion to Improve ASR on Dysarthric Speech
Syllable-based rhythm and voice conversion of dysarthric speech, trained without labels, improves LF-MMI ASR word error rates on Torgo, especially for severe speakers, but not for fine-tuned Whisper.