Reinforcement learning with word-error and meaning-preservation rewards adapts an LLM-based speech recognizer to disordered speech better than supervised fine-tuning in this study.
Clinical BERTScore: An Improved Measure of Automatic Speech Recognition Performance in Clinical Settings
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Automatic Speech Recognition (ASR) in medical contexts has the potential to save time, cut costs, increase report accuracy, and reduce physician burnout. However, the healthcare industry has been slower to adopt this technology, in part due to the importance of avoiding medically-relevant transcription mistakes. In this work, we present the Clinical BERTScore (CBERTScore), an ASR metric that penalizes clinically-relevant mistakes more than others. We demonstrate that this metric more closely aligns with clinician preferences on medical sentences as compared to other metrics (WER, BLUE, METEOR, etc), sometimes by wide margins. We collect a benchmark of 18 clinician preferences on 149 realistic medical sentences called the Clinician Transcript Preference benchmark (CTP) and make it publicly available for the community to further develop clinically-aware ASR metrics. To our knowledge, this is the first public dataset of its kind. We demonstrate that CBERTScore more closely matches what clinicians prefer.
citation-role summary
background 1
citation-polarity summary
fields
eess.AS 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Speech Recognition With LLMs Adapted to Disordered Speech Using Reinforcement Learning
Reinforcement learning with word-error and meaning-preservation rewards adapts an LLM-based speech recognizer to disordered speech better than supervised fine-tuning in this study.