Training an ASR language model with a mix of unidirectional, bidirectional masked, and corrupted-context objectives yields lower word error rates across shallow fusion and n-best rescoring than unidirectional training alone.
Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systems
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Large-scale language models (LLMs) such as GPT-2, BERT and RoBERTa have been successfully applied to ASR N-best rescoring. However, whether or how they can benefit competitive, near state-of-the-art ASR systems remains unexplored. In this study, we incorporate LLM rescoring into one of the most competitive ASR baselines: the Conformer-Transducer model. We demonstrate that consistent improvement is achieved by the LLM's bidirectionality, pretraining, in-domain finetuning and context augmentation. Furthermore, our lexical analysis sheds light on how each of these components may be contributing to the ASR performance.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
MTLM: Incorporating Bidirectional Text Information to Enhance Language Model Training in Speech Recognition Systems
Training an ASR language model with a mix of unidirectional, bidirectional masked, and corrupted-context objectives yields lower word error rates across shallow fusion and n-best rescoring than unidirectional training alone.