A representation-learning pretraining step, followed by brief CTC fine-tuning, yields lightweight Conformer ASR models with lower WER than from-scratch training in the paper's reported setup.
An Effective Training Framework for Light-Weight Automatic Speech Recognition Models
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Recent advancement in deep learning encouraged developing large automatic speech recognition (ASR) models that achieve promising results while ignoring computational and memory constraints. However, deploying such models on low resource devices is impractical despite of their favorable performance. Existing approaches (pruning, distillation, layer skip etc.) transform the large models into smaller ones at the cost of significant performance degradation or require prolonged training of smaller models for better performance. To address these issues, we introduce an efficacious two-step representation learning based approach capable of producing several small sized models from a single large model ensuring considerably better performance in limited number of epochs. Comprehensive experimentation on ASR benchmarks reveals the efficacy of our approach, achieving three-fold training speed-up and up to 12.54% word error rate improvement.
fields
cs.CV 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
An Effective Training Framework for Light-Weight Automatic Speech Recognition Models
A representation-learning pretraining step, followed by brief CTC fine-tuning, yields lightweight Conformer ASR models with lower WER than from-scratch training in the paper's reported setup.