A multi-stage training method for LLM-based ASR uses new entropy allocation metrics to achieve competitive benchmark performance with 2.3B parameters while mitigating hallucinations via better encoder-LLM decoupling.
Cr-ctc: Consistency regulariza- tion on ctc for improved speech recognition
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
Connectionist Temporal Classification (CTC) is a widely used method for automatic speech recognition (ASR), renowned for its simplicity and computational efficiency. However, it often falls short in recognition performance. In this work, we propose the Consistency-Regularized CTC (CR-CTC), which enforces consistency between two CTC distributions obtained from different augmented views of the input speech mel-spectrogram. We provide in-depth insights into its essential behaviors from three perspectives: 1) it conducts self-distillation between random pairs of sub-models that process different augmented views; 2) it learns contextual representation through masked prediction for positions within time-masked regions, especially when we increase the amount of time masking; 3) it suppresses the extremely peaky CTC distributions, thereby reducing overfitting and improving the generalization ability. Extensive experiments on LibriSpeech, Aishell-1, and GigaSpeech datasets demonstrate the effectiveness of our CR-CTC. It significantly improves the CTC performance, achieving state-of-the-art results comparable to those attained by transducer or systems combining CTC and attention-based encoder-decoder (CTC/AED). We release our code at https://github.com/k2-fsa/icefall.
citation-role summary
method 1
citation-polarity summary
years
2026 4roles
method 1polarities
use method 1representative citing papers
Trajectory-aware local temporal modeling before spatial aggregation plus CTC viseme supervision and EMA consistency lifts DVS-Lip word accuracy to 77.49%.
A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-level RAG for hotword customization.
MedASR is an open-source 105M-parameter ASR model achieving 58% relative WER reduction versus Whisper Large-v3 on medical dictation.
citing papers explorer
-
Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
A multi-stage training method for LLM-based ASR uses new entropy allocation metrics to achieve competitive benchmark performance with 2.3B parameters while mitigating hallucinations via better encoder-LLM decoupling.
-
TVTA: Trajectory-Aware Viseme-Guided Temporal Aggregation for Event-Based Lip Reading
Trajectory-aware local temporal modeling before spatial aggregation plus CTC viseme supervision and EMA consistency lifts DVS-Lip word accuracy to 77.49%.
-
NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
A 2.3B-parameter LLM-based ASR system achieves competitive recognition accuracy and reduced hallucination through a multi-stage training paradigm with asynchronous encoder updates, ASR-specialized RL, and phoneme-level RAG for hotword customization.
-
MedASR: An Open-Source Model for High-Accuracy Medical Dictation
MedASR is an open-source 105M-parameter ASR model achieving 58% relative WER reduction versus Whisper Large-v3 on medical dictation.