pith. sign in

arxiv: 2406.19674 · v1 · pith:RZFDABDCnew · submitted 2024-06-28 · 💻 cs.CL · cs.LG· cs.SD· eess.AS

Less is More: Accurate Speech Recognition & Translation without Web-Scale Data

Krishna C. Puvvada , Piotr \.Zelasko , He Huang , Oleksii Hrinchuk , Nithin Rao Koluguri , Kunal Dhawan , Somshubra Majumdar , Elena Rastorgueva
show 4 more authors
Zhehuai Chen Vitaly Lavrukhin Jagadeesh Balam Boris Ginsburg
This is my paper
classification 💻 cs.CL cs.LGcs.SDeess.AS
keywords dataspeechtranslationmodeltrainingdynamiclessmodels
0
0 comments X
read the original abstract

Recent advances in speech recognition and translation rely on hundreds of thousands of hours of Internet speech data. We argue that state-of-the art accuracy can be reached without relying on web-scale data. Canary - multilingual ASR and speech translation model, outperforms current state-of-the-art models - Whisper, OWSM, and Seamless-M4T on English, French, Spanish, and German languages, while being trained on an order of magnitude less data than these models. Three key factors enables such data-efficient model: (1) a FastConformer-based attention encoder-decoder architecture (2) training on synthetic data generated with machine translation and (3) advanced training techniques: data-balancing, dynamic data blending, dynamic bucketing and noise-robust fine-tuning. The model, weights, and training code will be open-sourced.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CTC-Seeded Token Edit Refinement for Non-Autoregressive Speech Recognition

    eess.AS 2026-06 unverdicted novelty 6.0

    CTC-seeded variable-length edit refinement with a diffusion-based Edit Flow decoder achieves WER reductions in non-autoregressive ASR using only two inference steps plus classifier-free guidance.

  2. BlasBench: An Open Benchmark for Irish Speech Recognition

    cs.CL 2026-04 conditional novelty 6.0

    BlasBench supplies an Irish-aware normalizer and scoring harness that enables reproducible ASR comparisons and exposes a 33-43 point generalization gap for fine-tuned models versus 7-10 points for massively multilingual ones.

  3. Frame-Aligned Fusion of Canary and WavLM for Non-Intrusive Intelligibility Prediction of Hearing-Aid-Processed Speech

    eess.AS 2026-05 unverdicted novelty 4.0

    Frame-aligned fusion of Canary and WavLM encoders, with WavLM temporally prepared via learnable strided convolution, outperforms other fusion strategies and reaches Eval RMSE 24.96 and Corr 0.796 on non-intrusive inte...