Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLM

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hard-Synth claims that cloning the speech an ASR model misrecognizes, wrapped around LLM-rewritten text, cuts WER on LibriSpeech dev/test-other by 6.5% and 4.4% with only 16.15 hours of synthetic audio.

desk verdict A plausible, well-analyzed ASR augmentation pipeline whose headline WER gains are real in direction but not statistically pinned down; worth reviewing, but the authors should add error bars and code. read the letter →

arxiv 2411.13159 v1 pith:H2DFXHE6 submitted 2024-11-20 cs.CL cs.SDeess.AS

Jiawei Yu , Yuang Li , Xiaosong Qiao , Huan Zhao , Xiaofeng Zhao , Wei Tang , Min Zhang , Hao Yang
show 1 more author
Jinsong Su
This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords automaticspeechrecognitiondataaugmentationzero-shottext-to-speechhardexampleselectionLLMtextrewritingsyntheticfilteringASRbiasreductionLibri
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hard-Synth is a data-augmentation recipe that tries to improve speech recognition by generating synthetic audio that is deliberately hard to recognize. It claims that cloning the acoustic style of utterances a weak ASR model fails on, and pairing that audio with LLM-rewritten transcripts, reduces word error on difficult speech more than random or clean synthetic speech does. On LibriSpeech with a 100-hour training set, the recipe yields relative WER reductions of $6.5\%$ and $4.4\%$ on dev/test-other for the Conformer model, using only 16.15 hours of synthetic audio. It also reports smaller gender and per-speaker WER gaps. The payoff, if true, is a low-cost way to improve ASR on noisy, accented, or uncommon speech without recording new real audio.

What carries the argument

The load-bearing device is the hard-prompt selector: a weak ASR model trained for about 15 epochs transcribes the training set with greedy CTC decoding, and utterances are ranked by character error rate (CER). The top-ranked audio clips, restricted to longer than 3 seconds, become prompts for VoiceCraft, an autoregressive zero-shot TTS model that re-synthesizes the prompt's accent, noise, timbre, and speaking speed around new text. LLM rewriting provides the new text, paraphrasing and restructuring the original transcripts so the synthetic set adds vocabulary and syntactic variety. A strong ASR model then filters the generated clips by CER against the intended text, keeping only those below a $10\%$ threshold. The choice of CER over WER is deliberate: a single wrong character can flip an entire word, so character-level error emphasizes acoustic difficulty rather than lexical content.

What would settle it

A decisive check would compare Hard-Synth against the same pipeline with 'cleaned' prompts, where the TTS output is forced to keep the rewritten text but drops the prompt's noise and prosody; if the test-other gain persists, the hard-prompt acoustics are not the active ingredient. The paper's own Table IV already shows random prompts lose to hard prompts by $0.28\%$ WER on test-other, so the sharper test is to manipulate prompt fidelity directly and see whether per-utterance WER gain tracks spectral similarity to the prompt.

Watch

Extended reading notes

Core claim

The paper's central claim is that an ASR model can be improved on hard speech by training on synthetic utterances that imitate the acoustic properties of the model's own failure cases. A weak recognizer is used to find the hardest real utterances, a zero-shot TTS model clones their speaking style, and an LLM supplies rewritten transcript text so the synthetic audio is not just acoustic repetition. The strongest supporting result is on LibriSpeech: with 100 hours of real training audio and 16.15 hours of synthetic audio, the Conformer's WER drops from $17.47\%$ to $16.34\%$ on dev-other and from $17.43\%$ to $16.67\%$ on test-other, and the Transformer improves across all subsets. The paper further claims that the required synthetic data is only about $16\%$ of the real data, that autoregressive TTS reproduces prompt acoustics better than a non-autoregressive flow-matching model, and that the method reduces gender and speaker-level WER disparities.

Load-bearing premise

The load-bearing premise is that high-CER utterances share a transferable acoustic difficulty, and that the TTS clones that exact difficulty instead of cleaning it up, so the synthetic clips teach the model about genuinely hard speech rather than about artifacts of the cloning process.

Editorial extensions

If this is right

  • Hard-Synth improves both Conformer and Transformer ASR models on every LibriSpeech subset, with the largest relative gains on the harder 'other' subsets: $6.5\%$ dev-other and $4.4\%$ test-other for Conformer.
  • Because the gains come from only 16.15 hours of synthetic audio, the recipe is data-efficient and adds negligible training-time cost; increasing synthetic volume to 30.87 hours actually hurts, consistent with a synthetic-to-real distribution gap.
  • Hard audio prompt selection matters: randomly chosen prompts give less improvement on test-other ($17.02\%$ versus $16.74\%$ WER), so the CER ranking, not just extra audio, is doing the work.
  • LLM rewriting contributes beyond acoustic cloning; adding rewritten text pushes the average WER from 11.61% to 11.43% for the Conformer, indicating both acoustic and semantic diversity help.
  • Hard-Synth reduces measured ASR bias, shrinking the male–female WER gap from $2.06\%$ to $1.66\%$ on test-other and lowering variance in WER across speakers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is right, the same two-stage loop can be re-run iteratively: retrain on the augmented set, re-rank the new model's hard utterances, and synthesize another round of clones, potentially yielding compounding gains on the long tail.
  • The results imply that human-perceived audio quality is the wrong yardstick for TTS in ASR augmentation; the useful property is prompt fidelity including noise, prosody, and speaking speed, which suggests a testable metric based on spectral similarity to the prompt rather than MOS.
  • Because high-CER utterances are often drawn from minority speaker groups, the ranking step acts as an unsupervised bias miner; a direct extension would seed the prompt pool with under-represented accents or noise conditions and track per-group WER variance.
  • The synthetic-to-real gap limiting larger augmentation volumes might be attacked by training the TTS model itself on hard prompts (the paper names this as future work), which would make the positive results at 16% real-data equivalent a lower bound rather than an upper bound.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Hard-Synth augments a low-resource LibriSpeech ASR training set (clean-100) by (1) rewriting the training transcripts with LLaMA3.1-8B, (2) selecting "hard" audio prompts as the top-CER utterances of a weakly trained ASR model, (3) synthesizing speech with the zero-shot TTS model VoiceCraft, and (4) filtering the synthetic audio by CER with a converged ASR model. The synthetic 16.15h set is mixed with real data to train ESPnet Transformer and Conformer models. The paper reports relative WER reductions of about 6.5%/4.4% on LibriSpeech dev/test-other for the Conformer, along with data-efficiency and bias-reduction claims, plus ablations over synthetic duration, CER threshold, prompt type, and TTS model choice.

Significance. If the reported effects are reliable, Hard-Synth is a practically attractive augmentation recipe: it uses no extra text corpora, introduces a simple hard-prompt criterion, and provides a useful AR-vs-NAR TTS comparison. The paper explicitly identifies an important limitation in Sections V-B and V-C, namely that high-quality TTS models tend to produce clean speech, and it evaluates with standard ESPnet recipes. The main weakness is evidentiary: the headline gains come from single-seed runs on one corpus, with no variance or significance reporting, and the advantages over simpler baselines such as random prompts are small (0.28% WER on test-other). The central claim should therefore be read as conditional until multi-run evidence is supplied.

major comments (3)
  1. [Tables III and IV; Section III-C] The central claim that Hard-Synth "significantly enhances" the Conformer rests on single training runs with no error bars, no seed count, and no significance test. The absolute effects are small: 17.43 to 16.67 WER on test-other for the full method, and only 17.02 to 16.74 WER between the random-prompt and hard-prompt conditions in Table IV. Run-to-run variation for a 34M-parameter Conformer trained over 70 epochs can plausibly exceed these margins. Please report multiple seeds (at least three, preferably five) with mean and standard deviation, or another valid uncertainty estimate; without this, the 6.5%/4.4% headline reduction is not established. The same single-run limitation applies to the bias-reduction claims in Table VII.
  2. [Section IV-B; Table IV] The augmentation hyperparameters T=15.38h and gamma=10% appear to be selected by comparing dev/test WER across conditions, and the weak-model epoch count (15) and minimum prompt duration (3s) are fixed without ablation. If these choices were made on the same dev/test sets that define the headline results, the reported improvements are post-selection estimates and can be inflated. A separate validation split, or an explicit statement that these choices were fixed a priori, is needed to support the claimed gains.
  3. [Section IV-A; Table IV] The hard-vs-random prompt comparison is not matched. Hard prompts are the top-CER utterances with a 3-second duration cutoff, while the "random prompt" condition appears to sample prompts from the full training set without matching duration, speaker, or CER distribution. The 0.28% WER advantage on test-other could therefore reflect prompt-length or speaker differences rather than acoustic hardness. Please match the random-prompt condition on duration and speaker distribution (or at least report the durations and speaker overlap), and preferably add a baseline that up-samples hard real utterances without TTS cloning, to isolate the contribution of the synthetic cloning step.
minor comments (5)
  1. [Section IV-C; Table V] The text says VoiceCraft's average WER is 11.34%, but Table V reports 11.43%; correct this numerical inconsistency, as it changes the stated size of the VoiceCraft/F5-TTS gap.
  2. [Section III-B] The 13% new-vocabulary statistic should define the vocabulary unit (word vs subword) and state how out-of-vocabulary words are computed; the paper should also report how many generated utterances were discarded by the CER filter, since only the final 9,196 utterances are described.
  3. [Section II-B] The claim that CER is preferable to WER for prompt selection because WER is more influenced by text distribution is plausible but not demonstrated; a short quantitative justification or reference would strengthen the argument.
  4. [Table VII and surrounding text] The text does not specify which Hard-Synth variant is evaluated (hard-only or hard+rewrite) or whether the bias metrics are averages over speakers or pooled over utterances; please clarify the configuration and aggregation.
  5. [General] No code, synthetic audio, or generation script is released, which limits reproducibility; consider releasing the prompt lists, the generated audio, and the filtering/generation configuration.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the synthetic-augmentation pipeline is evaluated on held-out LibriSpeech subsets, with no fitted parameter feeding the reported WER.

full rationale

Hard-Synth's derivation chain is self-contained with respect to its target metric. Section II-B selects audio prompts using CER from a weak ASR model on the original training set; Section II-C filters synthetic audio using CER from a converged ASR model; Section II-D rewrites training transcriptions with an LLM. None of these components takes the LibriSpeech dev/test-other WER values reported in Table III as an input, and the final WER is measured on held-out subsets. The synthetic corpus is a fixed augmentation set (16.15h), not an optimization variable fit to the reported WER; the hyperparameters in Table IV are chosen before the final comparison and do not define the target metric in terms of the selection CER. The paper cites external systems (VoiceCraft, F5-TTS, LLaMA, ESPnet, LibriSpeech) rather than relying on load-bearing self-citations. Concerns about single-seed training and hyperparameter selection on the test set are statistical validity issues, not circularity, and no equation or construction in the paper reduces the reported WER to its own inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. The free parameters are hyperparameters chosen by hand, and the axioms are domain assumptions about the benefit of synthetic speech and the faithfulness of prompt cloning. These are reasonable but not proven.

free parameters (4)
  • CER filtering threshold γ = 10%
    Set by hand; ablation shows 5% vs 10% nearly identical (Table IV).
  • Synthetic data duration T = 15.38h
    Best in ablation across 8.62/15.38/30.87h; chosen for main results.
  • Weak ASR training epochs = 15
    Chosen to be an under-trained model; no systematic search.
  • Minimum prompt duration = 3 seconds
    Clips shorter than 3s excluded as prompts, justified by lack of speech info.
assumptions (3)
  • domain assumption Synthetic speech from zero-shot TTS provides useful training signal for ASR
    Central premise of all TTS augmentation; invoked throughout Section I and II.
  • domain assumption CER from a weak ASR model identifies acoustically hard utterances
    Section II-B assumes high CER correlates with unusual prosody/noise, not just content difficulty.
  • domain assumption VoiceCraft preserves the hard acoustic properties of prompts
    Section IV-C and V-B rely on this; the paper shows spectrogram evidence but does not measure acoustic preservation objectively beyond sim-spk and speed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLM." pith.science (2026). https://pith.science/paper/H2DFXHE6

@misc{pith2026241113159,
  author       = {Pith},
  title        = {Pith review of: Hard-Synth: Synthesizing Diverse Hard Samples for ASR using Zero-Shot TTS and LLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2DFXHE6}},
  note         = {Machine review of arXiv:2411.13159}
}
read the original abstract

Text-to-speech (TTS) models have been widely adopted to enhance automatic speech recognition (ASR) systems using text-only corpora, thereby reducing the cost of labeling real speech data. Existing research primarily utilizes additional text data and predefined speech styles supported by TTS models. In this paper, we propose Hard-Synth, a novel ASR data augmentation method that leverages large language models (LLMs) and advanced zero-shot TTS. Our approach employs LLMs to generate diverse in-domain text through rewriting, without relying on additional text data. Rather than using predefined speech styles, we introduce a hard prompt selection method with zero-shot TTS to clone speech styles that the ASR model finds challenging to recognize. Experiments demonstrate that Hard-Synth significantly enhances the Conformer model, achieving relative word error rate (WER) reductions of 6.5\%/4.4\% on LibriSpeech dev/test-other subsets. Additionally, we show that Hard-Synth is data-efficient and capable of reducing bias in ASR.

Figures

Figures reproduced from arXiv: 2411.13159 by the authors.

Figure 1
Figure 1. System diagram of Hard-Synth. (a) Zero-shot TTS is employed [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Statistics of the synthetic dataset. B. Synthetic Data Generation We utilize the LibriSpeech-clean-100 [29] subset, consisting of 100 hours of speech data, for ASR training and data synthesis. The full dataset of 960 hours is not used, as we aim to simulate a low-resource scenario where data augmentation is more crucial. We utilize LLaMA3.1-8B [22] for text rewriting and employ an ASR model, trained for 15 epochs on… view at source ↗
Figure 3
Figure 3. Visualizations of spectrograms. VoiceCraft excels in replicating the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 28 canonical work pages

  1. [1]

    Attention-based models for speech recognition,

    Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio, “Attention-based models for speech recognition,” in Proc. of NeurIPS , Dec. 2015, pp. 577–585

    work page 2015

  2. [2]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    Alex Graves, Santiago Fern ´andez, Faustino Gomez, and J ¨urgen Schmid- huber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proc. of ICML , Jun. 2006, pp. 369–376

    work page 2006

  3. [3]

    Sequence transduction with recurrent neural networks,

    Alex Graves, “Sequence transduction with recurrent neural networks,” in Proc. of ICML , Edinburgh, Scotland, Jun. 2012

    work page 2012

  4. [4]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al., “Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,” arXiv preprint arXiv:2403.03100 , 2024

    arXiv 2024

  5. [5]

    Neural codec language models are zero-shot text to speech synthesiz- ers,

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al., “Neural codec language models are zero-shot text to speech synthesiz- ers,” arXiv preprint arXiv:2301.02111 , 2023

    arXiv 2023

  6. [6]

    Human per- ception of audio deepfakes,

    Nicolas M M ¨uller, Karla Pizzi, and Jennifer Williams, “Human per- ception of audio deepfakes,” in Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia , 2022, pp. 85– 91

    work page 2022

  7. [7]

    As good as a coin toss human detection of ai-generated images, videos, audio, and audiovisual stimuli,

    Di Cooke, Abigail Edwards, Sophia Barkoff, and Kathryn Kelly, “As good as a coin toss human detection of ai-generated images, videos, audio, and audiovisual stimuli,” arXiv preprint arXiv:2403.16760, 2024

    arXiv 2024

  8. [8]

    Task arithmetic can mitigate synthetic-to-real gap in automatic speech recognition,

    Hsuan Su, Hua Farn, Fan-Yun Sun, Shang-Tse Chen, and Hung-yi Lee, “Task arithmetic can mitigate synthetic-to-real gap in automatic speech recognition,” in Proc. of EMNLP, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, Eds., 2024

    work page 2024

Show all 38 references
  1. [9]

    Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator,

    Vladimir Bataev, Roman Korostik, Evgeny Shabalin, Vitaly Lavrukhin, and Boris Ginsburg, “Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator,” in Proc. of Interspeech, Naomi Harte, Julie Carson-Berndsen, and Gareth Jones, Eds., 2023

    2023

  2. [10]

    Using synthetic audio to improve the recognition of out-of-vocabulary words in end-to-end asr systems,

    Xianrui Zheng, Yulan Liu, Deniz Gunceler, and Daniel Willett, “Using synthetic audio to improve the recognition of out-of-vocabulary words in end-to-end asr systems,” in Proc. of ICASSP , 2021

    2021

  3. [11]

    Improving code-switching and named entity recognition in asr with speech editing based data augmentation,

    Zheng Liang, Zheshu Song, Ziyang Ma, Chenpeng Du, Kai Yu, and Xie Chen, “Improving code-switching and named entity recognition in asr with speech editing based data augmentation,” arXiv preprint arXiv:2306.08588, 2023

    2023 arXiv

  4. [12]

    SYNT++: utilizing imperfect synthetic data to improve speech recognition,

    Ting-Yao Hu, Mohammadreza Armandpour, Ashish Shrivastava, Jen- Hao Rick Chang, Hema Koppula, and Oncel Tuzel, “SYNT++: utilizing imperfect synthetic data to improve speech recognition,” in Proc. of ICASSP, 2022

    2022

  5. [13]

    V oicecraft: Zero-shot speech editing and text-to-speech in the wild,

    Puyuan Peng, Po-Yao Huang, Abdelrahman Mohamed, and David Harwath, “V oicecraft: Zero-shot speech editing and text-to-speech in the wild,” arXiv, 2024

    2024

  6. [14]

    F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,

    Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen, “F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching,” arXiv preprint arXiv:2410.06885 , 2024

    2024 arXiv

  7. [15]

    Text is all you need: Personalizing ASR models using controllable speech synthesis,

    Karren D. Yang, Ting-Yao Hu, Jen-Hao Rick Chang, Hema Swetha Koppula, and Oncel Tuzel, “Text is all you need: Personalizing ASR models using controllable speech synthesis,” in Proc. of ICASSP, 2023

    2023

  8. [16]

    Using personalized speech synthesis and neural language generator for rapid speaker adaptation,

    Yan Huang, Lei He, Wenning Wei, William Gale, Jinyu Li, and Yifan Gong, “Using personalized speech synthesis and neural language generator for rapid speaker adaptation,” in Proc. of ICASSP , 2020

    2020

  9. [17]

    Rapid RNN-T adaptation using personalized speech synthesis and neural language generator,

    Yan Huang, Jinyu Li, Lei He, Wenning Wei, William Gale, and Yifan Gong, “Rapid RNN-T adaptation using personalized speech synthesis and neural language generator,” in Proc. of Interspeech , Helen Meng, Bo Xu, and Thomas Fang Zheng, Eds., 2020

    2020

  10. [18]

    Enhancing low-resource asr through versatile tts: Bridging the data gap,

    Guanrou Yang, Fan Yu, Ziyang Ma, Zhihao Du, Zhifu Gao, Shiliang Zhang, and Xie Chen, “Enhancing low-resource asr through versatile tts: Bridging the data gap,” arXiv preprint arXiv:2410.16726 , 2024

    2024 arXiv

  11. [19]

    Synthesizing dysarthric speech using multi-speaker tts for dysarthric speech recognition,

    Mohammad Soleymanpour, Michael T. Johnson, Rahim Soleymanpour, and Jeffrey Berry, “Synthesizing dysarthric speech using multi-speaker tts for dysarthric speech recognition,” in Proc. of ICASSP , 2022

    2022

  12. [20]

    GPT-4 technical report,

    OpenAI, “GPT-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  13. [21]

    Glm-130b: An open bilingual pre-trained model,

    Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Peng Zhang, Yuxiao Dong, and Jie Tang, “Glm-130b: An open bilingual pre-trained model,” Proc. of ...

    2023

  14. [22]

    LLaMA: Open and efficient foundation language models,

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie- Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al., “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023

    2023 arXiv

  15. [23]

    Can generative large language models perform asr error correction?,

    Rao Ma, Mengjie Qian, Potsawee Manakul, Mark Gales, and Kate Knill, “Can generative large language models perform asr error correction?,” arXiv preprint arXiv:2307.04172 , 2023

    2023 arXiv

  16. [24]

    Rewritelm: An instruction- tuned large language model for text rewriting,

    Lei Shu, Liangchen Luo, Jayakumar Hoskere, Yun Zhu, Yinxiao Liu, Simon Tong, Jindong Chen, and Lei Meng, “Rewritelm: An instruction- tuned large language model for text rewriting,” Proc. AAAI, vol. 38, no. 17, pp. 18970–18980, Mar. 2024

    2024

  17. [25]

    69–80, Springer Nature Switzerland, 2023

    Yaxin Fan, Feng Jiang, Peifeng Li, and Haizhou Li, GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning , p. 69–80, Springer Nature Switzerland, 2023

    2023

  18. [26]

    Leveraging llm for augmenting textual data in code-switching asr: Arabic as an example,

    Sadeen Alharbi, Reem BinMuqbil, Ahmed Ali, Raghad AlOraini, Saiful Bari, Areeb Alowisheq, and Yaser Alonaizan, “Leveraging llm for augmenting textual data in code-switching asr: Arabic as an example,” in Proc. SynData4GenAI, 2024

    2024

  19. [27]

    Generating data with text-to-speech and large-language models for conversational speech recognition,

    Samuele Cornell, Jordan Darefsky, Zhiyao Duan, and Shinji Watanabe, “Generating data with text-to-speech and large-language models for conversational speech recognition,” CoRR, vol. abs/2408.09215, 2024

    2024 arXiv

  20. [28]

    Corpus synthesis for zero-shot ASR domain adaptation using large language models,

    Hsuan Su, Ting-Yao Hu, Hema Swetha Koppula, Raviteja Vemulapalli, Jen-Hao Rick Chang, Karren D. Yang, Gautam Varma Mantena, and Oncel Tuzel, “Corpus synthesis for zero-shot ASR domain adaptation using large language models,” in Proc. of ICASSP , 2024

    2024

  21. [29]

    Librispeech: an asr corpus based on public domain audio books,

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in Proc. of ICASSP, South Brisbane, Queensland, Australia, Apr. 2015, pp. 5206–5210

    2015

  22. [30]

    Attention is all you need,

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin, “Attention is all you need,” in Proc. NeurIPS, Long Beach, California, USA, Dec. 2017

    2017

  23. [31]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, et al., “Conformer: Convolution-augmented transformer for speech recognition,” Proc. of Interspeech , 2020

    2020

  24. [32]

    ESPnet: End-to-end speech processing toolkit,

    Shinji Watanabe, Takaaki Hori, Shigeki Karita, Tomoki Hayashi, Jiro Nishitoba, Yuya Unno, Nelson Enrique Yalta Soplin, Jahn Heymann, Matthew Wiesner, Nanxin Chen, Adithya Renduchintala, and Tsubasa Ochiai, “ESPnet: End-to-end speech processing toolkit,” in Proc. of Interspeech...

    2018

  25. [33]

    Specaugment: A simple data augmentation method for automatic speech recognition,

    Daniel S Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, and Quoc V Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” in Proc. of Interspeech, Graz, Austria, Sep. 2019, pp. 2613–2617

    2019

  26. [34]

    Hybrid CTC/attention architecture for end-to-end speech recognition,

    Shinji Watanabe, Takaaki Hori, Suyoun Kim, John R Hershey, and Tomoki Hayashi, “Hybrid CTC/attention architecture for end-to-end speech recognition,” IEEE Journal of Selected Topics in Signal Pro- cessing, vol. 11, no. 8, pp. 1240–1253, 2017

    2017

  27. [35]

    Mosnet: Deep learning based objective assessment for voice conversion,

    Chen-Chou Lo, Szu-Wei Fu, Wen-Chin Huang, Xin Wang, Junichi Yamagishi, Yu Tsao, and Hsin-Min Wang, “Mosnet: Deep learning based objective assessment for voice conversion,” in Proc. of Interspeech , 2019

    2019

  28. [36]

    WavLM: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, et al., “WavLM: Large-scale self-supervised pre- training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , 2022

    2022

  29. [37]

    Quantifying bias in automatic speech recognition,

    Siyuan Feng, Olya Kudina, Bence Mark Halpern, and Odette Scharen- borg, “Quantifying bias in automatic speech recognition,” arXiv preprint arXiv:2103.15122, 2021

    2021 arXiv

  30. [38]

    Robust speech recognition via large- scale weak supervision,

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large- scale weak supervision,” arXiv preprint arXiv:2212.04356 , 2022

    2022 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.