Pith. sign in

REVIEW 3 major objections 6 minor 38 references

A Non-autoregressive Model for Joint STT and TTS

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A jointly trained non-autoregressive model performs speech-to-text and text-to-speech, and with unpaired data plus iterative refinement it reaches parity with or beats the single-task baselines.

desk verdict A genuinely novel NAR joint STT/TTS architecture with a clean ablation, but the headline parity claim is confounded by unequal training data between joint and baseline models. read the letter →

arxiv 2501.09104 v2 pith:UJCMPK5A submitted 2025-01-15 cs.SD cs.AIeess.AS

Vishal Sunder , Brian Kingsbury , George Saon , Samuel Thomas , Slava Shechtman , Hagai Aronowitz , Eric Fosler-Lussier , Luis Lastras This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords non-autoregressivespeechrecognitiontext-to-speechmultimodalspeech-textmodeljointSTTandTTSiterativerefinementCTCalignmentself-supervisedtasksmask-predict
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether one model can perform both automatic speech recognition (speech-to-text, STT) and speech synthesis (text-to-speech, TTS) without generating output one token at a time. It claims the answer is yes: a fully non-autoregressive multimodal model, trained jointly on six tasks, can do both. The load-bearing additions are training on unpaired speech and text, and an iterative refinement pass at inference that feeds a partially masked prediction back into the model. With those, the joint model beats its STT-only baseline and roughly matches its TTS-only baseline on standard corpora. The paper reports that STT and TTS are not naturally complementary, which is why the extra tasks and refinement matter.

What carries the argument

The central object is the multimodal encoder that operates on the elementwise sum of two masked input streams: log-mel filterbank features for speech and upsampled CTC alignment embeddings for text. Identical-length, same-size representations are achieved by linear transforms and layer normalization. The model uses a CTC alignment (a frame-by-frame character labeling with blank tokens, produced from text by a duration model) as the text representation for TTS, predicted at inference by a duration predictor, and it uses two masking schedules for speech and one for text to enable self-supervised and mask-predict training. Iterative refinement is the other key mechanism: STT masks low-confidence characters and repeats, TTS progressively unmasks time and frequency bands, and the partially masked output is fed back as input for the next pass.

What would settle it

Train the same model with concatenation-plus-projection instead of elementwise addition for the two input streams, keeping all other hyperparameters fixed; if performance on either STT or TTS does not change, the additive fusion is doing the work, but if it improves noticeably, the addition is losing information and the joint-model claim rests on a fusion that can be bettered.

Watch

Extended reading notes

Core claim

The central claim is that a joint STT and TTS model can be trained in a fully non-autoregressive (NAR) manner, and that it achieves competitive performance on both tasks. The model is a multimodal encoder that takes as input the sum of log-mel speech features and upsampled CTC alignment embeddings derived from text; the same shared encoder feeds a CTC head for text output and an L1 regression head for speech output. It is trained on six tasks: STT, TTS, masked text reconstruction (T2T), masked speech reconstruction (S2S), speech-plus-partially-masked-text to text (ST2T), and speech-plus-text with masked speech to speech (ST2S). At inference, iterative refinement masks low-confidence characters for STT or gradually unmasks time/frequency regions for TTS, feeding the partial hypothesis back into the model. On LJSpeech, LibriTTS, and Librispeech, the full model with iterative refinement outperforms the STT-only baseline on word error rate and matches or exceeds the TTS-only baseline on UTMOS, intelligibility, and speaker similarity.

Load-bearing premise

The model assumes that simply adding the upsampled text-alignment embeddings to the log-mel speech features at the encoder input preserves enough modality-specific and cross-modal information for both tasks, a premise the paper does not analyze beyond end-task metrics.

Editorial extensions

If this is right

  • A single model can serve both speech recognition and speech synthesis applications, cutting the need for two separate systems and their separate training pipelines.
  • Non-autoregressive joint modeling avoids the slow, hallucination-prone sequential decoding of autoregressive models, making on-device or streaming use more practical.
  • Unpaired speech and text can be exploited during training, so the joint model can improve using raw text or audio without transcripts.
  • The iterative refinement procedure is task-agnostic in spirit: any output modality with a confidence signal (or a masking schedule) can be refined by feeding partial predictions back into the same encoder.
  • The finding that the two tasks are not complementary suggests that joint models need auxiliary self-supervised tasks to recover performance, which guides how future multimodal speech-text models should be trained.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the additive fusion is truly sufficient, a natural next step is scaling this architecture to larger paired and unpaired corpora; the current results use at most 50–100 hours of paired data, and scaling may shift the parity point.
  • The same mask-predict machinery could be extended to speech-to-speech translation or spoken language understanding by adding task-specific heads, since the encoder already operates on both modalities jointly.
  • The iterative refinement schedule for TTS (gradual unmasking in time and frequency) resembles diffusion-style generation; comparing this refinement to a diffusion head could reveal whether the gains come from the refinement loop or from the multimodal conditioning.
  • The paper's observation that STT and TTS are not complementary suggests a testable hypothesis: a deliberately complementary auxiliary task (e.g., predicting prosody or emphasis) might help TTS more than generic masked reconstruction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents a fully non-autoregressive (NAR) multimodal model for joint speech-to-text (STT) and text-to-speech (TTS). The model uses a shared conformer encoder that takes additive input from log-mel filterbank features and upsampled CTC alignments produced by a duration model; STT is trained with CTC loss and TTS with L1 loss on predicted log-mel features. In addition to the paired STT/TTS losses, the authors introduce self-supervised T2T and S2S losses on unpaired text/speech and two multimodal losses (ST2T, ST2S) that support an iterative mask-predict refinement at inference. Experiments on LJSpeech, LibriTTS, and Librispeech compare joint models with STT-only and TTS-only baselines; the final joint model with iterative refinement improves WER over the STT baseline and reaches UTMOS comparable to or better than the TTS baseline, with human MOS reported on a 40-sample LibriTTS subset. The central claim is that unpaired data and iterative refinement allow a joint NAR model to match or exceed unimodal baselines.

Significance. The contribution is timely and, if validated, useful: a single NAR model that performs both STT and TTS with weight sharing could offer faster, non-hallucinating inference relative to autoregressive multimodal systems such as SpeechT5. The paper's design is clearly specified, and the use of CTC alignments as the TTS input representation plus iterative refinement for both modalities is a novel combination. The authors are honest about task non-complementarity and about the low absolute MOS. However, the empirical validation as presented does not isolate the joint-NAR contribution from the effect of extra unpaired data and auxiliary losses, and no uncertainty quantification is provided for most metrics; these gaps must be addressed before the parity/outperform claim is established.

major comments (3)
  1. [Section III-C, Tables I and II] The comparison between the joint models and the unimodal baselines is confounded. Rows (4)-(6) and the Librispeech joint models are trained with 200-300 hours of unpaired speech/text and four additional auxiliary losses, while the STT-only and TTS-only baselines are trained only on the paired data. Table I row (3), which trains STT+TTS without these additions, already shows a clear TTS degradation (UTMOS 3.32 vs 3.82 on LJSpeech), so the improvement to 4.08 in row (6) could be due to the larger effective training set and auxiliary objectives rather than to the joint NAR architecture. To support the central claim, the authors should add matched baselines, e.g., a CTC STT model and a FastSpeech-style TTS model trained with the same unpaired data and the same auxiliary T2T/S2S losses, or at least an ablation in which the auxiliary tasks are applied to a single-task model.
  2. [Tables I, II, III and Figure 3] No error bars, confidence intervals, or significance tests are reported for WER, UTMOS, intelligibility, or speaker similarity. Several of the claimed differences are small (e.g., LJSpeech WER 5.07 vs 4.85; LibriTTS UTMOS 3.72 vs 3.82; Librispeech UTMOS 3.66 vs 3.68), and Table III shows overlapping intervals for MOS (2.84±0.07 vs 2.96±0.07). Without multiple training runs or an appropriate significance test, the 'parity or better' conclusion is not statistically supported; please report at least seed-based error bars or bootstrap confidence intervals on the test sets.
  3. [Section II-C and Figure 3] The main results in Tables I and II do not state the number of refinement iterations K used, and no test-set results for selecting K are shown. Because iterative refinement is one of the two ingredients credited for the final gains, the paper should specify K for each reported configuration and verify that the chosen K transfers from development to test. Relatedly, the TTS inference procedure is underspecified: when the speech input is 'fully masked' (pX=1.0), the length of the zero-vector sequence at test time must be determined by the duration model, but this is not stated explicitly in Section II-B.
minor comments (6)
  1. [Section II-A] The notation 'R is used to represent a sequence of repetitions' should be 'a sequence of repeat counts'; the same symbol is not formally defined as a tensor or list.
  2. [Section II-A] The masking example mapping 'C <mask>T' to ' CC<mask><mask>T ' is hard to follow; please spell out the blank/repetition bookkeeping step by step.
  3. [Section II-B] The T2T loss is written as CTC-Loss(softmax(OT2T), YT2T); please clarify the sequence lengths of OT2T and the target YT2T when the input is masked text with blanks.
  4. [Section III-C, Table III discussion] The statement that the added tasks 'improve the quality of the audio to some extent' is not supported by the overlapping MOS intervals; please soften the claim or provide a significance test.
  5. [Figure 1] The dashed arrows for iterative refinement are not defined in the caption; add a sentence explaining that the output logits or log-mel features are masked and fed back to the encoder input.
  6. [Abstract and Section I] The acronym 'STT' is used for automatic speech recognition, but it is nonstandard; define 'speech-to-text (STT)' at first use and consider using 'ASR' where appropriate.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; the central claims rest on external benchmarks and only minor non-load-bearing self-citations appear.

full rationale

This is an empirical systems paper rather than a derivation, and no load-bearing step reduces to its own inputs by construction. The main claims are evaluated on external corpora (LJSpeech, LibriTTS, Librispeech) with external metrics (WER, UTMOS, MOS, Whisper-based intelligibility, speaker similarity) and an external vocoder, so the reported numbers are not defined in terms of the model's own outputs or fitted parameters. The pseudo-CTC alignments for unpaired text are produced by a separately trained duration model and are used for an auxiliary masked-language-modeling objective, not for the reported STT/TTS evaluation. The iterative refinement procedure feeds the model's own partial hypotheses back as input, but that is an explicitly stated decoding strategy, not a hidden reuse of the target labels. Self-citations by IBM co-authors appear in related work and limitations ([12], [13], [36]) but are contextual and do not carry the central claim. The unmatched-training-data comparison between the joint model and unimodal baselines is an experimental confound worth noting, but it is a benchmark-design limitation rather than a circularity, since no equation or fitted parameter makes the claimed gains true by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central results rest on a handful of architectural and data assumptions. The model adds speech and text embeddings together without a learned fusion mechanism, a choice that is not justified beyond empirical results. It also assumes that CTC alignments, including pseudo-alignments for unpaired text, are adequate TTS conditioning. These assumptions are plausible and partially validated by the reported metrics, but they are not independently tested.

free parameters (4)
  • Duration loss weight alpha
    Used in LTTS = L1 + alpha*Ldur and LST2S = L1 + alpha*Ldur, but the numerical value is never specified in the paper. The central TTS performance depends on this weight.
  • Number of iterative refinement steps K = 4
    K is set to 4 based on saturation of dev-set performance (Figure 3). This is a hyperparameter chosen on development data.
  • Confidence threshold schedule for STT refinement = 0.99 to 0.9 linearly over K steps
    Used to decide which characters to mask in each refinement pass; chosen by hand without a tuning study.
  • Masking percentages pY and pX for ST2T/ST2S = uniform from {0.1, 0.25, 0.5, 0.75, 0.9}
    The masking ratios are sampled from a fixed set during training; these hyperparameters affect the auxiliary tasks and are not tuned per dataset.
assumptions (5)
  • domain assumption CTC alignments (characters interleaved with blanks and repeated by duration) are an adequate conditioning input for TTS.
    The TTS branch feeds EA into the shared encoder; if this representation lacks prosodic or phonetic detail, synthesized speech would be low quality. This assumption is introduced in Section II-A.
  • ad hoc to paper Simple elementwise addition of speech and text embeddings is sufficient input fusion for the multimodal encoder.
    The paper uses Z = MMEncoder(ˆX + ˆEA) with no learned gating or attention across modalities. This is a strong architectural choice not independently motivated. Section II-A.
  • domain assumption Pseudo-CTC alignments from a standalone duration model are valid for unpaired text in the T2T task.
    For text-only data, the paper states: 'We train a standalone duration model using the paired data which is used to generate pseudo-CTC alignments.' If these alignments are noisy, the T2T task may provide poor supervision. Section III-B.
  • domain assumption The model's greedy CTC confidence scores are sufficiently calibrated for masking decisions in iterative refinement.
    The refinement masks low-confidence characters based on average softmax probability. If these scores are miscalibrated, refinement could harm rather than help. Section II-C.
  • ad hoc to paper MaskX2, which masks entire trailing time and frequency regions, is a suitable schedule for TTS refinement.
    The schedule progressively unmask k/K of T and F. This is a heuristic with no theoretical grounding. Section II-C.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Non-autoregressive Model for Joint STT and TTS." pith.science (2026). https://pith.science/paper/UJCMPK5A

@misc{pith2026250109104,
  author       = {Pith},
  title        = {Pith review of: A Non-autoregressive Model for Joint STT and TTS},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJCMPK5A}},
  note         = {Machine review of arXiv:2501.09104}
}
read the original abstract

In this paper, we take a step towards jointly modeling automatic speech recognition (STT) and speech synthesis (TTS) in a fully non-autoregressive way. We develop a novel multimodal framework capable of handling the speech and text modalities as input either individually or together. The proposed model can also be trained with unpaired speech or text data owing to its multimodal nature. We further propose an iterative refinement strategy to improve the STT and TTS performance of our model such that the partial hypothesis at the output can be fed back to the input of our model, thus iteratively improving both STT and TTS predictions. We show that our joint model can effectively perform both STT and TTS tasks, outperforming the STT-specific baseline in all tasks and performing competitively with the TTS-specific baseline across a wide range of evaluation metrics.

Figures

Figures reproduced from arXiv: 2501.09104 by the authors.

Figure 1
Figure 1. Model overview. The dashed arrows from the output back to the input [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Effect of number of iterations of iterative refinement on STT (left) [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 14 canonical work pages

  1. [1]

    Almost unsupervised text to speech and automatic speech recognition,

    Y . Ren, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Almost unsupervised text to speech and automatic speech recognition,” in International conference on machine learning . PMLR, 2019, pp. 5410– 5419

    work page 2019

  2. [2]

    Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing,

    J. Ao, R. Wang, L. Zhou, C. Wang, S. Ren, Y . Wu, S. Liu, T. Ko, Q. Li, Y . Zhanget al., “Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 5723–5738

    work page 2022

  3. [3]

    Lauragpt: Listen, attend, understand, and regenerate audio with gpt,

    Q. Chen, Y . Chu, Z. Gao, Z. Li, K. Hu, X. Zhou, J. Xu, Z. Ma, W. Wang, S. Zheng et al., “Lauragpt: Listen, attend, understand, and regenerate audio with gpt,” arXiv preprint arXiv:2310.04673 , 2023

    arXiv 2023

  4. [4]

    Seamlessm4t-massively multilingual & multimodal machine translation,

    L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, P.-A. Duquenne, H. Elsahar, H. Gong, K. Heffernan, J. Hoffman et al. , “Seamlessm4t-massively multilingual & multimodal machine translation,” arXiv preprint arXiv:2308.11596 , 2023

    arXiv 2023

  5. [5]

    Speechverse: A large-scale generalizable audio language model,

    N. Das, S. Dingliwal, S. Ronanki, R. Paturi, D. Huang, P. Mathur, J. Yuan, D. Bekal, X. Niu, S. M. Jayanthi et al. , “Speechverse: A large-scale generalizable audio language model,” arXiv preprint arXiv:2405.08295 , 2024

    arXiv 2024

  6. [6]

    Viola: Conditional language models for speech recognition, synthesis, and translation,

    T. Wang, L. Zhou, Z. Zhang, Y . Wu, S. Liu, Y . Gaur, Z. Chen, J. Li, and F. Wei, “Viola: Conditional language models for speech recognition, synthesis, and translation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2024

    work page 2024

  7. [7]

    Funaudiollm: V oice understanding and generation foundation models for natural interaction between humans and llms,

    T. SpeechTeam, “Funaudiollm: V oice understanding and generation foundation models for natural interaction between humans and llms,” arXiv preprint arXiv:2407.04051 , 2024

    arXiv 2024

  8. [8]

    Owsm-ctc: An open encoder-only speech foundation model for speech recognition, translation, and language identification,

    Y . Peng, Y . Sudo, M. Shakeel, and S. Watanabe, “Owsm-ctc: An open encoder-only speech foundation model for speech recognition, translation, and language identification,” arXiv preprint arXiv:2402.12654 , 2024

    arXiv 2024

Show all 38 references
  1. [9]

    Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,

    A. Graves, S. Fern ´andez, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning , 2006, pp. 369–376

    2006

  2. [10]

    Fastspeech: Fast, robust and controllable text to speech,

    Y . Ren, Y . Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech: Fast, robust and controllable text to speech,” Advances in neural information processing systems , vol. 32, 2019

    2019

  3. [11]

    Joist: A joint speech and text streaming model for asr,

    T. N. Sainath, R. Prabhavalkar, A. Bapna, Y . Zhang, Z. Huo, Z. Chen, B. Li, W. Wang, and T. Strohman, “Joist: A joint speech and text streaming model for asr,” in 2022 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2023, pp. 52–59

    2022

  4. [12]

    Integrating text inputs for training and adapting rnn transducer asr models,

    S. Thomas, B. Kingsbury, G. Saon, and H.-K. J. Kuo, “Integrating text inputs for training and adapting rnn transducer asr models,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 8127–8131

    2022

  5. [13]

    Semi-autoregressive streaming asr with label context,

    S. Arora, G. Saon, S. Watanabe, and B. Kingsbury, “Semi-autoregressive streaming asr with label context,” in ICASSP 2024-2024 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 11 681–11 685

    2024

  6. [14]

    Align-refine: Non- autoregressive speech recognition via iterative realignment,

    E. A. Chi, J. Salazar, and K. Kirchhoff, “Align-refine: Non- autoregressive speech recognition via iterative realignment,” arXiv preprint arXiv:2010.14233, 2020

    2010 arXiv

  7. [15]

    Mask ctc: Non-autoregressive end-to-end asr with ctc and mask predict,

    Y . Higuchi, S. Watanabe, N. Chen, T. Ogawa, and T. Kobayashi, “Mask ctc: Non-autoregressive end-to-end asr with ctc and mask predict,” arXiv preprint arXiv:2005.08700, 2020

    2005 arXiv

  8. [16]

    Bert meets ctc: New formulation of end-to-end speech recognition with pre-trained masked language model,

    Y . Higuchi, B. Yan, S. Arora, T. Ogawa, T. Kobayashi, and S. Watanabe, “Bert meets ctc: New formulation of end-to-end speech recognition with pre-trained masked language model,” arXiv preprint arXiv:2210.16663 , 2022

    2022 arXiv

  9. [17]

    Bectra: Transducer-based end-to-end asr with bert-enhanced encoder,

    Y . Higuchi, T. Ogawa, T. Kobayashi, and S. Watanabe, “Bectra: Transducer-based end-to-end asr with bert-enhanced encoder,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  10. [18]

    Mask-conformer: Augmenting conformer with mask-predict decoder,

    Y . Higuchi, A. Rosenberg, Y . Wang, M. K. Baskar, and B. Ramabhadran, “Mask-conformer: Augmenting conformer with mask-predict decoder,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–8

    2023

  11. [19]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems , vol. 33, pp. 12 449– 12 460, 2020

    2020

  12. [20]

    Layer normalization,

    J. Lei Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” ArXiv e-prints, pp. arXiv–1607, 2016

    2016

  13. [21]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “Utmos: Utokyo-sarulab system for voicemos challenge 2022,” arXiv preprint arXiv:2204.02152 , 2022

    2022 arXiv

  14. [22]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

    2023

  15. [23]

    Espnet-spk: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models,

    J.-w. Jung, W. Zhang, J. Shi, Z. Aldeneh, T. Higuchi, B.-J. Theobald, A. H. Abdelaziz, and S. Watanabe, “Espnet-spk: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models,” arXiv preprint arXiv:2401.17230 , 2024

    2024 arXiv

  16. [24]

    The lj speech dataset,

    K. Ito and L. Johnson, “The lj speech dataset,” 2017

    2017

  17. [25]

    Libritts: A corpus derived from librispeech for text-to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” arXiv preprint arXiv:1904.02882 , 2019

    1904 arXiv

  18. [26]

    Librispeech: an asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2015, pp. 5206–5210

    2015

  19. [27]

    Libritts-r: A restored multi-speaker text-to-speech corpus,

    Y . Koizumi, H. Zen, S. Karita, Y . Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y . Zhang, W. Han, and A. Bapna, “Libritts-r: A restored multi-speaker text-to-speech corpus,” arXiv preprint arXiv:2305.18802 , 2023

    2023 arXiv

  20. [28]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu et al., “Conformer: Convolution-augmented transformer for speech recognition,” arXiv preprint arXiv:2005.08100 , 2020

    2005 arXiv

  21. [29]

    Ecapa-tdnn: Em- phasized channel attention, propagation and aggregation in tdnn based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa-tdnn: Em- phasized channel attention, propagation and aggregation in tdnn based speaker verification,” arXiv preprint arXiv:2005.07143 , 2020

    2005 arXiv

  22. [30]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” Advances in neural information processing systems , vol. 33, pp. 17 022–17 033, 2020

    2020

  23. [31]

    Specaugment: A simple data augmentation method for automatic speech recognition,

    D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” arXiv preprint arXiv:1904.08779 , 2019

    1904 arXiv

  24. [32]

    Audio augmentation for speech recognition

    T. Ko, V . Peddinti, D. Povey, and S. Khudanpur, “Audio augmentation for speech recognition.” in Interspeech, vol. 2015, 2015, p. 3586

    2015

  25. [33]

    Super-convergence: Very fast training of neural networks using large learning rates,

    L. N. Smith and N. Topin, “Super-convergence: Very fast training of neural networks using large learning rates,” in Artificial intelligence and machine learning for multi-domain operations applications , vol. 11006. SPIE, 2019, pp. 369–386

    2019

  26. [34]

    Rethinking the inception architecture for computer vision,

    C. Szegedy, V . Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 2818–2826

    2016

  27. [35]

    Regularization of neural networks using dropconnect,

    L. Wan, M. Zeiler, S. Zhang, Y . Le Cun, and R. Fergus, “Regularization of neural networks using dropconnect,” in International conference on machine learning. PMLR, 2013, pp. 1058–1066

    2013

  28. [36]

    Sequence noise injected training for end-to-end speech recognition,

    G. Saon, Z. T ¨uske, K. Audhkhasi, and B. Kingsbury, “Sequence noise injected training for end-to-end speech recognition,” in ICASSP 2019- 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 6261–6265

    2019

  29. [37]

    Relaxing the conditional independence assumption of ctc-based asr by conditioning on intermediate predictions,

    J. Nozaki and T. Komatsu, “Relaxing the conditional independence assumption of ctc-based asr by conditioning on intermediate predictions,” arXiv preprint arXiv:2104.02724 , 2021

    2021 arXiv

  30. [38]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.