Pith. sign in

REVIEW 4 major objections 5 minor 35 references

StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StreamMel claims that zero-shot text-to-speech can run as a single autoregressive Transformer over interleaved text and continuous mel-spectrogram frames, delivering the first audio packet in 0.01 seconds while staying close to offline…

desk verdict Plausible and novel single-stage streaming TTS with continuous mel, but the headline comparisons are under-specified enough that the SOTA claim needs a matched re-run. read the letter →

arxiv 2506.12570 v1 pith:4CWPP5BK submitted 2025-06-14 cs.SD cs.CLeess.AS

Hui Wang , Yifan Yang , Shujie Liu , Jinyu Li , Lingwei Meng , Yanqing Liu , Jiaming Zhou , Haoqin Sun
show 2 more authors
Yan Lu Yong Qin
This is my paper
classification cs.SDcs.CLeess.AS
keywords streamingtext-to-speechzero-shotTTScontinuousmel-spectrograminterleavedautoregressivemodelinglow-latencyspeechsynthesisspeakersimilarityLibricausalTransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

StreamMel sets out to show that streaming zero-shot text-to-speech does not need a separate acoustic stage or quantized codec tokens: one causal Transformer can consume an interleaved stream of phonemes and continuous mel-spectrogram frames and predict each new audio frame as text arrives. The paper reports that this single-stage design yields lower latency than existing streaming zero-shot systems—first audio in 0.01 seconds, or 0.04 seconds when a 25 ms upstream language-model delay is included—while matching or beating them on intelligibility and speaker similarity. On LibriSpeech test-clean, StreamMel reports WER-C 2.10, WER-H 2.76, WER-W 2.77, SIM-R 0.656, and SIM-O 0.622, with MOS 4.14 and SMOS 4.27, comparable to offline systems such as MELLE. If these results stand, real-time dialogue agents and simultaneous interpretation get a much simpler path to instant, speaker-preserving speech.

What carries the argument

The load-bearing object is the interleaved stream $z=[x_0,\ldots,x_{n-1},y_0,\ldots,y_{m-1},x_n,\ldots,y_m,\ldots]$, in which blocks of $n$ phoneme tokens alternate with blocks of $m$ mel-spectrogram frames at a fixed ratio. Text tokens are copied from the streaming input instead of predicted, and only mel frames are scored, so synthesis can start after a single text block and continue as more text arrives. The acoustic side is a 12-block causal Transformer decoder with a pre-net that projects mel frames into latent tokens; at each mel position the decoder outputs a mean and log-variance, samples a latent via reparameterization, projects to mel space, and predicts a stop probability. Training combines L1/L2 regression, a KL term toward a standard Gaussian prior, a spectrogram flux loss, and stop-prediction cross-entropy, which together keep the continuous predictions stable and termination flexible.

What would settle it

Run the cross-sentence evaluation on LibriSpeech test-clean with the stated 1:4 interleaving ratio and the same A100 hardware, with the audio-reduction factor and vocoder explicitly fixed: if the first packet does not arrive in roughly 0.01 seconds or WER-W is not near 2.77, the headline claim is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that an autoregressive Transformer trained on a fixed-ratio interleaving of text phonemes and continuous mel-spectrogram frames is a complete streaming zero-shot TTS system. Text tokens are copied in as they arrive rather than predicted, and the model only predicts the next mel frame, conditioned on all previous text and audio tokens; a lightweight latent decoder turns each predicted distribution into a mel frame and a stop probability. The paper argues this avoids two sources of latency and loss in prior streaming systems: the separate semantic-token stage followed by an acoustic flow model, and the vector quantization that converts audio into discrete tokens. Across continuation and cross-sentence evaluations on LibriSpeech, StreamMel is reported to beat the streaming baselines IST-LM and SMLLE on intelligibility and speaker similarity while reaching lower first-packet latency, and to stay within a small margin of offline, larger-data systems such as VALL-E 2 and MELLE trained on Libriheavy.

Load-bearing premise

The reported latency and quality numbers rest on one configuration—a fixed 1:4 text-to-audio interleaving ratio, a single NVIDIA A100 GPU, a 25-millisecond upstream language-model delay, and an audio-decoder setup whose details are not stated—so the exact numbers will not transfer if any of those change.

Editorial extensions

If this is right

  • Streaming zero-shot TTS collapses to a single causal language model; the separate semantic-token LM plus acoustic flow-matching stage can be dropped.
  • With one forward pass before the first audio packet, a live agent can begin speaking about 10 ms after text arrives, and about 40 ms when a 25 ms upstream LLM delay is included.
  • Continuous mel targets preserve speaker detail that quantization removes: StreamMel's SIM-O of 0.622 exceeds the streaming baselines and approaches offline MELLE trained on the same data.
  • The interleaving ratio is a direct knob on the latency-quality trade-off, with 1:4 the reported sweet spot; extreme ratios such as 1:5 sharply raise WER-C to 8.44.
  • A larger reduction factor lowers real-time factor (0.700 at r=1 to 0.179 at r=4 under 1:1 interleaving) at the cost of speaker similarity, making faster-than-real-time operation a tunable point.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same pattern—interleaving text with continuous target frames and predicting only those frames—could transfer to real-time speech-to-speech translation or dubbing, where incremental translated text drives low-latency audio; StreamMel names these applications as motivation but does not evaluate them.
  • Because the 0.04 s FPL-L number includes a fixed 25 ms delay from a 7B upstream LLM, end-to-end latency in a product would scale with the upstream model chosen, and a slower speech LLM would dominate the reported latency.
  • Continuous interleaving may also apply to other real-time generative tasks that currently quantize their signals, though that extension is not claimed in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. StreamMel is a streaming zero-shot TTS system that generates continuous mel-spectrograms with a single autoregressive Transformer decoder. Text phonemes and mel-spectrogram frames are interleaved at a fixed n:m ratio, and the model predicts mel frames frame-by-frame while text tokens are supplied incrementally. A latent sampling module and a stop predictor are added to the output head, following MELLE's design. The paper reports experiments on LibriSpeech for continuation and cross-sentence tasks, comparing against non-streaming and streaming baselines. The headline claims are that StreamMel is the first single-stage streaming zero-shot TTS system based on continuous representations and that it achieves state-of-the-art latency and speech quality among streaming systems, with FPL-A=0.01s, FPL-L=0.04s, WER-C 2.10, WER-H 2.76, WER-W 2.77, SIM-R 0.656, SIM-O 0.622, MOS 4.14 and SMOS 4.27 on LibriSpeech test-clean.

Significance. If the reported results are reproducible and the comparisons were matched, StreamMel would be a meaningful contribution: modeling continuous mel-spectrograms in a single streaming stage avoids the information loss of discrete codecs and the latency of a second acoustic stage. The paper gives a reasonably detailed architecture description, a clear loss decomposition, and useful ablations over the interleaving ratio, reduction factor, and number of latent samples. The provision of audio samples is a strength. However, the central empirical claim of state-of-the-art quality and latency is not yet established as written: the reduction factor for the main results is unspecified, the streaming baselines in Table IV are imported from a different paper with different training data and evaluation pipelines, the FPL numbers are internally inconsistent, and objective metrics have no error bars. These are fixable reporting and experimental issues, so the work is promising but needs major revision before the headline claims can be accepted.

major comments (4)
  1. [§III-A / §IV-A / Table VI] The reduction factor r used for the headline results is never stated. Section III-A specifies the interleaving ratio 1:4 and the loss weights, but r appears only in Table VI, where the sweep is performed under n:m=1:1. If the main model uses r=1, that should be stated explicitly; if another r is used, then the WER/SIM values in Tables I-IV and the real-time factors do not correspond to a reproducible configuration. This is load-bearing because the real-time claim depends on RTF and FPL, and the quality numbers depend on r.
  2. [Table IV / Table III] The state-of-the-art comparison is not matched. In Table IV, all baseline rows marked * are taken from [15], where models were trained on LibriTTS, Libriheavy, or Emilia, whereas StreamMel is trained on LibriSpeech. The WER-W comparison between StreamMel 2.77 and SyncSpeech 3.07 therefore conflates model quality with training corpus, ASR model, text normalization, and test subset. Table III similarly compares IST-LM trained on LibriTTS with LibriSpeech-trained models and omits WER-C and WER-W for IST-LM. Please provide matched baselines on the same test-clean subset with the same ASR pipeline, or explicitly re-label the comparison as indicative rather than as state-of-the-art.
  3. [§IV-A] The reported latency numbers are internally inconsistent. The text gives FPL-L = d_LLM + d_TTS with d_LLM = 25ms. With the reported FPL-L = 0.04s, this implies d_TTS = 15ms, whereas FPL-A = 0.01s implies d_TTS = 10ms. Please report d_TTS directly and define whether FPL includes the latent decoder and vocoder inference. Without this, the FPL comparisons in Table IV are not credible.
  4. [§III-B / §IV] Objective metrics are reported as point estimates without error bars or significance tests. For example, Table III shows StreamMel WER-C 2.10 vs MELLE (L) 2.21 and SIM-O 0.622 vs 0.625; these differences are small and could easily be within evaluation noise. Add confidence intervals or bootstrap significance tests for WER-C/WER-H/WER-W/SIM-R/SIM-O, and report the number of evaluation utterances.
minor comments (5)
  1. [§II-A, Eq. (2)] The floor function in Eq. (2) is typeset with an unreadable symbol; please rewrite with explicit floor brackets and define t' clearly for both branches of the formula.
  2. [Table VI] The row labels r=1,2,3,4 should clarify whether r denotes the number of mel frames predicted per decoding step and how r interacts with the n:m interleaving ratio; currently r is only defined for n:m=1:1.
  3. [Fig. 2] The x-axis 'sample times' is not explained; specify how the multiple latent samples are aggregated to produce the final audio (e.g., selection, averaging, or voting), since this determines the interpretation of the ablation.
  4. [§III-A] The mel-spectrogram extraction parameters and the vocoder configuration used for synthesis are not stated; provide these for reproducibility of the MOS and SIM results.
  5. [References] Several references are incomplete or lack version/venue information (e.g., [6], [7], [8], [12], [13]); please complete the bibliography entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: StreamMel's central claims rest on external benchmark measurements, not on fitted or self-derived quantities.

full rationale

StreamMel is an empirical system paper whose 'derivation chain' is architectural rather than mathematical. The interleaved sequence in Eq. (1)-(3) is a model definition, and the objective in Eq. (4) is a standard combination of regression, KL, flux, and stop losses; no equation reduces to its own input by construction. The headline results in Tables I-IV are measured on LibriSpeech test-clean against external baselines, including SyncSpeech numbers quoted from [15], so the central quality and latency claims are not implied by the model equations alone. The paper adopts MELLE's latent-sampling and KL-loss design with citation [11], and several baselines (IST-LM [13], SMLLE [14], FELLE [19]) come from the same Microsoft group; these are normal prior-work and baseline citations, not a uniqueness theorem or a fitted parameter renamed as a prediction. The unspecified reduction factor r for the main results and the unmatched baseline protocols are reproducibility and falsifiability concerns, not circularity. No fitted constant is used to generate a 'prediction' that is actually the fit itself, and the claimed first-single-stage continuous-mel contribution is not defined in terms of the experimental outcome. Therefore the paper is self-contained against external benchmarks and exhibits no significant circular reasoning.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces architectural components such as fill tokens, a latent decoder, and a stop predictor, but no new physical or ontological entities whose existence needs external evidence. These are learned model mechanisms, so the invented-entity ledger is empty.

free parameters (4)
  • interleaving ratio n:m = 1:4
    Chosen after ablations in Table V; quality degrades sharply at 3:1 and 1:5, so the headline results depend on this selection.
  • loss weights alpha, lambda, beta, gamma = 2, 0.05, 1, 0.5
    Set by hand in Section III-A; balancing reconstruction, KL, flux, and stop losses affects output quality.
  • reduction factor r = not stated for main results (1 to 4 explored)
    Controls frames per decoding step and RTF in Table VI; the main configuration is not pinned down.
  • sample times at inference = not stated for main results (1 to 6 explored)
    Fig. 2 shows WER and similarity improve with more samples; the reported scores do not say which count was used.
assumptions (5)
  • domain assumption LibriSpeech 960h contains enough speaker and prosody diversity for zero-shot generalization to test-clean speakers.
    Section III-A; the evaluation assumes training on LibriSpeech alone supports unseen-speaker similarity.
  • domain assumption Continuous mel frames can be decoded to natural audio by a lightweight MLP latent decoder plus an external vocoder, without explicit codec reconstruction loss.
    Sections II-B.3 and II-C; the mapping from sampled latent z to mel frames is learned with regression and KL losses, and waveform reconstruction is not described.
  • domain assumption The fixed interleaving position formula in Eq. (1)-(2) provides sufficient alignment between phonemes and mel frames without a separate duration model.
    Section II-A; streaming quality relies on this heuristic placement.
  • domain assumption ASR WER and WavLM speaker similarity are adequate objective proxies for intelligibility and speaker consistency.
    Section III-C; the objective evaluation section treats these metrics as valid for comparing TTS systems.
  • ad hoc to paper The upstream LLM delay d_LLM=25ms from Qwen-7B is representative for FPL-L latency.
    Section IV-A; this fixed value drives the FPL-L numbers and may not hold for larger or slower language models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling." pith.science (2026). https://pith.science/paper/4CWPP5BK

@misc{pith2026250612570,
  author       = {Pith},
  title        = {Pith review of: StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CWPP5BK}},
  note         = {Machine review of arXiv:2506.12570}
}
read the original abstract

Recent advances in zero-shot text-to-speech (TTS) synthesis have achieved high-quality speech generation for unseen speakers, but most systems remain unsuitable for real-time applications because of their offline design. Current streaming TTS paradigms often rely on multi-stage pipelines and discrete representations, leading to increased computational cost and suboptimal system performance. In this work, we propose StreamMel, a pioneering single-stage streaming TTS framework that models continuous mel-spectrograms. By interleaving text tokens with acoustic frames, StreamMel enables low-latency, autoregressive synthesis while preserving high speaker similarity and naturalness. Experiments on LibriSpeech demonstrate that StreamMel outperforms existing streaming TTS baselines in both quality and latency. It even achieves performance comparable to offline systems while supporting efficient real-time generation, showcasing broad prospects for integration with real-time speech large language models. Audio samples are available at: https://aka.ms/StreamMel.

Figures

Figures reproduced from arXiv: 2506.12570 by the authors.

Figure 1
Figure 1. StreamMel architecture. The model takes an interleaved sequence of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Effect of sample times on WER (left) and similarity metrics (right). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 24 canonical work pages

  1. [15]

    Syncspeech: Low-latency and efficient dual-stream text-to-speech based on temporal masked transformer,

    Z. Sheng, Z. Du, S. Zhanget al., “Syncspeech: Low-latency and efficient dual-stream text-to-speech based on temporal masked transformer,” 2025

    work page 2025

  2. [1]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [2]

    Audiolm: A language modeling approach to audio generation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Shar- ifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchi, and N. Zeghi- dour, “Audiolm: A language modeling approach to audio generation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2523–2533, 2023

    work page 2023

  4. [3]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” inThe Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=PqvMRDCJT9t

    2023

  5. [4]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020

    2020

  6. [5]

    Neural codec language models are zero-shot text to speech synthesizers,

    S. Chen, C. Wang, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li, L. He, S. Zhao, and F. Wei, “Neural codec language models are zero-shot text to speech synthesizers,”IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 705–718, 2025

    2025

  7. [6]

    V ALL-E 2: Neural codec language models are human parity zero-shot text to speech synthesizers,

    S. Chen, S. Liu, L. Zhouet al., “V ALL-E 2: Neural codec language models are human parity zero-shot text to speech synthesizers,” 2024

    work page 2024

  8. [7]

    CosyV oice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic to- kens,

    Z. Du, Q. Chen, S. Zhanget al., “CosyV oice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic to- kens,” 2024

    work page 2024

Show all 35 references
  1. [8]

    Seed-TTS: A family of high- quality versatile speech generation models,

    P. Anastassiou, J. Chen, J. Chenet al., “Seed-TTS: A family of high- quality versatile speech generation models,” 2024

    2024

  2. [9]

    Cosyvoice 3: Towards in-the-wild speech genera- tion via scaling-up and post-training,

    Z. Du, C. Gao, Y . Wang, F. Yu, T. Zhao, H. Wang, X. Lv, H. Wang, X. Shi, K. Anet al., “Cosyvoice 3: Towards in-the-wild speech genera- tion via scaling-up and post-training,”arXiv preprint arXiv:2505.17589, 2025

    2025 arXiv

  3. [10]

    Pseudo-autoregressive neural codec language models for efficient zero-shot text-to-speech synthesis,

    Y . Yang, S. Liu, J. Liet al., “Pseudo-autoregressive neural codec language models for efficient zero-shot text-to-speech synthesis,” 2025

    2025

  4. [11]

    Autoregressive speech synthesis without vector quantization,

    L. Meng, L. Zhou, S. Liuet al., “Autoregressive speech synthesis without vector quantization,” inProc. ACL, Vienna, 2025

    2025

  5. [12]

    Cosyvoice 2: Scalable streaming speech synthesis with large language models,

    Z. Du, Y . Wang, Q. Chenet al., “Cosyvoice 2: Scalable streaming speech synthesis with large language models,” 2024

    2024

  6. [13]

    Interleaved speech-text language models are simple streaming text to speech synthesizers,

    Y . Yang, Z. Ma, S. Liu, J. Li, H. Wang, L. Meng, H. Sun, Y . Liang, R. Xu, Y . Hu, Y . Lu, R. Zhao, and X. Chen, “Interleaved speech-text language models are simple streaming text to speech synthesizers,”

  7. [14]

    Zero-shot streaming text to speech synthesis with transducer and auto-regressive modeling,

    H. Sun, S. Hu, S. Liu, L. Meng, H. Wang, B. Han, Y . Yang, Y . Liu, S. Zhao, Y . Lu, and Y . Qian, “Zero-shot streaming text to speech synthesis with transducer and auto-regressive modeling,” 2025. [Online]. Available: https://arxiv.org/abs/2505.19669

    2025 arXiv

  8. [16]

    Discrete audio representation as an alternative to mel-spectrograms for speaker and speech recognition,

    K. C. Puvvada, N. Rao Koluguri, K. Dhawan, J. Balam, and B. Ginsburg, “Discrete audio representation as an alternative to mel-spectrograms for speaker and speech recognition,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP...

    2024

  9. [17]

    Autoregressive diffusion transformer for text-to-speech synthesis,

    Z. Liu, S. Wang, S. Inoueet al., “Autoregressive diffusion transformer for text-to-speech synthesis,” 2024

    2024

  10. [18]

    Autoregressive image gen- eration without vector quantization,

    T. Li, Y . Tian, H. Li, M. Deng, and K. He, “Autoregressive image gen- eration without vector quantization,”arXiv preprint arXiv:2406.11838, 2024

    2024 arXiv

  11. [19]

    FELLE: autoregressive speech synthesis with token-wise coarse-to-fine flow matching,

    H. Wang, S. Liu, L. Menget al., “FELLE: autoregressive speech synthesis with token-wise coarse-to-fine flow matching,” 2025

    2025

  12. [20]

    Ditar: Diffusion transformer autoregressive modeling for speech generation,

    D. Jia, Z. Chen, J. Chen, C. Du, J. Wu, J. Cong, X. Zhuang, C. Li, Z. Wei, Y . Wanget al., “Ditar: Diffusion transformer autoregressive modeling for speech generation,”arXiv preprint arXiv:2502.03930, 2025

    2025

  13. [21]

    Auto-encoding variational bayes,

    D. P. Kingma, M. Wellinget al., “Auto-encoding variational bayes,” 2013

    2013

  14. [22]

    Tacotron: Towards end- to-end speech synthesis,

    Y . Wang, R. J. Skerry-Ryan, D. Stantonet al., “Tacotron: Towards end- to-end speech synthesis,” inProc. Interspeech, Stockholm, 2017

    2017

  15. [23]

    Librispeech: an ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Poveyet al., “Librispeech: an ASR corpus based on public domain audio books,” inProc. ICASSP, South Brisbane, 2015

    2015

  16. [24]

    V ALL-E R: robust and efficient zero-shot text-to-speech synthesis via monotonic alignment,

    B. Han, L. Zhou, S. Liuet al., “V ALL-E R: robust and efficient zero-shot text-to-speech synthesis via monotonic alignment,” 2024

    2024

  17. [25]

    F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching,

    Y . Chen, Z. Niu, Z. Maet al., “F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching,” 2024

    2024

  18. [26]

    MaskGCT: Zero-shot text-to-speech with masked generative codec transformer,

    Y . Wang, H. Zhan, L. Liuet al., “MaskGCT: Zero-shot text-to-speech with masked generative codec transformer,” inProc. ICLR, Singapore, 2024

    2024

  19. [27]

    Ramp: Retrieval-augmented mos prediction via confidence-based dynamic weighting,

    H. Wang, S. Zhao, X. Zheng, and Y . Qin, “Ramp: Retrieval-augmented mos prediction via confidence-based dynamic weighting,” inINTER- SPEECH 2023, 2023, pp. 1095–1099

    2023

  20. [28]

    Ramp+: Retrieval-augmented mos prediction with prior knowledge integration,

    H. Wang, S. Zhao, X. Zheng, J. Zhou, X. Wang, and Y . Qin, “Ramp+: Retrieval-augmented mos prediction with prior knowledge integration,” IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 1520–1534, 2025

    2025

  21. [29]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wuet al., “Conformer: Convolution-augmented transformer for speech recognition,”arXiv preprint arXiv:2005.08100, 2020

    2005 arXiv

  22. [30]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W. N. Hsu, B. Bolte, Y . H. H. Tsaiet al., “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, 2021

    2021

  23. [31]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [Online]. Available: https://arxiv.org/abs/2212.04356

    2022 arXiv

  24. [32]

    WavLM: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chenet al., “WavLM: Large-scale self-supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, 2022

    2022

  25. [33]

    Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context,

    W. Kang, X. Yang, Z. Yaoet al., “Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context,” inProc. ICASSP, Seoul, 2024

    2024

  26. [34]

    Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,

    H. He, Z. Shang, C. Wanget al., “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation,” inProc. SLT, Macao, 2024

    2024

  27. [2024]

    Available: https://arxiv.org/abs/2412.16102

    [Online]. Available: https://arxiv.org/abs/2412.16102

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.