Pith. sign in

REVIEW 5 major objections 6 minor 36 references

Next Tokens Denoising for Speech Synthesis

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dragon-FM generates 48 kHz speech by predicting 2-second chunks of 25 tokens and denoising them in parallel.

desk verdict Plausible architecture, but the few-step quality claim is undercut by a contradictory Table 2 and missing FAD at the NFE values used for latency. read the letter →

arxiv 2507.22746 v2 pith:WS4GUIRV submitted 2025-07-30 cs.SD cs.CLeess.AS

Yanqing Liu , Ruiqing Xue , Chong Zhang , Yufei Liu , Gang Wang , Bohan Li , Yao Qian , Lei He
show 2 more authors
Shujie Liu Sheng Zhao
This is my paper
classification cs.SDcs.CLeess.AS
keywords text-to-speechflowmatchingautoregressivegenerationfinitescalarquantizationaudiocodeczero-shotspeechsynthesispodcast48kHz
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that autoregressive and flow-matching generation can be combined in a single text-to-speech model without giving up the main advantages of either. The model, Dragon-FM, predicts speech in 2-second chunks of 25 discrete tokens, using autoregressive prediction across chunks for global coherence and parallel flow-matching denoising inside each chunk for speed. If successful, this design would make high-quality, diverse, zero-shot speech synthesis much faster and better suited to long-form content like podcasts. The paper also argues that continuous flow-matching can reliably predict discrete tokens produced by finite scalar quantizers, bridging two feature-modeling traditions.

What carries the argument

The central objects are a 12.5 Hz finite scalar quantization (FSQ) codec that compresses 48 kHz audio into 25 tokens per 2 seconds, and a chunk-wise autoregressive flow-matching acoustic model that predicts these chunks. Within each chunk, mean-flow optimization enables few-step denoising, while across chunks the autoregressive loop preserves long-range coherence and leverages KV-cache. The FSQ tokenizer is what makes the compact frame rate possible while keeping reconstruction quality high enough for intelligible, speaker-similar speech.

What would settle it

Run the proposed codec on a standard speech corpus and measure word error rate and speaker similarity against EnCodec and DAC at matched bitrate; if the 12.5 Hz FSQ codec is clearly worse on intelligibility or timbre while the 16 Hz mel vocoder matches it, the claim that the codec preserves enough detail for the fast architecture would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that a text-to-speech system built on a 48 kHz codec with 12.5 tokens per second can generate high-fidelity, high-diversity zero-shot speech using a chunk-wise autoregressive structure where each 2-second chunk contains 25 tokens produced in parallel by flow-matching denoising. This is presented as a unification of AR and diffusion-style generation: the model keeps the key-value cache across chunks and uses bidirectional attention within each chunk, reducing both autoregressive steps and denoising steps dramatically. The paper further claims that continuous flow-matching models, without architectural changes, can predict FSQ-discrete tokens when the token embeddings are designed as continuous vectors, demonstrating a practical bridge between discrete and continuous feature modeling.

Load-bearing premise

The 12.5 Hz FSQ codec preserves enough acoustic detail for high-fidelity 48 kHz synthesis, because without that the entire chunk-AR design loses its quality basis.

Editorial extensions

If this is right

  • Two-second audio chunks can be generated with as few as 2 function evaluations per chunk, cutting the total number of function evaluations far below prior AR and diffusion TTS systems for equivalent audio length.
  • The 12.5 Hz token rate reduces self-attention complexity enough that long-form generation, such as full podcasts, becomes practical without sacrificing 48 kHz fidelity.
  • KV-caching across chunks yields low first-byte latency, approaching streaming systems, while parallel denoising inside chunks keeps the real-time factor competitive.
  • Because the acoustic model predicts discrete tokens with a continuous denoising objective, the same architecture can potentially handle both continuous and discrete modalities, opening a path to unified multimodal generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same chunk-AR-plus-flow-matching recipe could transfer to other modalities that admit a compact discrete representation, such as music or video tokens, wherever a low-token-rate codec exists.
  • The paper's TNFE comparison excludes text and speaker prompt processing; for very short utterances the prompt cost could dominate latency, so the speed advantage may be largest for long-form content rather than single sentences.
  • If the 12.5 Hz FSQ codec is the load-bearing component, then improving codec quality at even lower frame rates (e.g., 8 Hz) would strengthen the argument that compact discrete tokens can replace continuous mel features in diffusion-style TTS.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Dragon-FM, a zero-shot text-to-speech system that combines chunk-wise autoregressive prediction with parallel flow-matching denoising inside each chunk. Audio is encoded with a 48 kHz codec producing 12.5 FSQ tokens per second, and each 2-second chunk contains 25 tokens; AR steps act across chunks while mean-flow denoising generates the chunk in a few steps. The authors claim this unifies AR and diffusion benefits, yielding high-quality, diverse long-form podcasts with low latency and few function evaluations. Experiments report codec reconstruction (SIM/WER), FAD diversity, and TNFE analysis, but do not include listening tests or comparisons to recent TTS systems.

Significance. If the claims hold, Dragon-FM would be a notable efficiency advance for zero-shot TTS: the 12.5 Hz token rate is much lower than common codecs, chunk-wise AR reduces AR steps, and continuous flow-matching over FSQ embeddings is an interesting conceptual unification. The paper provides open demos and a clear TNFE analysis, and the codec design at 12.5 Hz is a useful data point. However, the empirical evidence is currently too thin to support the central quality and efficiency claims: no human evaluation, no strong baselines, no error bars, and an inconsistency in the key FAD table.

major comments (5)
  1. [§4.3.1, Table 2] The text states that Dragon-FM-A3 and A1 achieve a lower FAD score than Dragon-FM-A2, but the table lists A2 at 2.2, A1 at 2.4, and A3 at 2.6, so the numbers directly contradict the narrative. Because this table is the primary support for the claim that reducing flow-matching steps does not compromise diversity, the inconsistency must be corrected and the results re-evaluated.
  2. [§4.3.1 and §4.3.2] No FAD or other quality metric is reported for the 2- and 4-NFE configurations that are used in Table 3 and in the latency claims. Without quality measurements at the actual deployment NFE, the central efficiency-quality trade-off of Dragon-FM is not empirically supported.
  3. [§4.3 overall] The evaluation lacks any comparison with state-of-the-art zero-shot TTS systems such as VALL-E, E2, ARDiT, or Ditar, and includes no human listening tests (e.g., MOS, CMOS) or significance testing. The FAD differences in Table 2 are small (0.2 to 0.4 absolute) and may fall within noise; this is insufficient to establish the claimed 'high-quality zero-shot podcasts'.
  4. [§4.3.3, Table 4] The codec is only compared with internally trained mel-spectrogram vocoders; no comparison is made with established neural audio codecs such as EnCodec or DAC, or with other recent low-frame-rate codecs. Therefore the claim that the 12.5 Hz FSQ codec is 'high-fidelity' is not benchmarked against the field.
  5. [§1 and Abstract] The claim that 'continuous AR flow-matching can predict discrete tokens with finite scalar quantizers' is not directly evaluated; there is no analysis of token prediction accuracy or a comparison with a discrete cross-entropy model. The supporting evidence is only end-to-end audio metrics, which does not specifically validate this mechanistic claim.
minor comments (6)
  1. [§3.1] '48 kH audio' appears to be a typo for '48 kHz audio'.
  2. [Table 2 caption] The caption should capitalize 'Comparison' and should specify the units for the FM step column (number of function evaluations).
  3. [§4.2] The sentence about training for '2 epochs' and 'starting from 300,000 iterations' is ambiguous; please clarify whether 300,000 iterations is per epoch or total, and state the dataset size used.
  4. [§4.3.2, Eq. (1)] The equation for TNFE uses inconsistent spacing and the variables StepAR and NFEFM are not explicitly defined; please define them in the text and format the equation consistently.
  5. [Table 3] The two row groups (2-second and 32-second audio) would be clearer with explicit section headers, since the first row of each group repeats the model names without a separator.
  6. [Table 3 and Related Work] The paper cites both VALL-E [Wang et al., 2023] and VALL-E 2 [Chen et al., 2024] but occasionally uses 'V ALL-E' without specifying which version is used for the TNFE estimate; please clarify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Dragon-FM's chunk-AR/flow-matching design and FSQ codec are evaluated empirically, with self-citations used only as contextual baselines; the Table 2 inconsistency is a correctness-risk issue, not a circularity.

full rationale

The paper is an empirical system-design paper. The central claims—that a 12.5 Hz FSQ codec preserves fidelity, and that chunk-wise autoregression with parallel flow-matching reduces latency while maintaining quality—are supported by implemented models, ablation experiments, FAD, SIM/WER, and TNFE comparisons. No derivation reduces by construction to fitted values, and no load-bearing premise is justified solely by a self-citation. The FSQ quantizer is attributed to Mentzer et al. and the mean-flow step reduction to Geng et al., both external works; self-citations to VALL-E 2, NaturalSpeech, and related Microsoft TTS systems appear as contextual baselines or prior art, not as the justification for Dragon-FM's design. The TNFE definition in Eq. (1) is a stated convention, not a fitted quantity, and Table 3 follows directly from that definition. The internal contradiction in Table 2, where the text claims A1/A3 have lower FAD than A2 while the table shows A2 with the best FAD, is a real internal-consistency problem and a weakness in the empirical support, but it is not circular reasoning. Likewise, computing FAD on the same podcast distribution used for training is an evaluation-validity concern, not a circular derivation. Therefore, no circular step meeting the quoted-evidence threshold is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several domain assumptions about codec fidelity, metric validity, and few-step convergence. The paper provides direct evidence for codec fidelity only via internal reconstruction metrics (SIM, WER) and no external validation for the acoustic model. The free parameters are design choices (frame rate, chunk size, codebook size, NFE) that are ablated partially but not fully, and they are tuned on the same internal podcast distribution used for evaluation.

free parameters (5)
  • codec frame rate = 12.5 tokens/s
    Chosen to balance sequence length and reconstruction quality; evaluated at 12.5, 20, and 8.3 Hz in Table 4.
  • chunk size = 2 seconds (25 tokens)
    Chosen to reduce AR steps; ablated with 1-second chunks in Table 2.
  • FSQ codebook size (CodecB) = 100^5
    Selected as the default codec; alternatives 100^9 and 8.3 Hz variants are ablated in Table 4.
  • flow-matching steps (NFE) = 2 or 4
    Mean flow reduces NFE; Table 3 reports 2-4 steps for the final system, while Table 2 shows FAD for 6, 12, and 24 steps.
  • learning rate schedule = 1e-3 decaying to 1e-4 after 300k iterations
    Training detail that affects convergence but not the core architectural claim.
assumptions (5)
  • domain assumption Flow matching can be extended to discrete token embeddings treated as continuous vectors
    Section 3.1 'Discrete Tokens as Continuous Vectors' assumes that a continuous denoising model can classify token indices through learned embeddings, without direct classification experiments.
  • domain assumption FAD is a valid metric for comparing generated speech diversity and quality
    Section 4.3.1 uses FAD without validating it against human judgment or showing that FAD differences are perceptually meaningful.
  • domain assumption Mean flow provides one/few-step generation without quality loss
    Section 3.1 'Few Step Denoising' relies on mean flow (Geng et al., 2025) to reduce denoising steps to 2-4, but no ablation shows quality versus step count for the final system.
  • domain assumption The podcast dataset is representative and internal ASR transcriptions are accurate
    Section 4.1 says an internal ASR system transcribed 60,000 hours, noisy segments were filtered, and clips with irregular pitch or duration were excluded; no quality assessment of the transcriptions is provided.
  • standard math Transformer self-attention has quadratic complexity in sequence length
    Invoked in Section 3.1 to justify the need for short sequences; standard and uncontroversial.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Next Tokens Denoising for Speech Synthesis." pith.science (2026). https://pith.science/paper/WS4GUIRV

@misc{pith2026250722746,
  author       = {Pith},
  title        = {Pith review of: Next Tokens Denoising for Speech Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WS4GUIRV}},
  note         = {Machine review of arXiv:2507.22746}
}
read the original abstract

While diffusion and autoregressive (AR) models have significantly advanced generative modeling, they each present distinct limitations. AR models, which rely on causal attention, cannot exploit future context and suffer from slow generation speeds. Conversely, diffusion models struggle with key-value (KV) caching. To overcome these challenges, we introduce Dragon-FM, a novel text-to-speech (TTS) design that unifies AR and flow-matching. This model processes 48 kHz audio codec tokens in chunks at a compact rate of 12.5 tokens per second. This design enables AR modeling across chunks, ensuring global coherence, while parallel flow-matching within chunks facilitates fast iterative denoising. Thus, the model leverages KV-cache across chunks and utilizes bidirectional context within each chunk. Furthermore, it bridges continuous and discrete feature modeling, demonstrating that continuous AR flow-matching can predict discrete tokens with finite scalar quantizers. This efficient codec and fast chunk-autoregressive architecture also make the model highly effective for generating long-form content, such as podcasts. Experiments on podcast datasets demonstrate its capability to efficiently generate high-quality zero-shot podcasts.

Figures

Figures reproduced from arXiv: 2507.22746 by the authors.

Figure 1
Figure 1. Proposed Architecture. Dragon-FM autoregressively predicts chunk-level speech tokens, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the asymmetric codec architecture. The codec encodes 48 kHz speech into [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 6 canonical work pages

  1. [1]

    Better speech synthesis through scaling

    James Betker. Better speech synthesis through scaling. arXiv preprint arXiv:2305.07243,

    arXiv

  2. [5]

    High fidelity neural audio compression

    Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. arXiv preprint arXiv:2210.13438,

    arXiv

  3. [8]

    Improving dynamic object interactions in text-to-video generation with ai feedback

    Hiroki Furuta, Heiga Zen, Dale Schuurmans, Aleksandra Faust, Yutaka Matsuo, Percy Liang, and Sherry Yang. Improving dynamic object interactions in text-to-video generation with ai feedback. arXiv preprint arXiv:2412.02617,

    arXiv

  4. [9]

    Mean flows for one-step generative modeling

    Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. arXiv preprint arXiv:2505.13447,

    arXiv

  5. [11]

    Ditar: Diffusion transformer autoregressive modeling for speech generation

    Dongya Jia, Zhuo Chen, Jiawei Chen, Chenpeng Du, Jian Wu, Jian Cong, Xiaobin Zhuang, Chumin Li, Zhen Wei, Yuping Wang, et al. Ditar: Diffusion transformer autoregressive modeling for speech generation. arXiv preprint arXiv:2502.03930,

    arXiv

  6. [12]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100,

    arXiv

  7. [13]

    Making flow-matching-based zero-shot text-to-speech laugh as you like

    Naoyuki Kanda, Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Hemin Yang, Zirun Zhu, Min Tang, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, et al. Making flow-matching-based zero-shot text-to-speech laugh as you like. arXiv preprint arXiv:2402.07383,

    arXiv

  8. [14]

    istftnet: Fast and lightweight mel-spectrogram vocoder incorporating inverse short-time fourier transform

    Takuhiro Kaneko, Kou Tanaka, Hirokazu Kameoka, and Shogo Seki. istftnet: Fast and lightweight mel-spectrogram vocoder incorporating inverse short-time fourier transform. In ICASSP 2022- 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6207–6211. IEEE,

    work page 2022

Show all 36 references
  1. [15]

    Understanding ddpm latent codes through optimal transport

    Valentin Khrulkov, Gleb Ryzhakov, Andrei Chertkov, and Ivan Oseledets. Understanding ddpm latent codes through optimal transport. arXiv preprint arXiv:2202.07477,

    arXiv

  2. [16]

    Prompttts 2: Describing and generating voices with text prompt

    Yichong Leng, Zhifang Guo, Kai Shen, Xu Tan, Zeqian Ju, Yanqing Liu, Yufei Liu, Dongchao Yang, Leying Zhang, Kaitao Song, et al. Prompttts 2: Describing and generating voices with text prompt. arXiv preprint arXiv:2309.02285,

    arXiv

  3. [17]

    Investigating neural audio codecs for speech language model-based speech generation

    Jiaqi Li, Dongmei Wang, Xiaofei Wang, Yao Qian, Long Zhou, Shujie Liu, Midia Yousefi, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, et al. Investigating neural audio codecs for speech language model-based speech generation. arXiv preprint arXiv:2409.04016, 2024a. Naihan Li, Shujie L...

    arXiv

  4. [19]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022a. Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step is enough for high-quality diffu...

    arXiv

  5. [20]

    Delightfultts 2: End-to-end speech synthesis with adversarial vector-quantized auto-encoders

    Yanqing Liu, Ruiqing Xue, Lei He, Xu Tan, and Sheng Zhao. Delightfultts 2: End-to-end speech synthesis with adversarial vector-quantized auto-encoders. arXiv preprint arXiv:2207.04646 , 2022b. Zhijun Liu, Shuai Wang, Sho Inoue, Qibing Bai, and Haizhou Li. Autoregressive diffus...

    arXiv

  6. [21]

    Autoregressive speech synthesis without vector quantization

    Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, et al. Autoregressive speech synthesis without vector quantization. arXiv preprint arXiv:2407.08551,

    arXiv

  7. [22]

    Finite scalar quantization: Vq-vae made simple

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. arXiv preprint arXiv:2309.15505,

    arXiv

  8. [23]

    Scaling transformers for low-bitrate high-quality speech coding

    Julian D Parker, Anton Smirnov, Jordi Pons, CJ Carr, Zack Zukowski, Zach Evans, and Xubo Liu. Scaling transformers for low-bitrate high-quality speech coding. arXiv preprint arXiv:2411.19842,

    arXiv

  9. [24]

    Mo \ˆ usai: Text-to-music generation with long-context latent diffusion

    Flavio Schneider, Ojasv Kamal, Zhijing Jin, and Bernhard Schölkopf. Mo \ˆ usai: Text-to-music generation with long-context latent diffusion. arXiv preprint arXiv:2301.11757,

    arXiv

  10. [25]

    Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers

    Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. arXiv preprint arXiv:2304.09116,

    arXiv

  11. [26]

    Zero-shot streaming text to speech synthesis with transducer and auto-regressive modeling

    Haiyang Sun, Shujie Hu, Shujie Liu, Lingwei Meng, Hui Wang, Bing Han, Yifan Yang, Yanqing Liu, Sheng Zhao, Yan Lu, et al. Zero-shot streaming text to speech synthesis with transducer and auto-regressive modeling. arXiv preprint arXiv:2505.19669,

    arXiv

  12. [27]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805,

    arXiv

  13. [28]

    Improving and generalizing flow-based generative models with minibatch optimal transport

    Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint arXiv:2302.00482,

    arXiv

  14. [30]

    Felle: Autoregressive speech synthesis with token-wise coarse-to-fine flow matching

    Hui Wang, Shujie Liu, Lingwei Meng, Jinyu Li, Yifan Yang, Shiwan Zhao, Haiyang Sun, Yanqing Liu, Haoqin Sun, Jiaming Zhou, et al. Felle: Autoregressive speech synthesis with token-wise coarse-to-fine flow matching. arXiv preprint arXiv:2502.11128, 2025a. Hui Wang, Yifan Yang, ...

    arXiv

  15. [31]

    Pseudo-autoregressive neural codec language models for efficient zero-shot text-to-speech synthesis

    Yifan Yang, Shujie Liu, Jinyu Li, Yuxuan Hu, Haibin Wu, Hui Wang, Jianwei Yu, Lingwei Meng, Haiyang Sun, Yanqing Liu, et al. Pseudo-autoregressive neural codec language models for efficient zero-shot text-to-speech synthesis. arXiv preprint arXiv:2504.10352,

    arXiv

  16. [32]

    Lumos-1: On autoregressive video generation from a unified model perspective

    Hangjie Yuan, Weihua Chen, Jun Cen, Hu Yu, Jingyun Liang, Shuning Chang, Zhihui Lin, Tao Feng, Pengwei Liu, Jiazheng Xing, et al. Lumos-1: On autoregressive video generation from a unified model perspective. arXiv preprint arXiv:2507.08801,

  17. [33]

    Continuous speech tokens makes llms robust multi-modality learners

    Ze Yuan, Yanqing Liu, Shujie Liu, and Sheng Zhao. Continuous speech tokens makes llms robust multi-modality learners. arXiv preprint arXiv:2412.04917,

    arXiv

  18. [34]

    Boosting diffusion model for spectrogram up-sampling in text-to-speech: An empirical study

    Chong Zhang, Yanqing Liu, Yang Zheng, and Sheng Zhao. Boosting diffusion model for spectrogram up-sampling in text-to-speech: An empirical study. arXiv preprint arXiv:2406.04633,

    arXiv

  19. [35]

    Mixed-phoneme bert: Improving bert with mixed phoneme and sup-phoneme representations for text to speech

    Guangyan Zhang, Kaitao Song, Xu Tan, Daxin Tan, Yuzi Yan, Yanqing Liu, Gang Wang, Wei Zhou, Tao Qin, Tan Lee, et al. Mixed-phoneme bert: Improving bert with mixed phoneme and sup-phoneme representations for text to speech. arXiv preprint arXiv:2203.17190,

    arXiv

  20. [36]

    Speak foreign languages with your own voice: Cross-lingual neural codec language modeling

    Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling. arXiv preprint arXiv:2303.03926,

    arXiv

  21. [2017]

    Neural codec language models are zero-shot text to speech synthesizers

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111,

    arXiv

  22. [2019]

    Moboaligner: A neural alignment model for non-autoregressive tts with monotonic boundary search

    10 Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, Ming Liu, and Ming Zhou. Moboaligner: A neural alignment model for non-autoregressive tts with monotonic boundary search. arXiv preprint arXiv:2005.08528, 2020a. Naihan Li, Yanqing Liu, Yu Wu, Shujie Liu, Sheng Zhao, and Ming ...

    2005 arXiv

  23. [2020]

    Adaspeech: Adaptive text to speech for custom voice

    Mingjian Chen, Xu Tan, Bohan Li, Yanqing Liu, Tao Qin, Sheng Zhao, and Tie-Yan Liu. Adaspeech: Adaptive text to speech for custom voice. arXiv preprint arXiv:2103.00993,

    arXiv

  24. [2021]

    Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers

    Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. arXiv preprint arXiv:2406.05370,

    arXiv

  25. [2022]

    E2 tts: Embarrassingly easy fully non- autoregressive zero-shot tts

    Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, et al. E2 tts: Embarrassingly easy fully non- autoregressive zero-shot tts. arXiv preprint arXiv:2406.18009,

    arXiv

  26. [2023]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901,

    1901

  27. [2024]

    One step diffusion via shortcut models

    9 Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. arXiv preprint arXiv:2410.12557,

    arXiv

  28. [2025]

    Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment

    Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, and Furu Wei. Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment. arXiv preprint arXiv:2406.07855,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.