Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new system clones an unseen speaker across 14 languages from untranscribed reference audio, using a learnable speaker encoder and flow matching instead of reference transcripts.

desk verdict A competent engineering report with strong cross-lingual numbers, but the unaddressed training/eval corpus overlap and an inference-only continuation mode keep it from being conclusive. read the letter →

arxiv 2608.11650 v1 pith:YLLEDNKZ submitted 2026-08-12 cs.SD cs.CL

Huaxuan Wang , Huimin Wang , Ruiyu Zhang , Yingjie Li , Yitao Duan This is my paper · ORCID
classification cs.SDcs.CL
keywords zero-shotTTScross-lingualvoicecloningtranscript-freelearnablespeakerencoderself-supervisedspeechrepresentationsflowmatchingtext-to-semanticmultilingual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Confucius4-TTS is a multilingual zero-shot text-to-speech system that claims to clone an unseen speaker's voice from a short untranscribed reference clip in any of 14 languages. Its central thesis is that the reference transcript, which earlier systems need at inference time, can be replaced entirely by a speaker embedding extracted from self-supervised speech representations, without forced alignment or synthetic prompt pairs. The paper reports that this design reduces cross-lingual word error on the CV3-Eval benchmark to an average of 3.73% across six language directions and ranks first in overall human preference on three of four internal cross-lingual directions. A sympathetic reader should care because in-the-wild reference audio, especially for low-resource languages, is usually untranscribed; transcript-free cloning removes that bottleneck. The same model also offers a continuation-cloning mode when a transcript is available, trading slightly higher error for higher speaker similarity.

What carries the argument

The load-bearing mechanism is the learnable speaker encoder: an ECAPA-TDNN that consumes frame-level w2v-BERT 2.0 self-supervised representations of the reference audio and pools them into a fixed-size speaker embedding prepended to the T2S input sequence. Because the encoder is trained jointly with the T2S module and operates on SSL features, the embedding carries timbre without requiring the reference's transcript, forced alignment, or synthetic prompt pairs. It is what converts variable-length untranscribed audio into a clean conditioning vector that the rest of the pipeline can drive from.

What would settle it

Search the 500k-hour training corpus for speakers or utterances that also appear in CV3-Eval, X-Voice, Seed-TTS-eval, or MiniMax-MLS-Test (e.g., by speaker-embedding similarity); if overlap is found, re-run the reported WER and SIM measurements on a strictly disjoint held-out subset and check whether the margins over baselines survive.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a jointly trained speaker encoder operating on self-supervised speech representations (w2v-BERT 2.0 frames pooled by an ECAPA-TDNN) can serve as the sole conditioning signal for cross-lingual voice cloning, making the reference transcript unnecessary at inference time. The text-to-semantic module predicts semantic tokens from the target text plus this speaker embedding, while the semantic-to-acoustic module renders them into mel-spectrograms with conditional flow matching, a prompt mel-spectrogram, and a frozen speaker-verification embedding. The paper demonstrates the claim empirically: on CV3-Eval, the system reaches 3.73% average WER/CER across six directions, and on an internal cross-lingual set it attains the best average overall rank in a blinded human evaluation against recent open-source and commercial systems.

Load-bearing premise

The reported intelligibility and similarity numbers rest on the assumption that training and evaluation speech do not overlap at the speaker or utterance level, because the evaluation benchmarks are partly built from the same public corpora as the roughly 500k-hour training set and the paper does not state whether de-duplication was performed.

Editorial extensions

If this is right

  • Transcript-free cloning works across 14 languages, so in-the-wild audio without any annotation can be used as a voice reference.
  • On CV3-Eval, six directional pairs average 3.73% WER/CER, with particularly large intelligibility gains for Japanese and Korean references into Chinese (4.87% and 1.28% vs. 48.10% and 7.70% for CosyVoice 2).
  • Continuation cloning, conditioning on the reference transcript and its semantic tokens, consistently raises speaker similarity on MiniMax-MLS-Test across all 11 evaluated languages at a small intelligibility cost.
  • The human evaluation ranks the system first overall in three of four internal cross-lingual directions (en→zh, zh→ko, zh→ja) and second in the fourth (zh→en).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the SSL-based speaker encoder is the real carrier of cross-lingual transfer, a testable extension is whether the same architecture clones voices in languages outside the 14 training languages whenever w2v-BERT 2.0 provides coverage; the paper does not claim this, but the mechanism suggests it.
  • Since reference cloning trails continuation cloning in speaker similarity, a hybrid that conditionally includes reference semantic tokens only when the target prosody should match the reference could tighten the similarity–flexibility trade-off; that is an editorial suggestion, not the paper's result.
  • The transcript-free design eliminates forced alignment and synthetic prompt pairs from the training pipeline, which implies lower data-preparation cost for new languages; measuring that reduction directly would be a natural follow-up.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents Confucius4-TTS, a two-stage multilingual zero-shot TTS system (text-to-semantic and semantic-to-acoustic) that claims transcript-free voice cloning in both intra-lingual and cross-lingual settings across 14 languages. The T2S module uses a jointly trained ECAPA-TDNN speaker encoder over w2v-BERT 2.0 features, while the S2A module uses conditional flow matching with a DiT backbone. The authors report strong objective results on CV3-Eval, X-Voice, Seed-TTS-eval, and MiniMax-MLS-Test, plus a human evaluation on an internal set, and they release code, checkpoints, and demos. The central claim is that the model generalizes to unseen speakers without reference transcripts, with continuation cloning available when a reference transcript is present.

Significance. If the claims hold, this is a useful open-source contribution to the increasingly competitive area of multilingual zero-shot TTS. The paper's strengths are its clear architectural description, the use of several public benchmarks and multiple baseline systems, and the stated intent to release reproducible code and checkpoints. The transcript-free design via a learnable speaker encoder is a plausible and timely direction, and the reported gains on cross-lingual intelligibility, if validated, would be of interest to both practitioners and researchers. However, the significance is conditional on resolving the training/evaluation overlap concern, which affects all headline numbers.

major comments (3)
  1. [§4.1 and §4.2–§4.4] The paper does not state whether the approximately 500k-hour training corpus was de-duplicated against the public benchmarks used for evaluation. CV3-Eval, X-Voice, Seed-TTS-eval, and MiniMax-MLS-Test are built partly from Common Voice, FLEURS, and DiDiSpeech, yet Section 4.1 describes only a general filtering pipeline and does not list corpus sources or mention speaker- or utterance-level de-duplication. Since the central zero-shot claim depends on evaluation speakers being unseen during training, this is a load-bearing omission. The authors should either explicitly state the de-duplication procedure and its granularity, or re-run the evaluation on a benchmark guaranteed to be disjoint from training.
  2. [§3.3, Table 1, and Eq. (3)] Continuation cloning is presented as an inference capability of the same model, but the training sequence in Eq. (1) and the objective in Eq. (3) only cover the reference-cloning layout [e_r; H_txt; BOS] -> y. Table 1 introduces the continuation layout [e_r; H_txt,r; H_txt; BOS; y_r] -> y, yet the paper does not describe any training data, loss, or fine-tuning procedure that would teach the model to condition on a reference transcript and reference semantic tokens. Without such a description, the claim that 'the same model also supports continuation cloning' is not supported, and the continuation-cloning results in Tables 4 and 5 do not have a demonstrated mechanism. Please clarify whether the model was trained with both layouts or state that continuation cloning is a separate recipe requiring additional training.
  3. [Tables 2–6] No confidence intervals, significance tests, or inter-rater agreement statistics are reported anywhere, and the human evaluation in Section 4.5 does not state the number of raters or test utterances per condition. Several claims are based on small margins, for example the zh→en pronunciation rank in Table 6 (1.85 for Confucius4-TTS vs. 1.65 for VoxCPM2) and several WER differences below 0.3 points in Tables 2 and 3. Without uncertainty quantification, the claim of 'the best average overall rank' in human evaluation is not statistically established. I would ask the authors to add error bars or significance tests, and at minimum report the number of raters and items.
minor comments (5)
  1. [Abstract and Table 2] The abstract reports an 'average WER of 3.73% across six directions' on CV3-Eval, but Table 2 mixes CER (for Chinese targets) and WER (for English targets). Averaging these metrics into a single percentage is misleading; please report the CER and WER averages separately or state the mixed nature explicitly.
  2. [§4.1] The phrase 'roughly 1,000 hours of synthetic speech per language' is ambiguous: with 14 languages this sums to about 14k hours, which is a small fraction of the 500k-hour total, but the paper does not state the exact total or the fraction of synthetic data. Please clarify whether the 1,000 hours is per language or in total.
  3. [§2.2] The paper uses inconsistent and visually odd spacing in system names such as 'V ALL-E' and 'X-V oice'. These should be normalized to 'VALL-E' and 'X-Voice' for readability.
  4. [§4.4 and Table 5] For MiniMax-MLS-Test, the paper says 'continuation cloning consistently improves SIM across all 11 languages,' but Table 5(b) shows that for German the improvement is only 0.775 to 0.777, which is within the expected noise of the metric. Please qualify this statement or provide uncertainty estimates.
  5. [§4.5] The human evaluation section does not specify how many raters judged each utterance, how many utterances per direction, or whether raters overlap across directions. This information is needed to interpret the average ranks in Table 6.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the system is evaluated on external benchmarks and the training objectives do not presuppose the reported results.

full rationale

The paper's central claim, transcript-free cross-lingual zero-shot voice cloning, is supported by external public benchmarks (CV3-Eval, X-Voice, Seed-TTS-eval, MiniMax-MLS-Test) and a blind human evaluation, with measured WER/CER and SIM metrics. The T2S training objective (Eq. 3) is autoregressive cross-entropy over semantic tokens, and the S2A objective (Eq. 7) is flow-matching L1 loss; neither objective contains the benchmark results, and no fitted parameter is renamed as a prediction. The learnable speaker encoder is trained jointly with the T2S module, but this is a standard architectural choice and the evaluation still tests generalization to unseen reference speakers. The continuation-cloning mode is explicitly an additional inference recipe, not the claimed transcript-free result, and it is marked separately in the tables. No self-citation chain is load-bearing: the authors cite prior systems for architectural components (ECAPA-TDNN, w2v-BERT 2.0, MaskGCT, CAM++, BigVGAN), but those components are external and independently established, and the paper does not invoke any uniqueness theorem from its own prior work. The only substantive concern is potential training/evaluation overlap, because the approximately 500k-hour training corpus is not stated to be de-duplicated against Common Voice, FLEURS, or DiDiSpeech speakers used in the benchmarks. That is a data-hygiene and generalization risk, not circular reasoning: it does not make the derivation equivalent to its inputs by construction, and the paper's equations and benchmark protocol are not self-referential. Accordingly, no circular step can be quoted and exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The system does not introduce new physical or mathematical entities. The free parameters are standard engineering choices in the TTS pipeline. The axioms are typical domain assumptions of large-scale speech synthesis, plus one ad hoc assumption about the untrained continuation-cloning mode.

free parameters (5)
  • CFG strength alpha = 0.7
    Inference-time guidance strength in Eq. (8), chosen by hand. It affects the trade-off between synthesis quality and speaker similarity.
  • Euler solver steps = 25
    Number of ODE integration steps for S2A inference, chosen as a quality/speed compromise.
  • ASR filtering threshold = 2.5% (CER or WER)
    Data curation cutoff in Section 4.1 that retains only easily transcribable utterances, biasing the training distribution.
  • Synthetic speech per language = ~1000 hours
    Amount of synthetic data added per language in Section 4.1 to cover short and underrepresented utterances.
  • Training corpus size = ~500k hours
    Scale of the mixed real and synthetic training corpus; it is an engineering choice, not a derived quantity.
assumptions (5)
  • domain assumption Self-supervised speech representations (w2v-BERT 2.0) encode enough speaker identity independent of content and language to support cross-lingual cloning.
    Invoked in Section 3.2: SSL representations capture both content and speaker information. The paper provides no controlled analysis of language or content invariance.
  • domain assumption Pre-trained components (w2v-BERT 2.0, Mistral tokenizer, MaskGCT codec, CAM++ SV model, BigVGAN vocoder) perform as intended in the pipeline.
    All are used as frozen modules; their failure modes are not assessed in this paper.
  • ad hoc to paper The autoregressive T2S model can handle the continuation-cloning input layout at inference even though it was trained only in the reference-cloning layout.
    Section 3.3 says the T2S model is trained in the reference-cloning layout, yet continuation cloning is run at inference. The distribution shift is not validated.
  • domain assumption Whisper large-v3 and Paraformer provide trustworthy WER/CER measurements across the 14 evaluated languages.
    Used for all intelligibility metrics in Section 4.1; ASR errors propagate into reported numbers.
  • domain assumption Human rank scores from Section 4.5 are reliable without reported sample size or inter-rater agreement.
    Average ranks are reported with no statistical validation, so significance is unknown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder." pith.science (2026). https://pith.science/paper/YLLEDNKZ

@misc{pith2026260811650,
  author       = {Pith},
  title        = {Pith review of: Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YLLEDNKZ}},
  note         = {Machine review of arXiv:2608.11650}
}
read the original abstract

Recent advances in zero-shot text-to-speech (TTS) have substantially improved speech quality and voice cloning fidelity. However, many zero-shot TTS systems still depend on audio prompt transcripts at inference time. This dependency limits cross-lingual voice cloning, since in-the-wild reference audio is often untranscribed. In this technical report, we present Confucius4-TTS, a multilingual zero-shot TTS system that supports 14 languages and performs both intra-lingual and cross-lingual reference cloning without requiring transcripts of audio prompts. Confucius4-TTS follows a two-stage architecture, consisting of text-to-semantic (T2S) and semantic-to-acoustic (S2A) modules. The LLM-based T2S module uses a learnable speaker encoder to extract timbre features from self-supervised speech representations, and the conditional flow-matching S2A module converts the predicted semantic tokens into mel-spectrograms. The same model also supports continuation cloning when a reference transcript is available. Confucius4-TTS is trained on large-scale multilingual speech data. It achieves high intelligibility and speaker similarity on public benchmarks. On the CV3-Eval cross-lingual benchmark, Confucius4-TTS obtains an average WER of 3.73% across six directions. On our internal cross-lingual set, it achieves the best average overall rank in human evaluation among recent open-source and commercial systems. We release code, model checkpoints, and demos at https://github.com/netease-youdao/Confucius4-TTS.

Figures

Figures reproduced from arXiv: 2608.11650 by the authors.

Figure 1
Figure 1. Overall architecture of Confucius4-TTS, comprising the text-to-semantic module, the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 19 canonical work pages

  1. [1]

    Seed-TTS: A family of high-quality versatile speech generation models.arXiv preprint arXiv:2406.02430, 2024

    Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, et al. Seed-TTS: A family of high-quality versatile speech generation models.arXiv preprint arXiv:2406.02430, 2024

    arXiv 2024

  2. [2]

    Better speech synthesis through scaling.arXiv preprint arXiv:2305.07243, 2023

    James Betker. Better speech synthesis through scaling.arXiv preprint arXiv:2305.07243, 2023. 9

    arXiv 2023

  3. [3]

    XTTS: a massively multilingual zero-shot text-to-speech model.arXiv preprint arXiv:2406.04904, 2024

    Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. XTTS: a massively multilingual zero-shot text-to-speech model.arXiv preprint arXiv:2406.04904, 2024

    arXiv 2024

  4. [4]

    YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone

    Edresson Casanova et al. YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone. InICML, 2022

    work page 2022

  5. [5]

    WavLM: Large-scale self-supervised pre-training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022

    Sanyuan Chen et al. WavLM: Large-scale self-supervised pre-training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022

    work page 2022

  6. [6]

    F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching.arXiv preprint arXiv:2410.06885, 2024

    Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching.arXiv preprint arXiv:2410.06885, 2024

    arXiv 2024

  7. [7]

    w2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre-training

    Yu-An Chung, Yu Zhang, Wei Han, Chung-Cheng Chiu, James Qin, Ruoming Pang, and Yonghui Wu. w2v-BERT: Combining contrastive learning and masked language modeling for self-supervised speech pre-training. InIEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 244–250, 2021

    work page 2021

  8. [8]

    IndexTTS: An industrial-level controllable and efficient zero-shot text-to-speech system.arXiv preprint arXiv:2502.05512, 2025

    Wei Deng, Siyi Zhou, Jingchen Shu, Jinchao Wang, and Lu Wang. IndexTTS: An industrial-level controllable and efficient zero-shot text-to-speech system.arXiv preprint arXiv:2502.05512, 2025

    arXiv 2025

Show all 45 references
  1. [9]

    ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN based speaker verification

    Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. ECAPA-TDNN: Emphasized channel attention, propagation and aggregation in TDNN based speaker verification. InInter- speech, 2020

    2020

  2. [10]

    CosyV oice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens.arXiv preprint arXiv:2407.05407, 2024

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. CosyV oice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens.arXiv preprint arXiv:2407.05407, 2024

    2024 arXiv

  3. [11]

    CosyV oice 3: Towards in-the-wild speech generation via scaling-up and post-training.arXiv preprint arXiv:2505.17589, 2025

    Zhihao Du, Changfeng Gao, Yuxuan Wang, Fan Yu, Tianyu Zhao, Hao Wang, Xiang Lv, Hui Wang, Chongjia Ni, Xian Shi, et al. CosyV oice 3: Towards in-the-wild speech generation via scaling-up and post-training.arXiv preprint arXiv:2505.17589, 2025

    2025 arXiv

  4. [12]

    CosyV oice 2: Scalable streaming speech synthesis with large language models.arXiv preprint arXiv:2412.10117, 2024

    Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al. CosyV oice 2: Scalable streaming speech synthesis with large language models.arXiv preprint arXiv:2412.10117, 2024

    2024 arXiv

  5. [13]

    ElevenLabs multilingual text-to-speech, 2024

    ElevenLabs. ElevenLabs multilingual text-to-speech, 2024. Commercial system, https: //elevenlabs.io/

    2024

  6. [14]

    SooHwan Eom, Hee Suk Yoon, Eunseop Yoon, Mark Hasegawa-Johnson, and Chang D. Yoo. Transcript-free flow-matching text-to-speech via speech feature conditioning. InInterspeech, 2026

    2026

  7. [15]

    Fish audio speech-2, 2024

    Fish Audio. Fish audio speech-2, 2024. Commercial system,https://fish.audio/

    2024

  8. [16]

    CV3-Eval: The cross-lingual evaluation benchmark of CosyV oice 3

    FunAudioLLM Team. CV3-Eval: The cross-lingual evaluation benchmark of CosyV oice 3. https://github.com/FunAudioLLM/CV3-Eval, 2025

    2025

  9. [17]

    Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition.Proceedings of Interspeech, 2022

    Zhifu Gao, Shiliang Zhang, Ian McLoughlin, and Zhijie Yan. Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition.Proceedings of Interspeech, 2022

    2022

  10. [18]

    MOSS-TTS technical report.arXiv preprint arXiv:2603.18090, 2026

    Yitian Gong, Botian Jiang, Yiwei Zhao, Yucheng Yuan, Kuangwei Chen, Yaozhou Jiang, Cheng Chang, Dong Hong, Mingshu Chen, Ruixiao Li, et al. MOSS-TTS technical report.arXiv preprint arXiv:2603.18090, 2026

    2026

  11. [19]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  12. [20]

    Qwen3-TTS technical report.arXiv preprint arXiv:2601.15621, 2026

    Hangrui Hu, Xinfa Zhu, Ting He, Dake Guo, Bin Zhang, Xiong Wang, Zhifang Guo, Ziyue Jiang, Hongkun Hao, Zishan Guo, et al. Qwen3-TTS technical report.arXiv preprint arXiv:2601.15621, 2026. 10

    2026 arXiv

  13. [21]

    Jiang et al

    Albert Q. Jiang et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  14. [22]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Eric Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. InInternational Conference on Machine Learning (ICML), 2024

    2024

  15. [23]

    V oicebox: Text-guided multilingual universal speech generation at scale

    Matthew Le et al. V oicebox: Text-guided multilingual universal speech generation at scale. arXiv preprint arXiv:2306.15687, 2023

    2023 arXiv

  16. [24]

    BigVGAN: A universal neural vocoder with large-scale training

    Sang-gil Lee et al. BigVGAN: A universal neural vocoder with large-scale training. InICLR, 2023

    2023

  17. [25]

    IndexTTS 2.5 technical report.arXiv preprint arXiv:2601.03888, 2026

    Yunpei Li, Xun Zhou, Jinchao Wang, Lu Wang, Yong Wu, Siyi Zhou, Yiquan Zhou, and Jingchen Shu. IndexTTS 2.5 technical report.arXiv preprint arXiv:2601.03888, 2026

    2026 arXiv

  18. [26]

    Flow matching for generative modeling

    Yaron Lipman et al. Flow matching for generative modeling. InICLR, 2023

    2023

  19. [27]

    Cross-lingual F5-TTS: Towards language-agnostic voice cloning and speech synthesis.arXiv preprint arXiv:2509.14579, 2025

    Qingyu Liu, Yushen Chen, Zhikang Niu, Chunhui Wang, Yunting Yang, Bowen Zhang, Jian Zhao, Pengcheng Zhu, Kai Yu, and Xie Chen. Cross-lingual F5-TTS: Towards language-agnostic voice cloning and speech synthesis.arXiv preprint arXiv:2509.14579, 2025

    2025

  20. [28]

    Zero-shot voice conversion with diffusion transformers.arXiv preprint arXiv:2411.09943, 2024

    Songting Liu. Zero-shot voice conversion with diffusion transformers.arXiv preprint arXiv:2411.09943, 2024

    2024 arXiv

  21. [29]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR, 2019

    2019

  22. [30]

    Matcha-TTS: A fast TTS architecture with conditional flow matching

    Shivam Mehta et al. Matcha-TTS: A fast TTS architecture with conditional flow matching. arXiv preprint arXiv:2309.03199, 2023

    2023 arXiv

  23. [31]

    Attentive statistics pooling for deep speaker embedding

    Koji Okabe, Takafumi Koshinaka, and Koichi Shinoda. Attentive statistics pooling for deep speaker embedding. InProceedings of Interspeech, 2018

    2018

  24. [32]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InICCV, 2023

    2023

  25. [33]

    Qwen-audio-3.0-tts: Freely controllable and highly robust speech synthesis with multi-stage training paradigm.arXiv preprint, 2026

    Qwen Team. Qwen-audio-3.0-tts: Freely controllable and highly robust speech synthesis with multi-stage training paradigm.arXiv preprint, 2026

    2026

  26. [34]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InICML, 2023

    2023

  27. [35]

    Language models are unsupervised multitask learners.OpenAI blog, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 2019

    2019

  28. [36]

    Chatterbox-TTS

    Resemble AI. Chatterbox-TTS. https://github.com/resemble-ai/chatterbox, 2025. GitHub repository

    2025

  29. [37]

    Natu- ralspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers

    Kai Shen, Zeqian Ju, Xu Tan, Eric Liu, Yichong Leng, Lei He, Tao Qin, Jiang Bian, et al. Natu- ralspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  30. [38]

    Neural codec language models are zero-shot text to speech synthesizers.arXiv preprint arXiv:2301.02111, 2023

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. Neural codec language models are zero-shot text to speech synthesizers.arXiv preprint arXiv:2301.02111, 2023

    2023 arXiv

  31. [39]

    CAM++: A fast and efficient network for speaker verification using context- aware masking.arXiv preprint arXiv:2303.00332, 2023

    Hui Wang et al. CAM++: A fast and efficient network for speaker verification using context- aware masking.arXiv preprint arXiv:2303.00332, 2023

    2023 arXiv

  32. [40]

    MaskGCT: Zero-shot text-to-speech with masked generative codec transformer.arXiv preprint arXiv:2409.00750, 2024

    Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. MaskGCT: Zero-shot text-to-speech with masked generative codec transformer.arXiv preprint arXiv:2409.00750, 2024

    2024 arXiv

  33. [41]

    X-V oice: Enabling everyone to speak 30 languages via zero-shot cross-lingual voice cloning.arXiv preprint arXiv:2605.05611, 2026

    Rixi Xu, Qingyu Liu, Haitao Li, Yushen Chen, Zhikang Niu, Yunting Yang, Jian Zhao, Ke Li, Berrak Sisman, Qinyuan Cheng, et al. X-V oice: Enabling everyone to speak 30 languages via zero-shot cross-lingual voice cloning.arXiv preprint arXiv:2605.05611, 2026. 11

    2026 arXiv

  34. [42]

    MiniMax-Speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder.arXiv preprint arXiv:2505.07916, 2025

    Bowen Zhang, Congchao Guo, Geng Yang, Hang Yu, Haozhe Zhang, Heidi Lei, Jialong Mai, Junjie Yan, Kaiyue Yang, Mingqi Yang, et al. MiniMax-Speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder.arXiv preprint arXiv:2505.07916, 2025

    2025 arXiv

  35. [43]

    IndexTTS2: A breakthrough in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech

    Siyi Zhou, Yiquan Zhou, Yi He, Xun Zhou, Jinchao Wang, Wei Deng, and Jingchen Shu. IndexTTS2: A breakthrough in emotionally expressive and duration-controlled auto-regressive zero-shot text-to-speech. InProceedings of the AAAI Conference on Artificial Intelligence, 2026

    2026

  36. [44]

    V oxCPM2 technical report.arXiv preprint arXiv:2606.06928, 2026

    Yixuan Zhou, Guoyang Zeng, Xin Liu, Xiang Li, Renjie Yu, Jiancheng Gui, Jiaheng Wu, Ziyang Wang, Xudong Shen, Runchuan Ye, et al. V oxCPM2 technical report.arXiv preprint arXiv:2606.06928, 2026

    2026 arXiv

  37. [45]

    OmniV oice: Towards omnilingual zero-shot text-to-speech with diffusion language models.arXiv preprint arXiv:2604.00688, 2026

    Han Zhu, Lingxuan Ye, Wei Kang, Zengwei Yao, Liyong Guo, Fangjun Kuang, Zhifeng Han, Weiji Zhuang, Long Lin, and Daniel Povey. OmniV oice: Towards omnilingual zero-shot text-to-speech with diffusion language models.arXiv preprint arXiv:2604.00688, 2026. 12

    2026 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.