REVIEW 23 cited by
Better speech synthesis through scaling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In recent years, the field of image generation has been revolutionized by the application of autoregressive transformers and DDPMs. These approaches model the process of image generation as a step-wise probabilistic processes and leverage large amounts of compute and data to learn the image distribution. This methodology of improving performance need not be confined to images. This paper describes a way to apply advances in the image generative domain to speech synthesis. The result is TorToise -- an expressive, multi-voice text-to-speech system. All model code and trained weights have been open-sourced at https://github.com/neonbjb/tortoise-tts.
Forward citations
Cited by 23 Pith papers
-
FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model
FlexiSLM is the first spoken language model supporting dynamic and controllable frame rates on speech input and output, outperforming fixed-rate 7B models at high quality and enabling faster inference at lower rates l...
-
Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis
Sarashina2.2-TTS achieves SOTA kanji reading accuracy via data scaling and Joyo-kanji-targeted synthesis, introduces the Joyo Kanji Yomi Benchmark and Kana-CER metric, and shows stable cross-lingual performance.
-
An Evaluation Framework for Text-to-Speech Voice Reconstruction
The paper introduces a subjective-objective evaluation framework using Best Worst Scaling and a novel dual-reference distributional measure to better assess intelligibility versus speaker identity trade-offs in TTS vo...
-
X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning
X-Voice achieves zero-shot cross-lingual voice cloning across 30 languages by using IPA as a unified phonetic representation and a two-stage training process that first generates its own audio prompts then fine-tunes ...
-
Making Separation-First Multi-Stream Audio Watermarking Feasible via Joint Training
Jointly training the watermark embedder/detector with the source separator enables ~1% bit-error-rate recovery of per-stem watermarks after mixing and separation, where independent training yields 15–35%.
-
Next Tokens Denoising for Speech Synthesis
Dragon-FM generates speech autoregressively over two-second chunks while using flow matching inside each chunk, achieving fast synthesis at 12.5 discrete audio tokens per second.
-
SpeechFake: A Large-Scale Multilingual Speech Deepfake Dataset Incorporating Cutting-Edge Generation Methods
SpeechFake is a large-scale multilingual deepfake speech dataset with baseline experiments showing improved generalization to unseen generation methods.
-
TTS-1 Technical Report
A new TTS system family combines pre-training, supervised fine-tuning, and GRPO reinforcement learning with a 48 kHz codec to produce multilingual speech with in-context voice cloning.
-
Step-Audio 2 Technical Report
Step-Audio 2 integrates a latent audio encoder, reasoning-centric reinforcement learning, and discrete audio token generation into language modeling to deliver state-of-the-art performance on audio understanding and c...
-
DeePen: Penetration Testing for Audio Deepfake Detection
DeePen demonstrates that both production and academic audio deepfake detectors can be reliably deceived by simple signal processing attacks such as time-stretching or echo addition, with some attacks resistible via re...
-
Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
Seed-TTS models produce speech matching human naturalness and speaker similarity, with added controllability via self-distillation and reinforcement learning.
-
MLAAD: The Multi-Language Audio Anti-Spoofing Dataset
MLAAD provides a large-scale multi-language synthetic audio dataset for training and evaluating audio anti-spoofing models, showing better training performance than InTheWild and FakeOrReal and alternating superiority...
-
FlashTTS: Fast Streaming TTS with MTP Acceleration and X-pred Mean Flow Distillation
FlashTTS delivers a streaming TTS system using multi-track input processing and X-pred mean flow matching to reach 325 ms latency in two function evaluations while retaining zero-shot voice cloning.
-
X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning
X-Voice achieves zero-shot cross-lingual voice cloning across 30 languages via IPA-based training on 420K hours of data and a two-stage paradigm that synthesizes its own audio prompts for text-masked fine-tuning.
-
Enhancing Conversational TTS with Cascaded Prompting and ICL-Based Online Reinforcement Learning
A cascaded audio-prompting and ICL-based online RL method improves naturalness and expressivity in conversational TTS with reduced data needs.
-
Large Language Model Data Generation for Enhanced Intent Recognition in German Speech
LLM-generated German text data improves intent recognition for elderly German speakers, and the smaller German-focused LeoLM outperforms the much larger ChatGPT as a data generator.
-
Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space
A four-modality contrastive model with token-level alignment improves text, video, and newly introduced audio-to-motion retrieval on HumanML3D and KIT-ML.
-
Quantize More, Lose Less: Autoregressive Generation from Residually Quantized Speech Representations
QTTS models speech as sequences from a multi-codebook RVQ audio codec whose first codebook is trained with ASR supervision, aiming for higher-fidelity TTS than single-codebook systems.
-
De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attacks
Existing voice-protection perturbations succeed only against naive attackers; a phoneme-guided purification-refinement pipeline restores cloneability of protected speech for most VC models.
-
Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis
Rectified flows with a tunable sampling temperature offer the best naturalness-diversity trade-off among stochastic prosody predictors for text-to-speech.
-
StreamFlow: Streaming Flow Matching with Block-wise Guided Attention Mask for Speech Token Decoding
StreamFlow achieves streaming speech token decoding with 180 ms first-packet latency by using hierarchical block-wise attention masks in a DiT flow matching model.
-
JWB-DH-V1: Benchmark for Joint Whole-Body Talking Avatar and Speech Generation Version 1
A paper announcing a large-scale whole-body talking avatar benchmark and evaluation protocol, but with insufficient details to verify the dataset or the joint audio-video evaluation.
-
AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan
AT-ADD introduces standardized tracks and datasets for evaluating audio deepfake detectors on speech under real-world conditions and on diverse unknown audio types to promote generalization beyond speech-centric methods.
Discussion (0). Sign in to comment.