A FreeVC-style conditional VAE with mHuBERT-147 discrete units, mixed-style layer normalization, an augmentation similarity loss, and F0 cross-attention reports better emotion transfer and less source leakage than three baselines, at the cost of a higher word error rate.
VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Design
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Single-stage text-to-speech models have been actively studied recently, and their results have outperformed two-stage pipeline systems. Although the previous single-stage model has made great progress, there is room for improvement in terms of its intermittent unnaturalness, computational efficiency, and strong dependence on phoneme conversion. In this work, we introduce VITS2, a single-stage text-to-speech model that efficiently synthesizes a more natural speech by improving several aspects of the previous work. We propose improved structures and training mechanisms and present that the proposed methods are effective in improving naturalness, similarity of speech characteristics in a multi-speaker model, and efficiency of training and inference. Furthermore, we demonstrate that the strong dependence on phoneme conversion in previous works can be significantly reduced with our method, which allows a fully end-to-end single-stage approach.
citation-role summary
background 1
citation-polarity summary
fields
cs.SD 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion
A FreeVC-style conditional VAE with mHuBERT-147 discrete units, mixed-style layer normalization, an augmentation similarity loss, and F0 cross-attention reports better emotion transfer and less source leakage than three baselines, at the cost of a higher word error rate.