CAST benchmark shows language models infer correct word stress from discourse context but TTS systems frequently fail to produce it in speech.
Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
We present an extension to the Tacotron speech synthesis architecture that learns a latent embedding space of prosody, derived from a reference acoustic representation containing the desired prosody. We show that conditioning Tacotron on this learned embedding space results in synthesized audio that matches the prosody of the reference signal with fine time detail even when the reference and synthesis speakers are different. Additionally, we show that a reference prosody embedding can be used to synthesize text that is different from that of the reference utterance. We define several quantitative and subjective metrics for evaluating prosody transfer, and report results with accompanying audio samples from single-speaker and 44-speaker Tacotron models on a prosody transfer task.
verdicts
UNVERDICTED 5representative citing papers
Forward-backward decoding with divergence regularization and bidirectional decoder improves end-to-end TTS robustness and naturalness by addressing exposure bias via joint L2R/R2L training.
M3D-GAN introduces modality subnets, a shared computing body, and a universal attention module to enable translation across text, image, and speech modalities and their internal domains.
The paper outlines considerations, analogies from other fields, and recommendations for ML research norms on synthetic media to mitigate misuse risks without prescribing specific policies.
A methodology is proposed for emotional text-to-speech using emotional data collection, transfer-learning-based annotation of expressiveness features, and fine-tuning of a neutral TTS model.
citing papers explorer
-
Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmark
CAST benchmark shows language models infer correct word stress from discourse context but TTS systems frequently fail to produce it in speech.
-
Forward-Backward Decoding for Regularizing End-to-End TTS
Forward-backward decoding with divergence regularization and bidirectional decoder improves end-to-end TTS robustness and naturalness by addressing exposure bias via joint L2R/R2L training.
-
M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention
M3D-GAN introduces modality subnets, a shared computing body, and a universal attention module to enable translation across text, image, and speech modalities and their internal domains.
-
Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning
The paper outlines considerations, analogies from other fields, and recommendations for ML research norms on synthetic media to mitigate misuse risks without prescribing specific policies.
-
A Methodology for Controlling the Emotional Expressiveness in Synthetic Speech -- a Deep Learning approach
A methodology is proposed for emotional text-to-speech using emotional data collection, transfer-learning-based annotation of expressiveness features, and fine-tuning of a neutral TTS model.