REVIEW 7 cited by
StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Zero-shot voice conversion (VC) aims to transfer the timbre from the source speaker to an arbitrary unseen speaker while preserving the original linguistic content. Despite recent advancements in zero-shot VC using language model-based or diffusion-based approaches, several challenges remain: 1) current approaches primarily focus on adapting timbre from unseen speakers and are unable to transfer style and timbre to different unseen speakers independently; 2) these approaches often suffer from slower inference speeds due to the autoregressive modeling methods or the need for numerous sampling steps; 3) the quality and similarity of the converted samples are still not fully satisfactory. To address these challenges, we propose a style controllable zero-shot VC approach named StableVC, which aims to transfer timbre and style from source speech to different unseen target speakers. Specifically, we decompose speech into linguistic content, timbre, and style, and then employ a conditional flow matching module to reconstruct the high-quality mel-spectrogram based on these decomposed features. To effectively capture timbre and style in a zero-shot manner, we introduce a novel dual attention mechanism with an adaptive gate, rather than using conventional feature concatenation. With this non-autoregressive design, StableVC can efficiently capture the intricate timbre and style from different unseen speakers and generate high-quality speech significantly faster than real-time. Experiments demonstrate that our proposed StableVC outperforms state-of-the-art baseline systems in zero-shot VC and achieves flexible control over timbre and style from different unseen speakers. Moreover, StableVC offers approximately 25x and 1.65x faster sampling compared to autoregressive and diffusion-based baselines.
Forward citations
Cited by 7 Pith papers
-
StarVC: A Unified Auto-Regressive Framework for Joint Text and Speech Generation in Voice Conversion
StarVC is an autoregressive voice conversion model that generates text tokens before acoustic tokens, improving linguistic fidelity while retaining speaker similarity.
-
FlowDubber: Movie Dubbing with LLM-based Semantic-aware Learning and Flow Matching based Voice Enhancing
A dubbing system that combines an LLM, phoneme-level contrastive lip alignment, and flow matching with classifier-free guidance outperforms prior dubbing models on lip-sync and quality metrics.
-
MusFlow: Multimodal Music Generation via Conditional Flow Matching
MusFlow generates music from images, story texts, or captions by aligning all inputs into the CLAP audio embedding space and sampling with conditional flow matching.
-
REF-VC: Robust, Expressive and Fast Zero-Shot Voice Conversion with Diffusion Transformers
REF-VC is a zero-shot voice conversion system that random-erases redundant parts of speech-embedding features to stay robust to noise, and uses shortcut-distilled flow matching to convert speech in only four steps.
-
ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speech
ClapFM-EVC uses a contrastive emotion encoder and conditional flow matching to convert speech emotion from natural language prompts or reference audio, and reports state-of-the-art quality on a single-speaker Mandarin...
-
Enhancing Intelligibility for Generative Target Speech Extraction via Joint Optimization with Target Speaker ASR
A generative target-speech extractor trained jointly with a Whisper-based transcript-prediction loss achieves better intelligibility and competitive quality on Libri2Mix and WSJ0-2mix than discrete-token and mask-base...
-
EZ-VC: Easy Zero-shot Any-to-Any Voice Conversion
EZ-VC combines discrete units from a multilingual self-supervised encoder (Xeus) with an F5-TTS flow-matching decoder to achieve zero-shot any-to-any voice conversion, without text labels or multiple disentangling encoders.
Discussion (0). Continue with ORCID to comment.