Pith. sign in

REVIEW 16 cited by

OpenVoice: Versatile Instant Voice Cloning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.01479 v6 pith:ZDIOIGTM submitted 2023-12-03 cs.SD cs.LGeess.AS

Zengyi Qin , Wenliang Zhao , Xumin Yu , Xin Sun This is my paper
classification cs.SDcs.LGeess.AS
keywords voiceopenvoicecloninglanguagesmassive-speakerreferencespeakerstyles
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce OpenVoice, a versatile voice cloning approach that requires only a short audio clip from the reference speaker to replicate their voice and generate speech in multiple languages. OpenVoice represents a significant advancement in addressing the following open challenges in the field: 1) Flexible Voice Style Control. OpenVoice enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, in addition to replicating the tone color of the reference speaker. The voice styles are not directly copied from and constrained by the style of the reference speaker. Previous approaches lacked the ability to flexibly manipulate voice styles after cloning. 2) Zero-Shot Cross-Lingual Voice Cloning. OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive-speaker training set. Unlike previous approaches, which typically require extensive massive-speaker multi-lingual (MSML) dataset for all languages, OpenVoice can clone voices into a new language without any massive-speaker training data for that language. OpenVoice is also computationally efficient, costing tens of times less than commercially available APIs that offer even inferior performance. To foster further research in the field, we have made the source code and trained model publicly accessible. We also provide qualitative results in our demo website. OpenVoice has been used by more than 2M users worldwide as the voice engine of MyShell.ai

Discussion (0). Sign in to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Poly-SVC: Polyphony-Aware Singing Voice Conversion with Harmonic Modeling

    cs.SD 2026-05 unverdicted novelty 7.0 of 10

    Poly-SVC converts singing voices from polyphonic recordings while keeping melody, lyrics, and harmonies by combining CQT-based pitch extraction with a conditional flow matching diffusion decoder.

  2. V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Data

    cs.CR 2026-04 unverdicted novelty 7.0 of 10

    V.O.I.C.E is a new taxonomy that organizes synthetic voice risks into five categories and shows how they interact with exposure, visibility, and legal context using empirical incident data.

  3. X-VC: Zero-shot Streaming Voice Conversion in Codec Space

    eess.AS 2026-04 unverdicted novelty 7.0 of 10

    X-VC achieves zero-shot streaming voice conversion via one-step codec-space conversion with dual-conditioning acoustic converter and role-assignment training on generated paired data.

  4. JUST-DUB-IT: Video Dubbing via Joint Audio-Visual Diffusion

    cs.GR 2026-01 unverdicted novelty 7.0 of 10

    JUST-DUB-IT adapts a joint audio-visual diffusion model via LoRA to generate high-quality dubbed videos with translated audio and lip-synced facial motion.

  5. EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection

    eess.AS 2025-10 unverdicted novelty 7.0 of 10

    EchoFake is a new replay-aware dataset combining zero-shot TTS deepfakes and physical replay recordings to improve generalization of speech deepfake detection models over existing lab-focused datasets.

  6. Cloned Voices, Real Consequences: Evaluating Bias in Political Deepfake Detection for Electoral Integrity in Brazil

    eess.AS 2026-07 conditional novelty 6.0 of 10

    State-of-the-art audio deepfake detectors severely degrade on Brazilian Portuguese political speech, and the main source of performance gaps is the synthesis method, not demographic traits.

  7. AV-SyncBench: Decoupled Benchmarking of Temporal and Semantic Audio-Visual Synchronization

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    AV-SyncBench is a new benchmark dataset of 3,269 videos that separates temporal and semantic audio-visual synchronization assessment across voice, music, and sound scenarios.

  8. An Evaluation Framework for Text-to-Speech Voice Reconstruction

    eess.AS 2026-06 unverdicted novelty 6.0 of 10

    The paper introduces a subjective-objective evaluation framework using Best Worst Scaling and a novel dual-reference distributional measure to better assess intelligibility versus speaker identity trade-offs in TTS vo...

  9. Eroding Trust in Real Speech: A Large-Scale Study of Human Audio Deepfake Perception

    cs.SD 2026-05 unverdicted novelty 6.0 of 10

    Large-scale listening study of 35,532 judgments finds human accuracy on real audio fell from 72.7% to 64.1% since 2021 while fake detection remained stable, indicating a skepticism shift toward genuine speech.

  10. MimicLM: Zero-Shot Voice Imitation through Autoregressive Modeling of Pseudo-Parallel Speech Corpora

    cs.SD 2026-04 unverdicted novelty 6.0 of 10

    MimicLM achieves better naturalness in zero-shot voice imitation by autoregressively modeling pseudo-parallel data with synthetic sources and real targets, plus interleaved text-audio guidance and preference alignment.

  11. SpeechFake: A Large-Scale Multilingual Speech Deepfake Dataset Incorporating Cutting-Edge Generation Methods

    cs.SD 2025-07 conditional novelty 6.0 of 10

    SpeechFake is a large-scale multilingual deepfake speech dataset with baseline experiments showing improved generalization to unseen generation methods.

  12. Zero-VC: Zero-Lookahead Streaming Voice Conversion via Speaker Anonymization

    cs.SD 2026-06 unverdicted novelty 5.0 of 10

    Zero-VC applies speaker anonymization as a perturbation to achieve strictly causal zero-lookahead streaming voice conversion by balancing timbre leakage against prosodic utility.

  13. Talking Slide Avatars: Open-Source Multimodal Communication Approach for Teaching

    cs.HC 2026-04 conditional novelty 5.0 of 10

    The study documents an open-source pipeline that converts text scripts and portraits into talking-head videos to restore narrative and expressive elements in slide-based online teaching.

  14. Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASR

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    Combining LLM-based elderly-contextual paraphrasing with TTS synthesis using elderly speakers reduces word error rates in elderly ASR by up to 58% over standard Whisper baselines on English and Korean datasets.

  15. CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models

    cs.SD 2024-12 unverdicted novelty 5.0 of 10

    CosyVoice 2 delivers human-parity naturalness and near-lossless streaming speech synthesis by combining finite-scalar quantization, a streamlined pre-trained LLM, and chunk-aware causal flow matching on large multilin...

  16. Talking Slide Avatars: Open-Source Multimodal Communication Approach for Teaching

    cs.HC 2026-04 unverdicted novelty 3.0 of 10

    Presents an open-source pipeline using OpenVoice and Ditto-TalkingHead to generate talking slide avatars from scripts and images, framed as a multimodal communication model with practical and ethical guidelines for ed...

Pith tools