Pith. sign in

REVIEW 3 major objections 5 minor 66 references

Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read R-VC claims that zero-shot voice conversion can transfer both timbre and rhythm to an unseen target speaker while generating speech in as few as two sampling steps, matching or beating larger baselines in naturalness, intelligibility, and…

desk verdict Solid rhythm-controllable VC system, but the printed self-consistency equation is wrong as written; a referee must verify it before accepting the few-step claim. read the letter →

arxiv 2506.01014 v1 pith:QVUVDA5H submitted 2025-06-01 eess.AS cs.SD

Jialong Zuo , Shengpeng Ji , Minghui Fang , Mingze Li , Ziyue Jiang , Xize Cheng , Xiaoda Yang , Chen Feiyang
show 2 more authors
Xinyu Duan Zhou Zhao
This is my paper
classification eess.AScs.SD
keywords zero-shotvoiceconversionshortcutflowmatchingrhythmcontroldurationmodelingdiffusiontransformerHuBERTcontenttokensself-consistencytrainingfew-stepgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

R-VC claims that zero-shot voice conversion can transfer not only timbre but also rhythm to an unseen target speaker, and can do so in as few as two sampling steps. The system strips source speech down to deduplicated HuBERT content tokens, uses a masked generative transformer to re-time those tokens to the target speaker's speaking style, and then generates mel-spectrograms with a diffusion transformer trained by shortcut flow matching. On the paper's evaluations, R-VC matches the speaker similarity of larger baselines while improving word error rate, naturalness, and emotion style transfer, and it is about 2.8 times faster than its 10-step flow matching variant. If correct, this makes low-latency voice conversion that follows the target speaker's pacing rather than the source's.

What carries the argument

Shortcut flow matching is the efficiency engine: the decoder regresses a normalized shortcut $s(x_t, t, d)$ that points from the current noisy spectrogram to the next point after a step of size $d$, trained with an optimal-transport conditional flow matching loss at $d=0$ plus a self-consistency target $s(x_t,t,2d)=\frac12\bigl(s(x_t,t,d)+s(x'_{t+d},t,d)\bigr)$ so predictions stay consistent across step sizes. The other load-bearing piece is the masked generative transformer for duration, which uses mask-predict iterative decoding conditioned on deduplicated HuBERT tokens, unmasked context durations, and a speaker embedding to predict the unit durations that transfer the target's rhythm.

What would settle it

Inspect the released training loop for the self-consistency loss and verify that the second term of Eq. (7) uses time index $t+d$, i.e. $s(x'_{t+d}, t+d, d)$, and not $t$ as printed; if the code matches the printed form, retraining with that literal objective and measuring two-step word error rate, speaker similarity, and perceptual quality on the paper's test set would directly settle the claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that three design choices together yield a zero-shot voice converter that is both rhythm-controllable and efficient. First, perturbing the input waveform and discretizing HuBERT features with K-means, then deduplicating the tokens, removes timbre and prosody leakage from the content stream. Second, a non-autoregressive masked transformer predicts unit-level durations so the linguistic content is stretched or compressed to match the target speaker's rhythm; the paper reports 90.2% speaking-rate category accuracy and higher emotion transfer scores as evidence. Third, a Diffusion Transformer trained with shortcut flow matching conditions on the desired step size and a self-consistency loss, letting the model take large denoising jumps; the authors report two-step generation with a word error rate of 3.51 and speaker similarity essentially matching their ten-step decoder while running about 2.83 times faster.

Load-bearing premise

The two-step generation claim rests on the self-consistency identity in Eq. (7) being a valid training target; if its second term is evaluated at time $t$ instead of $t+d$ as printed, the shortcut target is mis-specified and the few-step results would not hold.

Editorial extensions

If this is right

  • Two-step R-VC reaches a real-time factor of 0.12 on a V100, making zero-shot voice conversion viable in latency-sensitive applications.
  • Because the duration model is trained on deduplicated content tokens, the same linguistic content can be re-timed to different target styles without retraining, demonstrated for slow, normal, and fast speaking rates.
  • Removing source prosody before content extraction reduces timbre leakage, so converted speech follows the target speaker's characteristics rather than the source's emotional style.
  • Shortcut flow matching keeps two-step WER within 0.04 points and speaker similarity within 0.001 of the ten-step variant, indicating few-step generation does not trade away intelligibility or similarity.
  • The 2.83x speedup over the ten-step CFM decoder closes the latency gap with non-diffusion voice converters while keeping quality near the vocoder upper bound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the self-consistency target is corrected and reproduced, the same step-size conditioning could speed up other flow-matching speech systems, such as text-to-speech and speech editing, without architectural change.
  • The deduplication-plus-duration-remodeling recipe suggests a general strategy for prosody transfer: any prosodic feature that survives token deduplication could in principle be re-targeted by a learned re-timing model.
  • The paper's rhythm-control evaluation measures only speaking rate; a natural extension would test finer prosodic contours such as pauses and stress, which would likely expose where unit-level duration prediction still errs.
  • Because the decoder separates a global speaker verification embedding from a masked target prompt, the model could be probed to see whether time-invariant timbre and time-varying style are truly independent in the generated audio.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes R-VC, a zero-shot voice conversion system that combines discrete HuBERT content tokens with data perturbation, a masked generative transformer for duration/rhythm modeling, and a diffusion-transformer decoder trained with shortcut flow matching. The central claim is that R-VC achieves state-of-the-art naturalness, intelligibility, and style transfer with comparable speaker similarity to larger models, while generating speech in as few as two sampling steps (a 2.83x speedup over 10-step CFM). The paper reports objective and subjective evaluations on LibriSpeech test-clean and the ESD dataset, plus ablations on the duration model, speaker conditioning, data perturbation, architecture, and the self-consistency mixing fraction.

Significance. If the method is correct, R-VC makes two useful contributions to zero-shot VC: explicit rhythm control via non-autoregressive duration modeling, and improved sampling efficiency via shortcut flow matching. The paper is generally well organized, includes informative ablations (Tables 4, 5, 8), reports both objective and subjective metrics, and provides a demo page with audio samples. The main technical risk is the self-consistency equation in Section 3.4 and Appendix B.2: as printed, it is mathematically inconsistent with the shortcut definition, which directly affects the load-bearing few-step generation claim. Because no code is released, the reader cannot determine whether the text is a typo or the implemented objective. With the identity corrected and the evaluation strengthened with error bars, the contributions would be credible and relevant to the speech synthesis community.

major comments (3)
  1. [Section 3.4, Eq. (7); Appendix B.2, Eq. (13)] The self-consistency identity has a wrong time index. From the shortcut definition x'_{t+d}=x_t+d*s(x_t,t,d), applying a second shortcut of size d gives x_t + d*s(x_t,t,d) + d*s(x'_{t+d}, t+d, d). Equating this to the 2d shortcut x_t + 2d*s(x_t,t,2d) yields s(x_t,t,2d)=1/2[s(x_t,t,d)+s(x'_{t+d},t+d,d)]. The printed equation and the starget formula in Appendix B.2 evaluate the second term at time t instead of t+d. If training uses the printed objective literally, the self-consistency target does not correspond to two consistent half-steps and the claimed NFE=2 performance is not a consequence of the stated loss; if it is a typo, the corrected identity must be stated explicitly for the method to be reproducible.
  2. [Section 4.2, Table 1; Section 4.3, Table 2] Objective metrics (WER, CER, SECS, UTMOS, EMO) are reported as point estimates without confidence intervals, bootstrap resampling, or significance tests. Several key comparisons rest on very small margins, e.g., R-VC NFE=2 has SECS 0.930 vs CosyVoice-VC 0.933 and UTMOS 4.10 vs 4.09; R-VC CFM WER 3.47 vs R-VC NFE=2 WER 3.51. Without variance estimates, the claim of 'surpassing' these baselines is not statistically supported. Please provide per-utterance error bars or bootstrap confidence intervals for all objective metrics.
  3. [Section 4.4, Table 3] The rhythm control evaluation lacks a baseline for comparison. The test set is imbalanced (2071 of 2620 samples are in the 'normal' class), so an always-normal predictor would already achieve 79.0% 'accuracy.' The per-class accuracies (86.0%, 92.3%, 80.7%) are more informative, but no error bars or significance tests are given, and no comparison is made to a source-rhythm baseline or to the duration model without speaker conditioning. Given that rhythm control is a central contribution, this analysis needs a stronger baseline and uncertainty quantification.
minor comments (5)
  1. [Section 3.5] There is a typo in 'shourtcut flow matching loss'; it should read 'shortcut.'
  2. [Section 4.2, text after Table 1] The text says the model surpasses baselines 'except inference latency,' but Table 1 shows R-VC NFE=2 has RTF 0.12 versus FACodec-VC 0.10; please clarify that R-VC is slightly slower than the fastest non-diffusion baseline while much faster than 10-step CFM.
  3. [Figure 3] The x-axis of Figure 3 should be labeled explicitly as the number of sampling steps (NFE), and the two curves should be distinguished in the legend with the same naming convention used in Table 1.
  4. [Abstract and Section 4.2] The abstract states the model 'surpasses [baselines] in terms of speech naturalness, intelligibility and style transfer performance,' but style transfer results appear only in Table 2 (ESD emotion transfer), not in Table 1; please cross-reference the correct table when making this claim.
  5. [Section 3.3] The iteration index t in the duration model equations (Eq. 3-5) conflicts with the continuous time t used in the shortcut flow matching section; consider renaming the iteration index to j or n to avoid confusion.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: R-VC's claims are grounded in external benchmarks and the shortcut objective is anchored by the d=0 flow-matching loss; self-citations are contextual rather than load-bearing.

full rationale

R-VC's central claims are not circular. The zero-shot quality claims are measured against external references: WavLM-based SECS, Whisper WER/CER, UTMOS, human QMOS/SMOS, Emotion2Vec emotion scores, and computed speaking-rate alignment; none of these metrics is a function of the model's own fitted parameters. The few-step generation claim rests on the shortcut self-consistency objective, which is a standard bootstrapping loss: the d=0 term is pinned to the OT-CFM vector field (x1-x0), so the self-consistency equation is not the sole target and the two-step output is not equivalent to a fitted input by construction. Self-citations (e.g., Zuo et al. 2025, Ji et al. 2024a, Ji et al. 2024b) appear in contextual descriptions of prompting, parallel decoding, and PPS metrics, but none is load-bearing for the main claims. The paper also includes an explicit limitation in Section 7 on duration-prediction instability, and Appendix B.2 Eq. (13) contains a likely time-index typo (the second shortcut should be evaluated at t+d, not t); that is a correctness and reproducibility risk, not a circularity. No code release is indicated, so the typo cannot be disambiguated from the manuscript alone, but this does not change the circularity verdict.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claims rest on standard pretrained components (HuBERT, WavLM, Whisper, HiFiGAN) plus a handful of hyperparameters chosen by hand. The most consequential premise is the self-consistency identity, which appears mis-stated in the paper. No new physical entities are introduced.

free parameters (5)
  • Number of ODE time steps N = 128
    Empirically chosen in Sec. 3.5 as smallest time unit for approximating the ODE; authors state smaller values degrade performance and they did not explore other configurations.
  • Self-consistency batch fraction k = 0.3
    Ablated in Appendix E; 30% chosen as trade-off between training efficiency and few-step performance.
  • Classifier-free guidance scale alpha = 0.7
    Set based on experimental results (Appendix C), not derived.
  • Masking schedule in duration model = p = sin(u), u~U[0, pi/2]
    Chosen sampling schedule; no comparison to alternatives.
  • Duration decoding iterations T = not specified
    The paper defines linear decay n = N*(T-t)/T but does not state the value of T used at inference.
assumptions (3)
  • domain assumption HuBERT K-means discrete tokens are a speaker-neutral representation of linguistic content
    The entire content pipeline relies on this; invoked in Sec. 3.2 and supported by external literature (Li et al., Van Niekerk et al.).
  • domain assumption Data perturbation (formant shifting, pitch randomization, equalizer) removes timbre and style information from the waveform
    Used in Sec. 3.2 and Appendix A; the paper provides no direct measurement of how much speaker/style information is removed.
  • ad hoc to paper Self-consistency property of shortcut models
    The training objective Eq. (7) assumes the identity s(xt,t,2d)=0.5(s(xt,t,d)+s(x'_{t+d},t,d)) holds; as written the time index is wrong, so this is an unverified premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching." pith.science (2026). https://pith.science/paper/QVUVDA5H

@misc{pith2026250601014,
  author       = {Pith},
  title        = {Pith review of: Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVUVDA5H}},
  note         = {Machine review of arXiv:2506.01014}
}
read the original abstract

Zero-Shot Voice Conversion (VC) aims to transform the source speaker's timbre into an arbitrary unseen one while retaining speech content. Most prior work focuses on preserving the source's prosody, while fine-grained timbre information may leak through prosody, and transferring target prosody to synthesized speech is rarely studied. In light of this, we propose R-VC, a rhythm-controllable and efficient zero-shot voice conversion model. R-VC employs data perturbation techniques and discretize source speech into Hubert content tokens, eliminating much content-irrelevant information. By leveraging a Mask Generative Transformer for in-context duration modeling, our model adapts the linguistic content duration to the desired target speaking style, facilitating the transfer of the target speaker's rhythm. Furthermore, R-VC introduces a powerful Diffusion Transformer (DiT) with shortcut flow matching during training, conditioning the network not only on the current noise level but also on the desired step size, enabling high timbre similarity and quality speech generation in fewer sampling steps, even in just two, thus minimizing latency. Experimental results show that R-VC achieves comparable speaker similarity to state-of-the-art VC methods with a smaller dataset, and surpasses them in terms of speech naturalness, intelligibility and style transfer performance.

Figures

Figures reproduced from arXiv: 2506.01014 by the authors.

Figure 1
Figure 1. Figure (a) illustrates the shortcut DiT-based flow matching decoder, which regresses the conditional [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The comparison of vanilla conditional flow [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Results with different inference steps. shown in [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The influence of references of different lengths [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: The UTMOS performance under different inference steps. manifesting in suboptimal generation results with fewer sampling steps similar to vanilla CFM. While increasing this proportion effectively enhances the model’s performance with fewer steps, it simulta￾neously intr…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 24 canonical work pages

  1. [1]

    Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick Von Platen, Yatharth Saraf, Juan Pino, et al. 2021. Xls-r: Self-supervised cross-lingual speech representation learning at scale. arXiv preprint arXiv:2111.09296

    arXiv 2021

  2. [2]

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449--12460

    2020

  3. [3]

    Zal \'a n Borsos, Rapha \"e l Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al. 2023 a . Audiolm: a language modeling approach to audio generation. IEEE/ACM transactions on audio, speech, and language processing, 31:2523--2533

    work page 2023

  4. [4]

    Zal \'a n Borsos, Matt Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. 2023 b . Soundstorm: Efficient parallel audio generation. arXiv preprint arXiv:2305.09636

    arXiv 2023

  5. [5]

    Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren G \"o lge, and Moacir A Ponti. 2022. Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. In International Conference on Machine Learning, pages 2709--2720. PMLR

    2022

  6. [6]

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. 2022. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315--11325

    work page 2022

  7. [7]

    Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. 2018. Neural ordinary differential equations. Advances in neural information processing systems, 31

    2018

  8. [8]

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505--1518

    2022

Show all 66 references
  1. [9]

    Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2024 a . F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arXiv:2410.06885

    2024 arXiv

  2. [10]

    Zhengyang Chen, Shuai Wang, Mingyang Zhang, Xuechen Liu, Junichi Yamagishi, and Yanmin Qian. 2024 b . Disentangling the prosody and semantic information with pre-trained model for in-context learning based zero-shot voice conversion. arXiv preprint arXiv:2409.05004

    2024 arXiv

  3. [11]

    Ha-Yeong Choi, Sang-Hoon Lee, and Seong-Whan Lee. 2023. Diff-hiervc: Diffusion-based hierarchical voice conversion with robust pitch generation and masked prior for zero-shot speaker adaptation. International Speech Communication Association, pages 2283--2287

    2023

  4. [12]

    Ha-Yeong Choi, Sang-Hoon Lee, and Seong-Whan Lee. 2024. Dddm-vc: Decoupled denoising diffusion models with disentangled representation and prior mixup for verified robust voice conversion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17862--17870

    2024

  5. [13]

    Hyeong-Seok Choi, Juheon Lee, Wansoo Kim, Jie Lee, Hoon Heo, and Kyogu Lee. 2021. Neural analysis and synthesis: Reconstructing speech from self-supervised representations. Advances in Neural Information Processing Systems, 34:16251--16265

    2021

  6. [14]

    Erica Cooper, Cheng-I Lai, Yusuke Yasuda, Fuming Fang, Xin Wang, Nanxin Chen, and Junichi Yamagishi. 2020. Zero-shot multi-speaker text-to-speech with state-of-the-art neural speaker embeddings. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal ...

    2020

  7. [15]

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. 2024. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407

    2024 arXiv

  8. [16]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  9. [17]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference o...

    2024

  10. [18]

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. 2024. One step diffusion via shortcut models. arXiv preprint arXiv:2410.12557

    2024 arXiv

  11. [19]

    Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. Mask-predict: Parallel decoding of conditional masked language models. arXiv preprint arXiv:1904.09324

    2019 arXiv

  12. [20]

    Yewei Gu, Zhenyu Zhang, Xiaowei Yi, and Xianfeng Zhao. 2021. Mediumvc: Any-to-any voice conversion using synthetic specific-speaker speeches as intermedium features. arXiv preprint arXiv:2110.02500

    2021 arXiv

  13. [21]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840--6851

    2020

  14. [22]

    Chin-Cheng Hsu, Hsin-Te Hwang, Yi-Chiao Wu, Yu Tsao, and Hsin-Min Wang. 2016. Voice conversion from non-parallel corpora using variational auto-encoder. In 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pages 1--6. IEEE

    2016

  15. [23]

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processin...

    2021

  16. [24]

    Shengpeng Ji, Ziyue Jiang, Hanting Wang, Jialong Zuo, and Zhou Zhao. 2024 a . Mobilespeech: A fast and high-fidelity framework for mobile zero-shot text-to-speech. arXiv preprint arXiv:2402.09378

    2024 arXiv

  17. [25]

    Shengpeng Ji, Jialong Zuo, Minghui Fang, Ziyue Jiang, Feiyang Chen, Xinyu Duan, Baoxing Huai, and Zhou Zhao. 2024 b . Textrolspeech: A text style control speech corpus with codec language text-to-speech models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Sp...

    2024

  18. [26]

    Ziyue Jiang, Jinglin Liu, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, et al. 2024. Mega-tts 2: Boosting prompting mechanisms for zero-shot speech synthesis. In The Twelfth International Conference on Learning Representations

    2024

  19. [27]

    Ziyue Jiang, Yi Ren, Ruiqi Li, Shengpeng Ji, Boyang Zhang, Zhenhui Ye, Chen Zhang, Bai Jionghao, Xiaoda Yang, Jialong Zuo, et al. 2025. Megatts 3: Sparse alignment enhanced latent diffusion transformer for zero-shot speech synthesis. arXiv preprint arXiv:2502.18924

    2025 arXiv

  20. [28]

    Ziyue Jiang, Qian Yang, Jialong Zuo, Zhenhui Ye, Rongjie Huang, Yi Ren, and Zhou Zhao. 2023. Fluentspeech: Stutter-oriented automatic speech editing with context-aware diffusion models. arXiv preprint arXiv:2305.13612

    2023 arXiv

  21. [29]

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. 2024. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100

    2024 arXiv

  22. [30]

    Heeseung Kim, Sungwon Kim, Jiheum Yeom, and Sungroh Yoon. 2023. Unitspeech: Speaker-adaptive speech synthesis with untranscribed data. arXiv preprint arXiv:2306.16083

    2023 arXiv

  23. [31]

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33:17022--17033

    2020

  24. [32]

    Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al. 2024. Voicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems, 36

    2024

  25. [33]

    Ann Lee, Hongyu Gong, Paul-Ambroise Duquenne, Holger Schwenk, Peng-Jen Chen, Changhan Wang, Sravya Popuri, Yossi Adi, Juan Pino, Jiatao Gu, et al. 2021. Textless speech-to-speech translation on real data. arXiv preprint arXiv:2112.08352

    2021 arXiv

  26. [34]

    Sang-Hoon Lee, Ha-Yeong Choi, Seung-Bin Kim, and Seong-Whan Lee. 2023. Hierspeech++: Bridging the gap between semantic and acoustic representation of speech by hierarchical variational inference for zero-shot speech synthesis. arXiv preprint arXiv:2311.12454

    2023 arXiv

  27. [35]

    Junjie Li, Yiwei Guo, Xie Chen, and Kai Yu. 2024. Sef-vc: Speaker embedding free zero-shot voice conversion with cross attention. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12296--12300. IEEE

    2024

  28. [36]

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. 2022. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747

    2022 arXiv

  29. [37]

    Eric Luhman and Troy Luhman. 2021. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388

    2021 arXiv

  30. [38]

    Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, Shiliang Zhang, and Xie Chen. 2023. emotion2vec: Self-supervised pre-training for speech emotion representation. arXiv preprint arXiv:2312.15185

    2023 arXiv

  31. [39]

    Shivam Mehta, Ruibo Tu, Jonas Beskow, \'E va Sz \'e kely, and Gustav Eje Henter. 2024. Matcha-tts: A fast tts architecture with conditional flow matching. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11341--11345. IEEE

    2024

  32. [40]

    Ziqian Ning, Qicong Xie, Pengcheng Zhu, Zhichao Wang, Liumeng Xue, Jixun Yao, Lei Xie, and Mengxiao Bi. 2023. Expressive-vc: Highly expressive voice conversion with attention fusion of bottleneck and perturbation features. In ICASSP 2023-2023 IEEE International Conference on A...

    2023

  33. [41]

    Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206--5210. IEEE

    2015

  34. [42]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195--4205

    2023

  35. [43]

    Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, Mikhail Kudinov, and Jiansheng Wei. 2021. Diffusion-based voice conversion with fast maximum likelihood sampling scheme. arXiv preprint arXiv:2109.13821

    2021 arXiv

  36. [44]

    Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. Mls: A large-scale multilingual dataset for speech research. arXiv preprint arXiv:2012.03411

    2020 arXiv

  37. [45]

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson, and David Cox. 2020. Unsupervised speech decomposition via triple information bottleneck. In International Conference on Machine Learning, pages 7836--7846. PMLR

    2020

  38. [46]

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. 2019. Autovc: Zero-shot voice style transfer with only autoencoder loss. In International Conference on Machine Learning, pages 5210--5219. PMLR

    2019

  39. [47]

    Kaizhi Qian, Yang Zhang, Heting Gao, Junrui Ni, Cheng-I Lai, David Cox, Mark Hasegawa-Johnson, and Shiyu Chang. 2022. Contentvec: An improved self-supervised speech representation by disentangling speakers. In International Conference on Machine Learning, pages 18003--18017. PMLR

    2022

  40. [48]

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. arXiv preprint arXiv:2204.02152

    2022 arXiv

  41. [49]

    David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, and Sanjeev Khudanpur. 2018. X-vectors: Robust dnn embeddings for speaker recognition. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5329--5333. IEEE

    2018

  42. [50]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063

    2024

  43. [51]

    Zhiyuan Tan, Jianguo Wei, Junhai Xu, Yuqing He, and Wenhuan Lu. 2021. Zero-shot voice conversion with adjusted speaker embeddings and simple acoustic features. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5964--5968. IEEE

    2021

  44. [53]

    Alexander Tong, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Kilian Fatras, Guy Wolf, and Yoshua Bengio. 2023 b . Conditional flow matching: Simulation-free dynamic optimal transport. arXiv preprint arXiv:2302.00482, 2(3)

    2023 arXiv

  45. [54]

    Aaron Van Den Oord, Oriol Vinyals, et al. 2017. Neural discrete representation learning. Advances in neural information processing systems, 30

    2017

  46. [55]

    Benjamin Van Niekerk, Marc-Andr \'e Carbonneau, Julian Za \" di, Matthew Baas, Hugo Seut \'e , and Herman Kamper. 2022 a . A comparison of discrete and soft speech units for improved voice conversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and S...

    2022

  47. [56]

    Benjamin Van Niekerk, Marc-Andr \'e Carbonneau, Julian Za \" di, Matthew Baas, Hugo Seut \'e , and Herman Kamper. 2022 b . A comparison of discrete and soft speech units for improved voice conversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and S...

    2022

  48. [57]

    Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023 a . Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111

    2023 arXiv

  49. [58]

    Zhichao Wang, Yuanzhe Chen, Lei Xie, Qiao Tian, and Yuping Wang. 2023 b . Lm-vc: Zero-shot voice conversion via speech generation based on language models. IEEE Signal Processing Letters

    2023

  50. [59]

    Da-Yi Wu, Yen-Hao Chen, and Hung-Yi Lee. 2020. Vqvc+: One-shot voice conversion by vector quantization and u-net architecture. arXiv preprint arXiv:2006.04154

    2020 arXiv

  51. [60]

    Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Xixin Wu, et al. 2023. Uniaudio: An audio foundation model toward universal audio generation. arXiv preprint arXiv:2310.00704

    2023 arXiv

  52. [61]

    Qian Yang, Jialong Zuo, Zhe Su, Ziyue Jiang, Mingze Li, Zhou Zhao, Feiyang Chen, Zhefeng Wang, and Baoxing Huai. 2024. Mscenespeech: A multi-scene speech dataset for expressive speech synthesis. arXiv preprint arXiv:2407.14006

    2024 arXiv

  53. [62]

    Jixun Yao, Yuguang Yang, Yi Lei, Ziqian Ning, Yanni Hu, Yu Pan, Jingjing Yin, Hongbin Zhou, Heng Lu, and Lei Xie. 2024. Promptvc: Flexible stylistic voice conversion in latent space driven by natural language prompts. In ICASSP 2024-2024 IEEE International Conference on Acoust...

    2024

  54. [63]

    Mingyang Zhang, Yi Zhou, Yi Ren, Chen Zhang, Xiang Yin, and Haizhou Li. 2024. Refxvc: Cross-lingual voice conversion with enhanced reference leveraging. IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2024

  55. [64]

    Kun Zhou, Berrak Sisman, Rui Liu, and Haizhou Li. 2021. Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 920--924. IEEE

    2021

  56. [65]

    Jialong Zuo, Shengpeng Ji, Minghui Fang, Ziyue Jiang, Xize Cheng, Qian Yang, Wenrui Liu, Guangyan Zhang, Zehai Tu, Yiwen Guo, et al. 2025. Enhancing expressive voice conversion with discrete pitch-conditioned flow matching model. arXiv preprint arXiv:2502.05471

    2025 arXiv

  57. [66]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  58. [67]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.