Pith. sign in

REVIEW 3 major objections 7 minor 44 references

RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RT-VC claims a real-time zero-shot voice conversion system that matches StreamVC's quality while cutting CPU latency to 61.4 ms, a 13.3 percent reduction.

desk verdict A credible streaming articulatory VC system whose headline latency win over StreamVC is not yet established as a like-for-like comparison. read the letter →

arxiv 2506.10289 v1 pith:WKHQCVTC submitted 2025-06-12 eess.AS cs.AI

Yisi Liu , Chenyang Wang , Hanjo Kim , Raniya Khan , Gopala Anumanchipalli This is my paper · ORCID
classification eess.AScs.AI
keywords voiceconversionzero-shotreal-timearticulatorycodingSPARCDDSPvocoderspeakerdisentanglementCPUlatency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RT-VC is a zero-shot voice conversion system that converts a new speaker's speech into a target speaker's voice without training on that speaker. The paper claims it achieves intelligibility, speaker similarity, and f0 consistency comparable to StreamVC, the current real-time state of the art, while reducing end-to-end CPU latency from 70.8 ms to 61.4 ms (13.3 percent). The low latency comes from a fully causal articulatory pipeline: a source extractor and acoustic-to-articulatory inverter predict vocal tract kinematics, and a differentiable DSP vocoder synthesizes speech from those features. If correct, this means real-time voice conversion on ordinary CPUs can be built on an articulatory, interpretable content representation rather than an opaque latent space.

What carries the argument

The load-bearing object is the articulatory feature space defined by Speech Articulatory Coding (SPARC): a speaker-agnostic, normalized set of vocal tract kinematics (EMA) that is supposed to carry content, plus a separate speaker embedding for timbre. Around it, three causal components make real-time operation possible: a source extractor that predicts pitch, periodicity, and loudness at 200 Hz; an EMA inverter that maps MFCCs to pseudo-EMA labels through 11 dilated causal convolutions; and a DDSP harmonic-plus-noise vocoder that synthesizes speech from these articulatory features, modulated by the speaker embedding via FiLM layers. The latency equation is $L = 32\ \mathrm{ms} + 15\ \mathrm{ms} + 14.4\ \mathrm{ms} = 61.4\ \mathrm{ms}$, where 32 ms is the spectral feature lookahead, 15 ms is the input chunk size, and 14.4 ms is the measured average processing time per chunk on an Apple M3 CPU.

What would settle it

Measure RT-VC and StreamVC on the same CPU, with the same audio chunking and the same intelligibility, similarity, f0 consistency, and latency protocol; if RT-VC's end-to-end latency is not below StreamVC's on that common benchmark, or if its WER and speaker similarity are worse than the reported margins, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that a causal articulatory coding pipeline can match the quality of the best real-time zero-shot voice conversion system while being faster on CPU. RT-VC builds on the Speech Articulatory Coding (SPARC) idea: linguistic content is encoded as normalized, speaker-agnostic vocal tract kinematics (EMA), and speaker identity is captured separately by a speaker embedding, so swapping the embedding transfers voice while preserving content. To make this streamable, RT-VC trains a causal source extractor and a causal acoustic-to-articulatory inversion model using pseudo-EMA labels from SPARC, and uses a DDSP harmonic-plus-noise vocoder for synthesis. Evaluated on unseen LibriTTS speakers with VCTK targets, it reports WER 6.69 percent versus StreamVC's 6.22 percent, Resemblyzer similarity 76.65 percent versus 77.81 percent, f0 Pearson correlation 0.865 versus 0.842, and 61.4 ms CPU latency versus 70.8 ms, with naturalness scores above 3.8.

Load-bearing premise

The comparison assumes StreamVC's published metrics (70.8 ms latency on its own hardware, plus WER, CER, Resemblyzer, and f0 values) are directly comparable to RT-VC's self-measured numbers, even though the two systems were not evaluated under identical hardware and protocols and the naturalness metrics differ.

Editorial extensions

If this is right

  • Zero-shot voice conversion can run at interactive speed on CPU-only servers, making browser-based conversion with dynamic target switching practical.
  • The causal source extractor and EMA inverter form a streaming articulatory front-end that could be reused for real-time articulatory feedback or speech-to-EMA applications beyond voice conversion.
  • DDSP vocoding directly from articulatory features is enough to match a strong learned-vocoder baseline in intelligibility, similarity, and f0 consistency, so heavy neural vocoding is not necessary for real-time quality.
  • The same articulatory representation that carries content may also transfer pitch range naturally through the median-pitch ratio scaling, preserving prosody while changing timbre.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper compares its own latency measurement on an Apple M3 CPU to StreamVC's published 70.8 ms figure, a same-benchmark replication on identical hardware would be needed to know whether the 13.3 percent gap holds or shifts.
  • The pseudo-EMA labels come from an English-only, English-articulation probe, so conversion quality on non-English or heavily accented input likely degrades; a multilingual evaluation would test how far the articulatory representation really generalizes.
  • The reported noise robustness applies to static noise above 20 dB SNR; babble, reverberation, or lower-SNR conditions are plausible failure points and are natural test extensions.
  • If the articulatory front-end is speaker-independent enough, manipulating EMA coordinates directly could enable speech edits such as emotion, speaking-rate, or clarity control, which the paper does not attempt.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. RT-VC is a streaming, zero-shot voice conversion system that replaces the offline SPARC encoder with a causal source extractor and EMA inverter, and uses a causal DDSP vocoder conditioned on a target-speaker embedding. The paper evaluates RT-VC on 377 LibriTTS test utterances converted to 6 VCTK target speakers, reporting naturalness (UTMOS/MOS), intelligibility (WER/CER), speaker similarity (Resemblyzer/SMOS), f0 correlation, and CPU latency. The central claim is that RT-VC matches StreamVC's conversion quality while reducing CPU latency from 70.8 ms to 61.4 ms (13.3%). A noise-robustness experiment and a web demo are also presented.

Significance. The potential contribution is a practical real-time VC system built from articulatory coding, with a concrete latency figure and a working demo. The causal, streamable redesign of the SPARC encoder and the integration of a streaming DDSP vocoder are sensible engineering steps, and the noise-robustness evaluation is a useful addition. The authors are transparent about limitations, including the language-specific nature of the pseudo-EMA labels and the dependence on recording quality. However, the headline comparison with StreamVC is currently not a controlled experiment: latency is measured on different hardware and with a different latency decomposition, and the quality comparison lacks a direct naturalness measurement and statistical significance metrics. If these comparison issues are addressed, the system would be a solid contribution to real-time VC.

major comments (3)
  1. [§3.6, Eq. (2); Table 1] The headline claim of a 13.3% CPU latency reduction is not based on a like-for-like comparison. RT-VC's 61.4 ms is computed as 32 ms lookahead + 15 ms chunk size + 14.4 ms processing time measured on an Apple M3 CPU, while StreamVC's 70.8 ms is taken directly from its publication without specifying the hardware, measurement protocol, or the exact decomposition of that figure. Because processing time is hardware-dependent and the two systems use different lookahead and chunking definitions, the reported ordering could change if both systems were measured on the same machine. The paper should either run StreamVC under the same conditions or explicitly present the comparison as indicative and report StreamVC's measurement details.
  2. [§5.3, Table 1] The claim that RT-VC maintains synthesis quality comparable to StreamVC is not fully supported. Naturalness is not directly compared: StreamVC has no UTMOS or MOS values in the table, and the paper states it could not reproduce StreamVC's DNSMOS result. In addition, WER, CER, Resemblyzer, and f0 PCC are reported as point estimates without error bars or significance tests, so the small observed differences (e.g., WER +0.47%, Resemblyzer –1.16%) cannot be distinguished from evaluation noise. Please provide confidence intervals or statistical tests for these metrics, and either obtain a comparable naturalness score for StreamVC or explicitly restrict the 'comparable quality' claim to intelligibility, similarity, and f0 consistency.
  3. [§3.3 and §5.1] The causal EMA inverter is trained on pseudo-EMA labels generated by the SPARC model, and the DDSP vocoder is also from prior work by the same group. The evaluation therefore demonstrates that RT-VC can approximate the offline SPARC pipeline in a streaming setting, but it does not independently validate the articulatory feature space as a conversion representation. The paper should state this limitation explicitly in the discussion, since it bears on how the interpretability and grounding claims are interpreted; the comparison with StreamVC remains valid for the final audio output, but the causal chain from speech to articulatory features to speech is only as strong as the quality of the pseudo labels.
minor comments (7)
  1. [§3.6] The paper should report the real-time factor (RTF) or throughput on the measured CPU, not just the per-chunk processing time, to allow reproducibility and to clarify what 'real-time' means beyond a single latency figure.
  2. [§5.2] The sentence 'we adopt the same evaluation protocol for all metrics except for naturalness and speaker similarity' is ambiguous because the table lists both automatic and subjective speaker similarity; please clarify which metrics are directly comparable to StreamVC and which are new measurements.
  3. [Figure 4] The noise robustness curves lack error bars and the number of utterances per SNR condition is not reported, so the trends cannot be assessed for statistical reliability.
  4. [Table 1] Please use footnotes or formatting to mark which values are taken from the StreamVC publication and which are new measurements, and state the hardware used for the latency row.
  5. [§3.3] The 'information perturbation technique' is described but the citation to Choi et al. (2021) is not given at the point of use; please add the citation in that sentence.
  6. [Title and Section 1] The title and the first paragraph contain a spurious space in 'V oice'; please correct the typo.
  7. [§5.1] The claim that all target speakers are 'unseen during training' should clarify whether the pretrained WavLM backbone used in the speaker encoder may have seen VCTK speakers, since that affects the zero-shot interpretation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; RT-VC is an empirical system evaluation against an external baseline, not a derivation that reduces to its own inputs.

full rationale

RT-VC's central claims are empirical: conversion quality is measured with standard external metrics (WER, CER, UTMOS, MOS, Resemblyzer, SMOS, f0 PCC) and latency is computed from a stated additive formula. The causal EMA inverter is trained to minimize an L1 loss against pseudo-EMA labels produced by the authors' prior SPARC inversion model, but this is supervised fitting of a learned predictor, not the derivation of a claimed result from that same result. The DDSP vocoder and SPARC components are cited from prior work by the same group, but the paper does not use those citations as the evidence for its central conclusion; the conclusion rests on comparisons to StreamVC, an external system, using independent audio-quality metrics. No equation in the paper equates a predicted quantity to its fitting target by construction, and no fitted parameter is renamed as a prediction. The reliance on StreamVC's published latency (70.8 ms) and the inability to reproduce its DNSMOS value raise measurement-protocol and comparability concerns, but those are correctness risks, not circularity. Self-citation is present in the construction of the system, but it is not load-bearing in any circular sense.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The system's central behavior rests on assumptions inherited from prior articulatory coding work, especially the validity of the pseudo-EMA labels and the linear affine universal articulatory space. The hand-chosen window and chunk sizes directly shape the headline latency figure. No new physical or theoretical entities are introduced.

free parameters (5)
  • window_size = 1024 samples (64ms at 16kHz)
    Chosen for spectral feature extraction; half of it sets the 32ms lookahead that directly contributes to the reported latency.
  • chunk_size = 15ms
    Selected for streaming; directly appears in the latency formula L = t_lookahead + t_chunksize + t_processing.
  • speaker_embedding_dim = 128
    Architecture choice for the speaker encoder output; affects conversion quality but is not derived from a theory.
  • num_dilated_conv_layers = 11
    Used in both the EMA inverter and the DDSP vocoder; a design choice inherited from the SoundStream-like architecture.
  • processing_time_per_chunk = 14.4ms on Apple M3 CPU
    Empirically measured average processing time; used to compute the headline 61.4ms latency, but it is hardware-dependent and not a universal constant.
assumptions (7)
  • domain assumption SPARC's articulatory feature space provides a naturally disentangled and interpretable representation separating content from speaker identity.
    The entire system inherits this from Cho et al. 2024b; if the articulatory space is not truly disentangled, the voice conversion quality claims fail. Invoked in Sections 1 and 3.1.
  • domain assumption The pseudo-EMA labels produced by the SPARC acoustic-to-articulatory inversion model are accurate enough to serve as ground truth for training a causal EMA inverter.
    Section 3.3; the authors rely on these labels without validating them against real EMA data in this paper, and the Limitations section notes the inversion is English-centric.
  • standard math The DDSP vocoder from Liu et al. 2024 achieves fast inference and high quality when conditioned on articulatory features.
    Adopted as a component in Section 3.5; treated as a reliable external building block from prior publication.
  • domain assumption The linear affine approximation between individual speakers' articulatory systems is sufficient for a universal articulatory space.
    Inherited from Cho et al. 2024a,b and referenced in Section 2.2; used to justify cross-speaker content invariance.
  • ad hoc to paper Pitch rescaling by the ratio of median pitches of target and source speakers is an adequate conversion of prosody.
    Equation (1) in Section 3.1 introduces this heuristic without empirical validation of its effect on naturalness or similarity.
  • domain assumption StreamVC's published metrics are reliable and comparable to RT-VC's own measurements.
    Sections 5.2 and 5.3 compare directly against StreamVC's reported numbers; the paper itself questions the reliability of StreamVC's DNSMOS, so this assumption is fragile.
  • domain assumption Causal convolutional networks with dilation provide sufficient context to approximate non-causal feature extraction without notable quality loss.
    Implicit in Sections 3.3 and 3.5; the causal approximations must not degrade quality, but no direct comparison against the non-causal SPARC system is shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding." pith.science (2026). https://pith.science/paper/WKHQCVTC

@misc{pith2026250610289,
  author       = {Pith},
  title        = {Pith review of: RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKHQCVTC}},
  note         = {Machine review of arXiv:2506.10289}
}
read the original abstract

Voice conversion has emerged as a pivotal technology in numerous applications ranging from assistive communication to entertainment. In this paper, we present RT-VC, a zero-shot real-time voice conversion system that delivers ultra-low latency and high-quality performance. Our approach leverages an articulatory feature space to naturally disentangle content and speaker characteristics, facilitating more robust and interpretable voice transformations. Additionally, the integration of differentiable digital signal processing (DDSP) enables efficient vocoding directly from articulatory features, significantly reducing conversion latency. Experimental evaluations demonstrate that, while maintaining synthesis quality comparable to the current state-of-the-art (SOTA) method, RT-VC achieves a CPU latency of 61.4 ms, representing a 13.3\% reduction in latency.

Figures

Figures reproduced from arXiv: 2506.10289 by the authors.

Figure 1
Figure 1. Training and conversion pipeline of RT-VC. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. DDSP vocoder architecture. 3.5 DDSP Vocoder We adopt the DDSP harmonic-plus-noise vocoder from (Liu et al., 2024) to enable fast inference. The model architecture is shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Screenshot of the RT-VC web demo interface. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: WER and UTMOS against input SNR for three types of additive noise: white, brown, and pink. with noise while the target remains clean. Noisy source speech is generated by adding white, pink, and brown noise at various signal-to-noise ratios (SNRs) to the original uttera…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 22 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ahmed Adel Attia, Yashish M Siriwardena, and Carol Espy-Wilson. 2024. Improving speech inversion through self-supervised embeddings and enhanced tract variables. In 2024 32nd European Signal Processing Conference (EUSIPCO), pages 306--310. IEEE

    work page 2024

  4. [4]

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449--12460

    2020

  5. [5]

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505--1518

    2022

  6. [6]

    Yu-Wen Chen, Kuo-Hsuan Hung, Shang-Yi Chuang, Jonathan Sherman, Wen-Chin Huang, Xugang Lu, and Yu Tsao. 2021. Ema2s: An end-to-end multimodal articulatory-to-speech system. In IEEE International Symposium on Circuits and Systems (ISCAS), pages 1--5

    work page 2021

  7. [7]

    Cheol Jun Cho, Abdelrahman Mohamed, Alan W Black, and Gopala K Anumanchipalli. 2024 a . Self-supervised models of speech infer universal articulatory kinematics. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12061--12065. IEEE

    work page 2024

  8. [8]

    Cheol Jun Cho, Peter Wu, Abdelrahman Mohamed, and Gopala K Anumanchipalli. 2023. Evidence of vocal tract articulation in self-supervised learning of speech. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

    2023

Show all 44 references
  1. [9]

    Prabhune, Dhruv Agarwal, and Gopala K

    Cheol Jun Cho, Peter Wu, Tejas S. Prabhune, Dhruv Agarwal, and Gopala K. Anumanchipalli. 2024 b . Coding speech through vocal tract kinematics. IEEE Journal of Selected Topics in Signal Processing, 18(8):1427--1440

    2024

  2. [10]

    Hyeong-Seok Choi, Juheon Lee, Wansoo Kim, Jie Lee, Hoon Heo, and Kyogu Lee. 2021. Neural analysis and synthesis: Reconstructing speech from self-supervised representations. Advances in Neural Information Processing Systems, 34:16251--16265

    2021

  3. [11]

    Ju-chieh Chou, Cheng-chieh Yeh, and Hung-yi Lee. 2019. One-shot voice conversion by separating speaker and content representations with instance normalization. arXiv preprint arXiv:1904.05742

    2019 arXiv

  4. [12]

    Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. 2024 a . Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407

    2024 arXiv

  5. [13]

    Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, et al. 2024 b . Cosyvoice 2: Scalable streaming speech synthesis with large language models. arXiv preprint arXiv:2412.10117

    2024 arXiv

  6. [14]

    Yingming Gao, Peter Birkholz, and Ya Li. 2024. Articulatory copy synthesis based on the speech synthesizer vocaltractlab and convolutional recurrent neural networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2024

  7. [15]

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processin...

    2021

  8. [16]

    Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. 2024. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100

    2024 arXiv

  9. [17]

    Hirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka, and Nobukatsu Hojo. 2018. Stargan-vc: Non-parallel many-to-many voice conversion using star generative adversarial networks. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 266--273. IEEE

    2018

  10. [18]

    Takuhiro Kaneko and Hirokazu Kameoka. 2018. Cyclegan-vc: Non-parallel voice conversion using cycle-consistent adversarial networks. In 2018 26th European Signal Processing Conference (EUSIPCO), pages 2100--2104. IEEE

    2018

  11. [19]

    Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, and Nobukatsu Hojo. 2019 a . Cyclegan-vc2: Improved cyclegan-based non-parallel voice conversion. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6820--6824. IEEE

    2019

  12. [20]

    Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, and Nobukatsu Hojo. 2019 b . Stargan-vc2: Rethinking conditional methods for stargan-based voice conversion. arXiv preprint arXiv:1907.12279

    2019 arXiv

  13. [21]

    Anton Kashkin, Ivan Karpukhin, and Svyatoslav Shishkin. 2022. Hifi-vc: High quality asr-based voice conversion. arXiv preprint arXiv:2203.16937

    2022 arXiv

  14. [22]

    Jong Wook Kim, Justin Salamon, Peter Li, and Juan Pablo Bello. 2018. Crepe: A convolutional representation for pitch estimation. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 161--165. IEEE

    2018

  15. [23]

    Miseul Kim, Zhenyu Piao, Jihyun Lee, and Hong-Goo Kang. 2023. Style modeling for multi-speaker articulation-to-speech. In ICASSP, pages 1--5

    2023

  16. [24]

    Yuma Koizumi, Heiga Zen, Shigeki Karita, Yifan Ding, Kohei Yatabe, Nobuyuki Morioka, Michiel Bacchiani, Yu Zhang, Wei Han, and Ankur Bapna. 2023. Libritts-r: A restored multi-speaker text-to-speech corpus. arXiv preprint arXiv:2305.18802

    2023 arXiv

  17. [25]

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33:17022--17033

    2020

  18. [26]

    Jingyi Li, Weiping Tu, and Li Xiao. 2023. Freevc: Towards high-quality text-free one-shot voice conversion. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

    2023

  19. [27]

    Jiachen Lian, Chunlei Zhang, and Dong Yu. 2022. Robust disentangled variational speech representation learning for zero-shot voice conversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6572--6576. IEEE

    2022

  20. [28]

    Yisi Liu, Bohan Yu, Drake Lin, Peter Wu, Cheol Jun Cho, and Gopala Krishna Anumanchipalli. 2024. Fast, high-quality and parameter-efficient articulatory synthesis using differentiable dsp. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 711--718. IEEE

    2024

  21. [29]

    Seung-won Park, Doo-young Kim, and Myun-chul Joe. 2020. Cotatron: Transcription-guided speech encoder for any-to-many voice conversion without parallel data. arXiv preprint arXiv:2005.03295

    2020 arXiv

  22. [30]

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. 2018. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32

    2018

  23. [31]

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Mark Hasegawa-Johnson, and David Cox. 2020. Unsupervised speech decomposition via triple information bottleneck. In International Conference on Machine Learning, pages 7836--7846. PMLR

    2020

  24. [32]

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. 2019. Autovc: Zero-shot voice style transfer with only autoencoder loss. In International Conference on Machine Learning, pages 5210--5219. PMLR

    2019

  25. [33]

    Kaizhi Qian, Yang Zhang, Heting Gao, Junrui Ni, Cheng-I Lai, David Cox, Mark Hasegawa-Johnson, and Shiyu Chang. 2022. Contentvec: An improved self-supervised speech representation by disentangling speakers. In International conference on machine learning, pages 18003--18017. PMLR

    2022

  26. [34]

    Yashish M Siriwardena and Carol Espy-Wilson. 2023. The secret source: Incorporating source features to improve acoustic-to-articulatory speech inversion. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

    2023

  27. [35]

    Lifa Sun, Kun Li, Hao Wang, Shiyin Kang, and Helen Meng. 2016. Phonetic posteriorgrams for many-to-one voice conversion without parallel data training. In 2016 IEEE International Conference on Multimedia and Expo (ICME), pages 1--6. IEEE

    2016

  28. [36]

    Benjamin Van Niekerk, Marc-Andr \'e Carbonneau, Julian Za \" di, Matthew Baas, Hugo Seut \'e , and Herman Kamper. 2022. A comparison of discrete and soft speech units for improved voice conversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Sign...

    2022

  29. [37]

    Haojie Wei, Xueke Cao, Tangpeng Dan, and Yueguo Chen. 2023. Rmvpe: A robust model for vocal pitch estimation in polyphonic music. arXiv preprint arXiv:2306.15412

    2023 arXiv

  30. [38]

    Anumanchipalli

    Peter Wu, Li-Wei Chen, Cheol Jun Cho, Shinji Watanabe, Louis Goldstein, Alan W Black, and Gopala K. Anumanchipalli. 2023. Speaker-independent acoustic-to-articulatory speech inversion. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processi...

    2023

  31. [39]

    Peter Wu, Paul Pu Liang, Jiatong Shi, Ruslan Salakhutdinov, Shinji Watanabe, and Louis-Philippe Morency. 2021. Understanding the tradeoffs in client-side privacy for downstream speech tasks. In 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and C...

    2021

  32. [40]

    Peter Wu, Shinji Watanabe, Louis Goldstein, Alan W Black, and Gopala Krishna Anumanchipalli. 2022. Deep speech synthesis from articulatory representations. In Interspeech

    2022

  33. [41]

    Junichi Yamagishi, Christophe Veaux, and Kirsten MacDonald. 2019. https://doi.org/10.7488/ds/2645 Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92) . [sound]

    2019 doi

  34. [42]

    Yang Yang, Yury Kartynnik, Yunpeng Li, Jiuqiang Tang, Xing Li, George Sung, and Matthias Grundmann. 2024. https://arxiv.org/abs/2401.03078 Streamvc: Real-time low-latency voice conversion . Preprint, arXiv:2401.03078

    2024 arXiv

  35. [43]

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495--507

    2021

  36. [44]

    Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. Libritts: A corpus derived from librispeech for text-to-speech. arXiv preprint arXiv:1904.02882

    2019 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.