Pith. sign in

REVIEW 2 major objections 7 minor 49 references

Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion

T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that expressive voice conversion improves when the content channel is forced to be linguistic via quantized multilingual speech units, and that the resulting system outperforms three VAE-based baselines in emotion and…

desk verdict A competent EVC integration with honest ablations, but the headline 'reduced source leakage' claim leans on an EER metric that shares its feature extractor with the model's own style encoder, so take that one number with a grain of salt. read the letter →

arxiv 2506.04013 v1 pith:H6AIBS54 submitted 2025-06-04 cs.SD cs.AIeess.AS

Seymanur Akti , Tuan Nam Nguyen , Alexander Waibel This is my paper
classification cs.SDcs.AIeess.AS
keywords expressivevoiceconversionsourcetimbreleakagedisentanglementdiscretespeechunitsmHuBERT-147mixed-layernormalizationzero-shotcross-lingual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Expressive voice conversion aims to move both who is speaking and how they sound from a target to a source utterance, and this paper's claim is that the job becomes easier when the content channel is forced to be purely linguistic. The authors adapt a FreeVC-style conditional variational autoencoder with three changes: quantized mHuBERT-147 discrete multilingual units as the content representation, mixed-style layer normalization that randomizes the style conditioning during training, and an augmentation-based similarity loss that pins content embeddings down across pitch perturbations. They also inject local F0 via cross-attention and enrich a single ECAPA-TDNN style embedding with global pitch and energy features. On ESD, Expresso, and LibriTTS test sets, the model reports higher emotion and speaker similarity than Hierspeech++, Consistency-VC, and X-E-Speech, with a lower equal-error rate on LibriTTS signalling reduced source speaker leakage. The payoff, if true, is that zero-shot expressive voice conversion can transfer emotions across speakers and even across languages without parallel emotional corpora, though the paper acknowledges a word-error-rate cost from the discrete bottleneck.

What carries the argument

The carrying object is the discrete-unit content bottleneck: mHuBERT-147 produces quantized tokens meant to encode linguistic content while discarding speaker and prosodic detail, and the system's premise is that such quantization removes more non-linguistic information than continuous features. Around that bottleneck sit three mechanisms that do the work: Mix-LN, which computes normalization scale and bias as $\gamma_{mix}(s)=\lambda\gamma(s)+(1-\lambda)\gamma(e_s)$ and $\beta_{mix}(s)=\lambda\beta(s)+(1-\lambda)\beta(e_s)$ with $e_s$ a batch-shuffled style embedding, making content embeddings style-agnostic; the perturbation-based similarity loss $L_{sim}=(1-\cos(e,e_{aug}))+(1-\cos(c,c_{aug}))$; and a local F0 encoder fused with content embeddings by multi-head cross-attention, plus a global style embedding $s=\lambda_{mel}e_{mel}+\lambda_{f0}e_{f0}+\lambda_{energy}e_{energy}$. The normalizing flow maps the posterior spectrogram distribution to the prior content distribution during training, and at inference the reverse flow produces style-injected latent codes for the HiFi-GAN waveform generator.

What would settle it

A concrete test: train the proposed model with the local F0 cross-attention and the global F0/energy branches disabled, keeping identical content units and style embedding, and measure emotion-embedding cosine similarity (EECS) on Expresso. If EECS stays close to the reported 85.3%, then pitch side channels are not what carries expressivity, contradicting the paper's mechanism; if it falls well below, the discrete units and style embedding are not sufficient for emotion transfer on their own.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that nearly all of the expressivity and speaker-transfer gains come from tightening the information bottleneck in three coordinated places: quantized mHuBERT-147 units for content, Mix-LN that mixes a sample's style embedding with a batch-shuffled one, and a similarity loss that keeps content embeddings of original and pitch-augmented audio close. Local F0 enters through cross-attention rather than summation, and the style embedding is a weighted sum of ECAPA-TDNN mel, F0, and energy embeddings. The headline numbers are 85.3% versus 82.0% emotion-embedding cosine similarity (EECS) on Expresso, 81.2% versus 73.1% speaker similarity (SECS) on Expresso, and 7.4% versus 14.6% equal-error rate (EER) on LibriTTS relative to Hierspeech++, which the authors read as better style adaptation and less source timbre leakage. Ablations attribute the gains to each added component, with Mix-LN raising intra-speaker style-embedding similarity from 0.719 to 0.748.

Load-bearing premise

The whole argument rides on the assumption that quantized mHuBERT-147 units strip away speaker and style information while still holding onto enough linguistic detail for synthesis; if those units also silently discard prosodic or phonetic content, the emotion transfer must come from the F0 and energy side channels and the disentanglement claim weakens.

Editorial extensions

If this is right

  • Zero-shot emotion transfer to unseen speakers improves: on Expresso and LibriTTS the model raises emotion and speaker embedding similarity while lowering the source-speaker equal-error rate.
  • Cross-lingual expressive conversion works without target-language training: an English-trained model transfers emotions to German (EmoDB) with higher ECA than baselines, though German-to-English intelligibility drops.
  • The ablation shows each component is load-bearing: F0 cross-attention, Mix-LN, and the similarity loss each raise emotion/speaker similarity and reduce leakage; Mix-LN also makes style embeddings more content-agnostic.
  • The price of the discrete bottleneck is intelligibility: word error rate rises to 7.98% versus Hierspeech++'s 5.01%, a trade-off the paper attributes to quantization removing some linguistic cues.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recipe — a discrete-unit bottleneck plus a perturbation-based invariance loss — could transfer to accent conversion or singing voice conversion, where source-leakage is the same failure mode and the loss depends only on content-embedding stability.
  • The cross-lingual result suggests a weaker form of zero-shot language transfer; a testable extension is training on non-English emotional data and checking whether the style encoder stays language-agnostic, since the German conversions here come with a large WER penalty.
  • The paper's own WER numbers place the method on a trade-off frontier: a hybrid that interpolates between mHuBERT discrete units and continuous features such as MMS could test whether disentanglement, rather than quantization alone, is what drives the gain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper proposes a zero-shot expressive voice conversion (EVC) system based on a conditional variational autoencoder. The key contributions are: using mHuBERT-147 discrete speech units as a content bottleneck to reduce source timbre leakage, applying mix-style layer normalization (Mix-LN) and an augmentation-based similarity loss to improve linguistic–acoustic disentanglement, fusing local F0 via cross-attention, and enriching a global style embedding with pitch and energy features. Experiments on ESD, Expresso, LibriTTS, and EmoDB compare against three VAE-based baselines (Consistency-VC, X-E-Speech, Hierspeech++), reporting objective metrics (WER, SECS, EECS, ECA, EER) and a small subjective MOS evaluation. The paper also demonstrates cross-lingual EVC from English to German and reports a component-wise ablation study.

Significance. If the reported results are reliable, the paper offers a useful combination of existing techniques for zero-shot EVC and introduces mHuBERT-147 discrete units to this task, with a plausible mechanism for reducing source leakage. The ablation study is directionally consistent with each claimed contribution, and the inclusion of a demo page and objective metrics across multiple datasets is a strength. However, the central claims of reduced source leakage and better disentanglement are weakened by two evaluation issues: the source-leakage metric shares its feature extractor with the proposed style encoder, and the content bottleneck causes substantially higher word error rates than all baselines, calling into question whether the disentanglement is achieved by preserving content or by discarding it.

major comments (2)
  1. [Section 4.1, Eq. (10), Table 1] The EER metric used to support the claim of reduced source style leakage is computed with the ECAPA-TDNN speaker verification model of reference [19], which is the same architecture family as the proposed style encoder in Eq. (10). Because the synthesized speech is conditioned on an ECAPA-derived target embedding, an ECAPA-based verifier will naturally judge the output as close to the target and far from the source, potentially inflating the reported improvement over Hierspeech++ (7.4% vs. 14.6% on LibriTTS). The paper should include an independent speaker-verifier evaluation (e.g., a different architecture such as WavLM-TDNN) or a direct source-leakage metric such as SECS between the synthesized audio and the source speaker, to support the headline claim.
  2. [Section 3.1, Tables 1 and 4] The discrete mHuBERT-147 content bottleneck leads to substantially higher WER than all baselines: 7.98% vs. 5.01% on ESD, 8.84% vs. 3.48% on LibriTTS, and 30.84% on German-to-English conversions. The paper acknowledges this as a limitation, but it also undermines the 'linguistic-acoustic disentanglement' claim in the abstract and title. If the content channel demonstrably discards usable linguistic and prosodic information, the model may achieve style transfer by degrading content rather than by cleanly separating it. The authors should report a content-preservation metric beyond WER (e.g., character error rate or intelligibility MOS) and discuss how the proposed system can be considered better disentangled when the content channel is so lossy.
minor comments (7)
  1. [Equation (4)] Define e_aug and c_aug explicitly; the text says 'content embeddings from the original and augmented samples remain close,' but the augmented versions of the unit embeddings and content embeddings are not formally introduced.
  2. [Section 3.1] The notation e_s for batch-wise shuffled style embeddings is easily confused with the unit embeddings e; consider using a different symbol such as s_shuf.
  3. [Tables 1 and 2] The caption or text should clarify that the 'Proposed' row in Table 2 is trained for 300k steps, while the 'Ours' row in Table 1 is trained for 1M steps, to prevent readers from comparing the two tables directly.
  4. [Section 4.2] The sentence 'In the zero-shot setting with unseen speakers from Expresso, our model shows slightly better emotion and speaker transfer' is vague; specify which metrics and numerical values support this statement.
  5. [Section 4.1, Table 3] The subjective evaluation uses only 15 users and no statistical significance tests; the authors should report the exact sample size per model per emotion and consider adding a significance test (e.g., Wilcoxon signed-rank) to support the MOS comparisons.
  6. [Section 3.5, Eq. (10)] Clarify whether the 512-dimensional embeddings are extracted separately from mel, F0, and energy contours and then fused, or whether the contours are concatenated before extraction.
  7. [Section 4.4, Table 4] The German-to-English WER of 30.84% is very high and limits the strength of the cross-lingual claim; a comparison with a model trained on German data, or at least a discussion of the expected degradation for an unseen language, would improve the presentation.

Circularity Check

1 steps flagged · score 3.0 of 10

Source-leakage claim rests on an EER computed with the same ECAPA-TDNN speaker-verifier family used as the model's style encoder; core emotion-transfer results are otherwise externally benchmarked.

  1. other [Section 3.5 (Eq. 10) and Section 4.1 (EER metric); results in Section 4.2, Table 1]
    "We use the ECAPA-TDNN model [19] for style encoding. ... EER: Equal error rate computed with a speaker verification model [19], with synthesized sample as query, source speech as negative, and target as positive. A lower EER signals better target speaker matching and less source speaker leakage."

    The headline claim of reduced source style leakage is supported by the 7.4% versus 14.6% EER on LibriTTS in Table 1. But the EER is computed with reference [19], the same ECAPA-TDNN model used as the proposed style encoder in Eq. 10. Because the decoder is conditioned on an ECAPA-derived target-style embedding, an ECAPA-based speaker verifier is biased to place the synthesized utterance near the target and away from the source, so low EER partly reflects matching the model's own conditioning representation rather than an independent measurement of source-timbre removal. The metric is not fully forced, since EER still depends on the whole generated waveform, but it is not an independent test of source leakage.

full rationale

The central derivation is self-contained: the model is trained with standard VAE/GAN losses (Eqs. 4-11) and compared against external baselines using external metrics (Whisper WER, Resemblyzer SECS, Emotion2Vec+ ECA/EECS). No predicted quantity is a fitted parameter renamed as a prediction, and the claimed emotion-transfer improvements are anchored to external benchmarks rather than to the model's own outputs. The one notable circular signal is evaluation-side: the EER used to support reduced source leakage is computed with the same ECAPA-TDNN model family used for style conditioning, so it partially measures how well the output matches the conditioning representation rather than an independent test of source-timbre removal. A minor self-citation [17] supports the discrete-unit premise, but the paper's own MMS ablation provides independent downstream evidence, so it does not rise to load-bearing circularity. Overall score 3.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

Nothing in the paper is derived; it is an architecture comparison. The claims rest on four domain assumptions and four unreported parameter sets. The most notable circular-adjacent dependency is the self-cited premise [17] (the authors' own SPSC 2024 study) that discrete units suppress speaker leakage, used as justification rather than re-tested here. The Mix-LN benefit is imported from GenerSpeech [18], a TTS paper. The style encoder assumes a single global embedding is content-agnostic, a premise the paper's own cosine-similarity measurement shows to be imperfect. No invented entities are introduced. The unreported hyperparameters (Beta shape for Mix-LN, fusion weights in Eq. 10, Parselmouth augmentation strength, F0 range) gate the ablation conclusions and reproduction.

free parameters (4)
  • Mix-LN Beta distribution shape = not reported
    Equations 1-3 sample lambda from a Beta distribution to mix shuffled style embeddings; the shape parameters control Mix-LN noise strength and are never reported, yet Mix-LN is a claimed contribution.
  • Style fusion weights lambda_mel, lambda_f0, lambda_energy = not reported
    Equation 10 fuses ECAPA, F0, and energy embeddings via trainable weights; the paper reports no learned values or sensitivity analysis, yet global F0 and energy enrichment is a claimed contribution.
  • Parselmouth augmentation parameters = not reported
    The similarity loss (Eq. 4) depends on hand-chosen pitch-range compression and pitch-shift amounts; without these values the L_sim ablation cannot be reproduced or interpreted.
  • F0 extraction range settings = not reported
    The Local F0 Encoder (Section 3.1) operates on frame-level F0 contours; the min/max frequency bounds and voicing thresholds are unstated, which matters for cross-speaker and cross-lingual conversion.
assumptions (5)
  • domain assumption Quantized mHuBERT-147 units remove non-linguistic information more effectively than continuous representations.
    Section 3.1 and the MMS ablation row in Table 2 rest on the authors' own prior finding [17]; it is an empirical premise, not re-derived here, and it drives the central leakage-reduction claim.
  • domain assumption Mix-LN from GenerSpeech [18] improves linguistic-acoustic disentanglement without corrupting content.
    Equations 1-3 import a method from text-to-speech style transfer; the paper assumes it transfers to voice conversion and that stochastic mixing does not erase linguistic units.
  • domain assumption A single global ECAPA-TDNN style embedding is content-agnostic enough for zero-shot cross-content transfer.
    Section 3.5 conditions the whole pipeline on one style vector; the paper's own measurement (cosine similarity 0.719 to 0.748 with Mix-LN) shows content contamination remains.
  • domain assumption Whisper WER and Emotion2Vec+ ECA are valid proxies for intelligibility and emotion of synthesized expressive speech.
    WER is computed on expressive, vocoded speech where ASR errors may reflect ASR brittleness rather than intelligibility; ECA inherits Emotion2Vec+ pretraining biases and thresholds.
  • standard math The KL divergence, normalizing flow, and GAN objectives (Eqs. 5-9) behave as in standard VITS/FreeVC training.
    The method leans on the standard VAE-ELBO-plus-adversarial recipe from VITS [14] and FreeVC [15]; the paper provides no check that the flow prior and posterior remain well-behaved with the F0 cross-attention modification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion." pith.science (2026). https://pith.science/paper/H6AIBS54

@misc{pith2026250604013,
  author       = {Pith},
  title        = {Pith review of: Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H6AIBS54}},
  note         = {Machine review of arXiv:2506.04013}
}
read the original abstract

Expressive voice conversion aims to transfer both speaker identity and expressive attributes from a target speech to a given source speech. In this work, we improve over a self-supervised, non-autoregressive framework with a conditional variational autoencoder, focusing on reducing source timbre leakage and improving linguistic-acoustic disentanglement for better style transfer. To minimize style leakage, we use multilingual discrete speech units for content representation and reinforce embeddings with augmentation-based similarity loss and mix-style layer normalization. To enhance expressivity transfer, we incorporate local F0 information via cross-attention and extract style embeddings enriched with global pitch and energy features. Experiments show our model outperforms baselines in emotion and speaker similarity, demonstrating superior style adaptation and reduced source style leakage.

Figures

Figures reproduced from arXiv: 2506.04013 by the authors.

Figure 1
Figure 1. Overall architecture of the proposed system. 3. Methodology We follow the architecture of FreeVC [15] and adapt it for EVC with significant modifications as overall architecture illustrated in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 38 canonical work pages

  1. [13]

    Simultaneous translation of open do- main lectures and speeches,

    A. Waibel and C. Fuegen, “Simultaneous translation of open do- main lectures and speeches,” Jan. 3 2012, uS Patent 8,090,570

    work page 2012

  2. [17]

    Nonpar- allel emotional speech conversion using vae-gan

    Y . Cao, Z. Liu, M. Chen, J. Ma, S. Wang, and J. Xiao, “Nonpar- allel emotional speech conversion using vae-gan.” inInterspeech, 2020

    work page 2020

  3. [19]

    Hiervst: Hierar- chical adaptive zero-shot voice style transfer,

    S.-H. Lee, H.-Y . Choi, H.-S. Oh, and S.-W. Lee, “Hiervst: Hierar- chical adaptive zero-shot voice style transfer,”Interspeech, 2023

    work page 2023

  4. [1]

    Conventional VC models perform well in replicating speaker identity but struggle when the target speech is highly expressive

    Introduction V oice conversion (VC) approaches aim to transform a source audio by transferring speaker characteristics from a target audio while preserving the source content. Conventional VC models perform well in replicating speaker identity but struggle when the target speech is highly expressive. Expressive voice con- version (EVC) expands on this by ...

    work page

  5. [2]

    These models typically required text supervi- sion

    Related Work EVC is a speech-to-speech task, where early approaches often relied on auto-regressive models trained with parallel speech data [8, 20]. These models typically required text supervi- sion. To reduce reliance on transcripts, [11] proposed a textless approach that extracts discrete speech units directly from au- dio and learns a translation net...

    work page

  6. [3]

    Methodology We follow the architecture of FreeVC [15] and adapt it for EVC with significant modifications as overall architecture illustrated in Fig. 1. The aim is to learn two distributions: one for linguis- tic featuresp=N(µ p, σp)and one for spectrogram features q=N(µ q, σq). During training, the normalizing flow maps the posterior distributionqto the ...

    work page

  7. [4]

    All datasets are English and total duration is around 228 hours with more than 920 speakers

    Experiments and Results For training, we used a combination of LibriTTS-100 [33], ESD [34] (English only), subset of GigaSpeech [35], and Ex- presso [36]. All datasets are English and total duration is around 228 hours with more than 920 speakers. We used 2 NVIDIA A600 GPUs for training with batch size of 64 for 1M steps. For the ablation study, we traine...

    work page

  8. [5]

    Conclusion In this work, we proposed a novel zero-shot EVC framework that enhances linguistic and acoustic feature disentanglement to particularly reduce source style leakage. Our approach inte- grates F0 injection with cross-attention, Mix-LN, mHuBERT- 147 units, perturbation-based similarity loss, and style embed- dings enriched with F0 and energy conto...

    work page

Show all 49 references
  1. [6]

    Acknowledgements The authors gratefully acknowledge support from the Ger- man Federal Ministry of Education and Research (BMBF) un- der grant 01EF1803B (RELATER), European Union’s Hori- zon research and innovation programme under grant 101135798 (Meetween), and KIT Campus Tran...

  2. [7]

    Styles2st: Zero-shot style transfer for direct speech-to- speech translation,

    K. Song, Y . Ren, Y . Lei, C. Wang, K. Wei, L. Xie, X. Yin, and Z. Ma, “Styles2st: Zero-shot style transfer for direct speech-to- speech translation,”Interspeech, 2023

    2023

  3. [8]

    Chil: Computers in the human interaction loop,

    A. Waibel, H. Steusloff, R. Stiefelhagenet al., “Chil: Computers in the human interaction loop,” 2005

    2005

  4. [9]

    Towards an open-domain social dialog system,

    M. Schmidt, J. Niehues, and A. Waibel, “Towards an open-domain social dialog system,”Dialogues with Social Robots: Enable- ments, Analyses, and Evaluation, pp. 271–278, 2017

    2017

  5. [10]

    Face-dubbing++: Lip-synchronous, voice preserv- ing translation of videos,

    A. Waibel, M. Behr, D. Yaman, F. I. Eyiokur, T.-N. Nguyen, C. Mullov, M. A. Demirtas, A. Kantarci, S. Constantin, and H. K. Ekenel, “Face-dubbing++: Lip-synchronous, voice preserv- ing translation of videos,” inICASSP Workshops, 2023

    2023

  6. [11]

    Findings of the iwslt 2024 evaluation campaign,

    I. S. Ahmad, A. Anastasopoulos, O. Bojar, C. Borg, M. Carpuat, R. Cattoni, M. Cettolo, W. Chen, Q. Dong, M. Federicoet al., “Findings of the iwslt 2024 evaluation campaign,”arXiv preprint arXiv:2411.05088, 2024

    2024 arXiv

  7. [12]

    and adapts it for jointly trained TTS and cross-lingual EVC and [29] introduces prosody extraction and fusing methods for improving VITS for EVC task. 1https://seymanurakti.github.io/evc/ arXiv:2506.04013v1 [cs.SD] 4 Jun 2025 Embedding Layer e eaug Mix Encoder 𝜇𝑝 𝜎𝑝 z' Posteri...

    2025 arXiv

  8. [14]

    Seamless: Multilingual expressive and streaming speech translation,

    L. Barrault, Y .-A. Chung, M. C. Meglioli, D. Dale, N. Dong, M. Duppenthaler, P.-A. Duquenne, B. Ellis, H. Elsahar, J. Haa- heimet al., “Seamless: Multilingual expressive and streaming speech translation,”arXiv preprint arXiv:2312.05187, 2023

    2023 arXiv

  9. [15]

    Limited data emotional voice conversion leveraging text-to-speech: Two-stage sequence-to- sequence training,

    K. Zhou, B. Sisman, and H. Li, “Limited data emotional voice conversion leveraging text-to-speech: Two-stage sequence-to- sequence training,”Interspeech, 2021

    2021

  10. [16]

    Nonparallel emotional speech conversion,

    J. Gao, D. Chakraborty, H. Tembine, and O. Olaleye, “Nonparallel emotional speech conversion,”Interspeech, 2019

    2019

  11. [18]

    Textless speech emotion conversion using discrete and decom- posed representations,

    F. Kreuk, A. Polyak, J. Copet, E. Kharitonov, T.-A. Nguyen, M. Rivi `ere, W.-N. Hsu, A. Mohamed, E. Dupoux, and Y . Adi, “Textless speech emotion conversion using discrete and decom- posed representations,”EMNLP, 2022

    2022

  12. [20]

    Expressive-vc: Highly expressive voice conversion with attention fusion of bottleneck and perturbation features,

    Z. Ning, Q. Xie, P. Zhu, Z. Wang, L. Xue, J. Yao, L. Xie, and M. Bi, “Expressive-vc: Highly expressive voice conversion with attention fusion of bottleneck and perturbation features,” in ICASSP, 2023

    2023

  13. [21]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” inICML, 2021

    2021

  14. [22]

    Freevc: Towards high-quality text-free one-shot voice conversion,

    J. Li, W. Tu, and L. Xiao, “Freevc: Towards high-quality text-free one-shot voice conversion,” inICASSP, 2023

    2023

  15. [23]

    mhubert-147: A compact multilingual hubert model,

    M. Z. Boito, V . Iyer, N. Lagos, L. Besacier, and I. Calapode- scu, “mhubert-147: A compact multilingual hubert model,”In- terspeech, 2024

    2024

  16. [24]

    V oice privacy- investigating voice conversion architecture with different bottle- neck features,

    S. Akti, T. N. Nguyen, Y . Liu, and A. Waibel, “V oice privacy- investigating voice conversion architecture with different bottle- neck features,” inProc. SPSC 2024, 2024, pp. 44–49

    2024

  17. [25]

    Generspeech: Towards style transfer for generalizable out-of-domain text-to- speech,

    R. Huang, Y . Ren, J. Liu, C. Cui, and Z. Zhao, “Generspeech: Towards style transfer for generalizable out-of-domain text-to- speech,”NeurIPS, 2022

    2022

  18. [26]

    Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,

    B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,” inInterspeech, 2020

    2020

  19. [27]

    Emotion intensity and its control for emotional voice conversion,

    K. Zhou, B. Sisman, R. Rana, B. W. Schuller, and H. Li, “Emotion intensity and its control for emotional voice conversion,”IEEE Transactions on Affective Computing, vol. 14, no. 1, 2022

    2022

  20. [28]

    Accent conversion using pre-trained model and synthesized data from voice conver- sion

    T.-N. Nguyen, N.-Q. Pham, and A. Waibel, “Accent conversion using pre-trained model and synthesized data from voice conver- sion.” inInterspeech, 2022

    2022

  21. [29]

    Improving pronunciation and accent conversion through knowledge distilla- tion and synthetic ground-truth from native tts,

    T. N. Nguyen, S. Akti, N. Q. Pham, and A. Waibel, “Improving pronunciation and accent conversion through knowledge distilla- tion and synthetic ground-truth from native tts,” inICASSP, 2025

    2025

  22. [30]

    Stargan for emo- tional speech conversion: Validated by data augmentation of end- to-end emotion recognition,

    G. Rizos, A. Baird, M. Elliott, and B. Schuller, “Stargan for emo- tional speech conversion: Validated by data augmentation of end- to-end emotion recognition,” inICASSP, 2020

    2020

  23. [31]

    Emocat: Language-agnostic emotional voice conver- sion,

    B. Schnell, G. Huybrechts, B. Perz, T. Drugman, and J. Lorenzo- Trueba, “Emocat: Language-agnostic emotional voice conver- sion,”arXiv preprint arXiv:2101.05695, 2021

    2021 arXiv

  24. [32]

    V oice conversion with just nearest neighbors,

    M. Baas, B. van Niekerk, and H. Kamper, “V oice conversion with just nearest neighbors,”Interspeech, 2023

    2023

  25. [33]

    Disentangling prosody representations with unsupervised speech reconstruction,

    L. Qu, T. Li, C. Weber, T. Pekarek-Rosin, F. Ren, and S. Wermter, “Disentangling prosody representations with unsupervised speech reconstruction,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023

    2023

  26. [34]

    Using joint train- ing speaker encoder with consistency loss to achieve cross-lingual voice conversion and expressive voice conversion,

    H. Guo, C. Liu, C. T. Ishi, and H. Ishiguro, “Using joint train- ing speaker encoder with consistency loss to achieve cross-lingual voice conversion and expressive voice conversion,” inASRU, 2023

    2023

  27. [35]

    X-e-speech: Joint training framework of non- autoregressive cross-lingual emotional text-to-speech and voice conversion,

    ——, “X-e-speech: Joint training framework of non- autoregressive cross-lingual emotional text-to-speech and voice conversion,” inInterspeech, 2024

    2024

  28. [36]

    Zse-vits: A zero-shot expressive voice cloning method based on vits,

    J. Li and L. Zhang, “Zse-vits: A zero-shot expressive voice cloning method based on vits,”Electronics, vol. 12, no. 4, 2023

    2023

  29. [37]

    Hierspeech++: Bridging the gap between semantic and acoustic representation of speech by hierarchical variational inference for zero-shot speech synthesis,

    S.-H. Lee, H.-Y . Choi, S.-B. Kim, and S.-W. Lee, “Hierspeech++: Bridging the gap between semantic and acoustic representation of speech by hierarchical variational inference for zero-shot speech synthesis,”arXiv preprint arXiv:2311.12454, 2023

    2023 arXiv

  30. [38]

    Vits2: Improving quality and efficiency of single-stage text-to-speech with adversarial learning and architecture design,

    J. Kong, J. Park, B. Kim, J. Kim, D. Kong, and S. Kim, “Vits2: Improving quality and efficiency of single-stage text-to-speech with adversarial learning and architecture design,”arXiv preprint arXiv:2307.16430, 2023

    2023 arXiv

  31. [39]

    Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,”Advances in neural information processing systems, vol. 33, 2020

    2020

  32. [40]

    Libritts: A corpus derived from librispeech for text- to-speech,

    H. Zen, V . Dang, R. Clark, Y . Zhang, R. J. Weiss, Y . Jia, Z. Chen, and Y . Wu, “Libritts: A corpus derived from librispeech for text- to-speech,”Interspeech, 2019

    2019

  33. [41]

    Seen and unseen emo- tional style transfer for voice conversion with a new emotional speech dataset,

    K. Zhou, B. Sisman, R. Liu, and H. Li, “Seen and unseen emo- tional style transfer for voice conversion with a new emotional speech dataset,” inICASSP, 2021

    2021

  34. [42]

    Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,

    G. Chen, S. Chai, G. Wang, J. Du, W.-Q. Zhang, C. Weng, D. Su, D. Povey, J. Trmal, J. Zhanget al., “Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio,” arXiv preprint arXiv:2106.06909, 2021

    2021 arXiv

  35. [43]

    Ex- presso: A benchmark and analysis of discrete expressive speech resynthesis,

    T. A. Nguyen, W.-N. Hsu, A. d’Avirro, B. Shi, I. Gat, M. Fazel- Zarani, T. Remez, J. Copet, G. Synnaeve, M. Hassidet al., “Ex- presso: A benchmark and analysis of discrete expressive speech resynthesis,”arXiv preprint arXiv:2308.05725, 2023

    2023 arXiv

  36. [44]

    Robust speech recognition via large-scale weak su- pervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak su- pervision,” inICML, 2023

    2023

  37. [45]

    Emotion2vec: Self-supervised pre-training for speech emotion representation,

    Z. Ma, Z. Zheng, J. Ye, J. Li, Z. Gao, S. Zhang, and X. Chen, “Emotion2vec: Self-supervised pre-training for speech emotion representation,”arXiv preprint arXiv:2312.15185, 2023

    2023 arXiv

  38. [46]

    The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english,

    S. R. Livingstone and F. A. Russo, “The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english,”PloS one, vol. 13, no. 5, 2018

    2018

  39. [47]

    A comparison of discrete and soft speech units for improved voice conversion,

    B. Van Niekerk, M.-A. Carbonneau, J. Za ¨ıdi, M. Baas, H. Seut ´e, and H. Kamper, “A comparison of discrete and soft speech units for improved voice conversion,” inICASSP, 2022

    2022

  40. [48]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandiet al., “Scaling speech technology to 1,000+ languages,”Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    2024

  41. [49]

    A database of german emotional speech

    F. Burkhardt, A. Paeschke, M. Rolfes, W. F. Sendlmeieret al., “A database of german emotional speech.” inInterspeech, 2005

    2005

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.