Training-Free Voice Conversion with Factorized Optimal Transport

Alexander Lobashev; Assel Yermekova; Maria Larchenko

REVIEW 3 major objections 5 minor 25 references

Training-Free Voice Conversion with Factorized Optimal Transport

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a factorized optimal transport map, replacing the nearest-neighbor search in the kNN-VC pipeline, gives high-quality any-to-any voice conversion from only 5–10 seconds of reference audio, with content preservation…

desk verdict A simple, training-free modification of kNN-VC that replaces nearest-neighbor regression with per-block linear optimal transport in sorted WavLM dimensions, and it delivers a large content-preservation win over kNN-VC on short references—though the evaluation protocol needs tightening. read the letter →

arxiv 2506.09709 v1 pith:Z633BMNE submitted 2025-06-11 cs.SD cs.CVcs.LGeess.AS

Alexander Lobashev , Assel Yermekova , Maria Larchenko This is my paper

classification cs.SDcs.CVcs.LGeess.AS

keywords voiceconversionoptimaltransportMonge-KantorovichLinearmapkNN-VCWavLMtraining-freecross-lingualcontentpreservation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the main weakness of kNN-VC—content quality collapsing when only a few seconds of reference audio are available—can be removed by replacing nearest-neighbor regression with a factorized optimal transport map. The map, built from the Monge-Kantorovich Linear solution for Gaussian distributions, is applied block-wise to WavLM embedding dimensions sorted by variance, so low-variance components are transported rather than ignored by distance computations. A sympathetic reader would care because the resulting MKL-VC is training-free, works with single 5–10 second utterances, and on LibriSpeech cuts word error rate from 32.292% to 8.131% at block size K=2, landing close to the trained FACodec baseline. The same recipe holds in cross-lingual German-French and low-resource language evaluations.

What carries the argument

The carrier of the argument is the factorized Monge-Kantorovich Linear (MKL) map, the analytic solution of quadratic optimal transport between two Gaussians: $$T(x)=\mu_2 + \$Sigma_1^{{-1/2}}$\left(\$Sigma_1^{{1/2}}$\Sigma_2\$Sigma_1^{{1/2}}$\right)^{1/2}\$Sigma_1^{{-1/2}}$(x-\mu_1).$$ Because a full transport plan in 1024 dimensions would be dominated by the few high-variance WavLM components, the method sorts dimensions by standard deviation, partitions them into N/K blocks of size K, assumes the source and reference covariances are approximately block-diagonal, and applies the K-dimensional MKL map per block. The Gaussianity of the per-block empirical distributions is supported by Wasserstein-2 distance measurements to matched normals, which are smallest for K=2.

What would settle it

Estimate the sorted 1024×1024 covariance of WavLM embeddings for a source and reference utterance and compare the average norm of off-diagonal K-blocks with the diagonal blocks; if off-diagonal norms are not negligible, the factorized map is not the true optimal transport for the full vector, and K=2's advantage over kNN-VC should be attributed to something other than distribution matching.

Watch

Extended reading notes

Core claim

The central claim is that matching the factored distributions of source and reference WavLM embeddings is a better conversion operator than copying nearest neighbors. Concretely, after sorting the 1024 embedding dimensions by standard deviation and partitioning them into blocks of size K, the method estimates each block's mean and covariance from the source and reference sequences and applies the analytical optimal transport map of Eq. (1) per block. The paper argues this preserves the information carried by low-variance dimensions, which remain near-Gaussian, and produces converted tokens that may be absent from the reference utterance; Table 2 reports total score 0.105 for K=2 versus 0.375 for kNN-VC, with WER dropping from 32.292% to 8.131%. Cross-lingual results on 1000 German-French FLEURS pairs place MKL-VC second to FACodec and far ahead of kNN-VC.

Load-bearing premise

The load-bearing premise is that after sorting WavLM dimensions by variance and splitting them into blocks, the source and reference embedding distributions are Gaussian with nearly block-diagonal covariances, so transporting each block independently yields the correct transport for the full vector.

Editorial extensions

If this is right

With a single 5–10 second reference utterance, MKL-VC on LibriSpeech reaches total score 0.105 (K=2) versus 0.375 for kNN-VC, putting it on par with the trained FACodec baseline while requiring no training.
The conversion keeps content intact: WER falls from 32.292% to 8.131% and CER from 18.877% to 3.846% on the same test pairs.
Cross-lingual German-French conversion on FLEURS ranks MKL-VC second only to FACodec, with total score 0.441 versus 1.156 for kNN-VC.
Because distribution matching can produce tokens not present in the reference, short references no longer need to cover every sound of the source utterance.
The single parameter K trades content preservation against speaker similarity: small K favors low WER/CER, while large K favors higher SIM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

For encoders other than WavLM, the factorized map's success is conditional on re-verifying Gaussianity and block-diagonality; if embeddings such as HuBERT show stronger cross-dimension correlations, per-block transport would likely need a different block selection or a low-rank correction.
A natural extension is adaptive blocking: choose block boundaries from the empirical covariance structure rather than from sorted standard deviations, which could improve speaker similarity at small K without sacrificing intelligibility.
At K=2 the method approaches per-dimension Gaussian standardization, which may explain the high intelligibility; comparing K=2 with explicit per-dimension mean-variance normalization would isolate what the covariance coupling in larger blocks adds.
The same distribution-matching operator could serve as data augmentation for low-resource speech synthesis, generating new prosodic variations from a few reference utterances.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline is that a very simple modification—replace kNN regression with per-block linear optimal transport in sorted WavLM dimensions—gives a large content-preservation improvement over kNN-VC on LibriSpeech and competitive numbers against FACodec. That is a real result, and it is new as far as I know. The factorized transport idea is clean, and the trimmed-kNN experiment that motivates it is a nice piece of evidence.

The paper also does several things right: it evaluates on 7800 LibriSpeech pairs plus 1000 cross-lingual FLEURS pairs, includes multiple baselines including a SinkVC reimplementation, ships code, and explicitly admits the method depends on WavLM-specific Gaussianity. The Wasserstein-distance check for Gaussianity is a good sanity signal, even if it is not a proof.

The soft spots are real but not fatal. The main one is evaluation protocol: K is the only tuning parameter, and K=2 was selected after looking at the test-set Table 2. No confidence intervals or per-speaker paired tests are given. With 7800 pairs, that is easy to add, and its absence weakens the precise claim. That said, the stress-test note says the headline is "really about a specific K," but the table shows K=8 and K=16 also beat kNN-VC by a wide margin (0.111 and 0.118 vs 0.375). So the qualitative claim—MKL-VC with a suitable K outperforms kNN-VC on short references—is robust across a range of K, even if the exact best K=2 number may be inflated.

Second soft spot: the block-diagonal covariance assumption is stated but not verified. The paper demonstrates per-block Gaussianity, but not independence across blocks. If cross-block correlations are strong, the factorized map is not the true optimal map. I would like to see a direct check of off-block correlation magnitude. This is a theoretical gap that could be addressed without rerunning the whole evaluation.

The cross-lingual subjective evaluation is thin: six native speakers, one Japanese reference, ranking instead of MOS. It is anecdotal but not misleading; the paper does not overstate it.

Verdict: for a voice-conversion researcher, this is worth a serious referee. The central empirical claim is substantial and likely true in direction, and the method is simple enough to reproduce. The paper needs a revision with held-out K selection or at least a discussion of selection bias, error bars, and a cross-block correlation check. I would engage with it.

Referee Report

3 major / 5 minor

Summary. The paper proposes MKL-VC, a training-free voice conversion method that operates on WavLM embeddings and replaces the k-nearest-neighbor regression of kNN-VC with a factorized linear optimal transport map. The authors sort WavLM dimensions by variance, split them into blocks of size K, compute the closed-form Gaussian optimal transport (Monge-Kantorovich Linear) map per block, and apply the direct product of these maps to convert source embeddings to target embeddings. Experiments on LibriSpeech test-clean with 7,800 pairs and on German-French FLEURS pairs show large reductions in WER/CER compared to kNN-VC at short reference durations, with Total scores competitive with the trained FACodec baseline. A small human evaluation in six languages ranks MKL-VC at or near the top.

Significance. If the results are reliable, the method is a significant practical simplification for any-to-any and cross-lingual voice conversion: it requires no training, uses only 5-10 seconds of reference audio, and improves content preservation over kNN-VC by a large margin. The analysis of WavLM's low-variance dimensions and the trimmed-kNN experiment are informative diagnostics. The code is publicly released, which supports reproducibility. However, the strength of the empirical claims is currently limited by the test-set selection of the hyperparameter K, the absence of confidence intervals, and the unverified block-diagonal covariance assumption.

major comments (3)

[§4.4, Table 2] The central comparison is compromised by test-set hyperparameter selection: the paper evaluates K ∈ {2,8,16,64,256} on the same 7,800-pair LibriSpeech test set and then reports K=2 as 'MKL, K=2 (ours)' with the best Total score. Because K is the method's only tuning parameter, the 0.105 Total score is an optimistically biased estimate; no held-out split or a-priori selection rule is described, and K=256 (Total 0.827) is far worse than kNN-VC (0.375). To support the claim that MKL-VC outperforms kNN-VC, the paper must specify how K is chosen (e.g., on a validation fold, by the Gaussianity criterion in Fig. 3 applied to training data, or as a fixed default) and report the performance of that chosen configuration.
[§3, Eq. (2)] The factorization claim is not validated. Section 3 states 'We assume covariance matrices Σ1 and Σ2 are approximately block-diagonal' after sorting dimensions by standard deviation, but no evidence for this assumption is provided. Figure 3 only checks the marginal Gaussianity of each block (via Wasserstein distance to a Gaussian with the same mean and covariance); it does not measure cross-block dependence. Consequently, the statement 'This map transports p0(x) to p1(x)' is exact only for exactly block-diagonal covariances; for the actual WavLM covariances, Eq. (2) is an approximation whose error is unquantified. Please add a direct test of the block-diagonal assumption (e.g., the norm of off-block entries relative to on-block entries) or revise the theoretical discussion to explicitly present Eq. (2) as a heuristic factorization.
[§4.3, Tables 2 and 3] No confidence intervals or significance tests are reported for any objective metric. With 7,800 LibriSpeech pairs, paired bootstrap intervals or per-speaker aggregated scores would be computationally cheap and would clarify whether the differences among the top rows of Table 2 (e.g., MKL K=2 Total 0.105 vs FACodec 0.106 vs FreeVC 0.113) are within noise. The absence of uncertainty quantification is especially important because the headline improvements over kNN-VC, while large in magnitude, are reported for a single test set and a single selected K.

minor comments (5)

[Abstract and §3] Typos and notation: 'optimal transport transport' appears in the sentence preceding Eq. (1), and the acronym MKL is used in the abstract before being defined in Section 3.
[§4.1] The FLEURS German-French construction is underspecified: 'combining every speaker from one dataset with every speaker from another' gives 10×10=100 speaker pairs, not the stated 1,000 samples. Please describe the number of utterances per speaker and the pairing procedure.
[Table 2] The table would be easier to interpret if a column indicated which systems are training-free (kNN-VC, SinkVC, MKL-VC) and which are trained (FACodec, FreeVC, Diff-VC), since 'training-free' is a central claim of the paper.
[Figure 3] The y-axis label 'Wasserstein Distance / (MKL dim)' is not explained; the normalization is likely intended to make distances comparable across K, but the formula is missing. Please define the plotted quantity and add the number of samples or error bars.
[§4.3] The Whisper-base ASR evaluation does not state the language setting used for German-French WER/CER calculation; please specify whether the language is forced to German/French or auto-detected, as this affects WER values.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the MKL map is an explicit external optimal-transport formula, the Gaussian assumption is independently validated, and the factorization is a stated approximation rather than a self-referential fit.

full rationale

The paper's derivation chain is self-contained. Eq. 1 is the standard Monge-Kantorovich Linear map with a published external source [15]; Eq. 2 is an explicit direct-product factorization under the stated block-diagonal covariance assumption. The Gaussianity of WavLM blocks is verified empirically via Wasserstein-2 distances in Fig. 3, not assumed by construction. No parameter inside the transport map is fitted to the evaluation data: K is a user-chosen block size, and the paper reports an ablation across K values rather than claiming the zero-parameter map predicts its own inputs. The trimmed-kNN experiment (Table 1) only establishes that a subset of WavLM dimensions dominates kNN distances; it does not enter the MKL-VC derivation. References are external (e.g., kNN-VC, FACodec, MKL), and no load-bearing self-citation chain is present. A legitimate statistical concern remains: K=2 appears to be selected on the same LibriSpeech test set used for the comparison, and no confidence intervals are reported, so the claimed superiority over kNN-VC may be optimistically biased. That is an evaluation-protocol weakness, not a circular derivation: the reported scores are not forced to equal the inputs by construction, and the claimed improvement does not reduce to a fitted parameter renamed as a prediction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests primarily on two empirical assumptions about WavLM embeddings, Gaussianity and block-diagonal covariance, and on the choice of K. No new physical or model entities are introduced. The method uses pretrained WavLM and HiFi-GAN as external components.

free parameters (1)

MKL dimension K = K=2 (chosen as best Total score on LibriSpeech test-clean, Table 2)
K is the block size for factorized transport. The paper evaluates K in {2, 8, 16, 64, 256} and reports K=2 as the best total score, so the headline result depends on this choice. Higher K raises SIM but lowers intelligibility.

assumptions (4)

domain assumption WavLM embedding vectors, after sorting by standard deviation and grouping into blocks, are approximately multivariate Gaussian within each block.
Used to justify the Monge-Kantorovich Linear formula (Eq. 1). Validated only indirectly via Wasserstein distances to a Gaussian in Fig. 3; the paper itself warns in the conclusion that Gaussianity must be verified for any new encoder.
domain assumption Covariance matrices of source and reference WavLM embeddings are approximately block-diagonal under the sorted-dimension partition.
The factorized map (Eq. 2) transports each block independently. The paper states this assumption without direct evidence, so correlations across blocks could make the factorized map suboptimal.
domain assumption WavLM cosine distances are meaningful for speech similarity, inherited from kNN-VC.
The motivation for distribution matching rests on the latent space geometry of WavLM, argued from Baas et al. and supported by the trimmed-kNN experiment in Table 1.
standard math The Monge-Kantorovich Linear map is the exact analytic solution of quadratic optimal transport between Gaussians.
This is the standard result from Pitie and Kokaram, referenced as [15], and is used as Eq. 1 without derivation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Training-Free Voice Conversion with Factorized Optimal Transport." pith.science (2026). https://pith.science/paper/Z633BMNE

@misc{pith2026250609709,
  author       = {Pith},
  title        = {Pith review of: Training-Free Voice Conversion with Factorized Optimal Transport},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z633BMNE}},
  note         = {Machine review of arXiv:2506.09709}
}

read the original abstract

This paper introduces Factorized MKL-VC, a training-free modification for kNN-VC pipeline. In contrast with original pipeline, our algorithm performs high quality any-to-any cross-lingual voice conversion with only 5 second of reference audio. MKL-VC replaces kNN regression with a factorized optimal transport map in WavLM embedding subspaces, derived from Monge-Kantorovich Linear solution. Factorization addresses non-uniform variance across dimensions, ensuring effective feature transformation. Experiments on LibriSpeech and FLEURS datasets show MKL-VC significantly improves content preservation and robustness with short reference audio, outperforming kNN-VC. MKL-VC achieves performance comparable to FACodec, especially in cross-lingual voice conversion domain.

Figures

Figures reproduced from arXiv: 2506.09709 by the authors.

**Figure 1.** Our Factorized MKL method performs distribution matching per sorted dimensions of WavLM encoder. Given that, one can justify the use of optimal transport theory. The main goal of optimal transport is to transform the source probability density to the target one with minimal cost, and the cost is assumed to be a distance. Here we suggest to modify the kNN-VC pipeline by replacing kNN regression with a linear optima… view at source ↗

**Figure 2.** Sorted standard deviations of WavLM embedding components across time dimensions, plotted on a log-log scale. Standard deviations were computed over concatenated sequences from the LibriSpeech train-clean-100 set. Most components have small standard deviations over time and thus do not significantly contribute to pairwise L2 and cosine distances [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Wasserstein-2 distance between WavLM embeddings distribution and normal distribution with the same mean and covariance matrix, divided by the MKL dimension. LibriSpeech, train-clean-100 set. within each group is approximately multivariate normal. Suppose we have two Gaussian distributions, p0(x) = N (x|µ1, Σ1) and p1(x) = N (x|µ2, Σ2). The MongeKantorovich Linear (MKL) map [15] defines the analytical solution of… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 18 canonical work pages

[1]

Training-Free Voice Conversion with Factorized Optimal Transport

Introduction Any-to-any voice conversion (VC) aims to change the voice identity of a speaker to match a reference voice without train- ing a specific model for each speaker-reference pair [1]. The linguistic content of the speaker’s utterance should remain un- changed, so the conversion targets only non-linguistic features (prosody) such as intonation, pi...

work page Pith review arXiv 2023
[2]

analysis- mapping-reconstruction

Related Work Despite advances in the field of voice conversion, any-to-any al- gorithms are not that common [4]. The most recent ones include FACodec [5], kNN-VC [2], Seed-VC 2, FreeVC [6], YourTTS [7], Diff-VC [8], and VQMIVC [9]. The standard scheme for voice conversion is “analysis- mapping-reconstruction” pipeline [1], also described as “encoder-conve...

work page 2024
[3]

Optimal Transport Maps are Good Voice Converters

Method Our method follows the common “encoder-converter-vocoder” architecture. Similarly to kNN-VC, we use the WavLM-Large model as the encoder [3] and the HiFi-GAN vocoder [13]. The conversion algorithm relies on the specific properties of the WavLM embeddings. These properties are discussed in more detail below. Structure of WavLM EmbeddingsWe observe t...

work page Pith review arXiv 2024
[4]

Datasets To evaluate any speaker to any speaker voice conversion we conduct our experiments on a LibriSpeech dataset [14], which consist of 40 speakers

Experiments 4.1. Datasets To evaluate any speaker to any speaker voice conversion we conduct our experiments on a LibriSpeech dataset [14], which consist of 40 speakers. To perform the voice conversion we con- structed 7800 content-reference pairs from the test-clean set of LibriSpeech [14] by taking 5 utterances for each speaker and converting them into ...

work page
[5]

MKL stands for Monge-Kantorovich Linear map- ping and is based on the exact solution of the quadratic opti- mal transport problem for Gaussian distributions

Conclusion In our work, we propose MKL-VC, a training-free modifica- tion of the kNN-VC pipeline for any-to-any cross-lingual voice conversion. MKL stands for Monge-Kantorovich Linear map- ping and is based on the exact solution of the quadratic opti- mal transport problem for Gaussian distributions. We show that MKL-VC solves the main issue of the origin...

work page
[6]

To prevent misuse, it is crucial to develop robust speech detection methods and avoid voice authentication in high-security applications

Broader Impact Statement Since VC models can synthesize speech with high speaker sim- ilarity, they carry potential risks, such as spoofing voice identi- fication systems or impersonating specific speakers. To prevent misuse, it is crucial to develop robust speech detection methods and avoid voice authentication in high-security applications

work page
[7]

An overview of voice conversion and its challenges: From statistical modeling to deep learning,

B. Sisman, J. Yamagishi, S. King, and H. Li, “An overview of voice conversion and its challenges: From statistical modeling to deep learning,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 132–157, 2020

work page 2020
[8]

V oice conversion with just nearest neighbors,

M. Baas, B. van Niekerk, and H. Kamper, “V oice conversion with just nearest neighbors,” inInterspeech 2023, 2023, pp. 2053– 2057

work page 2023

Show all 25 references

[9]

Wavlm: Large-scale self- supervised pre-training for full stack speech processing,

S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self- supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

2022
[10]

Overview of voice conversion methods based on deep learning,

T. Walczyna and Z. Piotrowski, “Overview of voice conversion methods based on deep learning,”Applied sciences, vol. 13, no. 5, p. 3100, 2023

2023
[11]

Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,

Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, E. Liu, Y . Leng, K. Song, S. Tanget al., “Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,” inForty-first International Conference on Machine Learning, 2024

2024
[12]

Freevc: Towards high-quality text-free one-shot voice conversion,

J. Li, W. Tu, and L. Xiao, “Freevc: Towards high-quality text-free one-shot voice conversion,” inICASSP 2023-2023 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

2023
[13]

Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,

E. Casanova, J. Weber, C. D. Shulby, A. C. Junior, E. G ¨olge, and M. A. Ponti, “Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone,” inInternational Con- ference on Machine Learning. PMLR, 2022, pp. 2709–2720

2022
[14]

Diffusion-based voice conversion with fast maxi- mum likelihood sampling scheme,

V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, M. S. Kudinov, and J. Wei, “Diffusion-based voice conversion with fast maxi- mum likelihood sampling scheme,” inInternational Conference on Learning Representations, 2022

2022
[15]

Vqmivc: Vector quantization and mutual information-based un- supervised speech representation disentanglement for one-shot voice conversion,

D. Wang, L. Deng, Y . T. Yeung, X. Chen, X. Liu, and H. Meng, “Vqmivc: Vector quantization and mutual information-based un- supervised speech representation disentanglement for one-shot voice conversion,”Interspeech 2021, 2021

2021
[16]

Synthesizing a choir in real-time using pitch synchronous over- lap add (psola)

N. Schnell, G. Peeters, S. Lemouton, P. Manoury, and X. Rodet, “Synthesizing a choir in real-time using pitch synchronous over- lap add (psola).” inICMC, 2000

2000
[17]

Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

2021
[18]

Sinkhorn distances: Lightspeed computation of op- timal transport,

M. Cuturi, “Sinkhorn distances: Lightspeed computation of op- timal transport,”Advances in neural information processing sys- tems, vol. 26, 2013

2013
[19]

Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,

J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial net- works for efficient and high fidelity speech synthesis,”Advances in neural information processing systems, vol. 33, pp. 17 022– 17 033, 2020

2020
[20]

Lib- rispeech: an asr corpus based on public domain audio books,

V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an asr corpus based on public domain audio books,” in2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210

2015
[21]

The linear monge-kantorovitch linear colour mapping for example-based colour transfer,

F. Pitie and A. Kokaram, “The linear monge-kantorovitch linear colour mapping for example-based colour transfer,” inIET 4th European Conference on Visual Media Production (CVMP 2007). IEE, 2007

2007
[22]

Fleurs: Few-shot learning evaluation of universal representations of speech,

A. Conneau, M. Ma, S. Khanuja, Y . Zhang, V . Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “Fleurs: Few-shot learning evaluation of universal representations of speech,” in2022 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2023, pp. 798–805

2023
[23]

Spoken language recognition using x-vectors

D. Snyder, D. Garcia-Romero, A. McCree, G. Sell, D. Povey, and S. Khudanpur, “Spoken language recognition using x-vectors.” in Odyssey, vol. 2018, 2018, pp. 105–111

2018
[24]

Robust speech recognition via large-scale weak supervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

2023
[25]

Moseley,Atlas of the World’s Languages in Danger

C. Moseley,Atlas of the World’s Languages in Danger. Unesco, 2010

2010

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Training-Free Voice Conversion with Factorized Optimal Transport

Introduction Any-to-any voice conversion (VC) aims to change the voice identity of a speaker to match a reference voice without train- ing a specific model for each speaker-reference pair [1]. The linguistic content of the speaker’s utterance should remain un- changed, so the conversion targets only non-linguistic features (prosody) such as intonation, pi...

work page Pith review arXiv 2023

[2] [2]

analysis- mapping-reconstruction

Related Work Despite advances in the field of voice conversion, any-to-any al- gorithms are not that common [4]. The most recent ones include FACodec [5], kNN-VC [2], Seed-VC 2, FreeVC [6], YourTTS [7], Diff-VC [8], and VQMIVC [9]. The standard scheme for voice conversion is “analysis- mapping-reconstruction” pipeline [1], also described as “encoder-conve...

work page 2024

[3] [3]

Optimal Transport Maps are Good Voice Converters

Method Our method follows the common “encoder-converter-vocoder” architecture. Similarly to kNN-VC, we use the WavLM-Large model as the encoder [3] and the HiFi-GAN vocoder [13]. The conversion algorithm relies on the specific properties of the WavLM embeddings. These properties are discussed in more detail below. Structure of WavLM EmbeddingsWe observe t...

work page Pith review arXiv 2024

[4] [4]

Datasets To evaluate any speaker to any speaker voice conversion we conduct our experiments on a LibriSpeech dataset [14], which consist of 40 speakers

Experiments 4.1. Datasets To evaluate any speaker to any speaker voice conversion we conduct our experiments on a LibriSpeech dataset [14], which consist of 40 speakers. To perform the voice conversion we con- structed 7800 content-reference pairs from the test-clean set of LibriSpeech [14] by taking 5 utterances for each speaker and converting them into ...

work page

[5] [5]

MKL stands for Monge-Kantorovich Linear map- ping and is based on the exact solution of the quadratic opti- mal transport problem for Gaussian distributions

Conclusion In our work, we propose MKL-VC, a training-free modifica- tion of the kNN-VC pipeline for any-to-any cross-lingual voice conversion. MKL stands for Monge-Kantorovich Linear map- ping and is based on the exact solution of the quadratic opti- mal transport problem for Gaussian distributions. We show that MKL-VC solves the main issue of the origin...

work page

[6] [6]

To prevent misuse, it is crucial to develop robust speech detection methods and avoid voice authentication in high-security applications

Broader Impact Statement Since VC models can synthesize speech with high speaker sim- ilarity, they carry potential risks, such as spoofing voice identi- fication systems or impersonating specific speakers. To prevent misuse, it is crucial to develop robust speech detection methods and avoid voice authentication in high-security applications

work page

[7] [7]

An overview of voice conversion and its challenges: From statistical modeling to deep learning,

B. Sisman, J. Yamagishi, S. King, and H. Li, “An overview of voice conversion and its challenges: From statistical modeling to deep learning,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 132–157, 2020

work page 2020

[8] [8]

V oice conversion with just nearest neighbors,

M. Baas, B. van Niekerk, and H. Kamper, “V oice conversion with just nearest neighbors,” inInterspeech 2023, 2023, pp. 2053– 2057

work page 2023