{"id":"4808d7f0-1b9d-4d6d-8772-868eb5205735","arxiv_id":"2504.13791","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A single-generator, three-discriminator GAN with a collective weighting rule and an optimal transport loss is claimed to improve non-parallel voice conversion over MaskCycleGAN-VC and MelGAN-VC.","lead":"This paper proposes CLOT-GAN-VC, a voice conversion model that uses one generator and three discriminators (a CNN, a Vision Transformer, and a conformer) with a weighted collective learning rule and an optimal transport loss. A generalist might read it to see whether mixing multiple discriminator architectures and optimal transport helps synthesized speech sound more natural.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central OT loss in Eq. (3) is algebraically inconsistent and cannot be implemented as stated: it cancels to an asymmetric four-term expression, and the specified Sinkhorn size N=4 conflicts with the stated mini-batch size of 1, so the paper's claimed mechanism is unsupported.","rationale":"The claim that this single-generator multi-discriminator OT-GAN outperforms existing VC models rests on two pillars: the collective weighting mechanism and the OT loss. Of these, the OT loss is the most technically distinctive and is explicitly credited by the ablation study (Table II, CLOT-GAN-VC(3)) for a substantial part of the improvement. The reader's REJECT verdict is supported by the Eq. (3) inconsistency, and I found no independent support that would offset it: no formal verification, no released training logs, and no statistical analysis of the MCD/MSD gaps. My concern is not that Eq. (3) disagrees with current consensus; it is that, on the face of the manuscript, the stated objective is not a coherent OT distance and the stated N=4 and mini-batch-size-1 values cannot both hold without a missing explanation. If a typesetting error is responsible, the burden is on the authors to state the implemented loss; without that, the central novelty is unverified. The paper does have useful material: code and sample links, a serious ablation structure, and a reasonable multi-discriminator architecture built from known components. But those do not establish the central OT mechanism as written. Hence the existing REJECT stands; I do not see a condition short of releasing and explaining the actual discriminator loss that would change it.","tokens_in":11413,"tokens_out":9568,"duration_ms":90681,"concrete_test":"Obtain the released implementation at shorturl.at/7X00d and locate the Sinkhorn/OT loss function. Run one debug pass with four labeled tensors X, X', Y, Y' and compare each computed OT term against Eq. (3). If the code reproduces Eq. (3), verify how N=4 is obtained with mini-batch size 1; if it does not reproduce Eq. (3), the paper's stated loss is not the trained loss. Independently, simplify Eq. (3) and check whether it equals a known four-sample Sinkhorn-divergence estimate such as Wc(X,Y') + Wc(X',Y) - Wc(X,X') - Wc(Y,Y'); if it does not, the printed loss is not a coherent distributional distance, and Table II's ablation cannot support the OT claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III 'Discriminator OT loss' defines the loss by Eq. (3) as Wc(X,X') + Wc(X,Y') + Wc(X',Y) + Wc(X',Y') - 2Wc(X,X') - 2Wc(Y,Y'). Combining like terms gives -Wc(X,X') + Wc(X,Y') + Wc(X',Y) + Wc(X',Y') - 2Wc(Y,Y'). This is not the Sinkhorn divergence between real and generated feature distributions: it contains a single negative real-real cost, a double negative fake-fake cost, a positive fake-fake cost, and it is not invariant under relabeling the two classes unless the within-class costs happen to coincide. No cited Sinkhorn/OT loss in [23], [24], or [32] has this form, and the text gives no derivation leading to it. The problem is not merely cosmetic: the ablation in Table II is used to attribute the reported gains to the OT loss, so the loss must be exactly what was trained. The implementation is also unclear: Eq. (1) uses a cost matrix C in R^{N x N} with 'N considered as 4', while Section IV-A fixes the training mini-batch size at 1; the paper does not say how four samples per batch are formed. If the code actually implements a different, valid OT loss, then Eq. (3) is a material misdescription of the method. If it implements Eq. (3), the objective is not the OT-bridging mechanism claimed. The central mechanism is therefore unverified as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CLOT-GAN-VC, a single-generator multi-discriminator GAN for non-parallel voice conversion. The generator follows the MaskCycleGAN-VC backbone, and the three discriminators (DCNN, ViT, conformer) are combined through a 'collective learning mechanism' that weights each discriminator's loss, where the discriminator loss is an optimal-transport (OT) loss based on the Sinkhorn algorithm. The authors evaluate the model on VCC 2018, CSTR-VCTK, and CMU-Arctic, reporting MCD/MSD and MOS against MaskCycleGAN-VC and MelGAN-VC, plus an ablation study on VCC 2018. The abstract claims that the model outperforms existing VC models in both objective and subjective assessments.","tokens_in":11742,"tokens_out":8257,"duration_ms":76312,"significance":"If the proposed method worked as described, the multi-discriminator collective-learning design and the OT-based training signal would be a reasonable and potentially useful direction for non-parallel VC, particularly the use of ViT and conformer discriminators. The paper includes an ablation study, reports MOS confidence intervals, and provides links to code and generated samples, which are useful practices. However, the mathematical specification of the central OT loss is internally inconsistent, the participation weights in the collective learning mechanism do not sum to one, and the empirical claims are overstated relative to the reported numbers. These issues are load-bearing because the novelty of the paper rests on the OT loss and the collective learning mechanism.","major_comments":[{"comment":"Equation (3) is not a valid or recognizable Sinkhorn/OT divergence as written. Combining like terms reduces it to Wc(X,Y') + Wc(X',Y) + Wc(X',Y') - Wc(X,X') - 2Wc(Y,Y'), which contains a negative within-class cost for the real class and a double-negative within-class cost for the fake class. No derivation is given, and the cited references [23], [24], and [32] do not contain this expression. This matters because the ablation in Table II is used to attribute the reported gains to the OT loss, so the trained loss must be exactly what is stated. In addition, Eq. (1) sets N=4 but Section IV-A states a training mini-batch size of 1; the paper does not explain how four samples per mini-batch are formed. If the implementation in fact uses a different OT loss, Eq. (3) is a material misdescription; if it uses Eq. (3), the loss is not the claimed OT-bridging mechanism. Please correct the equation, provide a derivation or citation, and clarify the mini-batch construction, or rerun the experiments with the intended loss.","section":"Section III, 'Discriminator OT loss', Eq. (3)"},{"comment":"The participation weights in Algorithm 1 are defined as alpha_k^y = (Ltot_y - Ld_k^y)/Ltot_y. Since Ltot_y = sum_j Ld_j^y, these weights sum to n-1, not to 1. With n=3, the weights sum to 2, so the final loss L_y = sum_k alpha_k^y Ld_k^y is not a weighted average as claimed in the text. The same issue applies to the beta weights in the generator loss. Please clarify whether normalization is intended and, if so, how it is performed; as written, the collective learning mechanism changes the loss scale and the interpretation of the weighting.","section":"Algorithm 1 and generator loss (Eqs. (4)-(5))"},{"comment":"The abstract states that CLOT-GAN-VC outperforms existing VC models, but Table I contains several counterexamples. For example, on CMU-Arctic F-F, CLOT-GAN-VC has MCD 8.66 versus 7.81 for MaskCycleGAN-VC; on CMU-Arctic M-F, CLOT-GAN-VC has MSD 1.41 versus 1.32 for MaskCycleGAN-VC; on CSTR-VCTK F-M, CLOT-GAN-VC has MCD 5.86 versus 5.26 for MelGAN-VC; and on CSTR-VCTK M-M, CLOT-GAN-VC has MSD 1.40 versus 1.38 for MaskCycleGAN-VC. No error bars, confidence intervals, or significance tests are reported for the objective metrics, so the claim that the model outperforms the baselines is not supported as stated.","section":"Section V-A, Table I"},{"comment":"Table III reports MOS values with 95% confidence intervals, but many intervals overlap with those of the baselines. For instance, VCC2018 M-M gives 3.57±0.53 for CLOT-GAN-VC versus 3.47±0.50 for MaskCycleGAN-VC, and CMU-Arctic M-M gives 3.22±0.09 versus 3.16±0.03. The text states that the proposed model is 'significantly higher' without any statistical significance test. With only 17 raters and overlapping intervals, the subjective superiority claim is not established. Please perform an appropriate significance test or temper the claim.","section":"Section V-B, Table III"}],"minor_comments":[{"comment":"The sentence 'The training, validation and evaluation (test) sets for each dataset comprised of 81, 35, 25 samples' is unclear: it does not specify whether these counts are per speaker pair, per utterance, or per dataset, and 81/35/25 is an unusual split. Please clarify.","section":"Section IV-A"},{"comment":"References [12] and [13] are duplicates (same title, authors, and conference). Please remove one.","section":"References"},{"comment":"The rank-based interpretation of Table II is informal and no error bars are given for the ablation runs, so the stated conclusions about which component contributes most are not statistically grounded.","section":"Section V-A, ablation discussion"},{"comment":"The mel-spectrogram and Grad-CAM panels in Figure 2 are extremely small, making it difficult to verify the claimed visual differences. Please enlarge the figure or provide higher-resolution views.","section":"Figure 2"},{"comment":"The code and demo links use short URLs (shorturl.at, tinyurl.com), which are not stable for archival purposes. Please provide persistent repository links or DOIs.","section":"Reproducibility links"}],"recommendation":"major_revision","confidential_remarks":"The central issues are the algebraic inconsistency of Eq. (3) and the batch-size conflict, which are fixable only if the authors can show that the public code actually implements a different, valid OT loss. If the code matches Eq. (3), the experiments would need to be rerun, which would push the manuscript toward rejection. The editor may want to request the exact loss implementation and training logs as part of the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is an honest engineering attempt—single generator, three heterogeneous discriminators (DCNN, ViT, conformer), and a weighted collective rule—but the central OT loss in Eq. (3) doesn't survive contact with the page. Combine like terms and you get −Wc(X,X′) + Wc(X,Y′) + Wc(X′,Y) + Wc(X′,Y′) − 2Wc(Y,Y′), which is not the Sinkhorn divergence or any OT loss I can find in the cited sources. On top of that, N=4 in Eq. (1) sits uneasily with the stated mini-batch size of 1; the paper never explains how four samples per class appear. That is load-bearing because the ablation in Table II is used to credit the OT loss for the gains.\n\nCredit where it's due: the multi-discriminator collective weighting is a real extension, and the authors do include an ablation, re-generated baseline samples, and links to code and audio. They are not hiding the rough edges.\n\nBut the evaluation is weaker than the abstract implies. Only two older baselines (MaskCycleGAN-VC, MelGAN-VC) are compared; objective metrics come without error bars or significance tests; and the MOS confidence intervals overlap with MaskCycleGAN-VC in several conditions, despite the claim of significant superiority. \"Outperforms existing VC models\" is an overclaim on this evidence.\n\nMy guess is that Eq. (3) is a typo and the actual implementation uses something like a proper Sinkhorn loss with different indices. If so, the paper is one careful revision away from being a decent contribution. As written, I wouldn't accept it. I would send it to a referee who can check the code, because the flaw is specific and fixable, and the underlying experiment is worth taking seriously.","headline":"A decent multi-discriminator GAN for voice conversion is undermined by an OT loss that does not algebraically hold together and an evaluation that is thinner than the abstract claims.","tokens_in":12296,"tokens_out":2653,"would_cite":false,"duration_ms":23829,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single-generator GAN with three differently structured discriminators and an optimal-transport loss is claimed to outperform existing non-parallel voice-conversion models on standard benchmark corpora.","keywords":["voice conversion","generative adversarial network","optimal transport","multiple discriminators","collective learning","Sinkhorn divergence","mel-spectrogram","non-parallel voice conversion"],"falsifier":"Run the exact training protocol with mini-batch size 1 and $N=4$, print the four vectors entering each $W_c$ term of Eq. (3), and compare the resulting MCD and MOS against a version trained with an $L_2$ discriminator loss under otherwise identical settings; if the improvement disappears, the claimed role of optimal transport is not supported.","tokens_in":11194,"feed_emoji":"🎙️","tokens_out":7079,"duration_ms":62656,"temperature":0.7,"pith_summary":"The paper aims to improve non-parallel voice conversion by replacing the usual single discriminator in a GAN with three differently structured discriminators: a deep CNN, a vision transformer, and a conformer. These discriminators are combined through a collective weighting rule, and the model also adds an optimal-transport loss that measures the gap between source and target mel-spectrogram distributions. The claim is that this combined design makes converted speech closer to the target speaker while preserving linguistic content, and the reported experiments on three standard benchmark corpora show lower Mel-Cepstral Distortion and Modulation Spectra Distance and higher Mean Opinion Scores than two prior GAN-based conversion systems. A sympathetic reader would care because the method targets a known weak point of current GAN voice converters: the naturalness gap between real and synthesized speech, and it claims to do so with a training scheme that works on small datasets. If the claim holds, multi-discriminator feedback plus distribution-level loss is a practical recipe for one-to-one non-parallel voice conversion.","feed_headline":"A three-discriminator GAN outperforms prior voice conversion","feed_subtitle":"Collective weighting and optimal-transport loss lower spectral distortion and raise naturalness on three benchmark corpora.","key_machinery":"The load-bearing mechanism is the collective learning rule paired with the optimal-transport discriminator loss. For each discriminator, the paper computes an individual loss $L_{d_k}$ and a total $L_{\\text{tot}} = \\sum_k L_{d_k}$; the participation weight is $\\alpha_k = (L_{\\text{tot}} - L_{d_k}) / L_{\\text{tot}}$, so the discriminator with the smallest loss gets the largest weight, and the final discriminator loss is $L = \\sum_k \\alpha_k L_{d_k}$. The optimal-transport loss is written as $L_d = W_c(X,X') + W_c(X,Y') + W_c(X',Y) + W_c(X',Y') - 2W_c(X,X') - 2W_c(Y,Y')$, where $W_c$ is the optimal transport cost with a cosine cost and $N=4$ Sinkhorn matchings. This mechanism tells each discriminator to focus on distribution mismatch rather than raw classification, and tells the generator to prioritize feedback from discriminators that are already doing well, which the paper interprets as a multiplayer cooperative game.","core_discovery":"The paper's central claim is that CLOT-GAN-VC, a single-generator GAN with three discriminators updated by a collective learning mechanism and trained with an optimal-transport discriminator loss, outperforms existing one-to-one non-parallel voice-conversion models on objective and subjective metrics. The collective mechanism weights each discriminator's loss inversely to its contribution to the total loss, so the generator receives a composite feedback signal from all three architectures. The optimal-transport loss, computed from cosine transport costs and Sinkhorn soft matchings on flattened feature embeddings, is intended to align the generated mel-spectrogram distribution with the target distribution rather than simply classifying real versus fake. Ablation results are used to attribute the gains specifically to the multi-discriminator setup, the weighted averaging, and the optimal-transport loss, with the relative importance varying by gender pair.","pith_inferences":["One consequence the paper does not test directly: swapping the optimal-transport discriminator loss onto a single-discriminator version would isolate whether the gain comes from the optimal-transport loss alone; the current ablations only remove optimal transport, multiple discriminators, and weighting one at a time from the full system, so the interaction between optimal transport and multiple di","If the collective weighting rule is sound, it should transfer to other speech-generation tasks with multiple feature extractors, such as speech enhancement or text-to-speech, where each discriminator could attend to a different acoustic property; that is an extension rather than a claim of this paper.","The stated mini-batch size of 1 and matching size $N=4$ leave an implementation question: where the four mini-batch vectors for the optimal-transport loss come from within a single sample. A reader checking the released code should verify this before trusting the optimal-transport explanation."],"forward_implications":["If the reported results hold, replacing a single discriminator with a diverse set of three architectures is a viable strategy for one-to-one non-parallel voice conversion without requiring parallel training data.","The ablation's ranking implies that the optimal-transport loss is the most important component for male-to-male and inter-gender conversion, while the multi-discriminator design matters most for female-to-female conversion; a practitioner could tune components by gender pair.","Because training used only 81 training samples per dataset, the method claims to be suitable for low-resource voice-conversion settings where large parallel corpora are unavailable.","The optimal-transport loss gives the generator a continuous distribution-level training signal, so the approach could reduce the over-smoothing and naturalness gap that the paper attributes to earlier GAN voice converters."],"supporting_citations":[{"why":"Supplies the generator and DCNN discriminator backbone and the main non-parallel voice-conversion baseline.","marker":"[14]"},{"why":"Provides the vision-transformer discriminator architecture used as one of the three classifiers.","marker":"[19]"},{"why":"Provides the conformer discriminator architecture used to capture local-global features.","marker":"[21]"},{"why":"Provides the optimal-transport and Sinkhorn divergence formulation that the discriminator loss is based on.","marker":"[24]"},{"why":"Defines the cosine cost function and the optimal-transport GAN discriminator loss structure used in Eq. (3).","marker":"[32]"},{"why":"The MelGAN-based voice-conversion system used as a comparison baseline.","marker":"[28]"},{"why":"Supplies the cycle-consistency and identity losses reused by the proposed model.","marker":"[16]"}],"fun_headline_variants":["CLOT-GAN: Multi-discriminator voice conversion with optimal transport","Three discriminators beat one for natural voice conversion","Collective learning GAN with OT loss improves voice conversion","Better voice conversion via collective multi-discriminator GAN","Optimal transport sharpens GAN-based voice conversion naturalness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the optimal-transport discriminator loss in Eq. (3) is correctly specified and can be computed with the stated Sinkhorn matching size $N=4$ under a mini-batch size of 1; if the formula or its implementation is inconsistent, the central explanation for the performance gain is unsupported.","fun_headline_variants_meta":{"raw":{"variants":["CLOT-GAN: Multi-discriminator voice conversion with optimal transport","Three discriminators beat one for natural voice conversion","Collective learning GAN with OT loss improves voice conversion","Better voice conversion via collective multi-discriminator GAN","Optimal transport sharpens GAN-based voice conversion naturalness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1590,"prompt_tokens":974,"completion_tokens":616,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":533}},"tokens_in":590,"tokens_out":616,"duration_ms":5353,"temperature":1.0,"reasoning_tokens":533,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:59:43.703579+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact training protocol with mini-batch size 1 and $N=4$, print the four vectors entering each $W_c$ term of Eq. (3), and compare the resulting MCD and MOS against a version trained with an $L_2$ discriminator loss under otherwise identical settings; if the improvement disappears, the claimed role of optimal transport is not supported.","supporting_citations":[{"cited_title":"Maskcyclegan-VC: Learning non-parallel voice conversion with filling in frames,","cited_arxiv_id":null,"evidence_quote":"Supplies the generator and DCNN discriminator backbone and the main non-parallel voice-conversion baseline."},{"cited_title":"ViT-GAN: Using vision transformer as discriminator with adaptive data augmentation,","cited_arxiv_id":null,"evidence_quote":"Provides the vision-transformer discriminator architecture used as one of the three classifiers."},{"cited_title":"CMGAN: Conformer-based metric gan for speech enhancement,","cited_arxiv_id":null,"evidence_quote":"Provides the conformer discriminator architecture used to capture local-global features."},{"cited_title":"Hilbert sinkhorn divergence for optimal transport,","cited_arxiv_id":null,"evidence_quote":"Provides the optimal-transport and Sinkhorn divergence formulation that the discriminator loss is based on."},{"cited_title":"FID- RPRGAN-VC: Fr ´echet inception distance loss based region-wise posi- tion normalized relativistic gan for non-parallel voice conversion,","cited_arxiv_id":null,"evidence_quote":"Supplies the cycle-consistency and identity losses reused by the proposed model."}],"review_version":1}