{"id":"f551cdca-d470-4c10-b09b-8185712efad1","arxiv_id":"2508.02038","paper_version":4,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Marco-Voice is a TTS system combining voice cloning and emotional speech generation via speaker-emotion disentanglement, contrastive learning, and a new Mandarin emotional dataset, with claimed quality gains over CosyVoice baselines.","lead":"Marco-Voice merges voice cloning and emotional speech synthesis into one TTS system, using speaker-emotion disentanglement and a new Mandarin emotional dataset. A reader might care because it targets two hard problems at once, controllable emotion and stable speaker identity, in a single pipeline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The universal-superiority claim is not supported by the reported evaluation: tiny A/B samples, absent CIs, and the paper's own AISHELL WER table contradict it.","rationale":"The reader's verdict is REJECT, and my pass does not change that. My reason is slightly different from the reader's named 'weakest assumption': rather than the disentanglement formula in Eq. 1, the load-bearing weakness is the evaluation itself. The strongest claim is an empirical universal-superiority assertion, so the necessary condition is valid, sufficiently powered, and internally consistent measurements. Those conditions are missing: no confidence intervals or rater counts, A/B samples too small to be informative, and an objective table that undercuts the claim. I agree with the reader's rationale on statistical rigor even though the named weakest assumption differs, hence 'partial.' The architectural components are plausible and the code/data release is in principle a positive, verifiable step, but those do not establish the headline performance claim. A concrete re-analysis of raw ratings and preference data, or an equivalent preregistered human evaluation, would settle the matter. Until such evidence is supplied, the preprint should remain rejected as an empirical claim, while the underlying system and dataset may still be worth pursuing.","tokens_in":10720,"tokens_out":6143,"duration_ms":75780,"concrete_test":"Obtain the raw per-item human ratings and the exact evaluation protocol (number of raters, number of utterances, instruction text) behind Tables 1-3; compute paired 95% bootstrap confidence intervals for each rated dimension and exact binomial confidence intervals for the A/B preference rates. If the Marco-Voice vs CosyVoice2 intervals overlap on any claimed dimension, or if the A/B win rate against either baseline is not significantly above 50%, the 'consistently outperforms' claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Marco-Voice 'consistently outperforms existing voice cloning systems across all evaluated dimensions' (Sec. 4.1) and is best on all emotional-generation aspects (Sec. 4.2). For this to hold, the evaluation must be statistically meaningful and internally consistent. It is neither. Tables 1 and 2 report mean ratings without any rater counts, item counts, confidence intervals, or significance tests, so the reported gaps cannot be distinguished from noise. The only preference data, Table 3, are n=20; 12/20 and 13/20 win rates are not significantly different from chance under an exact binomial test (two-sided p approximately 0.50 and 0.26, respectively). Moreover, the paper's own objective results directly contradict the 'consistently outperforms' wording: Table 5 shows CosyVoice1 at 3.0 WER on AISHELL versus 17.6 for Marco-Voice-v4, with Del&Ins 11 vs 218. Introducing CosyVoice1* (a same-data continued model) explains why continued training hurts WER, but it does not remove the direct comparison against the actual CosyVoice1 baseline. The empirical claim as written is therefore not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Marco-Voice, a unified text-to-speech system that combines voice cloning and emotional speech generation. The technical contributions are a rotational emotion embedding integration mechanism, a speaker-emotion disentanglement loss with in-batch contrastive learning, and a cross-attention module that injects emotion into the language-model output; the authors also introduce CSEMOTIONS, a Mandarin emotional speech dataset. The central claim is that Marco-Voice consistently outperforms existing voice cloning and emotional TTS systems across all evaluated dimensions, based on human ratings, A/B preference tests, and objective metrics on LibriTTS and AISHELL.","tokens_in":10972,"tokens_out":5221,"duration_ms":62630,"significance":"If fully supported, the proposed architecture and the CSEMOTIONS dataset would be useful resources for the expressive TTS community. The public release of code and data is a concrete strength. However, the empirical evidence presented is not statistically adequate and is in part internally inconsistent with the paper's own objective measurements, so the claimed substantial advance is not currently established.","major_comments":[{"comment":"The human evaluations that anchor the paper's central claim of consistent superiority lack basic statistical reporting: no rater counts, item counts, confidence intervals, or significance tests are given. The A/B preference test in Table 3 is based on 20 ratings per comparison, with win rates of 12/20 and 13/20; these are not significantly different from chance under a two-sided exact binomial test (p approximately 0.50 and 0.26). The claim that Marco-Voice is 'consistently preferred' is therefore not supported by the data as presented.","section":"Sec. 4.1-4.3, Tables 1-3"},{"comment":"The AISHELL objective results directly contradict the 'consistently outperforms' wording: Marco-Voice variants achieve WER between 15.9 and 18.2, while the CosyVoice1 baseline achieves 3.0, with Del & Ins counts about 20 times higher. The CosyVoice1* continuation model provides a post-hoc explanation for why continued training degrades WER, but it does not remove the direct comparison against the actual CosyVoice1 baseline that appears in the same table. The universal-superiority claim must be revised to account for this evidence.","section":"Sec. 4.4, Table 5"},{"comment":"The emotion recognition analysis uses emotion2vec_base_finetuned as the classifier, whereas the system's emotion embeddings are extracted using Emotion2vec (Figure 1). Evaluating the model with a classifier built on the same representation family that is used for conditioning introduces a likely positive bias for the proposed method; an independent emotion classifier or a human emotion-annotation study is needed to support the emotion-control claims.","section":"Sec. 4.4, Figures 2-5"},{"comment":"The core disentanglement mechanism assumes that the normalized difference between emotional and neutral embeddings from a pretrained emotion encoder captures emotional content while removing speaker identity. This assumption is not validated. If the difference direction is speaker-dependent or not linearly separable, the emotion-control mechanism may fail, and the cross-speaker aggregation in Eq. (2) would mix incompatible directions. The authors should provide an analysis showing that the emotion direction transfers across speakers, or temper the claims accordingly.","section":"Sec. 2.2, Eq. (1)"},{"comment":"The size of the CSEMOTIONS dataset is described inconsistently: the abstract says 'six professional speakers', while the introduction and Section 3.1 say 'ten professional native speakers'. This is a factual discrepancy in the dataset contribution and must be resolved.","section":"Abstract, Introduction, Sec. 3.1"}],"minor_comments":[{"comment":"There is a typo in 'intorduced'; it should be 'introduced'.","section":"Sec. 2.2"},{"comment":"The abstract contains the typo 'eemotional style'; it should be 'emotional style'.","section":"Abstract"},{"comment":"The loss weight λ_rot is mentioned in the implementation details, but Eq. (9) uses λ_contrast; the notation should be aligned to avoid confusion.","section":"Sec. 3.2 vs Eq. (9)"},{"comment":"Speaker similarity is reported as 0.8275 in Table 1 but as roughly 64 in Table 4 (SS SpeechBrain); the scale and the exact metric used in the human evaluation should be clarified.","section":"Tables 1 and 4"},{"comment":"The figures include a Marco-Voice-v5 model that is not defined or evaluated in Tables 4 and 5; the version numbering should be made consistent.","section":"Figures 2-5"},{"comment":"The limitation that paired emotional speech data is required is acknowledged, but the manuscript does not discuss how this requirement constrains the voice-cloning use case, which is one of the two main claimed capabilities.","section":"Sec. 5.2"}],"recommendation":"reject","confidential_remarks":"The manuscript is more of a technical report than a journal paper, and the central empirical claims are not supported by the evidence as presented. The A/B tests are underpowered, the human rating table lacks basic statistical metadata, and the objective AISHELL WER results directly contradict the 'consistently outperforms' statement. A substantially revised version with proper statistical reporting, an independent emotion evaluator, and carefully tempered claims could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about Marco-Voice. The genuinely useful part is the CSEMOTIONS dataset: about 10 hours of studio-quality Mandarin emotional speech across seven categories, from professional voice actors. The model work is a transparent extension of CosyVoice1 with an emotion embedding borrowed from EmoKnob, plus two new losses (cross-orthogonality and in-batch contrastive) and a cross-attention mechanism. The code and dataset are promised publicly. That is real contribution.\n\nWhat is not supported is the claim that Marco-Voice 'consistently outperforms' all baselines. The human evaluation tables report means without rater counts, item counts, confidence intervals, or significance tests. The A/B preference test is n=20 per comparison; 12/20 and 13/20 are not significantly above chance under a binomial test. The objective numbers on Mandarin are worse: CosyVoice1 gets 3.0 WER on AISHELL while Marco-Voice variants land between 15.9 and 18.2. The explanation about continued training on the same dataset and vocalized fillers may account for some degradation, but it does not erase the direct comparison. The emotion recognition figures rely on emotion2vec, the same embedding family used for conditioning, so those numbers are biased in the model's favor.\n\nThere are also sloppy internal inconsistencies: the abstract says six speakers, Sections 1 and 3.1 say ten; Section 3.2 reports λ_rot instead of λ_contrast. Fixable, but worth cleaning.\n\nThe disentanglement assumption in Eq. (1) — that the normalized difference between emotional and neutral embeddings removes speaker identity — is plausible but unproven. If that direction is speaker-dependent, the whole control mechanism weakens.\n\nWho should read it: anyone building emotional TTS or looking for a clean Mandarin emotional dataset. The evaluation needs to be redone before the claims can be trusted. I'd send it to review, but with the clear message that the human eval must be expanded and the WER gap addressed. As a preprint, treat the numbers as marketing, the dataset as the takeaway.","headline":"The dataset and system skeleton are worth having; the evaluation is too weak to support the claimed superiority, so treat the numbers as marketing and the dataset as the takeaway.","tokens_in":11511,"tokens_out":3615,"would_cite":true,"duration_ms":40876,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Marco-Voice claims voice cloning and emotional control can be unified in one TTS model, with the highest reported speaker similarity (0.8275) and best emotional expression among compared systems.","keywords":["text-to-speech synthesis","voice cloning","emotional speech generation","speaker–emotion disentanglement","contrastive learning","flow matching","emotional speech dataset"],"falsifier":"Compute the average emotion direction from female speakers only, then use it to synthesize the same emotion for male speakers; if emotion-recognition accuracy on the male output falls far below the female output, the direction is not speaker-independent and the central premise fails.","tokens_in":10523,"feed_emoji":"🎙️","tokens_out":7637,"duration_ms":74540,"temperature":0.7,"pith_summary":"This paper argues that the two long-standing goals of speech synthesis — cloning a given voice and making that voice speak with a chosen emotion — can be achieved in one unified model rather than through separate modules. The central idea is to separate speaker identity from emotional style in the learned embedding space: an emotion is represented as the normalized difference between an emotional utterance and a neutral utterance from the same speaker, and training forces speaker and emotion embeddings to be orthogonal while contrastive learning keeps different emotions apart. On top of that, a cross-attention mechanism lets the emotion vector modulate the language-model-generated acoustic tokens before a flow matching module renders the waveform. To support this, the authors built CSEMOTIONS, a 10-hour Mandarin emotional speech dataset from ten professional speakers covering seven emotion categories. The paper reports that Marco-Voice outperforms the CosyVoice baselines on every rated dimension, including the highest speaker similarity (0.8275) and the best emotional expression score (4.225).","feed_headline":"Marco-Voice unifies voice cloning and emotion control","feed_subtitle":"One model separates speaker identity from emotional style, beating CosyVoice on expressiveness and similarity.","key_machinery":"The load-bearing object is the rotational emotion embedding: the normalized difference vector $v_e^i = (u_e^i - u_n^i)/\\|u_e^i - u_n^i\\|$ between a pretrained emotion encoder's representations of an emotional utterance and a neutral utterance from the same speaker, averaged over speakers to obtain $e = \\frac{1}{N}\\sum_i v_e^i$. This vector is treated as a direction in embedding space that carries the emotional content while cancelling speaker identity. Two training losses enforce the disentanglement: a cross-orthogonality loss that drives batch-wise speaker and emotion embeddings to be perpendicular, and an in-batch contrastive loss that separates emotion embeddings belonging to different emotional states. A cross-attention layer then uses $e$ as the query against the language model's token sequence as keys and values, so the emotion signal modulates the acoustic tokens before the conditional flow matching module synthesizes speech.","core_discovery":"The paper is trying to establish that emotional content and speaker identity can be separated in the embedding space of a pretrained emotion encoder, and that this separation is enough to control both dimensions in a single text-to-speech system. For each speaker, an emotion direction is the normalized difference between the emotional and neutral embeddings of that speaker, $v_e^i = (u_e^i - u_n^i)/\\|u_e^i - u_n^i\\|$, and the system uses the average of such directions over speakers as the emotion embedding. A cross-orthogonality loss makes the speaker and emotion embedding matrices perpendicular in each batch, an in-batch contrastive loss pushes different emotions apart, and a cross-attention layer uses the emotion embedding as a query over the language model's acoustic tokens before a flow-matching module renders the waveform. The reported result is that Marco-Voice outperforms CosyVoice1 and CosyVoice2 on every rated dimension, including the highest speaker similarity (0.8275) and the best emotional expression (4.225), in both human ratings and automatic metrics.","pith_inferences":["Beyond the paper, the same paired-difference recipe could be applied to other voice attributes beyond emotion — accent, speaking rate, formality, or even health state — whenever neutral and attribute-bearing samples from the same speaker are available.","The paper's own gender analysis suggests the speaker-independence of emotion directions may be weaker for male voices; a natural extension is to estimate emotion directions per gender or to add a gender-conditioned correction before averaging.","Because the approach needs paired emotional-neutral speech from the same speaker, scaling to new languages will require either more paired data or a self-supervised way to estimate a neutral anchor; this is a bottleneck the paper acknowledges as a limitation."],"forward_implications":["A single model can clone a voice and then re-render that voice in any of the seven supported emotions, so downstream systems no longer need separate voice-conversion and emotion-transfer modules.","Because the emotion embedding is a continuous direction rather than a discrete one-hot label, the same mechanism can in principle produce intermediate emotional intensities by scaling the rotation, not just categorical emotions.","The new CSEMOTIONS dataset gives the Mandarin speech community ten hours of studio-quality, seven-emotion speech from ten professional speakers, with parallel prompts in Chinese and English.","In the paper's reported evaluations, Marco-Voice's speaker similarity (0.8275) exceeds CosyVoice1 (0.700) and CosyVoice2 (0.605), and its emotional-expression rating (4.225) exceeds CosyVoice2 (3.240)."],"supporting_citations":[{"why":"Supplies the normalized emotional-minus-neutral difference that defines the rotational emotion embedding (Eq. 1).","marker":"[Chen et al., 2024a]"},{"why":"CosyVoice is the base architecture and training checkpoint on which Marco-Voice is built.","marker":"[Du et al., 2024a]"},{"why":"CosyVoice2 is the primary baseline for emotional speech generation and voice cloning comparisons.","marker":"[Du et al., 2024b]"},{"why":"ESD provides the main public emotional training corpus used alongside CSEMOTIONS.","marker":"[Zhou et al., 2022]"},{"why":"Provides the conditional flow matching formulation used in the acoustic synthesis module.","marker":"[Lipman et al., 2023]"},{"why":"Supplies the in-batch contrastive learning idea that separates emotion embeddings.","marker":"[Gao et al., 2021]"},{"why":"SpeechBrain supplies the speaker model used for the speaker-similarity metric.","marker":"[Ravanelli et al., 2024]"},{"why":"LibriTTS supplies the English prompts used for objective evaluation.","marker":"[Zen et al., 2019]"},{"why":"AISHELL-3 supplies the Mandarin prompts used for objective evaluation.","marker":"[Shi et al., 2021]"},{"why":"ERes2Net supplies the second speaker-verification model for similarity scoring.","marker":"[Chen et al., 2024b]"}],"fun_headline_variants":["Marco-Voice splits speaker identity from emotion for better TTS","Emotion and speaker disentangled in one speech synthesis model","Marco-Voice beats CosyVoice on expressiveness and speaker similarity","Contrastive learning enables independent emotion and identity control","Voice cloning and emotion control unified in Marco-Voice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole framework rests on the assumption that subtracting a speaker's neutral embedding from their emotional embedding leaves a direction that encodes only the emotion, independent of who the speaker is; if that direction is speaker-dependent or not linearly separable, the disentanglement and emotional control collapse.","fun_headline_variants_meta":{"raw":{"variants":["Marco-Voice splits speaker identity from emotion for better TTS","Emotion and speaker disentangled in one speech synthesis model","Marco-Voice beats CosyVoice on expressiveness and speaker similarity","Contrastive learning enables independent emotion and identity control","Voice cloning and emotion control unified in Marco-Voice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000282,"raw_usage":{"total_tokens":1676,"prompt_tokens":958,"completion_tokens":718,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":634}},"tokens_in":574,"tokens_out":718,"duration_ms":7593,"temperature":1.0,"reasoning_tokens":634,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:11:34.582100+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the average emotion direction from female speakers only, then use it to synthesize the same emotion for male speakers; if emotion-recognition accuracy on the male output falls far below the female output, the direction is not speaker-independent and the central premise fails.","supporting_citations":[],"review_version":1}