{"id":"95804fdc-c0e0-4ce9-b09a-5f2c6193877a","arxiv_id":"2608.11590","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CookVoice is a single compact model that generates both speech and singing, aligning content, style, and prosody at the frame level and reporting stronger style and pitch controllability than larger unified baselines.","lead":"CookVoice is a single, compact computer model that can generate both spoken and sung voice from text, style descriptions, reference voices, and pitch contours. Its main selling point is fine-grained control over style and melody with far fewer parameters and faster generation than larger unified systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"S-SIM's style encoder [29] is from the same research group as CookVoice; without evidence that this metric is impartial across systems, the headline style-controllability advantage is not established.","rationale":"The reader's weakest assumption is also the most load-bearing concern I can identify. The paper's principal differentiator is style controllability, and the only direct evidence for that is S-SIM. Since the style encoder in Section 4.4 is self-cited [29] and comes from the same group that developed CookVoice's style conditioning, the metric's impartiality is not independently established. This does not mean the result is wrong; it means the central claim is not yet supported beyond reasonable doubt. I do not see a stronger, more concrete flaw: the F0-based metrics are definitionally tied to the conditioning signal, the efficiency numbers are reported with a reasonable RTF and ODE-step ablation, and the architecture is described in enough detail to be plausible. The main remaining weaknesses are the S-SIM confound and the lack of listener-count/significance detail, which the conditional verdict already captures. Therefore the reader's verdict should stand unchanged, pending an independent style-similarity evaluation.","tokens_in":21375,"tokens_out":10844,"duration_ms":117977,"concrete_test":"Recompute the S-SIM column of Table 2 with an independently published style or speaker embedding model (e.g., ECAPA-TDNN or WavLM-TDNN embeddings), using the same generated audio and reference voices, and test whether CookVoice's advantage over Vevo2 (91.65 vs 75.11 in TTS; 95.00 vs 88.09 in TTSV) persists beyond the reported standard deviations. Additionally, report the correlation between AutoSIFT-space S-SIM and the independent-space S-SIM across all baselines; a weak correlation would indicate that the original metric is not measuring a generalizable notion of style similarity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of stronger style controllability is almost entirely carried by the S-SIM column of Table 2: CookVoice reaches 91.65% (TTS) and 95.00% (TTSV), versus 75.11% and 88.09% for the strongest baselines. Section 4.4 defines S-SIM as cosine similarity between style embeddings from a pre-trained style encoder, citing [29]. That encoder is from the same research group and belongs to the same style-representation lineage as the ParaMeta/AutoSIFT work [10, 29] that also underlies CookVoice's own voice-style conditioning. If the metric's embedding space is shaped by the same data, labels, or objectives used to train CookVoice's style encoder, the metric will systematically favor CookVoice over Vevo2 and other baselines that were not optimized in that space. The manuscript does not rule this out: the encoder's training data, its overlap with CookVoice's training data, and its independence from the conditioning scheme are not reported. Because the headline 'stronger style controllability' collapses if S-SIM is biased, this is the load-bearing assumption. The prosody and efficiency comparisons are less vulnerable, since F0-CORR is an acoustic signal-level measure and the ODE-step behavior is described in Appendix C, but those do not rescue the style-superiority claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents CookVoice, a non-autoregressive flow-matching DiT framework that unifies speech and singing voice generation by decomposing the voice into content, prosody, and style and expanding these control signals to the acoustic frame level. Style can be provided as a text prompt or a reference voice, and prosody can be provided as discrete lexical or note tokens or as a continuous F0 contour. A random condition-switching strategy is used during training so that a single model can support multiple task combinations. Experiments compare CookVoice against CosyVoice, F5-TTS, IndexTTS, ParaStyleTTS, Vevo2, DiffSinger, StyleSinger, TCSinger, and Vevo1.5. The paper claims comparable generation quality to baselines while providing stronger style similarity (S-SIM) and prosody controllability (F0-CORR), with only 43.51M parameters and inference using as few as 4 ODE steps.","tokens_in":21685,"tokens_out":9289,"duration_ms":99735,"significance":"The proposed design is clean and practically motivated: frame-level alignment of content, style, and prosody is an interpretable way to obtain fine-grained control, and random condition switching is an elegant mechanism for multi-task training. Objective prosody metrics such as F0-CORR and F0-RMSE are grounded in acoustic signals, and the efficiency analysis addresses a real deployment concern. However, the headline style-controllability claim rests almost entirely on S-SIM computed with an in-house style encoder from the same research group, subjective quality claims are not supported by statistical methodology, and the multi-task capability is not empirically demonstrated beyond TTS and TTSV. If these evaluation concerns are addressed, CookVoice would be a solid contribution to unified voice generation; as presented, the strongest claims are not yet adequately supported.","major_comments":[{"comment":"","section":"§4.4, Eq. (9), Table 2"},{"comment":"The subjective MOS and MC-MOS evaluations lack essential methodology: the number of listeners, the number of utterances per system, and any significance testing are not reported. In the TTS condition, CookVoice's best MOS (3.98) is numerically below several baselines (IndexTTS 4.42, F5-TTS 4.35, CosyVoice 4.30, Vevo2 4.30), so the abstract's claim of 'comparable generation quality' is not supported by the table as it stands. In the TTSV condition, the difference between CookVoice (3.40) and Vevo2 (3.42) is too small to interpret without significance tests. Please report listener counts, confidence intervals, and paired significance tests for the MOS and MC-MOS comparisons.","section":"§5.1, Table 3"},{"comment":"The paper's central multi-task claim (contribution 1 and Section 3.5) lists 13 tasks including voice conversion, singing voice conversion, speech and singing editing, sketch-to-voice, and humming-to-voice, but the experiments in Section 5 only evaluate TTS and TTSV. As written, support for these additional tasks is purely conceptual, resting on the claim that condition-switching allows the same model to handle them. Please provide at least a small proof-of-concept evaluation (e.g., objective speaker-similarity for VC, or qualitative demos with clear task definitions) for the principal non-TTS tasks, or explicitly delimit the empirical claims to TTS, TTSV, and style/prosody-controllable speech and singing. Without this, the unified multi-task contribution is overstated.","section":"Table 5, §5"},{"comment":"The claimed parameter efficiency ('only 43.51M parameters') appears to count only the trainable DiT-S generation backbone and not the frozen HiFi-GAN-style autoencoder or the style encoder that are part of the full generation pipeline. Since the parameter counts for baselines in Table 4 may use a different accounting convention, the efficiency comparison may be inconsistent. Please report a parameter breakdown for CookVoice (generation model, AE encoder and decoder, style encoder) and state whether the same accounting convention is applied to all baselines. In addition, report the real-time factor for the full pipeline, including latent encoding, ODE integration, and waveform decoding.","section":"§2.2, §5.5, Table 4"}],"minor_comments":[{"comment":"The relative-percentage annotations for F0-RMSE are ambiguous because lower is better, yet the table mixes downward and upward arrows without stating which direction indicates an improvement.","section":"Table 2"},{"comment":"The main text mentions WER, PhoER, and ProER, but the full per-language results appear only in Appendix F; please add a pointer to that table at the relevant point in Section 5.","section":"§4.4, Appendix F"},{"comment":"The metric is called M-CMOS in Appendix B but MC-MOS in Tables 3 and 4; please use one consistent name throughout the paper.","section":"Appendix B vs. Tables 3–4"},{"comment":"The voiced-frame set U is used in Eq. (5) without a formal definition; please define U as the set of voiced frames and state how voiced frames are detected.","section":"§4.2, Eq. (5)"},{"comment":"The sentence beginning 'Which leads to an observable style-sim degradation...' is a grammatical fragment; please merge it with the preceding sentence.","section":"§5.3"},{"comment":"The caption reads 'Redder indicate stronger performance'; this should be 'Redder colors indicate stronger performance'.","section":"Figure 2"},{"comment":"Vevo2 appears twice in the table (once for TTS and once for TTSV); consider merging the rows or adding a footnote to explain that the same model is listed in both task groups.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The most serious concern is the circularity risk surrounding S-SIM: the headline controllability comparison uses a style encoder from the same research group and potentially the same data lineage as CookVoice. Please ask the authors to provide an external or human-correlated validation of the style-similarity metric; without it, the main contribution is difficult to verify. Also check whether the parameter-efficiency comparison uses consistent accounting across systems, as this affects the paper's second headline claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CookVoice is a genuine practical effort: a 43.51M-parameter non-autoregressive model that unifies TTS, singing, voice conversion, and editing through frame-level conditioning, with RTF 0.04 and solid ODE-step analysis. The flexible alignment—content, prosody (discrete tokens or continuous F0), and style (text prompt or reference voice) all expanded to frame level in a flow-matching DiT—is a clean design, and the mixed speech/singing training with random condition switching is a sensible way to get multi-task behavior. The efficiency and F0-CORR gains over Vevo2 and others are real and worth attention.\n\nThe soft spots are equally clear. The headline style-controllability claim rests almost entirely on S-SIM, computed with the authors' own AutoSIFT encoder [29], same lineage as the ParaMeta style representations underlying CookVoice. The paper does not disclose the encoder's training data, its overlap with CookVoice's training data, or any evidence of impartiality across systems. That makes the style-superiority claim unproven—this is the load-bearing weakness and the stress-test concern is on target. Second, the abstract's 'comparable generation quality' overstates things: CookVoice's best TTS MOS is 3.98, below 4.30+ for several baselines; only the singing MOS is comparable. Third, subjective tests lack listener counts and significance tests, so the MOS differences should be read cautiously.\n\nI find the architecture itself sound and the paper mostly honest—it states its small-data, small-model limitations plainly. The math is standard flow-matching, and the citations are appropriate. The main fix is to evaluate style similarity with an independent encoder or, at minimum, disclose the AutoSIFT training details and show that the metric does not favor CookVoice. The efficiency and architecture contributions are likely to survive that check.\n\nWho is this for? Speech/audio researchers interested in efficient unified TTS/singing systems; it's a useful case study in both practical design and evaluation-metric circularity.\n\nThis paper deserves peer review, not a desk reject. The right outcome is likely major revision: add listener counts and significance tests, soften the abstract, and re-verify or replace the S-SIM measurement.","headline":"A practical unified TTS/singing framework with real efficiency gains, but its style-controllability lead rests on an in-house metric that needs independent verification.","tokens_in":22193,"tokens_out":4653,"would_cite":true,"duration_ms":49019,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CookVoice claims a single non-autoregressive model with frame-level alignment can generate both speech and singing voice, matching larger baselines on quality while exceeding them on style and prosody control.","keywords":["voice generation","text-to-speech","text-to-singing voice","style control","prosody control","flow matching","diffusion transformer","non-autoregressive"],"falsifier":"Re-score the same generated samples with human style-similarity ratings or with an independently trained style encoder; if CookVoice's roughly 16-percentage-point TTS and 7-percentage-point TTSV S-SIM lead over the strongest baselines shrinks or reverses, the frame-level controllability advantage is not established.","tokens_in":21173,"feed_emoji":"🎤","tokens_out":12187,"duration_ms":118257,"temperature":0.7,"pith_summary":"CookVoice claims that a single non-autoregressive (parallel) model can generate both speech and singing voice by decomposing every voice into content, prosody, and style, and by aligning all control signals to individual acoustic frames. If this is right, one 43.51-million-parameter system replaces task-specific text-to-speech, singing, style-transfer, and voice-editing models without sacrificing quality and with finer-grained control than larger autoregressive systems. The paper reports style similarity of 91.65% for speech and 95.00% for singing, pitch-following correlation of 0.7102 and 0.8425, and real-time inference with 4–8 ODE steps. The unification matters because a user can decide what is said, how it sounds, and where the pitch comes from within one model rather than switching between isolated task systems.","feed_headline":"One 43M-parameter model speaks and sings with frame-level control","feed_subtitle":"CookVoice claims stronger style and pitch control than larger baselines, in real time.","key_machinery":"The central mechanism is a flexible frame-level alignment strategy. Phoneme durations, discrete prosody tokens, style embeddings, and a continuous F0 contour are each expanded or projected so that every control signal is represented as a sequence of length $T$, the same number of frames as the target latent spectrogram; the expanded style, content, and prosody tensors are concatenated along the feature dimension into a conditioning tensor $C \\in \\mathbb{R}^{3D \\times T}$ that a flow-matching DiT (a transformer trained to regress the vector field carrying Gaussian noise to the target latent audio) uses as cross-attention conditioning to regress the vector field $Y_1 - Y_0$. Task differences reduce to duration-expansion rules, learned or predicted for speech and score-derived for singing, and to a training-time condition-switching scheme that randomly samples style from text or voice and prosody from discrete tokens or continuous F0, so no task-specific heads or objectives are required. The prosody encoder removes the voice-level mean from F0, so the explicit contour carries only relative melody, avoiding entanglement with the style embedding.","core_discovery":"CookVoice's central claim is that frame-level alignment, not model scale or autoregressive decoding, is what enables precise style and prosody control across speech and singing. The model decomposes a voice into content (phonemes), prosody (lexical tones, stresses, MIDI notes, or a reference F0 contour), and style (text prompt or reference voice), expands each control signal to the acoustic frame length, concatenates them, and feeds the result to a flow-matching diffusion transformer that generates a latent acoustic embedding. Because the same conditioning machinery serves every task, text-to-speech, text-to-singing, style-controllable generation, voice mimicry, voice conversion, and voice editing are all just different combinations of the same three factors. In the reported evaluations, the best configurations reach style similarity of 91.65% for TTS and 95.00% for TTSV, F0 correlation of 0.7102 and 0.8425, and MOS of 3.98 for TTS and 3.40 for TTSV, with 43.51M parameters and real-time inference.","pith_inferences":["The paper reports experiments for TTS and TTSV; a direct test of the unified claim would be to measure whether the same frame-level conditioning also improves the listed but unevaluated tasks of voice conversion, speech/singing editing, sketch-to-voice, and humming-to-voice.","Because the prosody encoder deliberately strips the voice-level mean from F0 and leaves absolute pitch to the style embedding, the design predicts that shifting a reference contour's mean while keeping style fixed should not change the generated pitch range; supplying a plain reference F0 with a different mean would test this decoupling directly.","The reported optimum of 4–8 ODE steps is task-dependent: intelligibility is best near 4 steps while style and prosody fidelity keep improving to 8, so a deployment could pick the step count by which objective matters more.","The authors note the model has not been scaled; if the architecture's ceiling is capacity, training the same design with a larger DiT backbone and more data could plausibly close the remaining MOS gap to large baselines while keeping the efficiency and controllability profile. This is an untested scaling prediction, not a claim of the paper."],"forward_implications":["A single 43.51M-parameter model can cover TTS, text-to-singing, style-controllable generation, voice mimicry, voice conversion, and voice editing by toggling which of content, style, and prosody conditions are active.","Users can combine style from a text prompt or reference voice with prosody from discrete tokens (tones, stresses, MIDI notes) or a continuous reference F0 contour in any pairing.","At 4–8 ODE steps the system generates audio faster than real time, using about 5% of the parameters, about 20% of the GPU memory, and less than 1% of the inference time of the closest unified speech-and-singing baseline, while reporting higher style and prosody controllability.","Voice-based style plus continuous F0 gives the strongest reported controllability (S-SIM 91.65/95.00, F0-CORR 0.7102/0.8425 for TTS/TTSV), while text-style plus discrete prosody still improves on the baselines by 41.48% and 13.11% S-SIM respectively.","Inference step count is a real trade-off in the reported results: style and prosody fidelity improve up to roughly 8 steps, while intelligibility peaks near 4 steps, so the model's operating point depends on whether content clarity or style adherence matters more."],"supporting_citations":[{"why":"Defines the autoencoder architecture that maps spectrograms to latent embeddings and back, giving the model its generation target.","marker":"[9]"},{"why":"Supplies the DiT backbone used as the flow-matching acoustic model.","marker":"[11]"},{"why":"Defines the optimal-transport flow-matching objective and the ODE sampling procedure used at inference.","marker":"[12]"},{"why":"Provides the FFT content-encoder blocks and duration-predictor approach adapted for phoneme expansion.","marker":"[13]"},{"why":"Supplies the pre-trained phoneme alignments and duration predictor used to supervise speech durations, and is also the paralinguistic-style baseline.","marker":"[3]"},{"why":"Converts text and lyrics into phonemes and discrete prosody tokens (tones, stresses) for content and prosody encoding.","marker":"[19]"},{"why":"Provides the frozen sentence encoder that turns text style prompts into semantic embeddings for text-based style control.","marker":"[17]"},{"why":"Supplies the pre-trained style encoder used to compute the S-SIM style similarity metric for all models.","marker":"[29]"},{"why":"Transcribes generated audio for the WER, PhoER, and ProER intelligibility metrics.","marker":"[28]"},{"why":"The closest unified speech-and-singing baseline; its quality, controllability, and efficiency numbers anchor the main comparisons.","marker":"[8]"}],"fun_headline_variants":["One 43M model for speech, singing, and style control","CookVoice: unified voice generation with frame-level control","Six voice tasks, one tiny model: CookVoice at 43M params","Real-time voice generation: CookVoice speaks and sings with 43M","Frame-level alignment makes CookVoice a voice generation all-rounder"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The controllability claims rest on a style-similarity metric computed by a single pre-trained style encoder; if that encoder is biased toward this model's own conditioning design, the reported advantage over the baselines would not hold under an independent measure.","fun_headline_variants_meta":{"raw":{"variants":["One 43M model for speech, singing, and style control","CookVoice: unified voice generation with frame-level control","Six voice tasks, one tiny model: CookVoice at 43M params","Real-time voice generation: CookVoice speaks and sings with 43M","Frame-level alignment makes CookVoice a voice generation all-rounder"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000929,"raw_usage":{"total_tokens":4022,"prompt_tokens":1029,"completion_tokens":2993,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":2903}},"tokens_in":645,"tokens_out":2993,"duration_ms":22991,"temperature":1.0,"reasoning_tokens":2903,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:34:31.834102+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-score the same generated samples with human style-similarity ratings or with an independently trained style encoder; if CookVoice's roughly 16-percentage-point TTS and 7-percentage-point TTSV S-SIM lead over the strongest baselines shrinks or reverses, the frame-level controllability advantage is not established.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the autoencoder architecture that maps spectrograms to latent embeddings and back, giving the model its generation target."},{"cited_title":"First, due to re- source constraints, CookV oice has not yet been scaled up","cited_arxiv_id":null,"evidence_quote":"Supplies the DiT backbone used as the flow-matching acoustic model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained phoneme alignments and duration predictor used to supervise speech durations, and is also the paralinguistic-style baseline."},{"cited_title":"Vevo2: A unified and controllable framework for speech and singing voice generation,","cited_arxiv_id":null,"evidence_quote":"Converts text and lyrics into phonemes and discrete prosody tokens (tones, stresses) for content and prosody encoding."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The closest unified speech-and-singing baseline; its quality, controllability, and efficiency numbers anchor the main comparisons."}],"review_version":1}