{"id":"d5ad1f35-c0e4-4682-b47e-cc1b5e4110d6","arxiv_id":"2501.04256","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A sketch-conditioned diffusion TTS model that turns coarse user-drawn pitch and energy trends into natural, precisely controlled speech.","lead":"DrawSpeech lets users control speech synthesis by drawing rough pitch and energy sketches, and a diffusion model fills in the detailed contours. The paper reports natural and expressive speech with fine-grained prosody control on a single-speaker English dataset, though the evidence has some statistical and baseline weaknesses.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"User-drawn sketches are not in the training distribution: the central 'any sketch' controllability claim is untested.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the method relies on the sketch-to-contour predictor generalizing from automatically smoothed contours (training) to user-drawn sketches (inference). The paper's evaluations do not test the user-drawn distribution: Table I and Table III use automatically extracted sketches from reference speech, and Figure 3 uses author-drawn sketches without quantitative metrics. Thus the central claim of user-friendly, precise control over arbitrary user sketches is not yet supported. This concern is external validity rather than internal inconsistency, and it is addressable with a proper user study, so it justifies a conditional verdict rather than rejection. The paper does provide released code and audio samples, which would enable the proposed test directly, and the architecture itself is reasonable. The p-value and baseline issues noted by the reader are secondary; the distribution mismatch is the most load-bearing because it attacks the fundamental premise of the control signal.","tokens_in":8531,"tokens_out":3111,"duration_ms":31474,"concrete_test":"Recruit N=20 naive users who are not authors; for 10 fixed sentences each, ask them to draw a pitch sketch expressing a specified emphasis target (e.g., emphasize 'stole'). Use the released DrawSpeech implementation with the user-drawn sketch as control, synthesize speech, extract pitch, and compute the paper's SC metric between the user sketch and synthesized pitch, and RMSE between predicted and synthesized pitch. Compare to the same metrics when the condition is an automatically Savitzky-Golay-smoothed sketch of a reference utterance with the same intended emphasis. If mean user-drawn SC is lower than auto-smoothed SC by more than 0.5 or RMSE increases by >20%, the distribution-mismatch concern is confirmed; if not, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section II-A defines training sketches as Savitzky-Golay smoothed ground-truth pitch/energy contours (Pske, Eske from P,E). The sketch-to-contour predictor in Section II-B is trained exclusively on these smoothed natural contours. At inference, users are expected to draw 'any prosody sketch'; however, human-drawn sketches are piecewise-linear, often contain sharp corners, arbitrary plateaus, and may encode emphasis patterns that do not occur in natural speech. The paper provides no evidence that human sketches are in-distribution for the predictor. The RMSE evaluation in Table III conditions on sketches extracted from reference speech (not user-drawn); Figure 3 is a qualitative demonstration with sketches drawn by the authors; the SC metric in Table I is also applied to automatically extracted sketches. Consequently, the central claim that users can precisely control fine-grained prosody by drawing arbitrary sketches is not supported by the experiments. If the predictor degrades on out-of-distribution sketches, the claimed controllability fails even though the architecture and baselines remain unchanged.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DrawSpeech, a text-to-speech system that conditions synthesis on a prosody sketch: a smoothed version of the pitch and energy contours. The sketch is intended to be an easy, user-drawn control signal that conveys the desired prosodic trend. The architecture consists of a sketch extractor that produces smoothed contours from ground truth during training, a sketch-to-contour predictor that reconstructs detailed pitch and energy contours from the sketch and text, and a latent diffusion model conditioned on the contours, sketches, and text. The experiments on LJSpeech report that DrawSpeech achieves a MOS of 4.49 under sketch conditions, an SC of 4.30, and lower RMSE than text-only synthesis when reconstructing reference prosody. The paper claims human-level quality and fine-grained, user-friendly prosody control.","tokens_in":8694,"tokens_out":3940,"duration_ms":38308,"significance":"If the method works as claimed, the prosody-sketch control signal is a genuinely novel and user-friendly paradigm for fine-grained prosody control in TTS, addressing a well-known limitation of both reference-speech prompting and natural-language description. The paper provides a clear architecture, open-source implementation, and public audio samples, which are strengths for reproducibility. However, the experimental evidence does not yet substantiate the strongest claim that arbitrary user-drawn sketches can precisely control prosody, and the ground-truth comparison is overstated. The core idea is defensible and likely of interest to the speech-synthesis community, but the presented support is incomplete.","major_comments":[{"comment":"The central claim that users can draw 'any prosody sketch' to achieve precise control is not supported by the evaluation. Training sketches are produced by Savitzky-Golay smoothing of ground-truth contours, but all quantitative evaluations (Tables I and III) use sketches extracted from reference speech, not user-drawn sketches. Figure 3 uses a few author-drawn sketches but is only a qualitative illustration. This leaves open the possibility that the sketch-to-contour predictor does not generalize to human-drawn sketches, which typically contain sharp corners, piecewise-linear segments, and emphasis patterns not present in smoothed natural contours. Please add a user study where participants draw sketches for a set of sentences and measure the correlation (SC) or the objective match between the drawn sketch and the synthesized speech's prosody, or at least provide a quantitative analysis of the distribution gap between SG-smoothed contours and user sketches.","section":"Section II-A and Section III-B"},{"comment":"The statement that sketch-conditioned DrawSpeech 'even slightly surpasses the ground truth speech' is not supported by the reported p-value of 0.058. A Wilcoxon rank-sum test with p > 0.05 indicates no statistically significant difference, so the correct interpretation is that DrawSpeech is comparable to ground truth, not superior. The text should be revised to avoid the misleading 'surpass' language, and the p-value should be interpreted as evidence of non-inferiority rather than superiority.","section":"Table II and Section III-B-2"},{"comment":"The comparison with FastSpeech 2 and NaturalSpeech 2 is not a head-to-head evaluation of the proposed control signal, because these baselines are given sketches without any training adaptation for such inputs. The resulting low MOS scores are expected and do not establish that DrawSpeech is superior for sketch-based control. The fine-tuned FastSpeech 2 (FT) is a more meaningful baseline, but it still integrates the proposed sketch extractor and predictor, so the comparison is not fully apples-to-apples. To strengthen the significance of the method, the authors should also compare DrawSpeech against state-of-the-art reference-based and text-based prosody control methods, not only against models that are not designed for sketches.","section":"Section III-B-1, Table I"},{"comment":"The Savitzky-Golay filter parameters (window size and polynomial order) are not reported. These parameters define the smoothness of the training sketches and therefore directly influence the sketch-to-contour predictor's behavior and the meaning of the control signal. Without these parameters, the method is not fully reproducible and the reader cannot assess whether the chosen smoothing is appropriate for the claimed user-friendly sketch interface. Please specify the filter configuration.","section":"Section II-A and Section III-A-2"}],"minor_comments":[{"comment":"There are typos in the figure: 'V ocoder' should be 'Vocoder' and 'Ptich' should be 'Pitch'.","section":"Figure 1"},{"comment":"The text 'I did’t say you stole the money' contains a typo; it should be 'I didn’t say you stole the money'.","section":"Figure 3 caption"},{"comment":"The phrase 'It’s much easier for users to draw a pitch sketch' should use 'It is' for formal style, and the sentence beginning 'because it relieve users’ burden' has a subject-verb agreement error; it should be 'because it relieves users’ burden'.","section":"Section II-A"},{"comment":"The symbol P is used both for the pitch contour (Section II-A) and for the sketch-to-contour predictor (Eq. 2), which is confusing. Please rename the predictor, e.g., to F, to avoid ambiguity.","section":"Section II-B, Eqs. (1) and (2)"},{"comment":"The definition of the sketch correlation (SC) metric is incomplete: it is not stated how many samples were rated per listener, whether the listeners could see the sketch while rating, or how the ratings were aggregated. Please clarify the protocol.","section":"Section III-A-4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and timely topic in expressive TTS, and the sketch-based control paradigm is a reasonable contribution. The main issues are experimental: the lack of direct evaluation with user-drawn sketches, the overstated ground-truth comparison, and the unfair baseline comparison. These are addressable within the scope of a revision. I recommend major revision rather than rejection, provided the authors add the missing experiments and adjust the claims accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick take on DrawSpeech. The core idea is genuinely new: instead of reference audio or natural-language prompts, the user draws a rough pitch/energy sketch, and a latent diffusion model fills in the detailed contours. That's a real gap in the TTS controllability toolbox, and the paper makes a serious attempt to fill it. The architecture is sensible — phoneme-level contours, a sketch-to-contour predictor, LDM with VAE — and the authors ship code and audio samples, which is more than many papers in this area do.\n\nThe good stuff: the training process with Savitzky-Golay smoothed natural contours is a clean way to get sketches for free, and the fine-tuned FastSpeech 2 baseline is a fair comparison. The MOS of 4.49 with sketch conditions is respectable, and the qualitative Figure 3 does show the intended emphasis effects.\n\nNow the soft spots, in order of importance. The biggest one is the 'any sketch' claim. The model is trained on smoothed versions of natural contours. Human-drawn sketches are likely piecewise-linear, with sharp corners and flat plateaus that don't resemble Savitzky-Golay output. The paper gives no evidence that real users' sketches are in-distribution. The only hand-drawn examples are by the authors, and the RMSE experiment uses sketches extracted from reference speech. So the central controllability claim — that users can draw arbitrary sketches and get the prosody they want — is untested. This is exactly the stress-test concern, and I think it lands.\n\nSmaller but real: the p-value in Table II. They report 0.058 for DrawSpeech vs. ground truth and conclude it 'slightly surpasses' ground truth. That's backwards. p > 0.05 means no significant difference, not superiority. The right takeaway is 'statistically indistinguishable,' which is still fine for a human-level claim, but the wording needs correcting.\n\nThe SC metric is a subjective rating of 'alignment' — okay as a sanity check, but not a precise controllability measure. And the baselines without fine-tuning are given sketches in a way they aren't designed for; the fine-tuned version mitigates this, but the table could be clearer about which comparison matters.\n\nNone of these are fatal. The idea is worth a serious referee. A revision with a user study (even 10 people drawing sketches), a corrected statistical wording, and a clearer separation of 'imitate this reference' versus 'obey this drawing' would get it there. I'd send it to review, expecting major revisions.","headline":"A genuinely new sketch-based prosody control for TTS, held back by a missing user study and a misread p-value.","tokens_in":9239,"tokens_out":3022,"would_cite":false,"duration_ms":29215,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-10T21:36:49.864306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}