{"id":"cbf53725-d0b6-45d4-b2af-4e75516ce632","arxiv_id":"2507.12015","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"EME-TTS combines weakly supervised emphasis labels, variance-based pitch and duration features, and an attention block to synthesize emotional speech with controllable, clearly audible word emphasis.","lead":"Researchers built a text-to-speech system that lets users control which words are emphasized while also setting the emotion of the voice, using pseudo-labels and an attention adjustment block. The work targets a practical gap: emotional voices often blur or shift intended emphasis, and this design tries to keep emphasized words clear across emotions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EmphaClass pseudo-labels are unvalidated on ESD; if they mislabel emotional prosody as emphasis, the supervision in §2.3 and the EAT evaluation rest on a shifting reference.","rationale":"The reader's weakest assumption is exactly the one I would stress-test first: EmphaClass pseudo-labels on ESD are the ground truth for every emphasis-related loss and for the EPE mask, and no validation is reported. The paper's own Section 2.2 says 'highly reliable' but gives no agreement, no per-emotion analysis, and no comparison with human annotation. On acted emotional data, the risk is concrete: surprise is described in Section 3.2.1 as having phrase-final pitch rises that make listeners mislocate emphasis, and the same acoustic pattern could produce false-positive emphasis labels in a recognizer trained on a different benchmark. If that happened, Eqs. (3)-(6) would encode the wrong targets and Table 1 would not measure what the abstract claims. The LLM used at inference is a second unvalidated input; with no prompt or evaluation, the pipeline cannot be reproduced. Neither issue makes the method impossible; both are fixable with validation and reporting, so CONDITIONAL remains the right verdict rather than ACCEPT or REJECT. My read therefore leaves the reader's verdict unchanged.","tokens_in":8439,"tokens_out":5285,"duration_ms":64023,"concrete_test":"On a stratified sample of 150 ESD utterances (30 per emotion), have at least three annotators mark emphasized words and compare majority human labels with EmphaClass word-level outputs per emotion; report precision/recall, Cohen's kappa, and the confusion pattern for surprise/happy. Also extract LLM emphasis predictions for the same EAT test sentences and score them against the human majority labels. If per-emotion agreement with humans is below a pre-registered threshold (e.g., kappa < 0.6) or shows systematic false positives on emotional prosody, retrain EME-TTS with human-corrected labels and re-run the Table 1 EAT. This single check settles whether the emphasis positions the model is trained and evaluated on are the positions human listeners perceive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim presupposes that the emphasis positions used for training are true emphasis positions. Section 2.2 assumes EmphaClass, trained on EmphAssess, transfers to ESD and asserts it yields 'highly reliable emphasis pseudo-labels' without reporting any validation on ESD. These labels are hard supervision: Eqs. (3)-(6) compute pitch and duration variance targets only inside pseudo-labeled emphasis regions, and the EPE mask in Eq. (10) uses the same start/end positions. ESD is acted emotional speech, so surprise and happy utterances contain large pitch excursions and phrase-final lengthening; if EmphaClass systematically flags those prosodic events as emphasis, the model learns a recognizer artifact rather than word-level emphasis. The EAT in §3.2.1 then measures listener agreement with that artifact-defined target, not with independently perceived emphasis. No per-emotion agreement, inter-annotator statistics, or error analysis is given. At inference, the LLM emphasis predictor is also unspecified, with no prompt or accuracy evaluation, so even a positive EAT cannot be cleanly attributed to the acoustic model's emphasis-emotion link.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EME-TTS, an emotional text-to-speech system built on EmoSpeech, which adds variance-based emphasis features and an Emphasis Perception Enhancement (EPE) block. Emphasis positions are obtained as pseudo-labels by running the EmphaClass recognizer on the ESD dataset, and the model is conditioned on both emotion labels and emphasis positions. At inference, an unspecified large language model predicts emphasis positions from text and emotion. The experiments compare EME-TTS with and without the EPE block, EmoSpeech, and CosyVoice2 on emphasis recognition accuracy, emotion recognition accuracy, expressiveness preference, and MOS/NISQA. The central claim is that EME-TTS, combined with LLM-based emphasis prediction, yields more natural emotional speech while preserving stable and distinguishable target emphasis across emotions.","tokens_in":8581,"tokens_out":3872,"duration_ms":47746,"significance":"The interaction between emphasis control and emotion in TTS is a timely and underexplored problem, and the proposed two-question framing is useful. If the claims were fully supported, EME-TTS would provide a practical control interface that couples emphasis positions and emotion labels in a single acoustic model, and the EPE block would be a concrete contribution. The manuscript also has several strengths: a multi-task evaluation protocol, inclusion of objective and subjective measures, and a public demo page with synthesized samples. However, the current evidence is not yet sufficient to establish the central claim: the pseudo-labeling step is unvalidated on the target corpus, the LLM component is unspecified, the listener-level statistics are missing, and no emphasis-aware baseline is included. These gaps are load-bearing because they affect the validity of the emphasis and emotion accuracy results.","major_comments":[{"comment":"Section 2.2 asserts that EmphaClass yields 'highly reliable emphasis pseudo-labels' on ESD, but no validation on ESD is reported. These labels are used as hard supervision for the variance targets in Eqs. (3)-(6) and for the EPE mask in Eq. (10), and they also define the expected answers in the Emphasis Accuracy Test. ESD is acted emotional speech, so prosodic events such as surprise pitch rises or phrase-final lengthening could be systematically misclassified as emphasis by a recognizer trained on EmphAssess. Please report per-emotion agreement between EmphaClass and human annotation on a held-out ESD subset, including inter-annotator statistics and error analysis; without this, the supervision and the EAT evaluation rest on an unverified transfer assumption.","section":"§2.2, Eqs. (3)-(6), Eq. (10)"},{"comment":"The paper's central claim is that EME-TTS 'when combined with large language models for emphasis position prediction' produces more natural emotional speech, but the LLM component is not specified: no prompt template, model version, or inference procedure is given, and no evaluation of the predicted emphasis positions is reported. As a result, the EAT and EEPT results cannot be cleanly attributed to the acoustic model; they may reflect LLM quality rather than the proposed architecture. Please provide the exact LLM prompt and a quantitative evaluation of the predicted emphasis positions against human judgments or a held-out EmphaClass reference.","section":"§3.2.2 and Abstract"},{"comment":"The main support for the emphasis and emotion claims consists of small differences in accuracy scores measured with only 11 listeners and 80 items per task, but the paper reports no confidence intervals, significance tests, or listener-level variance. For example, the EAT mean is 0.78 vs. 0.73, and the EAT-EMO mean is 0.67 vs. 0.58, with per-emotion values that fluctuate considerably (e.g., happy accuracy 0.32 in Table 3). Please report paired significance tests (e.g., Wilcoxon signed-rank or bootstrap) and confidence intervals for each table, and state the chance level and the number of candidate words in the EAT task.","section":"Tables 1 and 3, §3.2.1-3.2.2"},{"comment":"No emphasis-aware baseline is included. The comparison to EmoSpeech and CosyVoice2 addresses emotion accuracy, but the paper's central emphasis-control claim is supported only by the ablation of the EPE block. To isolate the contribution of the EPE block and the variance-based emphasis modeling, the EAT should also be run on an existing emphasis-controllable TTS system (e.g., the variance-feature approach of [19] adapted to the same emotion-conditioned backbone) under identical conditions. Without such a baseline, the reader cannot tell whether the reported emphasis accuracy comes from the proposed components or from the underlying FastSpeech2/EmoSpeech backbone.","section":"§3.2.1 and Tables 1-3"},{"comment":"The emphasis intensity control relies on two free parameters: the strength value in the Emphasis Adapter (set to 0.2) and the normalization range [0,2] for variance features. No sensitivity analysis or justification is given for either choice. Since these parameters directly control the magnitude of emphasis modulation, a small sweep (or at least a rationale based on preliminary listening) is necessary to establish that the reported gains over the w/o EPE variant are not an artifact of a particular parameter setting.","section":"§2.3, §2.4, §3.1"}],"minor_comments":[{"comment":"The per-emotion numbers in Table 1 are visually misaligned; the column boundaries are unclear, making the table hard to read.","section":"Table 1"},{"comment":"The abbreviations EAT and EAT-EMO are easily confused; renaming the emotion task (e.g., ERT for emotion recognition test) would improve clarity.","section":"§3.2.2"},{"comment":"The MOS and NISQA results are reported only as overall means; given the paper's emphasis on per-emotion effects, a per-emotion breakdown would be more informative.","section":"§3.2.4"},{"comment":"The sentence 'EME-TTS demonstrate its effectiveness' contains a subject-verb agreement error; it should be 'demonstrates'.","section":"§4"},{"comment":"The paper does not release code or trained models; the online demo page is useful but not sufficient for reproducibility of the reported quantitative results.","section":"General"},{"comment":"Reference [29] is the GPT-4 technical report, but the text never states which LLM and version was actually used for inference; this should be stated explicitly.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the proposed direction is interesting, but the experimental evidence is currently too preliminary. The main risks are the unvalidated pseudo-labeling transfer and the unspecified LLM component; both are addressable with additional experiments and analysis. I would encourage the authors to strengthen the statistics and add an emphasis-aware baseline before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this is a genuinely novel combination of known pieces. It takes variance-based emphasis features from Seshadri et al., adds a pretrained emphasis recognizer (EmphaClass) to pseudo-label an emotional corpus, and slots a small attention-masking block (EPE) into an EmoSpeech-style acoustic model. The targeted gap—how emphasis and emotion interact in synthesis—is real and understudied, and the architecture is simple enough to be usable. Credit where due: the subjective tasks are well designed, especially the emphasis accuracy test where listeners identify the emphasized word rather than rate prominence, and the surprise condition honestly exposes a known confound (sentence-final pitch rises). The contextualized expressiveness ranking is a nice addition. The conclusion also admits gains are limited for some emotions, which is candid.\n\nThe soft spots are real but mostly fixable. The headline numbers are thin: mean EAT 0.73 to 0.78 with 11 listeners and no confidence intervals or significance tests; surprise goes 0.55 to 0.64, which is an effect but a small one. The bigger problem, and the one the stress-test correctly flags, is the pseudo-label chain. EmphaClass is asserted to give \"highly reliable\" labels on ESD, but no validation, agreement, or error analysis is reported. ESD is acted emotional speech where surprise and happiness are naturally marked by large pitch excursions; if EmphaClass systematically flags those as emphasis, both the training supervision (Eqs. 3–6, the EPE mask) and the EAT evaluation are measuring a recognizer artifact, not true emphasis. The LLM emphasis predictor is also a black box—no prompt, no accuracy evaluation—so the end-to-end claim can't be cleanly attributed to the acoustic model. Finally, there is no comparison with an emphasis-only baseline like Seshadri et al., so we don't know whether EPE is what actually helps or whether the variance features already carry the effect.\n\nNone of this is fatal. The architecture is coherent and the demo may well support the perceptual claim. But as written, the evidence doesn't meet the abstract's promise of \"stable and distinguishable\" emphasis across emotions. I'd send this to peer review—the problem is worth solving and the approach is a step forward—but I'd condition acceptance on pseudo-label validation, significance testing, and at least one emphasis-only baseline. A serious referee should engage with it.","headline":"Plausible new combination of emphasis control and emotional TTS, but the unvalidated pseudo-labels and small subjective effects make the central claim conditional.","tokens_in":9177,"tokens_out":2275,"would_cite":false,"duration_ms":27982,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EME-TTS keeps stressed words clear across five emotions.","keywords":["Emotional Speech Synthesis","Emphasis Control","Emotion Expressiveness","Weakly Supervised Labeling","Variance-Based Prosody","Attention Modulation","Expressive TTS"],"falsifier":"Hand-label a small sample of the emotional test set for emphasis, compute per-emotion agreement between human labels and the pseudo-labeler, and check whether emphasis recognition accuracy drops when the model is trained with human labels instead of pseudo-labels; if the pseudo-labels disagree sharply on surprise or happy speech, the reported emphasis control partly measures pitch-range artifacts rather than true prominence.","tokens_in":8179,"feed_emoji":"🗣️","tokens_out":9720,"duration_ms":94968,"temperature":0.7,"pith_summary":"The paper sets out to show that emotional tone and word emphasis can be controlled in a single speech synthesis model instead of being handled separately. It builds an emotional text-to-speech system trained with automatically generated emphasis labels and prosodic variance features, and adds an emphasis-perception block that strengthens the attention given to designated words. The reported experiments indicate that this combination makes emotional speech more expressive and keeps listeners able to identify the intended stressed word across angry, happy, sad, surprise, and neutral conditions. If the claim holds, a user could specify both the emotion and the words to stress, and the model would deliver natural speech without emphasis being drowned out by the emotion.","feed_headline":"EME-TTS keeps stressed words clear across five emotions","feed_subtitle":"The model ties emotion labels to emphasis positions, so the right words stand out without losing the emotion.","key_machinery":"The load-bearing mechanism is the Emphasis Perception Enhancement (EPE) block, a transformer-style block that re-weights attention in the regions marked as emphasized. Its Emphasis Adapter adds a scaled mask to the attention weights at the start and end of the emphasized word, so the model can enhance perceivable prominence through attention rather than by directly shaping pitch or energy. The same pipeline also uses variance-based pitch and duration features, computed as local averages in emphasized regions minus sentence-level averages, which tell the acoustic model how far to push prosody above the sentence norm.","core_discovery":"The central claim is that there is a learnable, controllable link between emphasis and emotion, and that EME-TTS exploits it. The framework takes emotion labels and emphasized-word positions as two conditioning signals. During training, emphasis positions are obtained as pseudo-labels from a pre-trained emphasis recognizer; the model then predicts pitch and duration variance features that highlight the emphasized region, while the Emphasis Perception Enhancement (EPE) block re-weights its attention with a mask over those positions and integrates emotion through conditional normalization. At inference, a large language model is used to choose the emphasis positions from text and emotion. The paper reports that the resulting speech is perceived as more emotionally expressive, especially with surrounding context, and that target emphasis remains recognizable across all five emotions, with the largest gains on surprise, where pitch rises at sentence end otherwise mislead listeners.","pith_inferences":["The paper fixes the emphasis strength at 0.2; a natural extension would be to expose that scalar as a user dial and measure whether emphasis clarity and emotion accuracy trade off as it varies.","Because the LLM's emphasis predictions are not separately evaluated, one could test how sensitive the emotion outcomes are to prediction errors by comparing LLM-chosen positions against human-chosen positions on the same test sentences.","The same weakly supervised pipeline could be applied to other expressive dimensions, such as contrastive focus or question focus, if a suitable recognizer supplies pseudo-labels.","An interesting open question is whether emphasis positions themselves differ by emotion in natural speech; the current design treats them as an input rather than learning emotion-specific emphasis tendencies."],"forward_implications":["Listeners can identify the intended emphasized word from synthesized speech across all five emotions, with mean recognition accuracy rising from 0.73 without the EPE block to 0.78 with it.","Emphasis control improves perceived emotion accuracy: subjective emotion recognition averaged 0.67 for EME-TTS versus 0.58 for the base emotional TTS model.","The expressiveness gain is strongest when the generated sentence appears with surrounding context, suggesting emphasis needs semantic support to enhance emotion.","Emphasis does not degrade synthesis quality: the model's MOS and NISQA scores sit above the base model and above the version without the EPE block.","The increased duration of emphasized regions is credited with improving objective recognition of sad emotion, which rose to 0.61 from 0.54 for the base model."],"supporting_citations":[{"why":"Supplies the variance-based emphasis modeling assumption that pitch and duration are the primary emphasis cues, which the paper extends.","marker":"[19]"},{"why":"Provides the emphasis recognizer whose pseudo-labels are used to supervise the variance predictors and the emphasis mask.","marker":"[22]"},{"why":"Supplies the emotional speech corpus used for training, validation, and testing of all models.","marker":"[23]"},{"why":"Provides the base emotional TTS architecture that EME-TTS modifies with variance features and the EPE block.","marker":"[9]"},{"why":"Supplies the conditional layer normalization design used to inject emotion into the EPE blocks.","marker":"[25]"},{"why":"Represents the large language model used at inference to predict emphasis positions from text and emotion.","marker":"[29]"},{"why":"Serves as a comparison baseline for emotion accuracy in the subjective and objective evaluations.","marker":"[28]"},{"why":"Provides the emotion recognition model used for the objective emotion accuracy measurement.","marker":"[30]"},{"why":"Supplies the objective naturalness score used to compare synthesis quality.","marker":"[31]"}],"fun_headline_variants":["EME-TTS: unlocking the emphasis-emotion link","Emphasis and emotion: TTS that keeps stressed words clear","LLM picks emphasis spots for clearer emotional TTS","Weakly supervised model ties emphasis to emotion in TTS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training pipeline assumes that the pre-trained emphasis recognizer's automatic labels on the emotional speech database are accurate enough to act as ground truth for where emphasis actually falls.","fun_headline_variants_meta":{"raw":{"variants":["EME-TTS: unlocking the emphasis-emotion link","Emphasis and emotion: TTS that keeps stressed words clear","LLM picks emphasis spots for clearer emotional TTS","Weakly supervised model ties emphasis to emotion in TTS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2569,"prompt_tokens":863,"completion_tokens":1706,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":1639}},"tokens_in":479,"tokens_out":1706,"duration_ms":18632,"temperature":1.0,"reasoning_tokens":1639,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:55:59.921238+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hand-label a small sample of the emotional test set for emphasis, compute per-emotion agreement between human labels and the pseudo-labeler, and check whether emphasis recognition accuracy drops when the model is trained with human labels instead of pseudo-labels; if the pseudo-labels disagree sharply on surprise or happy speech, the reported emphasis control partly measures pitch-range artifacts rather than true prominence.","supporting_citations":[{"cited_title":"Daft- exprt: Cross-speaker prosody transfer on any text for expressive speech synthesis,","cited_arxiv_id":null,"evidence_quote":"Supplies the variance-based emphasis modeling assumption that pitch and duration are the primary emphasis cues, which the paper extends."},{"cited_title":"Lex- ical stress perception as a function of acoustic properties and the native language of the listener,","cited_arxiv_id":null,"evidence_quote":"Provides the emphasis recognizer whose pseudo-labels are used to supervise the variance predictors and the emphasis mask."},{"cited_title":"Emphatic speech generation with conditioned input layer and bidirectional lstms for expressive speech synthesis,","cited_arxiv_id":null,"evidence_quote":"Supplies the emotional speech corpus used for training, validation, and testing of all models."},{"cited_title":"Glow-tts: A genera- tive flow for text-to-speech via monotonic alignment search,","cited_arxiv_id":null,"evidence_quote":"Provides the base emotional TTS architecture that EME-TTS modifies with variance features and the EPE block."},{"cited_title":"Emphasis control for parallel neural tts,","cited_arxiv_id":null,"evidence_quote":"Supplies the conditional layer normalization design used to inject emotion into the EPE blocks."},{"cited_title":"Em- phAssess : a prosodic benchmark on assessing emphasis transfer in speech-to-speech models,","cited_arxiv_id":null,"evidence_quote":"Serves as a comparison baseline for emotion accuracy in the subjective and objective evaluations."},{"cited_title":"Hierarchical rep- resentation and estimation of prosody using continuous wavelet transform,","cited_arxiv_id":null,"evidence_quote":"Provides the emotion recognition model used for the objective emotion accuracy measurement."},{"cited_title":"Adaspeech 4: Adaptive text to speech in zero-shot scenar- ios,","cited_arxiv_id":null,"evidence_quote":"Supplies the objective naturalness score used to compare synthesis quality."}],"review_version":1}