{"id":"7c56b7b3-f832-46d5-99a0-a57bf624951e","arxiv_id":"2608.00722","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Experience-Calibrated Contrastive Decoding, a training-free decoding method that strengthens text alignment signals, reduces speech hallucination errors across four LM-based TTS models and nine languages.","lead":"This paper introduces a new decoding rule, ECCD, that reduces content errors in AI text-to-speech by comparing what the speech model predicts with and without the target text. Across four speech models it cuts word/character error rates by up to 55.6% and a listening test preferred it over native decoding.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline WER/CER gains lack confidence intervals or significance tests, so the 'all settings' claim is statistically untested.","rationale":"The reader's weakest assumption concerns the text-ablated proxy pA, which is a conceptual limitation that the paper explicitly acknowledges ('removing text may itself induce a conditioning shift'). This would threaten the theoretical framing and the I_i/G_i analyses, but not necessarily the empirical observation that ECCD lowers WER: the method could in principle work even if pA is not a perfectly clean 'experience' distribution. By contrast, the headline quantitative claim is directly about WER/CER reductions, and those are reported without any uncertainty quantification. The consistency across many settings and models is encouraging, but for small-effect rows (e.g., GLM-TTS) the difference could plausibly be noise. The hyperparameter sweep on test-hard adds a mild selection effect. A bootstrap or McNemar test directly probes the robustness of the central claim and would settle whether the 'all settings' assertion is justified. The large, consistent improvements for Llasa and the positive CMOS from a human listening test provide independent support, so a REJECT verdict is not warranted; CONDITIONAL remains appropriate pending uncertainty quantification.","tokens_in":13667,"tokens_out":10747,"duration_ms":96549,"concrete_test":"Bootstrap the test utterances for each Table 1 row: resample utterances with replacement for 10,000 iterations, recompute WER/CER for ECCD and the native baseline on each resample, and form 95% confidence intervals for the ECCD-minus-baseline difference. Alternatively, apply a paired McNemar test on utterance-level error counts. If the confidence interval includes zero or the test is non-significant in any SeedTTS-Eval setting, the 'all settings' claim should be qualified to those settings where significance is established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ECCD reduces WER/CER in all SeedTTS-Eval settings and 24/25 CV3-Eval settings rests solely on single test-set point estimates. Table 1 reports no confidence intervals, standard errors, or significance tests for any WER/CER value; the only variance-reported metric is UTMOS. For small absolute gains (e.g., GLM-TTS test-zh CER 1.01→0.97, test-en WER 2.12→1.94), the improvement may be within utterance-level sampling noise. Additionally, the hyperparameter sensitivity sweep in Figure 2 was performed on SeedTTS-Eval test-hard, the same split used for the headline CosyVoice2 results; although α=1,k=25 is a fixed default rather than the swept optimum, the lack of a held-out validation set leaves a selection effect unquantified. If the improvements are not statistically distinguishable from zero in some settings, the universal 'all settings' claim is unsupported, even though the large Llasa reductions (e.g., 26.26→14.27 CER on test-hard) would likely remain significant.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Experience-Calibrated Contrastive Decoding (ECCD), a training-free decoding method for LM-based text-to-speech that contrasts the full-condition next-token distribution p_E against a text-ablated distribution p_A. ECCD retains the original expert score, applies a positive-only alignment enhancement within a top-k plausible set, and scales the enhancement by an experience compatibility coefficient. The authors report WER/CER reductions across four TTS models (CosyVoice2, CosyVoice3, Llasa, GLM-TTS) on SeedTTS-Eval and CV3-Eval, a listening-test CMOS gain of +0.644, and an information-theoretic analysis of alignment influence around hallucination onsets. The paper claims the first adaptation of contrastive decoding to autoregressive acoustic-token generation and interprets the results as evidence that conditional-information control is a promising route for mitigating speech hallucinations.","tokens_in":13854,"tokens_out":6837,"duration_ms":61601,"significance":"If the empirical claims hold, ECCD is a simple, model-agnostic, training-free intervention that could complement existing training-based hallucination mitigations. The evaluation is unusually broad: four models, nine languages, both objective and subjective metrics, a component ablation, and a mechanism analysis. The paper also states its limitations explicitly, including the conditioning-shift caveat for p_A and the approximate nature of the ASR-based onset alignment. The main gap is statistical: the universal 'all settings' claim rests on point estimates without confidence intervals or significance tests, and some reported gains are very small. The mechanism analysis is informative but not fully independent of the proposed method. With additional statistical support and a clearer treatment of the proxy-validity and hyperparameter-selection questions, this would be a valuable contribution.","major_comments":[{"comment":"The headline claim that ECCD 'reduces WER/CER in all SeedTTS-Eval settings and 24 of 25 CV3-Eval settings' is supported only by single point estimates without confidence intervals or significance tests. Several improvements are small (e.g., GLM-TTS test-zh CER 1.01→0.97 and test-en WER 2.12→1.94) and may lie within utterance-level or ASR variability. I recommend reporting bootstrap or per-utterance confidence intervals, or paired significance tests (e.g., McNemar on utterance-level errors), and stating which settings remain significant; this is needed to substantiate the universal claim.","section":"Experiments, Table 1 and Tables 2–3"},{"comment":"The amateur distribution p_A is obtained by removing all text conditions, but the paper itself notes that 'removing text may itself induce a conditioning shift.' Because both ECCD's enhancement and the I_i/G_i measures are built on the p_E/p_A contrast, the interpretation of the gains—and of the onset analysis—as alignment-specific depends on p_A being a valid experience proxy. I ask for a control experiment or sensitivity analysis, such as comparing p_A against a condition with uninformative text or a model whose text conditioning is separately ablated, to rule out artifacts from altered prosody, silence, or degenerate outputs.","section":"Method, Eq. (2)"},{"comment":"The hyperparameter sweep and the component ablation are run on SeedTTS-Eval test-hard, the same split on which the headline CosyVoice2 results are reported. Although α=1,k=25 is presented as a fixed default, the sweep itself shows α=1,k=15 would give lower CER and higher SS on this split, so the degree to which the reported results benefit from selection on the test set is unquantified. A held-out validation split should be used for any tuning, or the authors should show that the default performs comparably on an independent split.","section":"Experiments, Figure 2 and Table 1"},{"comment":"The I_i and G_i statistics are computed from the same p_E/p_A contrast that ECCD modifies, so the observation that G_i is lower at first-error boundaries than at matched correct boundaries is not an independent test of the proposed onset mechanism; it could reflect a property of the contrast itself (e.g., conditioning-shift artifacts) rather than alignment support. The paper's caveat that the analysis is 'consistent with, but does not causally establish' the hypothesis is welcome, but the section should explicitly state this non-independence and, ideally, support the interpretation with a reference measure not derived from the same contrast (e.g., attention maps or forced alignment).","section":"Mechanism analysis, Tables 5–6"}],"minor_comments":[{"comment":"I_i is an unbounded KL divergence, and the text says it 'replaces' 1−C_i in Eq. (8) without describing normalization; please specify the exact substitution and any normalization used, since this directly affects reproducibility.","section":"Equation (8) and ablation of ECCD_I"},{"comment":"The dashed segments after offset 5 are described as unreliable because sample counts decrease, but no counts or thresholds are given; please include a sample-size profile or a minimum-count note so the reader can judge the reliability of those segments.","section":"Figure 1"},{"comment":"The paper does not state whether code or audio samples will be released; given the simplicity of the training-free method, providing these would materially help replication.","section":"Reproducibility"},{"comment":"The extracted text contains frequent missing spaces (e.g., 'acoustictokenselection'), and some model names are used inconsistently; please ensure the camera-ready text is properly spaced and model names are consistent throughout.","section":"General presentation"}],"recommendation":"major_revision","confidential_remarks":"To the editor: This is a methodologically interesting paper with broad evaluation, but the statistical overclaiming and the unresolved proxy-validity question need to be addressed before publication. I would not recommend rejection; a major revision with additional statistical analysis, a held-out validation, and a clearer discussion of the p_A proxy would bring it to publishable quality."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first real attempt I've seen to apply contrastive decoding to autoregressive acoustic-token generation in TTS, and it's not a toy. The ECCD score in Eq. (8) is new: an expert anchor, a positive-only log-ratio boost, and an experience compatibility coefficient that scales the boost. It's training-free and model-agnostic, and the experiments cover four models and nine languages. The headline reductions are real where they matter: Llasa CER on test-hard drops from 26.26% to 14.27%, and the listening test shows +0.644 CMOS over native decoding. That is a substantial result for a decoding-time intervention.\n\nThe paper earns credit for running the right ablations. Conventional CD makes things worse—CER goes up, speaker similarity drops, and the output temporally compresses to 66% of native token count. Adding the expert anchor reverses the CER degradation; positive-only enhancement brings CER to 6.43%; the ECC calibration restores temporal ratios. That sequence is exactly what their design story predicts. The I_i/G_i analysis is also careful: they see a larger shortfall in G_i than I_i at first-error boundaries, and they explicitly say the analysis is consistent with, not proof of, an alignment shortfall cause. They also flag the text-ablated proxy's conditioning-shift problem in the same section, which is the main theoretical caveat and they don't hide it.\n\nThe soft spots are real but non-fatal. Most importantly, the WER/CER tables have no confidence intervals or significance tests. For a claim like \"all SeedTTS-Eval settings,\" point estimates alone are not enough; the GLM-TTS improvements (1.01 to 0.97 CER on zh) could easily be utterance-level noise. The large Llasa reductions would presumably survive a bootstrap, but the small ones need variance reporting. The hyperparameter sweep on the test-hard split is a minor selection-effect risk—they use a fixed default, but they also note that k=15 would be better, so a held-out validation protocol would clean that up. And the abstract's \"preserves the original expert distribution\" is loose; the method modifies the score, it doesn't keep pE unchanged. Wording fix, not a conceptual flaw.\n\nWho's this for? Speech synthesis researchers who care about inference-time control, and anyone building LM-based TTS in production. It deserves a serious referee and, after the error-bar and wording revisions, a probably-accept. Missing code is a minus, but the equation-level specification is sufficient to reimplement.\n\nI'd bring it to our reading group and cite it if I were doing TTS decoding work. Send it to review.","headline":"A training-free decoding-time fix for LM-TTS hallucinations that mostly works and deserves real referees once the missing error bars are addressed.","tokens_in":14429,"tokens_out":2668,"would_cite":true,"duration_ms":23356,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free decoding method that contrasts a speech LM's text-conditioned and text-free next-token predictions cuts content errors in LM-based TTS by up to 55.6% across four models and in 24 of 25 multilingual settings.","keywords":["speech hallucination","contrastive decoding","LM-based text-to-speech","decoding-time control","alignment information","experience information","acoustic token generation","training-free mitigation"],"falsifier":"Compute $I_i$ and $G_i$ on a large set of utterances with reliable ASR labels and compare first-error boundaries with matched correct boundaries under the same token-rate mapping; if the matched correct-minus-error differences in $G_i$ do not replicate or flip sign, the onset-shortfall claim fails. Alternatively, apply ECCD to a corpus where text removal is known to alter prosody without causing content errors: if the CER gains disappear or reverse while lower-temperature sampling still helps, the alignment-support story is wrong.","tokens_in":13459,"feed_emoji":"🔊","tokens_out":9238,"duration_ms":64897,"temperature":0.7,"pith_summary":"This paper claims that a substantial class of speech hallucinations in language-model text-to-speech begins as a local shortfall in text-derived alignment support at a vulnerable token transition, and that this shortfall can be corrected at decoding time without retraining. It introduces Experience-Calibrated Contrastive Decoding (ECCD), which contrasts the same speech model's next-token predictions with and without text conditions, keeps the original expert distribution as an anchor, and adds only positive alignment enhancement, scaled by an experience-compatibility coefficient. Across four speech LMs, ECCD lowers word/character error rates by up to 55.6% in all SeedTTS-Eval settings and in 24 of 25 multilingual CV3-Eval settings, with a listening-test CMOS gain of +0.644. If these results hold, decoding-time conditional-information control is a viable complement to architecture- and training-based hallucination mitigation.","feed_headline":"One decoding tweak cuts TTS speech errors by 55.6%","feed_subtitle":"Contrasting text-conditioned and text-free scores fixes content errors across four speech models with no retraining.","key_machinery":"The load-bearing object is the ECCD score of Eq. (8): $S_{\\text{ECCD}}(x_i)=\\log p_E(x_i)+\\alpha(1-C_i)[\\log(p_E(x_i)/p_A(x_i))]_+$ inside the expert-defined top-$k$ plausible set $V_{\\text{head}}$, and simply $\\log p_E(x_i)$ outside it. The contrast between the full-conditioned expert $p_E$ and the text-ablated amateur $p_A$ — same model, same acoustic history, only the text conditions removed — is the operational signal of incremental alignment support. $C_i$, the experience compatibility coefficient, is the total probability mass $p_A$ assigns to the top-$k$ expert-plausible set: a large $C_i$ means the experience proxy already agrees with the expert's candidates, so the intervention is attenuated, while a small $C_i$ triggers stronger positive enhancement. The design deliberately avoids conventional contrastive decoding's move of treating amateur support as negative evidence, because $p_A$ carries pronunciation, prosody, duration, and continuity information that must be preserved for fluent speech.","core_discovery":"The paper's central claim is that hallucination onset in LM-based TTS occurs when the selected acoustic token receives insufficient relative alignment support from the text, even when distribution-level text influence is present; once the off-target token enters the history, the error propagates. Operationally, the paper defines the full-condition distribution $p_E(x_i)=p(x_i\\mid x_{<i},a,T_p,T_t)$ and the text-ablated distribution $p_A(x_i)=p(x_i\\mid x_{<i},a)$, and treats their discrepancy as alignment influence. ECCD then modifies the next-token score inside an expert-defined top-$k$ set: $S_{\\text{ECCD}}(x_i)=\\log p_E(x_i)+\\alpha(1-C_i)[\\log(p_E(x_i)/p_A(x_i))]_+$, with the plain expert score $\\log p_E(x_i)$ used outside the set. The method keeps the expert distribution as an anchor, applies only the positive part of the contrast, and scales that correction by $(1-C_i)$, where $C_i$ is the total probability mass $p_A$ assigns to the expert-plausible set. The paper reports that this reduces content errors across four models in all SeedTTS-Eval subsets and in 24 of 25 CV3-Eval language/subset combinations, and that the per-token measures $I_i$ (forward KL from expert to ablated distribution) and $G_i$ (the selected token's log-ratio) are lower at first-error boundaries than at matched correct boundaries, supporting an onset-specific alignment shortfall.","pith_inferences":["The text-ablated amateur distribution could be repurposed as an online hallucination-onset detector: tokens where $G_i$ is unusually low relative to a running baseline may pinpoint the transitions where intervention matters most, a use the paper's averaged analysis does not itself establish.","The same design pattern — expert anchoring, positive-only contrast, and set-level compatibility calibration — may transfer to other autoregressive generation tasks where the ablated 'amateur' distribution carries useful regularities rather than pure noise, such as image-token or audio-language models.","The paper's fixed default of $\\alpha=1$, $k=25$ is not optimal in its own hyperparameter sweep, where $\\alpha=1$, $k=15$ improves both CER and speaker similarity, so per-model tuning could yield larger gains than the headline numbers.","Because $C_i$ measures set-level agreement between experience and expert candidates, it may also serve as a per-step measure of text-history conflict, potentially useful for diagnosing which linguistic units (gaps versus sustained regions) are most vulnerable."],"forward_implications":["ECCD can be applied to any autoregressive acoustic-token language model without architectural changes or additional training, since it only modifies next-token scores before the model's native filtering and penalty operations.","The method transfers across languages and speaker conditions: error reductions appear in 24 of 25 model/subset combinations across nine languages, suggesting that decoding-time alignment enhancement generalizes beyond a single training distribution.","Lower-temperature sampling does not reproduce ECCD's gains and even raises test-en word error rate for one model at temperature 0.75, so the improvement is not just distribution sharpening.","Conventional contrastive decoding is harmful for speech: in the ablation it raises CER, lowers speaker similarity, and compresses utterance duration to roughly two-thirds of native, whereas ECCD's components restore most of that temporal structure.","The measured signatures — lower $I_i$ and $G_i$ at first-error boundaries than at matched correct boundaries, with higher values inside complete error regions — imply that alignment support returns during propagation without restoring correct content, which distinguishes onset from propagation."],"supporting_citations":[{"why":"Introduces contrastive decoding, the expert-amateur framework ECCD modifies for acoustic-token generation.","marker":"(Li et al. 2023)"},{"why":"Provides the SeedTTS-Eval benchmark and its test-en, test-zh, and test-hard subsets used for the main evaluations.","marker":"(Anastassiou et al. 2024)"},{"why":"Provides CosyVoice3 and the multilingual zero-shot CV3-Eval benchmark, including the kana-conversion note used to interpret the Japanese exception.","marker":"(Du et al. 2025b)"},{"why":"Provides CosyVoice2, the baseline model for the lower-temperature comparison, ablation studies, and listening test.","marker":"(Du et al. 2024)"},{"why":"Represents an architecture- and training-based alignment mitigation approach that ECCD is positioned against.","marker":"(Du et al. 2025a)"},{"why":"Documents that hallucinated speech remains fluent and speech-like, which motivates ECCD's experience-preservation design.","marker":"(Liu et al. 2025)"},{"why":"Supplies Whisper-large-v3, the ASR system used to compute WER/CER for non-Chinese languages.","marker":"(Radford et al. 2023)"},{"why":"Supplies Paraformer-zh, the ASR system used to compute CER for Chinese.","marker":"(Gao et al. 2022)"}],"fun_headline_variants":["ECCD: a training-free fix that slashes TTS speech errors","No retraining, just a contrast: TTS errors cut by 55.6%","Decoding-time control: contrastive logic reduces TTS hallucinations","ECCD: up to 55.6% fewer content errors in TTS, no retraining","Alignment versus experience: a single contrast step reduces TTS hallucination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the text-ablated prediction $p_A(x_i \\mid x_{<i}, a)$ is a faithful operational proxy for experience-oriented prediction, so that the contrast $\\log(p_E/p_A)$ isolates text-derived alignment support; if dropping the text instead changes prosody, silence, or generative quality in ways unrelated to alignment, the enhancement and the measured $I_i$ and $G_i$ signatures could be chasing an artifact.","fun_headline_variants_meta":{"raw":{"variants":["ECCD: a training-free fix that slashes TTS speech errors","No retraining, just a contrast: TTS errors cut by 55.6%","Decoding-time control: contrastive logic reduces TTS hallucinations","ECCD: up to 55.6% fewer content errors in TTS, no retraining","Alignment versus experience: a single contrast step reduces TTS hallucination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000986,"raw_usage":{"total_tokens":4275,"prompt_tokens":1131,"completion_tokens":3144,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":747,"completion_tokens_details":{"reasoning_tokens":3041}},"tokens_in":747,"tokens_out":3144,"duration_ms":19965,"temperature":1.0,"reasoning_tokens":3041,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:17:15.729530+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute $I_i$ and $G_i$ on a large set of utterances with reliable ASR labels and compare first-error boundaries with matched correct boundaries under the same token-rate mapping; if the matched correct-minus-error differences in $G_i$ do not replicate or flip sign, the onset-shortfall claim fails. Alternatively, apply ECCD to a corpus where text removal is known to alter prosody without causing content errors: if the CER gains disappear or reverse while lower-temperature sampling still helps, the alignment-support story is wrong.","supporting_citations":[],"review_version":2}