{"id":"8923c72a-4f91-4f44-9367-318799a8ab14","arxiv_id":"2505.13830","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A codec-token denoiser that predicts only the first two token groups of clean audio enables noise-robust zero-shot TTS, matching clean-prompt performance.","lead":"A new speech denoiser operates on compressed audio tokens instead of raw waveforms, and it is paired with an LLM-based text-to-speech system so voices can be cloned from noisy prompts. The approach reports cleaner, more speaker-similar synthesized speech than the standard pipeline of running a separate speech enhancement model first.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'outperforms SOTA' claim for the codec denoiser is not yet established: SE results rely on DNSMOS without error bars and omit the closest token-domain baselines (MaskSR, Genhancer), so the claimed advantage may not hold.","rationale":"Reading in good faith, the paper proposes a sensible pipeline and its NR-LauraTTS results are promising: the clean-prompt comparison in Table 3 is well designed, audio samples are provided, and the computational-efficiency argument is concrete. My concern is not with novelty or honesty but with the strength of the evidence for the specific claim that the codec denoiser outperforms SOTA SE methods. That claim has two load-bearing conditions: the comparison set must include the relevant SOTA methods, and the metric must be stable enough to support the reported differences. Both conditions are questionable. The omission of token-domain baselines is especially significant because the paper's central design decision, predicting only the first two RVQ groups, is motivated by the difficulty and complexity of predicting all VQ groups, yet no all-VQ token-domain model is evaluated. Without such a comparison, the observed DNSMOS advantage over signal-domain models cannot be attributed to the two-group design rather than to differences in architecture or training difficulty. The DNSMOS-only evaluation without error bars compounds this, as the margins over MP-SENet are small. These issues are concrete and fixable, and they do not invalidate the TTS-side comparison, so I do not move the verdict to reject; the conditional verdict remains appropriate. The reader's weakest_assumption focused on the sufficiency of the first two RVQ groups; my concern is adjacent but centered on the comparison and statistical support for the 'outperforms SOTA' claim, so I mark partial agreement.","tokens_in":8778,"tokens_out":8392,"duration_ms":82517,"concrete_test":"Evaluate MaskSR and Genhancer on the same LibriTTS-R/WHAM! noisy test set (SNR 0-20 dB) using their official checkpoints or models retrained on the paper's noisy-clean training set, and compare DNSMOS, PESQ, STOI, and FLOPs against the proposed codec denoiser and MP-SENet. Report 95% confidence intervals or paired bootstrap intervals over test utterances for all metrics. If the codec denoiser does not significantly beat the best token-domain baseline on DNSMOS and at least one conventional objective metric, the 'outperforms SOTA SE' claim should be narrowed or withdrawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim that the codec denoiser 'outperforms SOTA SE methods' rests on Table 1, which reports only DNSMOS P.835 scores with no error bars, no significance tests, and no conventional SE metrics (PESQ, STOI, SI-SNR). The largest margin over MP-SENet is 0.06 on SIG and OVRL and 0.02 on BAK; given known utterance-level and run-to-run variance of DNSMOS, these differences may not be reliable. More importantly, the only comparators, CMGAN and MP-SENet, are signal-domain models. The token-domain SE methods cited as related work, MaskSR [18] and Genhancer [19], are never evaluated, even though they are the direct competitors for a discrete-codec-token denoiser and typically predict all RVQ groups. The paper's own motivation for predicting only the first two groups is that all-VQ prediction is harder and more complex, but this is never tested against an all-VQ token-domain model. If Genhancer or MaskSR achieves comparable or better DNSMOS, the two-group design and the 'outperforms SOTA' claim would need substantial revision. A similar statistical weakness affects Table 3: the TTS comparison to LauraTTS+MP-SENet shows small differences (MOS 4.02 vs 4.01, WER 2.44 vs 2.54) with overlapping confidence intervals, yet the abstract states NR-LauraTTS 'surpasses' the SE-based approach.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a noise-robust zero-shot TTS system, NR-LauraTTS, in which a codec-based denoiser operates in the discrete acoustic token domain before the prompt is fed to the LLM-based TTS backbone LauraTTS. The codec denoiser uses an Encodec encoder to obtain all RVQ token groups, a Conformer token denoiser that predicts only the first two clean token groups, and an embedding refiner that estimates the summed clean acoustic embedding, which is then decoded into a waveform or used directly as the acoustic prompt for LauraTTS. Experiments compare the codec denoiser with CMGAN and MP-SENet using DNSMOS P.835 and FLOPs, and compare NR-LauraTTS with LauraTTS under clean, noisy, and MP-SENet-enhanced prompts using MOS, SMOS, SECS, WER, CER, and additional FLOPs. The authors report that the codec denoiser outperforms state-of-the-art SE methods and that NR-LauraTTS approaches clean-prompt performance with lower complexity than a signal-domain SE front-end.","tokens_in":9117,"tokens_out":5290,"duration_ms":47310,"significance":"If the reported results are robust, the paper makes a useful contribution: it demonstrates that denoising in the token domain is a viable and efficient alternative to signal-domain enhancement for LLM-based zero-shot TTS, and the two-group prediction design is well aligned with LauraTTS's prompting mechanism. The paper has several concrete strengths: the architecture is clearly described, the comparison includes a clean-prompt upper bound and a strong signal-domain SE baseline, the evaluation is performed on a held-out test set with no circularity in the loss or test procedure, and a public demo page is provided. The main weaknesses are that the closest token-domain SE baselines are not evaluated, the SE results are reported without confidence intervals or conventional metrics, and the wording of the TTS claims is stronger than the small differences in Table 3 support. These issues are addressable and do not invalidate the central idea, but they currently limit the reliability of the headline claims.","major_comments":[{"comment":"The claim that the codec denoiser 'outperforms state-of-the-art SE methods' is not established because the closest competitors, the token-domain SE models MaskSR [18] and Genhancer [19], are cited in Section 2.2 but never evaluated. These are the models whose design decision (predicting tokens from all RVQ groups) the authors explicitly argue against, so a comparison with them is necessary to support both the SOTA claim and the two-group design. Please add these baselines, or restrict the claim to 'outperforms CMGAN and MP-SENet'.","section":"§4.1, Table 1"},{"comment":"The SE evaluation reports only DNSMOS P.835 scores with no confidence intervals, significance tests, or conventional objective metrics such as PESQ, STOI, or SI-SNR. The reported margins over MP-SENet are small (SIG 3.62 vs. 3.56, BAK 4.11 vs. 4.09, OVRL 3.36 vs. 3.30), and DNSMOS is known to vary across utterances and noise conditions. Please report error bars or statistical tests and, ideally, corroborating objective metrics before claiming superiority.","section":"§4.1, Table 1"},{"comment":"The abstract and conclusion state that NR-LauraTTS 'surpasses' the approach using additional SE models, but Table 3 shows MOS 4.02 vs. 4.01 with identical 95% confidence intervals, WER 2.44 vs. 2.54, and CER 1.27 vs. 1.41; only SMOS and SECS are numerically higher. No paired significance tests are reported, so the 'surpasses' wording is too strong. Either provide significance testing or rewrite the claim to say 'comparable to the SE-based approach with improved speaker similarity'.","section":"§4.2, Table 3"},{"comment":"The design premise that the first two RVQ groups contain sufficient acoustic information (speaker identity and essential prosody) is supported only by the DNSMOS ablation in Table 2, whose differences are small (OVRL 3.36 for two groups vs. 3.35 for one and four groups, 3.33 for eight) and are reported without variance. This does not establish that two groups are sufficient for faithful voice cloning or for the embedding refiner. Please add a direct analysis of token-group information content, evaluate downstream TTS speaker similarity when different numbers of groups are predicted, or present the two-group choice as an empirical design choice with appropriate uncertainty rather than as a validated property of Encodec.","section":"§2.1, Table 2"}],"minor_comments":[{"comment":"The sentence 'we utilized the 580-hour training set of the 15-hour development set' appears to contain a grammatical error; presumably the 580-hour training set and the 15-hour development set were both used.","section":"§3.1"},{"comment":"The number of embedding-refiner Conformer blocks is denoted N_ER in the methodology but reported as 'NCV' in Section 3.1; please make the notation consistent.","section":"§2.2.2 and §3.1"},{"comment":"The DNSMOS results are given without indicating the number of test utterances or the SNR breakdown of the test set; please report the test-set size and, if possible, per-condition results.","section":"Tables 1 and 2"},{"comment":"The statement that 'the VQs of the codec act as information bottlenecks, allowing the quantized discrete acoustic tokens to filter out some noise' is presented as an explanation but is not directly tested; consider presenting it as a hypothesis rather than a confirmed mechanism.","section":"§4.1"},{"comment":"The figures are dense and the two-group output is not immediately easy to identify; adding explicit labels in the captions for the RVQ dimension and the 'first two groups' path would improve readability.","section":"Figures 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is reasonable and the architecture is sound, but the strength of the claims currently exceeds the evidence: the omission of token-domain SE baselines and the absence of statistical analysis in Tables 1 and 3 are the main obstacles. These are fixable within the scope of the manuscript, so I recommend major revision rather than rejection. I also note that MP-SENet is from the authors' own prior work; using it as a baseline is legitimate, but the omission of MaskSR and Genhancer may be perceived as selective unless a convincing justification or comparison is provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a sensible, compact extension of token-domain speech enhancement to LLM-based zero-shot TTS, and the TTS-side evaluation mostly supports the main claim. The broader claim that the codec denoiser beats SOTA SE methods is not established by the evidence given.\n\nWhat's new: predicting only the first two RVQ groups plus an embedding refiner, trained as a denoiser on Encodec tokens and plugged into LauraTTS. Prior token-domain SE methods like MaskSR and Genhancer predict all groups. The two-group design is a real efficiency choice, and the FLOPs difference is large (9.96 G vs 31.68/38.93 G). That is a useful result if it holds.\n\nWhat I like: the TTS experiments include clean-prompt, noisy-prompt, and an SE front-end, with both objective and subjective metrics. NR-LauraTTS lands close to clean-prompt LauraTTS on MOS, SMOS, WER, and SECS, and does better than LauraTTS+MP-SENet on speaker similarity with much lower added compute. The ablation over the number of predicted groups is informative, even if the conclusion drawn from it is a bit strong.\n\nWhere I would push back: the SE comparison in Table 1 is thin. It reports only DNSMOS, no error bars, no significance test, and the margins over MP-SENet are 0.02–0.06, which is likely within DNSMOS variance. The two baselines are signal-domain; the closest token-domain methods are mentioned in the intro but never evaluated. So 'outperforms SOTA SE methods' overstates what is shown. The choice of two predicted groups is selected from Table 2, and the metric differences between one, two, and four groups are tiny, so the 'sufficient acoustic information' conclusion is inferred rather than demonstrated. There is also no code or checkpoint release, which makes reproducibility harder to check. These are fixable issues: add the token-domain baselines, report standard SE metrics and error bars, and soften the abstract.\n\nThe citation pattern is fine. Using MP-SENet as a baseline is a legitimate choice even though it comes from the same group; it is a real published method, not a strawman. The paper is squarely aimed at the speech synthesis/SE community. I would send it to review; it deserves referee time, but I would expect the authors to tighten the SE claims before publication.","headline":"A compact token-domain denoiser for LLM-based zero-shot TTS that largely delivers on the TTS side, though the 'outperforms SOTA SE' claim is not established by the evidence.","tokens_in":9672,"tokens_out":2065,"would_cite":true,"duration_ms":18936,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Token-domain denoising restores LLM-based zero-shot TTS quality from noisy prompts.","keywords":["zero-shot TTS","noise-robust TTS","speech enhancement","codec denoising","discrete acoustic tokens","residual vector quantization","LLM-based TTS","embedding refiner"],"falsifier":"Take a pre-trained codec denoiser and feed it prompts corrupted by a noise source absent from its training data, such as background music or room reverberation at a signal-to-noise ratio of -5 dB, then synthesize speech and compare speaker similarity and word error rate against the TTS system given the clean prompt. A significant drop would indicate that the two-group token representation is insufficient under distribution shift; a more direct test is to measure, via information-theoretic probing, how much speaker identity is retained in the first two token groups relative to later groups.","tokens_in":8581,"feed_emoji":"🎤","tokens_out":9663,"duration_ms":83675,"temperature":0.7,"pith_summary":"LLM-based zero-shot text-to-speech systems preserve the acoustic environment of the supplied prompt, so a noisy recording of a few seconds degrades the synthesized voice and intelligibility. This paper argues that the cleanest place to attack that problem is the discrete acoustic-token domain, not the waveform. It proposes a codec-based denoiser whose token denoiser predicts the first two groups of clean acoustic tokens from all groups of noisy tokens, with an embedding refiner that reconstructs the summed clean embedding for waveform recovery. Paired with an LLM-based TTS system, the enhanced tokens restore synthesis to the level of a clean prompt across subjective and objective metrics, while also beating two speech enhancement baselines in perceptual quality at roughly a third of the compute. The central claim is that the first two residual-vector-quantizer groups of the codec carry enough speaker and prosodic information to serve both denoising and TTS prompting.","feed_headline":"Clean tokens, not waveforms, restore TTS on noisy prompts","feed_subtitle":"A two-group codec denoiser matches clean-prompt quality at a fraction of the compute.","key_machinery":"The carrier of the argument is the codec denoiser, built around a pre-trained audio codec with $K$ residual vector quantizers (RVQs). It has three learned components: a token denoiser, which sums the embeddings of all $K$ noisy token groups and passes them through $N_{\\mathrm{TD}}$ Conformer blocks to predict the probability distribution of the first two clean token groups, followed by an argmax; an embedding refiner, which concatenates the predicted first-two-group embedding with the summed noisy embedding and uses $N_{\\mathrm{ER}}$ Conformer blocks to predict the summed clean embedding $\\sum_{k=1}^K E(\\hat{y})_k$; and the codec decoder, which turns that embedding into the enhanced waveform. Training jointly optimizes a cross-entropy loss on the two predicted token groups and an embedding-refinement loss combining $\\ell^1$ and Frobenius terms, with teacher forcing that randomly feeds clean tokens. The load-bearing design choice is restricting prediction to the first two RVQ groups, which keeps the prediction task easy and, the paper reports, is also the point of highest denoising quality.","core_discovery":"The paper's central discovery is that residual vector quantization itself acts as a noise bottleneck: the codec's first two token groups retain the speaker and prosodic essentials, and later groups add reconstruction detail that is both harder to predict and unnecessary for the TTS prompt. The authors train a Conformer token denoiser to map every group of noisy tokens to the first two groups of clean tokens, then an embedding refiner to predict the summed clean embedding from those tokens plus the noisy full embedding, and finally the codec decoder restores the clean waveform. In their comparisons, predicting exactly two token groups gives the best perceptual scores; predicting more groups lowers quality and raises FLOPs, and the two-group design outperforms two strong signal-domain speech enhancement baselines. Integrated into the TTS system, the denoised tokens yield synthesized speech that matches the clean-prompt reference on naturalness, speaker similarity, and intelligibility, while a waveform-enhancement front end leaves a speaker-similarity gap.","pith_inferences":["If the two-group bottleneck is the real source of the gain, the same token-denoising front end should transfer to other codec-conditioned generative models, such as voice conversion or speech-to-speech translation, with little re-tuning.","The method's noise coverage in training and testing comes from distinct noise corpora and a limited SNR range, so its real-world robustness depends on the noise types seen; adversarial or non-stationary noise would be a stress test the paper does not run.","The claim that the first two groups contain essential acoustic information is inferred from an ablation, not directly measured; a probe of token-group information content would tell whether the design generalizes to codecs with different VQ counts or codebook sizes.","Because the embedding refiner receives the noisy summed embedding as a condition, the model may learn to suppress noise rather than reconstruct clean content; comparing its behavior on clean inputs would reveal whether it introduces unnecessary alterations."],"forward_implications":["The integrated TTS system, given only a noisy prompt, produces speech whose naturalness, speaker similarity, and intelligibility match the same system given a clean prompt, closing the gap left by waveform-level enhancement.","The codec denoiser scores higher on a perceptual speech-quality metric than two signal-domain speech enhancement baselines while needing roughly a third of their FLOPs.","Predicting only the first two codec token groups yields better denoised quality than predicting 4, 8, 16, or 32 groups, making the extra groups pure cost in this design.","Because the enhanced tokens are used directly as the TTS prompt, the approach avoids the artifacts that signal-domain enhancement injects into the prompt, which is why speaker similarity stays high.","The full noise-robust pipeline adds 1.10 GFLOPs over the TTS model, far below the 38.93 GFLOPs charged by the waveform-enhancement front end."],"supporting_citations":[{"why":"Defines the LauraTTS backbone, whose practice of predicting only the first two codec token groups from an acoustic prompt establishes the two-group prompting convention this paper adopts.","marker":"[4]"},{"why":"Supplies the neural audio codec whose residual vector quantization produces the discrete acoustic tokens used for both denoising and TTS conditioning.","marker":"[9]"},{"why":"Provides one of the two signal-domain speech enhancement baselines whose perceptual scores and FLOPs the codec denoiser is compared against.","marker":"[13]"},{"why":"Provides the other signal-domain baseline, also used as the waveform-enhancement front end in the TTS comparison.","marker":"[14]"},{"why":"Supplies the Conformer block architecture used as the backbone of both the token denoiser and the embedding refiner.","marker":"[20]"},{"why":"Supplies the clean speech recordings used to construct the noisy-clean training and validation pairs for the codec denoiser.","marker":"[22]"},{"why":"Supplies the noise samples used to build the noisy training set with signal-to-noise ratios between -5 and 15 dB.","marker":"[23]"},{"why":"Supplies the noise samples used to build the noisy test set with signal-to-noise ratios between 0 and 20 dB.","marker":"[24]"},{"why":"Provides the DNSMOS P.835 perceptual metric used to score the denoised speech.","marker":"[26]"}],"fun_headline_variants":["Two-group token denoise beats waveform enhancement for TTS","Codec tokens, not waveforms, clean up zero-shot TTS prompts","First two acoustic tokens suffice for noise-robust TTS","LLM TTS resists noise by denoising tokens, not audio","Token denoiser keeps TTS voice crisp on noisy inputs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the premise that the first two residual vector quantization groups of the codec already contain enough clean acoustic information to preserve speaker identity and essential prosody, for both waveform reconstruction and TTS prompting; if more groups are actually needed, the denoiser and the conditioned TTS both degrade.","fun_headline_variants_meta":{"raw":{"variants":["Two-group token denoise beats waveform enhancement for TTS","Codec tokens, not waveforms, clean up zero-shot TTS prompts","First two acoustic tokens suffice for noise-robust TTS","LLM TTS resists noise by denoising tokens, not audio","Token denoiser keeps TTS voice crisp on noisy inputs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000214,"raw_usage":{"total_tokens":1419,"prompt_tokens":930,"completion_tokens":489,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":399}},"tokens_in":546,"tokens_out":489,"duration_ms":4162,"temperature":1.0,"reasoning_tokens":399,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:09:13.310875+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a pre-trained codec denoiser and feed it prompts corrupted by a noise source absent from its training data, such as background music or room reverberation at a signal-to-noise ratio of -5 dB, then synthesize speech and compare speaker similarity and word error rate against the TTS system given the clean prompt. A significant drop would indicate that the two-group token representation is insufficient under distribution shift; a more direct test is to measure, via information-theoretic probing, how much speaker identity is retained in the first two token groups relative to later groups.","supporting_citations":[{"cited_title":"Results on the SE Task The experimental results of the SE task are depicted in Table 1","cited_arxiv_id":null,"evidence_quote":"Defines the LauraTTS backbone, whose practice of predicting only the first two codec token groups from an acoustic prompt establishes the two-group prompting convention this paper adopts."},{"cited_title":"DINO-VITS: Data-efficient zero-shot TTS with self-supervised speaker verification loss for noise robustness,","cited_arxiv_id":null,"evidence_quote":"Provides one of the two signal-domain speech enhancement baselines whose perceptual scores and FLOPs the codec denoiser is compared against."},{"cited_title":"How bad are artifacts?: Analyzing the impact of speech enhancement errors on asr,","cited_arxiv_id":null,"evidence_quote":"Supplies the clean speech recordings used to construct the noisy-clean training and validation pairs for the codec denoiser."},{"cited_title":"Learning to enhance or not: Neural network-based switching of enhanced and observed signals for overlapping speech recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the noise samples used to build the noisy training set with signal-to-noise ratios between -5 and 15 dB."},{"cited_title":"MaskSR: Masked language model for full-band speech restoration,","cited_arxiv_id":null,"evidence_quote":"Supplies the noise samples used to build the noisy test set with signal-to-noise ratios between 0 and 20 dB."}],"review_version":1}