{"id":"a8a87762-0607-4aae-9873-b2022452c8cf","arxiv_id":"2507.17735","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A token-based accent normalization pipeline using HuBERT discrete tokens, TTS-synthesized targets, and flow matching improves accent conversion over a frame-to-frame baseline.","lead":"This paper describes an accent normalization system that converts foreign-accented English to native-like speech using discrete speech tokens and non-parallel training data. It reports better naturalness, accentedness, and speaker similarity than a frame-to-frame baseline, plus two ways to preserve utterance duration.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test set reuses training sentences: L2-ARCTIC's shared prompts leak sentence content into training, so the reported gains may not reflect generalization to unseen text.","rationale":"The reader's weakest assumption concerns the correctness of synthetic target tokens and deduplication. That is a legitimate internal-method concern, but the paper's subjective results provide some independent support even if the objective accentedness metrics are circular. The sentence-leakage issue is more load-bearing because it undermines all evaluation results, including the subjective MUSHRA and WER, by allowing the model to benefit from content memorization. If the concrete test shows the superiority disappears on unseen sentences, the central claim is substantially weakened. If the superiority persists, the paper's main conclusion stands. Therefore the appropriate verdict remains CONDITIONAL: the method is promising but the evidence does not yet demonstrate generalization to new content, and the current evaluation should be corrected with a sentence-disjoint split before the claim is treated as established. I do not recommend REJECT because the methodology is coherent and the disclosed high-WER limitation is honest; the concern is testable and addressable.","tokens_in":9016,"tokens_out":7298,"duration_ms":81393,"concrete_test":"Re-run the full evaluation with a sentence-disjoint split: train on a subset of L2-ARCTIC prompts (e.g., 120 sentences) and hold out a different set of 80 prompts never seen in training, keeping the same held-out speaker per accent. Compare WER, subjective NAT/ACT/SIM, and objective metrics between the proposed method and the baseline on these unseen sentences. If the gap narrows or reverses, the current evaluation overstates generalization to new content.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a token-based, non-parallel accent normalization pipeline outperforms a frame-to-frame baseline. The evidence for this is produced under a test split that holds out one speaker per accent but does not hold out sentences (Sec. 4.1: 'Since L2-ARCTIC uses the same prompt sentences across speakers, we construct the data splits: 50 sentences for validation and 80 for testing, holding out one unseen speaker per accent.'). Because the same 80 test prompts appear in training utterances from the other speakers, the conversion model has already seen the exact target token sequences (from TTS-synthesized targets for those sentences) during training. This means the reported WER, objective accentedness metrics, and subjective MUSHRA scores measure performance on content presented in training, not on new utterances. A model with the capacity of the proposed autoregressive token converter could partially memorize content-specific mappings, inflating both objective and subjective results. This leakage affects the entire evaluation, not just one metric, making it more load-bearing than the synthetic-target circularity noted by the reader. The paper does not claim generalization to unseen sentences, but 'accent normalization' and practical applications such as dubbing implicitly require handling new text; the current evidence does not establish that.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces TokAN, an accent-normalization pipeline built on self-supervised discrete tokens. Source speech is encoded into HuBERT tokens, deduplicated, converted by an autoregressive token-to-token model, and synthesized via a flow-matching token-to-Mel module followed by a vocoder. The conversion model is pre-trained with a BART-style denoising objective, conditioned on accent embeddings, and trained with CTC phone supervision; targets are HuBERT tokens extracted from TTS-synthesized native speech with voice cloning. The authors evaluate on six L2 accents (Arabic, Chinese, Hindi, Korean, Spanish, Vietnamese) plus native American English, comparing against a frame-to-frame TTS-guided baseline. They report improvements in naturalness, accentedness, and speaker similarity, and propose two total-duration-preservation methods. They also present a token-level phoneme distribution analysis and ablations.","tokens_in":9227,"tokens_out":7338,"duration_ms":74547,"significance":"The proposed method is relevant to accent conversion and speech synthesis: operating on discrete tokens rather than frame-level features is a sensible way to reduce sensitivity to synthetic-target acoustic quality, and the modular design allows separate training of the converter and synthesizer on different corpora. The release of code and demo samples, the multi-accent comparison, the ablation study, and the interpretable token-distribution analysis are concrete strengths. That said, the evaluation as presented has a content-leakage problem in the data split and a circularity in the objective accentedness metrics; these issues affect the central claim and must be resolved before the reported superiority can be regarded as established.","major_comments":[{"comment":"The split is speaker-disjoint but not sentence-disjoint: because L2-ARCTIC uses the same prompt sentences across speakers, the 80 test sentences also occur in the training utterances of the remaining speakers for each accent. Under the training pipeline of Sec. 3.3, the conversion model is trained on token pairs for those exact sentences (source L2 tokens and TTS-synthesized native tokens). An autoregressive token converter can therefore memorize sentence-specific target token sequences rather than learn a general accent mapping. Since the WER, objective accentedness, and MUSHRA/BWS evaluations in Sec. 5 are all performed on these 80 prompts, the reported gains may substantially overstate performance on unseen text. Please re-run the evaluation with a sentence-disjoint split (excluding the test prompts from training for all speakers) and report whether the conclusions survive.","section":"Sec. 4.1 (data splits)"},{"comment":"The objective accentedness metrics (ΔPPG, FD, F0 corr.) are computed as distances to \"the synthetic targets\" produced by Matcha-TTS with voice cloning, and those same TTS-synthesized targets are the training targets for the conversion model (Sec. 3.3). The metrics therefore measure how closely the output matches the very distribution the model was optimized to produce, which is circular and can inflate apparent accent-reduction gains even if the synthetic targets are imperfectly native. Please evaluate accentedness against real native references (e.g., native ARCTIC recordings) or independent human accentedness ratings, and discuss the discrepancy, if any.","section":"Sec. 4.4 (objective accentedness metrics)"}],"minor_comments":[{"comment":"The method is called \"non-parallel,\" but the conversion training pairs each L2 utterance with a TTS-synthesized target of the same sentence, so the pairs are content-parallel even if not acoustically parallel. Please clarify the terminology.","section":"Sec. 3.3 and throughout"},{"comment":"Removing pre-training improves FD (from 35.24 to 30.83) while worsening WER (from 16.25 to 27.59). The text discusses only the WER degradation; please explain the FD behavior or discuss why FD is not trustworthy in this comparison.","section":"Table 4"},{"comment":"The text reports the absolute duration difference for \"ours w/ dur. control\" (0.064 s, 1.64%) but does not report the corresponding number for \"ours w/ dur. scaling\"; since scaling forces total duration to match by construction, please state this explicitly.","section":"Sec. 5.3"},{"comment":"With only one held-out speaker per accent, the accent-wise results in Tables 2 and 3 may be strongly speaker-dependent; consider reporting per-speaker results or using multiple test speakers.","section":"Sec. 4.1"},{"comment":"The proprietary 300-hour Chinese-accented ASR dataset is not described; please provide corpus details or state its availability.","section":"Sec. 4.2 and Sec. 4.3"},{"comment":"\"Comarison\" should be \"Comparison.\"","section":"Figure 3 caption"},{"comment":"The MUSHRA and BWS ratings are reported with confidence intervals but no significance tests; please clarify which pairwise differences are statistically significant.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":"The sentence-level leakage in Sec. 4.1 is serious enough that the revision should be re-reviewed after re-evaluation with a sentence-disjoint split. The circular objective accentedness metrics also need to be addressed. The paper has useful ideas and code, so major revision rather than reject seems appropriate if the authors can provide leakage-free results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the evaluation split leaks sentence content. L2-ARCTIC uses the same prompt sentences across speakers, and the test set is 80 sentences from held-out speakers. Those same sentences appear in training utterances from the other speakers, and the TTS-synthesized targets for those sentences are in the training token pairs. So the conversion model has seen the exact target token sequences during training. That affects WER, the objective accentedness metrics, and the MUSHRA samples—not just one number. The reader's conditional verdict didn't catch this; it is the more serious problem and undermines the central claim that the method outperforms the baseline on unseen content.\n\nWhat is genuinely good: the pipeline is a coherent combination of deduplicated HuBERT tokens, a BART-style autoregressive converter, and flow-matching synthesis. The ablations show the expected importance of pre-training and phone supervision. The duration-control experiments are a useful extra, and the authors are honest about the high WER limitation. Public code and a demo help reproducibility, even though the 300-hour proprietary dataset blocks exact replication.\n\nThe synthetic-target circularity the reader flagged is real but secondary. The objective accentedness metrics (ΔPPG, FD, F0 corr.) are computed against the same TTS-synthesized targets used for training, so they measure how well the model matches synthetic speech, not native speech. Also, the objective metrics in Table 1 have no error bars or significance tests, which makes the claimed gains hard to interpret.\n\nProportionately, this is a solid engineering study with a flawed evaluation design. The core idea is not novel—TTS-guided and token-based accent conversion already exist—but this specific combination is a reasonable incremental step. The paper does not establish generalization to new sentences, which is practically required for dubbing or language learning. A proper split that holds out sentences, not just speakers, would fix the main weakness.\n\nWho this is for: speech researchers working on accent conversion, voice conversion, or TTS-guided normalization. It deserves a serious referee because the method is sensible and the failure mode is fixable. My recommendation: send it to peer review, but require the authors to redo the evaluation with held-out sentences and to report objective metrics with uncertainty. The current evidence does not support the 'superior performance' claim as stated.","headline":"A token-based accent normalization pipeline with a sensible design, but the evaluation leaks sentence content because L2-ARCTIC shares prompts across speakers; the central comparison needs a stricter split.","tokens_in":9810,"tokens_out":2164,"would_cite":false,"duration_ms":24652,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Discrete speech tokens can normalize foreign accents without parallel training data.","keywords":["accent normalization","accent conversion","self-supervised discrete tokens","HuBERT","non-parallel data","flow matching","duration control","voice conversion"],"falsifier":"Take test sentences that exist in both accented and native recordings, run the pipeline, and score the converted output against the real native audio rather than the TTS-generated targets; if the accent-normalization claim is correct, peak-aligned measures such as phonetic posteriorgram distance, frame distortion, and F0 correlation should improve against real native speech too. A second check is to train with synthetic targets from several different TTS voices and see whether converted token distributions stay consistent; large swings would indicate the token converter is memorizing TTS artifacts rather than genuine native pronunciations.","tokens_in":8783,"feed_emoji":"🗣️","tokens_out":5081,"duration_ms":48643,"temperature":0.7,"pith_summary":"This paper claims that foreign-accented English can be normalized to native-like speech by working entirely with discrete self-supervised speech tokens, without any parallel corpus. The proposed pipeline extracts HuBERT tokens from accented input, runs an autoregressive token-to-token conversion model that maps accented token sequences onto native token sequences, and synthesizes the final waveform with a flow-matching token-to-Mel system. Training the conversion model on deduplicated tokens, where consecutive identical tokens are merged, lets it learn phonetic accent mappings while ignoring the duration and prosody of synthetic TTS targets, which the authors argue avoids the error accumulation of frame-to-frame baselines. Evaluations across six L2 accents and American English report better naturalness, lower perceived accentedness, higher speaker similarity, and lower word error rates than a frame-to-frame TTS-guided baseline. The paper also contributes two methods for preserving the original utterance duration, targeting dubbing applications.","feed_headline":"Discrete speech tokens erase accents without parallel data","feed_subtitle":"HuBERT token-to-token conversion beats a frame-to-frame TTS baseline on naturalness, accentedness, and voice identity.","key_machinery":"The load-bearing object is the HuBERT discrete token sequence with its k-means codebook of 1000 units, used after deduplication of consecutive identical tokens. Deduplication strips duration and most prosodic information while preserving phonetic identity, so the autoregressive conversion model trained with CTC phone supervision and accent-conditioned embeddings learns accent-to-accent token mappings without being distracted by synthetic target duration. The second stage is a non-autoregressive flow-matching synthesizer with a duration predictor, speaker embeddings from Resemblyzer, classifier-free guidance, and BigVGAN vocoding, which maps normalized tokens back to Mel-spectrograms and ultimately to speech.","core_discovery":"The central claim is that discrete self-supervised tokens, quantized with HuBERT and k-means clustering, carry enough phonetic content to support accent conversion, and that converting tokens to tokens rather than frames to frames is what makes non-parallel training viable. Deduplication removes duration information, so the conversion model learns a pure phonetic mapping from accented to native pronunciations, while a separately trained non-autoregressive synthesizer recovers duration and timbre from speaker embeddings. On subjective and objective measures across Arabic, Chinese, Hindi, Korean, Spanish, Vietnamese, and American English, the system outperforms a frame-to-frame diffusion baseline in naturalness, accentedness reduction, intelligibility, and speaker similarity. Phoneme-level token distribution analysis shows converted Chinese-accented speech moving closer to native token distributions on known mispronunciation patterns. The authors also report that duration-aware flow-matching prediction keeps total duration within about 1.6 percent of the source, while direct scaling is preferred subjectively.","pith_inferences":["If deduplicated tokens really separate phonetics from duration and prosody, the same two-stage recipe should transfer to other paralinguistic traits, such as emotion, dialect, or speaking style, provided a synthesizer can re-inject those traits.","A direct test of the synthetic-target bias would be to score converted speech not only against TTS targets but against real native recordings of the same sentences; the paper's objective accentedness metrics inherit whatever phonetic errors the TTS voice introduces.","The Hindi accent underperformed in intelligibility, which the authors attribute to retroflex consonants not captured by the 1000-unit token inventory; a larger or linguistically informed codebook is a testable extension.","Because WER remains elevated after conversion, a practical system might combine token-level accent normalization with a spelling or pronunciation-guiding secondary task, something the paper does not explore."],"forward_implications":["Accent conversion can be trained without paired parallel speech, using only a TTS system to generate synthetic targets and a separate native dataset for the synthesizer.","Because conversion happens in token space, the quality of synthetic TTS targets matters less than in frame-to-frame methods, mitigating error accumulation from voice cloning and duration modeling.","The reported gains on naturalness, accentedness, and speaker similarity over the frame-to-frame baseline hold across all six tested L2 accents, with the largest intelligibility gains for Chinese and Vietnamese.","Duration preservation via direct scaling or a total-duration-aware flow-matching predictor makes the pipeline usable in dubbing, at a small cost in naturalness relative to free duration.","The authors note a limitation: converted speech still has higher word error rates than source speech, so practical use awaits more robust tokenizers."],"supporting_citations":[{"why":"Supplies the HuBERT self-supervised discrete tokens that carry the phonetic content used throughout the pipeline.","marker":"[21]"},{"why":"Establishes the use of discrete SSL tokens for accent conversion with small-scale paired data, which this work extends to non-parallel settings.","marker":"[13]"},{"why":"Introduces TTS-guided training without parallel data, the baseline family this paper builds on and improves.","marker":"[16]"},{"why":"Provides the frame-to-frame diffusion baseline that the proposed token-based method is compared against.","marker":"[19]"},{"why":"Matcha-TTS serves both as the flow-matching synthesizer architecture and as the native TTS used to generate synthetic targets.","marker":"[29]"},{"why":"Provides the flow-matching framework used in the token-to-Mel synthesizer and in the total-duration-aware duration predictor.","marker":"[28]"},{"why":"Supplies the L2-ARCTIC dataset with six accented English corpora used for training and evaluation.","marker":"[37]"},{"why":"BigVGAN is the vocoder used to synthesize the final waveform from the generated Mel-spectrogram.","marker":"[40]"}],"fun_headline_variants":["Token-based accent normalization beats frame baseline","Discrete tokens convert accents without parallel speech","Self-supervised tokens improve accent conversion","Accent neutralization via token-to-token flow matching"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that native TTS plus voice cloning produces target HuBERT token sequences that are phonetically correct, and that deduplicating consecutive identical tokens removes duration and prosody without discarding accent-relevant phonetic detail; if either fails, the conversion model learns a distorted accent mapping and the objective accentedness scores, which are computed against those same synthetic targets, overstate the improvement.","fun_headline_variants_meta":{"raw":{"variants":["Token-based accent normalization beats frame baseline","Discrete tokens convert accents without parallel speech","Self-supervised tokens improve accent conversion","Accent neutralization via token-to-token flow matching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1214,"prompt_tokens":839,"completion_tokens":375,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":319}},"tokens_in":455,"tokens_out":375,"duration_ms":4408,"temperature":1.0,"reasoning_tokens":319,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:17:41.066139+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take test sentences that exist in both accented and native recordings, run the pipeline, and score the converted output against the real native audio rather than the TTS-generated targets; if the accent-normalization claim is correct, peak-aligned measures such as phonetic posteriorgram distance, frame distortion, and F0 correlation should improve against real native speech too. A second check is to train with synthetic targets from several different TTS voices and see whether converted token distributions stay consistent; large swings would indicate the token converter is memorizing TTS artifacts rather than genuine native pronunciations.","supporting_citations":[{"cited_title":"Matcha-tts: A fast tts architecture with conditional flow match- ing,","cited_arxiv_id":null,"evidence_quote":"Supplies the L2-ARCTIC dataset with six accented English corpora used for training and evaluation."},{"cited_title":"Convert and speak: Zero- shot accent conversion with minimum supervision,","cited_arxiv_id":null,"evidence_quote":"Supplies the HuBERT self-supervised discrete tokens that carry the phonetic content used throughout the pipeline."},{"cited_title":"Foreign accent con- version by synthesizing speech from phonetic posteriorgrams","cited_arxiv_id":null,"evidence_quote":"Establishes the use of discrete SSL tokens for accent conversion with small-scale paired data, which this work extends to non-parallel settings."},{"cited_title":"Tts-guided train- ing for accent conversion without parallel data,","cited_arxiv_id":null,"evidence_quote":"Introduces TTS-guided training without parallel data, the baseline family this paper builds on and improves."},{"cited_title":"Zero-shot foreign accent conversion without a native reference,","cited_arxiv_id":null,"evidence_quote":"Provides the frame-to-frame diffusion baseline that the proposed token-based method is compared against."},{"cited_title":"Hubert: Self-supervised speech repre- sentation learning by masked prediction of hidden units,","cited_arxiv_id":null,"evidence_quote":"Matcha-TTS serves both as the flow-matching synthesizer architecture and as the native TTS used to generate synthetic targets."},{"cited_title":"BigVGAN: A universal neural vocoder with large-scale train- ing,","cited_arxiv_id":null,"evidence_quote":"BigVGAN is the vocoder used to synthesize the final waveform from the generated Mel-spectrogram."}],"review_version":1}