{"id":"c6c202f9-e174-498c-a4e7-5e50b9e8d592","arxiv_id":"2412.01040","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Training a fake-voice detector with Indonesian and Thai accented English added to native speech cuts non-native equal error rate from roughly 36-43% to 9-15%.","lead":"The authors built a new dataset of native, Indonesian-accented, and Thai-accented English speech with synthetic spoof attacks, and compared fake-voice detectors trained on native-only versus accent-mixed data. Adding the accented data to training cut non-native detection errors from more than 35% EER to below 9% EER, showing that accent coverage, not just algorithm choice, is the key to spoofing detection in multilingual settings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Non-native test spoof is generated with the same three attack systems used to build the Combined CM's training set, so the reported gain may be attack-algorithm familiarity, not accent generalization; a held-out-generator evaluation is needed.","rationale":"Good-faith reading: the paper is a scoped empirical study with a new dataset, standard features/classifiers, and visible large effects; the in-domain improvement of Combined CM over Native CM in Table II is real as reported. The reader's weakest assumption correctly locates the risk: the non-native test spoof is produced by the same generators (and partly same speaker-derived features) as the non-native training spoof, so the measured improvement may not reflect general capability on non-native spoofing. I agree with that assessment. The most decisive way to resolve it is a held-out attack evaluation; a size-matched native-only training control would also separate the data-quantity confound. Because the paper currently provides neither, the conditional verdict is appropriate: the central claim should not be relied on until this generalization check is run. No evidence of internal inconsistency in the main numbers was found, though the dataset-size sentence in Section III-A ('4,000 utterances... eliminated 10...') appears to be a typo for the 7,990-utterance total.","tokens_in":8966,"tokens_out":8437,"duration_ms":79190,"concrete_test":"Generate a non-native English evaluation set with a held-out attack system not among SpeechT5/FreeVC/WORLD (e.g., VITS or HiFi-GAN TTS, or another VC), using the same Indonesian/Thai speakers and recording setup; evaluate the existing Combined and Native CMs on it. If the Combined CM's margin over Native CM collapses (e.g., minDCF rises toward the >0.79 range seen for Native CM), the reported advantage is largely explained by training/test generator overlap. As a complementary control, train a Native CM on native English spoof generated by SpeechT5/FreeVC/WORLD and evaluate it on the non-native held-out set; this separates 'unseen algorithm' from 'unseen accent' as the cause of Native CM's failure. Release the resulting score files and decision thresholds so the comparison can be checked.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the evaluation of the central claim that adding non-native data is what improves non-native spoof detection. In Experiment 2 the Combined CM is trained on native plus non-native spoof, and the non-native spoof in the test set is generated by exactly the three systems used to create that non-native training spoof (SpeechT5, FreeVC, WORLD, Section III-B). The Native CM, by contrast, was trained only on ASVspoof5 native spoof attacks (Section V-A), so the non-native test attacks are also largely unseen algorithms for it. Thus the Native-vs-Combined comparison changes both the accent domain and the attack-algorithm overlap simultaneously. If a large part of the improvement comes from having seen these generators during training, the conclusion that accent coverage is the driver would not generalize to unseen spoofing attacks. The paper itself does not report any held-out attack condition, and the SpeechT5 subset additionally draws x-vector/bottleneck features from the collected bonafide non-native speaker data, further reducing the gap between training and test conditions. A secondary confound is that the Combined CM also trains on roughly twice as many utterances, so gains could reflect data quantity rather than accent domain. This is an external-validity concern, not an internal inconsistency: the in-domain numbers in Table II may be correct as reported, but they do not yet establish the accent-domain explanation claimed in the abstract and conclusions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses spoofing countermeasure (CM) performance on non-native English speech from Indonesian and Thai speakers. The authors construct the ENIT dataset, containing 7,990 bonafide utterances from native English speakers and 7,990 from non-native speakers, plus spoof utterances generated from ASVspoof5 native data and, for non-native data, from SpeechT5, FreeVC, and WORLD. They train two types of CMs (Native and Combined) using MFCC/LFCC/CQCC features with CatBoost, XGBoost, and GMM classifiers. Experiment 1 shows that Native CMs perform poorly on non-native evaluation (minDCF > 0.79, EER > 35% for tree-based models). Experiment 2 shows that Combined CMs substantially improve non-native detection (best CQCC+CatBoost: minDCF 0.19, EER 8.56%) with only modest native degradation. The paper concludes that incorporating non-native speech data into training is necessary for robust spoofing detection in non-native contexts.","tokens_in":9212,"tokens_out":5628,"duration_ms":46841,"significance":"If the central claim is sustained, the paper would make a useful contribution: a new non-native spoofing dataset, a clear demonstration of the domain gap, and strong baselines for future work. The strengths are the explicit focus on an understudied population, the use of standard features/classifiers and challenge metrics (minDCF, EER), and the large effect sizes that are easy to reproduce. However, the current evidence does not yet separate accent-domain coverage from attack-algorithm familiarity or from increased training data quantity. The dataset contribution and baseline numbers remain valuable regardless, but the interpretation in the abstract and conclusions needs additional controlled experiments. No formal circularity is present; the concern is external validity of the measurement.","major_comments":[{"comment":"The central Native-vs-Combined comparison on non-native speech is confounded by attack-algorithm overlap. The non-native test spoof is generated with the same three systems (SpeechT5, FreeVC, WORLD) used to create the non-native training spoof; for SpeechT5, roughly 2,000 training utterances use x-vector and bottleneck features extracted from the collected non-native bonafide speaker data (Section III-B). The Native CM, by contrast, was trained only on ASVspoof5 native spoof attacks (Section V-A). Therefore the non-native test attacks are unseen algorithms for Native CM but seen for Combined CM, so the improvement could reflect attack familiarity rather than accent-domain capability. To support the claim that adding non-native speech data is what drives the gain, the authors should add a held-out attack condition (e.g., generate non-native test spoof with unseen TTS/VC systems) or an attack-matched ablation (e.g., train Native CM on the same three attack algorithms using native speech only). Without such a control, the conclusion in the abstract and Section VII is an overinterpretation of the in-domain results.","section":"III-B / V-B, Table II"},{"comment":"The Combined CM is trained on both native and non-native training sets, roughly doubling the number of training utterances relative to Native CM. The reported improvement could therefore be an effect of training-set size rather than of the accent domain. The authors should control for data quantity, for example by subsampling an equal-size native training set for Native CM or by adding an equivalent amount of native-only data to the Native CM, to isolate the contribution of non-native data. This is a second confound for the central claim and should be addressed in the revised manuscript.","section":"V-B / V-C"},{"comment":"All experiments use a single random seed (42) and report point estimates without error bars, confidence intervals, or significance tests. Claims such as \"significantly improved\" in Section VI and the identification of a \"best\" system are therefore not statistically supported. At minimum, the authors should run multiple seeds for the tree-based classifiers and report mean and standard deviation, or use a paired test across seeds. This is important for between-feature and between-classifier comparisons, though the main Native-vs-Combined gap is large enough that it may survive such analysis.","section":"V-A, Table II"}],"minor_comments":[{"comment":"The description of native bonafide sampling is ambiguous: the text first says \"randomly selected 4,000 utterances\" and then Table I reports 7,990 native bonafide utterances. Please clarify the sampling procedure and reconcile the numbers.","section":"III-A"},{"comment":"The table caption should explain the role of the separator '/' in the speaker-count columns; the text defines it only later in Section III-B.","section":"Table I"},{"comment":"The GMM classifier description says separate GMMs are learned for bonafide and spoof data, but it is not stated whether the likelihood ratio is score-normalized; the 100% EER for MFCC with GMM in Table II suggests a possible implementation or thresholding issue. Please clarify or check the GMM scoring pipeline.","section":"IV-B, Table II"},{"comment":"The sentence \"This behavior may be due to the increased variability introduced by the non-native speech data within the training sets\" is speculative; consider removing it or supporting it with a small analysis of native performance variability.","section":"VI"},{"comment":"Reference [5], \"ASVspoof 5: Crowdsourced data, deepfakes and adversarial attacks at scale,\" is listed as submitted; please update to the published version if available.","section":"II"},{"comment":"The authors should state whether the ENIT dataset and generation code will be made publicly available, as this is central to the dataset contribution and to reproducibility.","section":"III / VII"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the venue and makes a useful dataset contribution. The main concern is not internal inconsistency but external validity: the headline improvement could be explained by attack-algorithm overlap or data quantity. I would like to see the authors run the proposed controlled experiments before publication. The paper would also benefit from multiple seeds and a more careful treatment of the GMM degenerate results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper gives the community something it didn't have — an English native plus Indonesian/Thai accented spoof corpus (ENIT) with TTS/VC attacks — and shows that a countermeasure trained only on ASVspoof5 native speech collapses on non-native speech (minDCF > 0.79, EER > 35%), while adding the non-native training data brings the best system down to minDCF 0.19, EER 8.56%. The effect sizes are large and the native-vs-non-native gap is probably real.\n\nThe new content is the corpus and the first systematic evaluation of native-trained CMs on those accents. The methodological recipe is standard (MFCC/LFCC/CQCC + CatBoost/XGBoost/GMM), which is fine for a baseline paper.\n\nThe main weakness is that the central comparison doesn't isolate accent. The Combined CM's non-native training spoof is generated with the same three systems — SpeechT5, FreeVC, WORLD — used to make the non-native evaluation spoof, and for SpeechT5, the x-vector/bottleneck features partly come from the same non-native speaker pool. The Native CM, trained only on ASVspoof5, likely hasn't seen those exact generators. So the reported improvement could be attack-algorithm familiarity rather than accent coverage. Also, the Combined CM trains on roughly twice as many utterances, so data quantity is a second confound. The paper doesn't report a held-out-generator condition, so the abstract's claim that incorporating non-native data is what matters is not yet established.\n\nOther soft spots: no error bars or significance tests (single seed), the dataset and code aren't released, and there's an internal inconsistency in Section III-A — the text says 4,000 utterances were selected for native bonafide, but the table sums to 7,990. That needs fixing before the numbers can be trusted.\n\nWhat holds up: the in-domain numbers in Table II are clear and the effect is large. The dataset, if released, would be a useful resource for the anti-spoofing community. The paper deserves refereeing, but it should be conditional on addressing the confound with a held-out attack generator, adding error bars, releasing the corpus, and fixing the inconsistency.\n\nRecommendation: serious editor should send it to peer review; expect heavy revision rather than desk rejection.","headline":"Useful new dataset and a striking in-domain result, but the accent-generalization claim is confounded by attack-generator overlap; needs a held-out-attack experiment and error bars.","tokens_in":9782,"tokens_out":3729,"would_cite":false,"duration_ms":30590,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Spoofing countermeasures trained only on native English fail on Indonesian and Thai accented speech, while adding non-native training data brings the best system's EER from above 35% down to 8.56%.","keywords":["spoofing countermeasure","non-native speech detection","Indonesian accent English","Thai accent English","voice conversion attack","text-to-speech attack","cepstral features","gradient boosting"],"falsifier":"Run the Combined CM on non-native test spoofs produced by an attack algorithm that was absent from the non-native training set (leave-one-generator-out), using non-native speakers whose voices were never used to generate training spoofs; if minDCF and EER rise back toward Native CM levels, the paper's conclusion that adding non-native data is sufficient would be undermined.","tokens_in":8743,"feed_emoji":"🎙️","tokens_out":6884,"duration_ms":54908,"temperature":0.7,"pith_summary":"This paper asks whether a spoofing countermeasure trained only on native English can detect text-to-speech and voice-conversion attacks in English spoken with Indonesian and Thai accents. It constructs a new dataset, ENIT, with native and non-native bonafide recordings and spoof utterances generated by SpeechT5, FreeVC, and WORLD, then trains three feature-classifier combinations on either native-only or native-plus-non-native data. The answer it defends is that native-only training performs poorly on non-native speech, with minDCF above 0.79 and EER mostly above 35%, while adding non-native speech to training improves non-native detection dramatically: the best system (CQCC features with CatBoost) reaches minDCF 0.19 and EER 8.56% at a small cost to native accuracy. The paper's central claim is that accent-domain coverage in the training data, rather than a more sophisticated model, is what drives non-native spoof detection performance here.","feed_headline":"Accented training data cuts non-native spoof-detection error to 8.6%","feed_subtitle":"Native-only training gives >35% EER on these accents; adding them to training cuts EER to 8.6%.","key_machinery":"The load-bearing object is the ENIT dataset, which pairs English-native speech from ASVspoof 5 with newly recorded Indonesian and Thai accented English, and generates spoof utterances using three systems: SpeechT5 (speech synthesis and voice conversion), FreeVC (one-shot voice conversion), and WORLD (vocoder-based synthesis). The argument works by comparing two training regimes, a Native CM trained only on native speech and a Combined CM trained on native plus non-native speech, while keeping front-end features (MFCC, LFCC, CQCC) and classifiers (CatBoost, XGBoost, GMM) identical. The performance gap between the two regimes on the non-native evaluation set is the evidence for the paper's claim.","core_discovery":"The central discovery is that the population of training speakers, not the choice of feature or classifier, is the dominant factor in detecting spoofed non-native speech in this setup. A countermeasure trained solely on native English degrades sharply when tested on Indonesian and Thai accented English, with error rates above 35% even for the best classifiers; the same classifiers trained on a combined native and non-native corpus detect non-native spoofs far more reliably, and the best configuration, CQCC with CatBoost, cuts the non-native EER to 8.56% while only slightly hurting native performance. The paper interprets this as demonstrating the necessity of domain-specific training data for non-native accents.","pith_inferences":["Because the non-native test spoofs are generated by the same three attack systems used in training, the measured advantage of Combined over Native is an in-domain comparison; a leave-one-generator-out test could show a smaller gap.","If the accent-coverage interpretation is right, the same recipe, adding target-accent data to training without changing the model, should transfer to other under-resourced accents such as Vietnamese or Filipino English, as long as attack diversity is kept similar.","The small native-side degradation suggests a data-balancing or domain-adaptation approach might recover native accuracy while keeping non-native gains, a direction the paper does not explore."],"forward_implications":["Deploying a countermeasure built only on native English speech on Indonesian or Thai accented users risks error rates above 35%; such systems should be retrained or augmented with accent-matched data.","Adding non-native speech to training yields large gains on non-native evaluation (average minDCF improvement around 0.41 and EER improvement around 30% relative), with only a small native-side penalty.","Simple hand-crafted cepstral features with gradient-boosted tree classifiers are sufficient to obtain these gains; the best configuration is CQCC with CatBoost.","ENIT provides a baseline dataset for future work on non-native spoofing countermeasures, including deep-learning models that the paper leaves to future work."],"supporting_citations":[{"why":"supplies the native English bonafide and spoof utterances that seed the Native and Combined training sets, plus the minDCF and EER evaluation protocol.","marker":"[5]"},{"why":"one of the three generators of non-native spoof data; SpeechT5 with swapped x-vector and bottleneck features creates roughly 5200 synthetic utterances.","marker":"[14]"},{"why":"one of the three non-native spoof generators; FreeVC performs one-shot voice conversion on the non-native speech.","marker":"[19]"},{"why":"one of the three non-native spoof generators; WORLD synthesizes speech with F0, aperiodicity, and spectral envelope plus randomized speed and F0 manipulation.","marker":"[22]"},{"why":"defines CQCC, the feature set used by the best-performing system in the comparison.","marker":"[26]"},{"why":"CatBoost gradient boosting is the classifier used in the best-performing CQCC+CatBoost configuration.","marker":"[27]"},{"why":"XGBoost provides the main competing classifier achieving near-identical competitive results.","marker":"[28]"},{"why":"provides the feature extraction implementation (Matlab toolbox and Smileslab) used to compute the front-end features.","marker":"[31]"}],"fun_headline_variants":["Native-only training fails on accented spoofs: 35% EER vs 8.6%","For spoof detection, training speakers matter more than features","Accent-aware training cuts spoof detection error to 8.56%","Non-native spoofs require non-native training data","Spoof detection: adding accented speech to training beats new features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim assumes that detecting spoofs generated by the same three systems (SpeechT5, FreeVC, WORLD) used to build the non-native training spoof, with largely the same non-native speaker pool feeding the SpeechT5 features, predicts how the countermeasure will perform against unseen attack algorithms and different non-native speakers.","fun_headline_variants_meta":{"raw":{"variants":["Native-only training fails on accented spoofs: 35% EER vs 8.6%","For spoof detection, training speakers matter more than features","Accent-aware training cuts spoof detection error to 8.56%","Non-native spoofs require non-native training data","Spoof detection: adding accented speech to training beats new features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1470,"prompt_tokens":867,"completion_tokens":603,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":505}},"tokens_in":483,"tokens_out":603,"duration_ms":5438,"temperature":1.0,"reasoning_tokens":505,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:45:27.990414+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Combined CM on non-native test spoofs produced by an attack algorithm that was absent from the non-native training set (leave-one-generator-out), using non-native speakers whose voices were never used to generate training spoofs; if minDCF and EER rise back toward Native CM levels, the paper's conclusion that adding non-native data is sufficient would be undermined.","supporting_citations":[{"cited_title":"ASVspoof 5: Crowdsourced data, deep- fakes and adversarial attacks at scale,","cited_arxiv_id":null,"evidence_quote":"supplies the native English bonafide and spoof utterances that seed the Native and Combined training sets, plus the minDCF and EER evaluation protocol."},{"cited_title":"SpeechT5: Unified- Modal Encoder-Decoder Pre-Training for Spoken Lan- guage Processing,","cited_arxiv_id":null,"evidence_quote":"one of the three generators of non-native spoof data; SpeechT5 with swapped x-vector and bottleneck features creates roughly 5200 synthetic utterances."},{"cited_title":"FreeVC: Towards High- Quality Text-Free One-Shot V oice Conversion,","cited_arxiv_id":null,"evidence_quote":"one of the three non-native spoof generators; FreeVC performs one-shot voice conversion on the non-native speech."},{"cited_title":"WORLD: A vocoder-based high-quality speech synthesis system for real-time applications,","cited_arxiv_id":null,"evidence_quote":"one of the three non-native spoof generators; WORLD synthesizes speech with F0, aperiodicity, and spectral envelope plus randomized speed and F0 manipulation."},{"cited_title":"An Explainability Study of the Constant Q Cepstral Coefficient Spoofing Countermeasure for Automatic Speaker Verification,","cited_arxiv_id":null,"evidence_quote":"defines CQCC, the feature set used by the best-performing system in the comparison."},{"cited_title":"CatBoost: unbiased boosting with categorical features,","cited_arxiv_id":null,"evidence_quote":"CatBoost gradient boosting is the classifier used in the best-performing CQCC+CatBoost configuration."},{"cited_title":"XGBoost: A Scalable Tree Boosting System,","cited_arxiv_id":null,"evidence_quote":"XGBoost provides the main competing classifier achieving near-identical competitive results."}],"review_version":1}