{"id":"fd734c60-e4b7-4eb5-b100-9e81935beac8","arxiv_id":"2502.03559","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Across six self-supervised speech models and ten deepfake datasets, the first 4-12 transformer layers match full-model fake audio detection performance, reducing parameters by at least half.","lead":"This paper tests whether only a few early layers of speech AI models can detect fake audio as well as the full models, across English, Chinese, Spanish, songs, and acoustic scenes. It finds that lower layers carry the most useful signals, so truncated models can be much cheaper and faster with little loss in accuracy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 4-6/10-12 layer recommendation hinges on sub-1-point average EER gaps chosen post hoc with no reported variance; a hold-out/CI check is needed.","rationale":"The paper's practical claim is that truncated SSL models using only 4-6 layers (small) or 10-12 layers (large) perform as well as, or better than, the full model while halving parameters. For this to hold as stated, the superiority of the truncated configurations would need to be a real, reproducible effect rather than an artifact of choosing the best of several configurations on the same evaluation sets. The direct truncation experiments in Tables 1, 3, and 4 are a genuine strength, and the paper provides public code and models. However, Table 4's exact optimal ranges are selected post hoc on the same data used for evaluation, the margins over the full model are below 1 EER point, and no variance is reported despite three seeds. The paper itself acknowledges that full models win on SceneFake and PartialSpoof, weakening 'consistently.' A leave-one-dataset-out or confidence-interval check would settle whether the ranges generalize. The reader's conditional verdict already captures this concern, so no change to the verdict is needed.","tokens_in":16729,"tokens_out":6817,"duration_ms":63831,"concrete_test":"Run a paired bootstrap or per-seed standard-deviation analysis on Table 4's EER differences, and perform a leave-one-dataset-out selection: for each held-out dataset, choose the layer count that minimizes average EER on the remaining nine datasets, then evaluate on the held-out dataset. If the chosen counts vary across folds, or if the held-out performance of the selected counts is not better than the full model beyond the noise floor, the specific layer-range recommendation should be downgraded from 'optimal' to 'comparable.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 4's recommended layer ranges are selected after seeing all datasets, and the margins over the full model are tiny: for Small models, 6 layers gives an average EER of 20.98% versus 21.74% for the full 12-layer model (a 0.76-point gap), while 4 layers is 21.01%; for Large models, 12 layers gives 17.78% versus 17.97% for the full 24-layer model (a 0.19-point gap), and WavLM-Large's full model is actually 0.09 points better than its 12-layer version. These averages are dominated by the hardest datasets (ADD23 EERs around 45-75%), and the paper reports only the mean of 3 seeds with no standard deviations or per-dataset confidence intervals. The exact '4-6' and '10-12' ranges are therefore not distinguishable from selection noise, especially because the optimal count varies by model family (e.g., Hubert-Small's best is 4 layers, WavLM-Large's best is the full model). The qualitative conclusion that truncated models are broadly comparable may survive, but the word 'consistently' and the precise optimal-layer recommendation are not supported by the reported evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a layer-wise analysis of six frozen SSL front-ends (Wav2Vec2, Hubert, WavLM in small/large versions) for audio deepfake detection across ten diverse datasets spanning multiple languages and scenarios (full, partial, song, scene). The authors learn softmax-weighted combinations of transformer-layer outputs (Eq. 1-2) with two back-end classifiers (FFN and AASIST), and additionally train truncated models using only the first 2-12 layers. The central claim is that lower layers provide the most discriminative features and that reduced-layer models using 4-6 layers (small) or 10-12 layers (large) achieve comparable or better average EER than the full models. The paper reports mean EER over three seeds for each configuration.","tokens_in":17009,"tokens_out":3789,"duration_ms":35595,"significance":"If the main claim is supported, the paper would have practical value: it quantifies how much of a frozen SSL model can be discarded for deepfake detection, potentially reducing compute and memory substantially. The experimental scope is a strength: six front-ends, two back-ends, and ten datasets spanning languages and deepfake types, with direct truncated-model comparisons rather than only correlational weight analysis. The authors also release code and models. However, the headline quantitative recommendation (4-6 and 10-12 layers) is built on very small average-EER differences computed on the same evaluation sets that were used to choose the ranges, and no variance or significance testing is reported. The qualitative trend that lower layers contain useful discriminative information is plausible and partially confirmed by the truncated models, but the paper's stronger wording ('consistently', 'optimal') is not fully supported by the evidence.","major_comments":[{"comment":"The recommended reduced-layer ranges (4-6 for small, 10-12 for large) are selected based on the same evaluation sets used to report final performance, and no error bars, standard deviations, confidence intervals, or significance tests are provided. The reported mean EER over 3 seeds leaves differences such as the 6-layer small A_VGout of 20.98% vs the full model's 21.74%, and the 12-layer large 17.78% vs full 17.97%, within the range of selection noise. In fact, Table 4 shows that for WavLM-Large the full model is 0.09 points better than the 12-layer version, and for several model/back-end pairs (e.g., Wav2Vec-Small-AASIST, WavLM-Large-AASIST) the full model is the best configuration overall. The average EER is dominated by the hardest datasets (ADD23, with EERs around 45-75%), while easier datasets like LA19 have EERs near 0.5%, so the pooled average may not reflect per-dataset behavior. The qualitative conclusion that truncated models are broadly comparable may survive, but the precise optimal-layer recommendation and the word 'consistently' require a hold-out evaluation or at least bootstrap confidence intervals over datasets and seeds.","section":"§4.2, §5.3, Table 4"},{"comment":"The abstract states that 'lower layers consistently provide the most discriminative features,' but the paper's own results contain notable exceptions. For SceneFake, the full models for both Wav2Vec-Small and WavLM-Small consistently outperform their reduced-layer versions (by about 0.5-1.5% EER), and for PartialSpoof the full models consistently outperform reduced-layer models, as acknowledged in §5.2. In Table 4, Wav2Vec2-Large-FFN and WavLM-Large-AASIST full models achieve the best A_VGin. The qualitative finding that several reduced-layer configurations match or beat the full model on multiple datasets is valuable, but the claim of universal consistency should be softened to 'often' or 'in most datasets', and the exceptions should be explicitly factored into the conclusion.","section":"Abstract, §5.2"},{"comment":"The layer-importance analysis uses the learned softmax weights w_l from Eq. (2) to conclude that lower layers are 'more important.' However, these weights are not identifiable as causal contributions: the outputs of different transformer layers are highly correlated, and the weights are jointly optimized with a classifier, so any linear recombination of correlated features can produce many equivalent weight distributions. The truncated-model experiments in §5.2 provide a more direct measure of the value of lower layers, and they are the strongest evidence in the paper. The weight-based analysis should be presented as a heuristic indicator rather than a direct measurement of layer importance, or supplemented with an ablation that removes individual layers (e.g., leave-one-out) to confirm the interpretation.","section":"§3, §5.1, Figures 2-3"}],"minor_comments":[{"comment":"The caption text 'A VERAGE' contains an unintended space; it should read 'AVERAGE'.","section":"Figure 2, Figure 3"},{"comment":"There are repeated typos in model names and back-end names: 'Wav2Vev2' instead of 'Wav2Vec2', 'ASSIST' instead of 'AASIST', and 'Wav2Vec-Small' vs 'Wav2Vec2-Small' are used inconsistently.","section":"Tables 1 and 3 and text"},{"comment":"The notation A_VGin and A_VGout is confusing: A_VGin is the average over datasets within a model, and A_VGout is the average over models, but the subscripts 'in' and 'out' are not intuitive. Please define these more explicitly in the text and table caption.","section":"§5.3, Table 4"},{"comment":"The sentence 'Tables 1 and 3 show a10.95% difference' is missing a space after 'a'; rephrase to avoid ambiguity (e.g., 'a difference of 10.95%').","section":"§6"},{"comment":"The paper states 'Each experiment is repeated 3 times with different seeds for reliable results and the average results are reported,' but no standard deviations or variance measures are given anywhere. Reporting at least the standard deviation of EER over the three seeds would make the reliability of the small average differences assessable.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a broad empirical study with a useful practical message, but the headline recommendation is currently supported by post-hoc selection and missing variance analysis. The central trend (lower layers are useful, truncated models are roughly competitive) may well be correct, but the specific 4-6/10-12 ranges and the word 'consistently' need to be either statistically justified or substantially softened. This is within the scope of a major revision rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper earns its keep on the qualitative claim: across six SSL models and ten datasets spanning multilingual, partial-fake, song, and scene scenarios, lower transformer layers carry most of the discriminative signal for audio deepfake detection, and models truncated to roughly half their layers match or slightly beat full-model average EER. That is a useful, practical result, and the experiments are real: they train reduced-layer models from scratch, not just reweight features, and they release code and models. The limitations section is also honest about scope.\n\nThe soft spot is the precision of the headline. The specific recommendations—4-6 layers for small models, 10-12 for large—are chosen after looking at the same evaluation sets used to report final EER, and the margins are small: the gap between the best small configuration and the full model is about 0.76 points EER, and for large models the 12-layer configuration beats the full model by 0.19 points, with WavLM-Large's full model actually 0.09 points better than its 12-layer version. No standard deviations or confidence intervals are reported, and the averages are dominated by hard datasets like ADD23. So the exact numbers are not distinguishable from selection noise, even though the general trend—truncated models are comparable—is probably right. The word 'consistently' in the abstract overstates it.\n\nThe citation pattern is fair: they cite the prior layer-wise work they build on, and the novelty is the systematic scope rather than a new mechanism.\n\nWho is this for? Practitioners who want cheap deepfake detectors and researchers studying where SSL features live. It deserves a serious referee, but I'd expect revision: per-dataset confidence intervals or a held-out layer-count selection, and toning down the optimal-layer claim to 'typically.' If that's done, it's a solid contribution worth citing.\n\nRecommendation: send to peer review.","headline":"A broad, useful ablation of SSL layers for deepfake detection, whose precise layer-count recommendation outruns the reported statistics.","tokens_in":17502,"tokens_out":3558,"would_cite":true,"duration_ms":31338,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Audio deepfake detection can drop most of a self-supervised speech model's layers and still match or beat full-model accuracy.","keywords":["audio deepfake detection","self-supervised learning","layer-wise analysis","Wav2Vec2","HuBERT","WavLM","model compression","equal error rate"],"falsifier":"Train a truncated model, say Wav2Vec2-Small-6, and its full counterpart on one corpus, fix the layer count before looking at test results, and evaluate on a held-out fake-audio generator or language not among the paper's ten datasets; if the full model clearly wins on several such sets, or if the learned layer weights spread toward upper layers, the 'lower layers suffice' claim fails.","tokens_in":16547,"feed_emoji":"🎧","tokens_out":6779,"duration_ms":58494,"temperature":0.7,"pith_summary":"The paper argues that, across six self-supervised speech models and ten datasets spanning English, Chinese, and Spanish, plus partial, song, and scene fakes, the information that separates real speech from deepfakes is concentrated in the lower transformer layers. It therefore claims that a detector can discard most of a model's upper layers, keeping only the first 4 to 6 layers of small models or 10 to 12 layers of large models, and still match, and often beat, the full model's equal error rate. If right, this makes audio deepfake detection substantially cheaper and faster without sacrificing accuracy, and it reframes what SSL models actually learn for this task: local artifact patterns rather than global utterance abstractions.","feed_headline":"Lower layers match full models at audio deepfake detection","feed_subtitle":"Keeping only the first few transformer layers still catches deepfakes, so detectors can run faster and on lighter hardware.","key_machinery":"The load-bearing object is the weighted layer-aggregation front end, $h_{\\mathrm{final}} = \\sum_{l=1}^{L} w_l h_l$, where $h_l$ are the per-layer transformer outputs and the weights $w_l$ are learnable and softmax-normalized. By freezing the SSL front-end and reading the trained weights, the paper obtains an empirical importance score for each layer; by then evaluating truncated models that keep only the first $k$ layers, it converts those scores into a parameter-reduction recipe. The two back-end heads, a two-layer feedforward network with attentive statistical pooling and AASIST, a graph-based audio anti-spoofing classifier, act as controls showing the lower-layer preference is not specific to one classifier.","core_discovery":"The central discovery is a layer-wise importance profile for frozen SSL front-ends: learned softmax weights assigned to each transformer layer, averaged over datasets, concentrate in layers 1-6 for small models and 1-12 for large models, with the strongest weights around layers 4-7. When the model is truncated to these lower layers and retrained with either a lightweight feedforward head or the AASIST graph-attention anti-spoofing head, its averaged EER over ten datasets is 20.98% for the best small six-layer configuration versus 21.74% for full small models, and 17.78% for the best large twelve-layer configuration versus 17.97% for full large models. The paper interprets this as evidence that lower layers capture local, sub-band, and short-time artifacts left by speech synthesis and voice conversion, while upper layers carry less task-relevant abstraction.","pith_inferences":["Editorial extension: the layer-count recommendation is selected on the same ten datasets used for reporting, so a stronger test would fix 4-6 and 10-12 layers beforehand and validate on a new language or a new generator; the paper does not run that held-out protocol.","Editorial extension: if the lower-layer preference reflects local synthesis artifacts, then truncated detectors should degrade most on attacks that hide artifacts at longer time scales or in global prosody, and comparing truncation behavior across latest neural codec and audio-language-model generators would test this.","Editorial extension: because large models matter most on the hardest recent datasets, the practical recipe may be a small truncated model for low-cost screening plus a large truncated model only when budget allows, rather than one universal layer count."],"forward_implications":["Detectors built from the first 4-6 layers of small models or 10-12 layers of large models cut parameter count by at least half while keeping average EER within roughly 0.3-0.7 points of the full model.","Lower-layer truncation improves generalization on several benchmarks, for example Wav2Vec2-Small-6 with a feedforward head reaches 0.48% EER on ASVspoof2019 LA and Wav2Vec2-Large-12 with AASIST reaches 0.21%, both beating their full models.","The reduction holds consistently across English, Chinese, and Spanish and across full, partial, song, and scene fakes, so it is not an artifact of one language or one attack type.","The main exceptions are partial-fake and scene-fake data, where full models still outperform truncated ones on PartialSpoof and SceneFake, though by small margins, indicating that upper layers matter when artifact location or global context is involved."],"supporting_citations":[{"why":"Supplies the Wav2Vec2 architecture and contrastive pre-training used as one of the six front-end models.","marker":"Baevski et al., 2020"},{"why":"Supplies HuBERT and its masked-prediction pre-training objective, the second front-end family tested.","marker":"Hsu et al., 2021"},{"why":"Supplies WavLM, the third front-end family, including its denoising auxiliary task.","marker":"Chen et al., 2022"},{"why":"Provides the lightweight feedforward back-end and the practice of reading hidden states from specific layers, which this paper generalizes to a full layer profile.","marker":"Martín-Doñas and Álvarez, 2022"},{"why":"Provides the AASIST graph-attention classifier used as the second back-end control.","marker":"Jung et al., 2022"},{"why":"Establishes fine-tuning Wav2Vec2 for spoof detection, the baseline this paper extends to layer-wise truncation.","marker":"Tak et al., 2022"},{"why":"Prior evidence that a single lower layer's hidden states help spoof-aware detection, broadened here into a systematic layer analysis.","marker":"Lee et al., 2022"},{"why":"Contributes the PartialSpoof dataset for partial-fake evaluation, one of the scenarios where upper layers still contribute.","marker":"Zhang et al., 2021"}],"fun_headline_variants":["Lower layers alone match full model deepfake detection","Deepfake detection: low layers do the heavy lifting","Skip upper SSL layers, still spot deepfakes","Truncate SSL models for faster deepfake detection","Low SSL layers carry deepfake detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The recommended layer counts are chosen after seeing results on the same benchmark sets, and the average that drives them is itself pulled hardest by datasets with EERs near 60 percent, so the whole recommendation rests on that post-hoc average being representative of the deepfake world.","fun_headline_variants_meta":{"raw":{"variants":["Lower layers alone match full model deepfake detection","Deepfake detection: low layers do the heavy lifting","Skip upper SSL layers, still spot deepfakes","Truncate SSL models for faster deepfake detection","Low SSL layers carry deepfake detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001406,"raw_usage":{"total_tokens":5651,"prompt_tokens":883,"completion_tokens":4768,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":4696}},"tokens_in":499,"tokens_out":4768,"duration_ms":30059,"temperature":1.0,"reasoning_tokens":4696,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T04:30:51.013743+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a truncated model, say Wav2Vec2-Small-6, and its full counterpart on one corpus, fix the layer count before looking at test results, and evaluate on a held-out fake-audio generator or language not among the paper's ten datasets; if the full model clearly wins on several such sets, or if the learned layer weights spread toward upper layers, the 'lower layers suffice' claim fails.","supporting_citations":[],"review_version":1}