{"id":"54f4068b-c8af-43e6-9f27-84f54b9b379a","arxiv_id":"2506.10683","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adding SE attention blocks to a small CNN raises deepfake detection accuracy from 91.13% to 94.14% on the StyleGAN subset of DFFD, but the result rests on a single run with questionable comparisons.","lead":"A compact convolutional network with squeeze-and-excitation attention reports 94.14% accuracy and 0.985 AUC on the StyleGAN subset of the Diverse Fake Face Dataset. The claim of competitiveness with state-of-the-art detectors is weakened by cross-dataset comparisons and an inconsistent evaluation protocol.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evaluation protocol in Section IV.A is internally contradictory (80/20 split vs. 10-fold cross-validation), so the headline 94.14% accuracy and 0.985 AUC, and the 3.01-point SE gain over CNN, are not reproducible without a stated single protocol and variance estimates.","rationale":"The reader's weakest_assumption correctly identifies the contradictory evaluation protocol in Section IV.A as the load-bearing issue; I agree. The paper's strongest claim is a concrete empirical result, and the only evidence for it is the unreported experiment behind Table I and Fig. 7. If the evaluation protocol is ambiguous, the headline numbers cannot be interpreted, and the claimed SE-block improvement over the plain CNN cannot be separated from run-to-run noise. I also note that Table II compares AUC on different datasets, so the 'comparable to state-of-the-art' language is unsupported even if the StyleGAN/DFFD numbers are correct. This does not mean the method is wrong; a lightweight SE+CNN could plausibly reach such accuracy on StyleGAN images, and the reported 56.18 MB model size is consistent with a compact architecture. But no code, no seeds, no per-fold results, and no same-dataset baseline comparison are provided. The reader's CONDITIONAL verdict is the appropriate response: the authors should specify one protocol, report variance, and rerun the comparison fairly. My analysis does not move that verdict, so I recommend UNCHANGED.","tokens_in":8121,"tokens_out":4487,"duration_ms":53167,"concrete_test":"Obtain the exact training and evaluation code from the authors and rerun both CNN and CNN+SE under one explicit protocol: 10-fold cross-validation on the same 38,000 images, with the same optimizer, batch size, and 100 epochs, reporting per-fold accuracy and AUC for each model. Compute a paired comparison across folds (e.g., Wilcoxon signed-rank test). If the CNN+SE model is not significantly better than CNN, or if the per-fold spread makes the reported 94.14% accuracy and 0.985 AUC ambiguous, the central SE advantage and headline accuracy are not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends entirely on the reported empirical numbers, but Section IV.A says both \"we use train test split from sklearn to split the numpy arrays into 80% for training and 20% for testing\" and \"The model is trained for 100 epochs with 10-fold cross-validation.\" These are different evaluation protocols, and no per-fold or per-seed results are given. It is therefore unknown whether 94.14% and AUC 0.985 come from one held-out 20% split, from an average over cross-validation folds, or from some combination. The key within-paper comparison, CNN (91.13%) vs. CNN+SE (94.14%), is a 3.01-point gap on a binary task with 38,000 images; without fold-level or repeated-seed variation, this gap is within normal training stochasticity for a 100-epoch CNN and cannot be reliably attributed to the SE block. In addition, Table II's \"comparable performance\" claim compares AUC values from different datasets (DFDC vs. StyleGAN/DFFD), so it does not support the abstract's statement that the model is competitive with existing methods. The architecture is plausible and the reported numbers may be real, but as written the evidence does not pin down which experiment produced them.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a lightweight CNN augmented with Squeeze-and-Excitation (SE) blocks for deepfake image detection, targeting a subset of the Diverse Fake Face Dataset (StyleGAN-generated fake images paired with FFHQ real images). The architecture combines five convolutional layers with SE blocks inserted after batch normalization, and the authors report 94.14% classification accuracy and an AUC-ROC of 0.985 on a held-out test set, alongside a comparison with a plain CNN baseline that achieves 91.13%. The stated contributions are a compact SE-CNN model and a demonstration that SE attention improves detection accuracy with minimal computational overhead. The paper also compares its AUC against prior work on different datasets, claiming competitive performance at a reduced model size.","tokens_in":8401,"tokens_out":2083,"duration_ms":25439,"significance":"If the empirical claims are reliable, the result would be a modest but useful data point: a relatively lightweight CNN with SE blocks can match heavier architectures on a deepfake detection benchmark at substantially lower model size. The paper provides a straightforward ablation (CNN vs. CNN+SE) on a public dataset, and the reported absolute numbers are plausible for the StyleGAN/FFHQ subset of DFFD. However, the current evidence is insufficient to support the central claim because the evaluation protocol is contradictory and the cross-dataset comparison in Table II is not a valid basis for a competitiveness claim. The work would be strengthened by a clear, reproducible evaluation protocol with variance estimates and by restricting comparisons to same-dataset evaluations.","major_comments":[{"comment":"The evaluation protocol is internally contradictory: the text states both \"we use train test split from sklearn to split the numpy arrays into 80% for training and 20% for testing\" and \"The model is trained for 100 epochs with 10-fold cross-validation.\" These are distinct protocols, and the reported 91.13% vs. 94.14% accuracy and the 0.985 AUC cannot be interpreted without knowing which protocol produced them, or whether the numbers are from a single split, a fold average, or a best fold. The central empirical claim is therefore not reproducible as written. Please specify a single protocol, report per-fold and per-seed results, and provide mean and standard deviation for all reported metrics.","section":"Section IV.A, Dataset and Implementation"},{"comment":"Table II compares AUC values across different datasets (DFDC, 140k, StyleGAN/DFFD) and treats them as directly comparable. AUC is dataset-dependent, so the statement \"our CNN with SE block demonstrates superior performance compared to existing methods\" is not supported: the comparison conflates model quality with dataset difficulty and class balance. Likewise, the model-size comparison between the proposed model (56.18 MB) and WM (1722.85 MB) is only meaningful if both are measured on the same dataset and with the same counting methodology. Please either restrict comparisons to results on the same dataset or explicitly frame the table as indicative context rather than as evidence of superiority.","section":"Table II, Comparison of Performance"},{"comment":"The AUC reporting is inconsistent: the abstract and text report 0.985, while Section IV.B says \"achieved an AUC-ROC score of 98.5\" and Fig. 7 reports per-class AUCs of 0.99 (class 0) and 0.98 (class 1). If the per-class values are averaged, the result would be 0.985, but the text does not state this. More importantly, the AUC-ROC is a global scalar for a binary classifier, so per-class AUCs (which are equivalent to one-vs-rest AUCs) need to be tied to the decision threshold or probability outputs. Please clarify how the 0.985 AUC was computed and whether it corresponds to the model's probability output before thresholding.","section":"Section IV.B, Results and Discussion and Abstract"},{"comment":"The 3.01 percentage-point accuracy gap between CNN (91.13%) and CNN+SE (94.14%) is presented as the key benefit of the SE block, but no variance estimates are given. With 38,000 images and training for 100 epochs, the gap could be within training stochasticity (different initializations, data shuffling, and the unspecified split). Without repeated runs or significance testing, the improvement attributable to the SE block is not established. Please report results over multiple seeds (at least three) and state the standard deviation, and if 10-fold cross-validation was also run, report the fold-level accuracy distribution.","section":"Table I and Section IV.A"}],"minor_comments":[{"comment":"The text says the images are \"re-scaled and saved as numpy arrays\" and then split using sklearn; please state the exact preprocessing (e.g., pixel scaling to [0,1] or standardization) and whether any data augmentation was applied, since Fig. 3 mentions augmentation but no augmentation details are given in the implementation section.","section":"Section IV.A"},{"comment":"The sentence \"The 3 X 3 filter is used over 1 X 1 filter or 5 X 5 filter because of its efficiency in feature learning\" is unclear and seems to be a fragment; also \"He Normal\" should be \"He normal\" throughout, and \"prepocessed\" in Section III.C is a typo.","section":"Section III.A"},{"comment":"The last sentence of Section II.B ends with \"A summary of the dee\" which is an incomplete sentence; the section appears to be cut off mid-word.","section":"Section II.B"},{"comment":"The AUC-ROC curve should show a single curve for the binary classifier; the figure text reports separate AUCs per class, which is nonstandard for binary classification and should be clarified or corrected.","section":"Section IV.B, Fig. 7"},{"comment":"The table header \"AUC\" should specify that values are percentages (e.g., 90.9 vs. 98.5), and the proposed model's AUC should be reported consistently as 0.985 or 98.5, not both.","section":"Table II"},{"comment":"The paper compares the proposed model with only a handful of prior methods and omits several standard baselines on DFFD (e.g., the original DFFD paper reports on the same dataset); citing and comparing with same-dataset baselines would substantially strengthen the evaluation.","section":"Overall"}],"recommendation":"major_revision","confidential_remarks":"The paper is a workmanlike empirical study but currently lacks the methodological precision required for publication. The contradictory evaluation protocol and the invalid cross-dataset comparison in Table II are fixable with a revised experimental section and a more cautious comparison. I do not see evidence of fabrication, but the reported numbers need to be pinned down to a single reproducible pipeline. The scope is appropriate for a workshop or a short conference paper if the evaluation is cleaned up; for a journal, the contribution would need stronger benchmarking on multiple datasets and a more thorough comparison with same-dataset state-of-the-art."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a small empirical study that puts SE blocks into a simple CNN and tests it on the StyleGAN subset of DFFD. The headline numbers are plausible, but as written the evaluation protocol is internally contradictory, so I can't tell which split produced the 94.14% accuracy and 0.985 AUC. That alone makes the central claim unreproducible.\n\nWhat's good: The paper is honest about its building blocks. It cites Hu et al. for SE, and it explicitly mentions Roy et al.'s prior use of SE blocks for deepfake detection on FaceForensics++. The architecture is described in enough detail to reimplement: five conv layers with 32–512 filters, SE block after batch norm, max pooling, Adam, categorical cross-entropy. The baseline comparison to a plain CNN without SE is the right kind of ablation. The model-size comparison in Table II is at least a real number.\n\nThe soft spots are bigger. Section IV.A says both '80% for training and 20% for testing' and 'trained for 100 epochs with 10-fold cross-validation.' Those are different protocols. No per-fold or per-seed numbers are given, so the 3-point gap between CNN (91.13%) and CNN+SE (94.14%) is within normal stochasticity for a 100-epoch CNN on 38k images. I can't attribute it to the SE block. Table II compares AUC numbers across datasets—DFDC, 140k, and the StyleGAN subset of DFFD—as if they were interchangeable. They are not. Matching WM's 98.5 AUC on DFDC with your 98.5 on StyleGAN/DFFD is not 'comparable performance'; it's a cross-dataset coincidence. Also, no code or data is released, and the learning rate is reported as 0.001 in Section III and 0.0001 in Section IV.A—another inconsistency.\n\nThe novelty is low: this is a reimplementation of known attention blocks on a relatively easy benchmark. That's fine for a workshop paper, but not for a strong venue.\n\nMy recommendation: don't send this to referees in its current form. The evaluation must be clarified, rerun with a single consistent protocol, and compared against baselines on the same dataset. If the authors do that, it could become a conference-workshop-tier paper. Right now the claims are not reproducible.","headline":"A plausible but irreproducible small SE+CNN deepfake detector; the evaluation protocol is contradictory and the cross-dataset comparison is invalid.","tokens_in":8956,"tokens_out":3458,"would_cite":false,"duration_ms":37691,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small CNN with squeeze-and-excitation attention spots deepfakes at 94.14% accuracy, matching the AUC of a model 30 times its size.","keywords":["Deepfake detection","Squeeze-and-Excitation","attention mechanism","lightweight CNN","StyleGAN","Diverse Fake Face Dataset","channel-wise feature recalibration","face forgery detection"],"falsifier":"Train the exact CNN+SE architecture and the plain CNN on the same 38,000-image StyleGAN/FFHQ subset under a fixed 80/20 split with multiple seeds. If the SE model's mean accuracy does not exceed the plain CNN's by roughly three percentage points in non-overlapping confidence intervals, or if the 94.14% accuracy and 0.985 AUC do not reproduce, the central claim is refuted.","tokens_in":7928,"feed_emoji":"🎭","tokens_out":4291,"duration_ms":42006,"temperature":0.7,"pith_summary":"This paper claims that a deliberately small convolutional network, upgraded with squeeze-and-excitation (SE) attention blocks, can detect AI-generated fake faces almost as reliably as much larger deepfake detectors. On a subset of the Diverse Fake Face Dataset built from StyleGAN images and Flickr-Faces-HQ real faces, the model reports 94.14% accuracy and a 0.985 AUC-ROC score. The design point is efficiency: at 56.18 MB, the model is roughly 30 times smaller than a competing detector with the same AUC. The paper also reports that the SE version outperforms its own plain-CNN baseline by about three percentage points of accuracy, and the authors argue that channel-wise recalibration is what lets a small model focus on subtle generation artifacts. That combination of near-state-of-the-art accuracy in a deployable-size model is the reason a sympathetic reader would care.","feed_headline":"Lightweight CNN with SE blocks hits 94% deepfake accuracy","feed_subtitle":"A 56 MB model reaches 0.985 AUC on StyleGAN faces, matching a 1.7 GB competitor.","key_machinery":"The load-bearing component is the Squeeze-and-Excitation (SE) block, a channel-attention module borrowed from image classification. It computes a per-channel descriptor via global average pooling, then uses two dense layers to produce a set of channel weights, and rescales the input feature maps by those weights. The authors place the block after batch normalization in the four deepest convolution stages, so the network can emphasize informative channels and suppress irrelevant ones at each scale. The rest of the architecture is a conventional sequential CNN with 3x3 kernels, ReLU activation, He initialization, max pooling, and a final dense classifier.","core_discovery":"The central claim is that inserting SE blocks into a five-layer sequential CNN turns a modest deepfake detector into one that matches the AUC of a much heavier solution. The authors' model processes 224x224 face images through five convolutional layers with 32 to 512 filters, batch normalization, max pooling, and an SE block after normalization in four of the layers; the SE block squeezes each channel to a scalar by global average pooling, passes it through two dense layers (ReLU then sigmoid), and rescales the original feature map channel by channel. On 38,000 balanced StyleGAN/FFHQ images, this pipeline yields 94.14% accuracy and per-class AUC-ROC of 0.99 (fake) and 0.98 (real), versus 91.13% accuracy for the same CNN without SE. The authors interpret the gain as evidence that channel-wise recalibration captures manipulation-specific texture cues, and that a lightweight architecture can thereby achieve results comparable to existing cutting-edge methodologies.","pith_inferences":["The headline comparison is cross-dataset: the 0.985 AUC for the proposed model comes from StyleGAN/DFFD, while the WM baseline's 0.985 comes from the DFDC video dataset; matching numbers do not imply matching performance on a shared benchmark, and a head-to-head test on DFDC or FaceForensics++ would be needed to validate the efficiency claim.","The paper reports both an 80/20 split and 10-fold cross-validation without stating which produced the reported numbers; if the 94.14% figure is from a single split, the variance across folds could easily exceed the three-point gap over the baseline, so the superiority of the SE version is not yet established.","The authors' stated plan to test CBAM and scaled dot-product attention on more datasets suggests the same experimental setup could be used to identify which attention mechanism gives the best accuracy-per-parameter trade-off for deepfake detection."],"forward_implications":["A 56 MB model reaching 0.985 AUC on StyleGAN faces suggests that lightweight detectors can be competitive with much larger models, making on-device and real-time deepfake screening feasible.","The three-point accuracy gain over the plain CNN attributes a concrete, reproducible benefit to SE blocks in this detection task, not just in image classification.","If the result transfers across generators, channel-wise recalibration could become a standard cheap addition to forensic classifiers.","The reported model size reduction from 1722.85 MB to 56.18 MB at equal AUC implies that most parameters in large detectors are unnecessary for this data distribution."],"supporting_citations":[{"why":"Supplies the squeeze-and-excitation block method, including the squeeze, excitation, and scaling operations that the proposed architecture adapts.","marker":"[11]"},{"why":"Provides the Diverse Fake Face Dataset, from which the StyleGAN fake images used in the experiments are drawn.","marker":"[8]"},{"why":"Provides the Flickr-Faces-HQ real-face images that form the 'real' class in the balanced 38,000-image test set.","marker":"[18]"},{"why":"Supplies the Fake-buster baseline, a self-attention CNN whose 90.9 AUC on the DFDC dataset the paper claims to surpass.","marker":"[19]"},{"why":"Supplies CNN and SVM baselines on a 140k-image dataset that the proposed model is compared against in terms of AUC.","marker":"[20]"},{"why":"Supplies the WM baseline, the equal-AUC competitor whose 1722.85 MB model size is contrasted with the proposed model's 56.18 MB size.","marker":"[21]"}],"fun_headline_variants":["SE blocks shrink deepfake detector to 56 MB, holds 94%","94% deepfake catch with tiny CNN and SE attention","Channel attention makes lightweight CNN match big models","Small CNN with SE block hits 0.985 AUC on deepfakes","SE attention boosts CNN to 94% on StyleGAN fakes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The performance claim rests on the assumption that the reported evaluation is well-defined and reproducible; since the paper states both an 80/20 split and 10-fold cross-validation without saying which produced the numbers, and reports no variance, the 94.14% accuracy and the three-point gain over the baseline cannot be verified as stated.","fun_headline_variants_meta":{"raw":{"variants":["SE blocks shrink deepfake detector to 56 MB, holds 94%","94% deepfake catch with tiny CNN and SE attention","Channel attention makes lightweight CNN match big models","Small CNN with SE block hits 0.985 AUC on deepfakes","SE attention boosts CNN to 94% on StyleGAN fakes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000492,"raw_usage":{"total_tokens":2431,"prompt_tokens":973,"completion_tokens":1458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":1370}},"tokens_in":589,"tokens_out":1458,"duration_ms":11590,"temperature":1.0,"reasoning_tokens":1370,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:19:54.487831+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the exact CNN+SE architecture and the plain CNN on the same 38,000-image StyleGAN/FFHQ subset under a fixed 80/20 split with multiple seeds. If the SE model's mean accuracy does not exceed the plain CNN's by roughly three percentage points in non-overlapping confidence intervals, or if the 94.14% accuracy and 0.985 AUC do not reproduce, the central claim is refuted.","supporting_citations":[{"cited_title":"On the de- tection of digital face manipulation,","cited_arxiv_id":null,"evidence_quote":"Provides the Diverse Fake Face Dataset, from which the StyleGAN fake images used in the experiments are drawn."},{"cited_title":"A style-based generator architecture for generative adversarial networks,","cited_arxiv_id":null,"evidence_quote":"Provides the Flickr-Faces-HQ real-face images that form the 'real' class in the balanced 38,000-image test set."},{"cited_title":"Fake- buster: A lightweight solution for deepfake detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the Fake-buster baseline, a self-attention CNN whose 90.9 AUC on the DFDC dataset the paper claims to surpass."},{"cited_title":"Deepfake Detection Analyzing Hybrid Dataset Utilizing CNN and SVM","cited_arxiv_id":"2302.10280","evidence_quote":"Supplies CNN and SVM baselines on a 140k-image dataset that the proposed model is compared against in terms of AUC."},{"cited_title":"kaggle-dfdc: Deepfake Detection Challenge (DFDC) solu- tion,","cited_arxiv_id":null,"evidence_quote":"Supplies the WM baseline, the equal-AUC competitor whose 1722.85 MB model size is contrasted with the proposed model's 56.18 MB size."}],"review_version":1}