{"id":"7c9fc1ee-7ad0-40a5-81b1-9f072da75b97","arxiv_id":"2412.00073","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"CNN and DenseNet detectors were benchmarked on new Stable Diffusion 2.1/3.0, blurred, prompt-varied, and LoRA-modified CIFAKE datasets, revealing version-shift, blur, and LoRA vulnerabilities.","lead":"Detectors trained on one version of Stable Diffusion lose accuracy on newer versions, and simple blurring can drop fake-image detection to chance levels; DenseNet appears more robust. The paper also releases seven new benchmark datasets that stress-test AI-image detectors.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 9's CNN baseline copies the diagonal of Table 3, not the CIFAKE-SD2.1-trained row (81.89, 95.23, 84.30), so the DenseNet mitigation claim lacks a valid comparison until corrected.","rationale":"I agree with the reader's weakest assumption. The paper's empirical vulnerability results—cross-version drops, blur collapse, LoRA drop—are concrete and self-consistent enough to stand as observations, and the prompt-robustness finding is an interesting negative result. The problem is the proposed-solution section. Table 9's CNN row matching the diagonal of Table 3 is an internal contradiction, not a matter of taste or outside consensus. A single model trained on CIFAKE-SD2.1 cannot simultaneously have accuracies 93.67, 95.23, 96.84 per Table 9 and 81.89, 95.23, 84.30 per Table 3. The most plausible reading is a copy-paste error from the diagonal, but the paper as written does not provide a valid baseline for the DenseNet cross-version comparisons. The blur comparison in Table 10 is less affected, but the headline claim of consistent DenseNet superiority is unsupported until the table is fixed and repeated runs are reported. This does not require rejecting the paper; it requires revision and a re-run, so the verdict should remain conditional.","tokens_in":10888,"tokens_out":5395,"duration_ms":43701,"concrete_test":"Download the released datasets and code, then rerun the comparison with a single CIFAKE-SD2.1-trained CNN and the same CIFAKE-SD2.1-trained DenseNet on the same CIFAKE, CIFAKE-SD2.1, CIFAKE-SD3.0, and CIFAKE-SD2.1-Blurred test splits. Report mean and standard deviation over at least three seeds. Also re-tabulate Table 9 using the Table 3(a) SD2.1 CNN row (81.89, 95.23, 84.30) to see whether DenseNet superiority persists; if the corrected deltas are much smaller or reverse on CIFAKE or CIFAKE-SD3.0, the mitigation claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing evidence for the proposed solution is Table 9, which claims to compare DenseNet and CNN models both 'trained on CIFAKE-SD2.1' across CIFAKE, CIFAKE-SD2.1, and CIFAKE-SD3.0. The CNN row in Table 9 is 93.67, 95.23, 96.84. That is not the CIFAKE-SD2.1-trained CNN row from Table 3(a), which is 81.89, 95.23, 84.30. Instead it exactly copies the diagonal of Table 3(a): the values obtained by three different models, each trained and tested on the same dataset. As printed, the baseline is three different models, not one SD2.1-trained CNN. Therefore DenseNet's 97.23 and 98.78 on CIFAKE and CIFAKE-SD3.0 are compared against 93.67 and 96.84, inflating the apparent gains. The correct deltas against a single SD2.1-trained CNN would be 15.34 and 14.48 points, not 3.56 and 1.94, but those are also based on an invalid table and need verification. Table 10's blur comparison is consistent with Table 4 and is less damaged, but the headline 'DenseNet mitigates vulnerabilities' relies on both tables. Without a corrected, same-split baseline and variance estimates, the central proposed-solution claim is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates a CNN-based AI-image detector (Bird and Lotfi's CIFAKE classifier) on CIFAKE variants generated with Stable Diffusion 1.4, 2.1, and 3.0, as well as on versions subjected to Gaussian blurring, prompt changes, negative prompts, resolution changes, and a custom LoRA fine-tune. It reports that the CNN is robust to prompt and resolution variations but vulnerable to generator-version shifts, blur, and LoRA tuning, and it proposes a modified DenseNet121 as a more robust detector. The main empirical results include a version-overfitting matrix (Table 3), a stark blur-induced collapse of fake-image accuracy from 98.10% to 49.90% (Table 4), a LoRA-induced drop to 78.18% (Table 8), and a claimed DenseNet improvement (Tables 9-10).","tokens_in":11214,"tokens_out":4728,"duration_ms":44195,"significance":"If the findings hold, the paper is a useful measurement of detector fragility under realistic generator evolution and is notable for shipping public code and datasets. The version-overfitting matrix in Table 3 is internally coherent, the blur result in Table 4(b) is a striking and falsifiable observation, and the DenseNet proposal is a concrete, low-cost architectural mitigation. However, the significance is conditional because the central DenseNet comparison in Table 9 is internally inconsistent and because the absence of variance estimates and training details prevents the reader from distinguishing real robustness gains from run-to-run noise or unreported hyperparameter choices.","major_comments":[{"comment":"The CNN baseline row in Table 9 is not the CIFAKE-SD2.1-trained model stated in the header. The printed CNN values (93.67, 95.23, 96.84) are exactly the diagonal of Table 3(a), i.e., three different models each trained and tested on the same dataset. The actual CIFAKE-SD2.1-trained CNN row from Table 3(a) is (81.89, 95.23, 84.30). Consequently, the DenseNet-versus-CNN deltas reported for CIFAKE and CIFAKE-SD3.0 are comparing against the wrong baselines, and the claim in Section 5 that DenseNet 'outperformed the CNN model across all evaluation scenarios' is not supported by the table as printed. A corrected baseline, evaluated on identical splits and with the same protocol, is required before the DenseNet mitigation claim can be assessed.","section":"Table 9 (Section 4.2)"},{"comment":"No training details are given for the DenseNet121 models: the paper does not report epochs, learning rate, batch size, optimizer, data augmentation, or the exact architectural modifications beyond a generic Figure 2. Since the CNN hyperparameters are specified in Section 3, the absence of the corresponding DenseNet hyperparameters makes the headline comparison unreproducible and leaves open the possibility that the reported gains are due to different training schedules rather than the architecture.","section":"Section 4.2, DenseNet experiments (Tables 9-10)"},{"comment":"Every accuracy reported in the paper is a single run with no standard deviation, confidence interval, or number of seeds. Several conclusions, such as prompt invariance (Table 6: 95.23 vs. 95.36 vs. 95.26) and image-size insensitivity (Table 5: a 1.33-point drop), rest on differences that are smaller than typical run-to-run variation for small 32x32 classifiers. Repeated trials or error bars are needed to distinguish genuine robustness from stochasticity.","section":"Section 4.2, Tables 3-10"},{"comment":"The LoRA experiment is presented as evidence that fine-tuning degrades detection, but the LoRA configuration is underspecified: no rank, learning rate, number of steps, or validation of the photorealism effect are reported. The paper also asserts without justification that the MIT-Adobe FiveK LoRA is representative of adversarial photorealism fine-tuning. As printed, the result in Table 8 describes only this one custom configuration and cannot support the general claim that 'LoRA can effectively tune Stable Diffusion' to evade detectors.","section":"Section 4.2, CIFAKE-SD2.1-LoRA"}],"minor_comments":[{"comment":"The sentence 'Code and is available at ...' contains a grammar error; it should read 'Code is available at ...'.","section":"Section 4, first paragraph"},{"comment":"The text says 'Table 9 and 9 combine results for DenseNet and CNN models'; this should refer to Tables 9 and 10, and the typo should be fixed.","section":"Section 5, Using DenseNets"},{"comment":"The Gaussian blur sigma formula is malformed as printed. The standard OpenCV sigma formula is sigma = 0.3*((ksize-1)*0.5 - 1) + 0.8; please ensure the reported sigma value and the implementation match.","section":"Section 4, Equation (1)"},{"comment":"The claim that the original CIFAKE study 'did not provide details on key training parameters' is contradicted by the paper's later statement that missing details were filled with 'reasonable assumptions'; acknowledging this explicitly would clarify the scope of the replication claim.","section":"Section 2.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an independently produced, code-and-data-released empirical study. The main obstacle is not the experimental concept but the numerical integrity of Table 9: as printed, the CNN baseline contradicts the paper's own Table 3, and this table is load-bearing for the proposed DenseNet solution. A corrected baseline, DenseNet training details, and repeated trials would be needed before the central claim can be accepted. I would not reject outright, because the vulnerability measurements in Tables 3, 4, and 8 are coherent and potentially valuable, and the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Justin,\n\nYou should know two things about arXiv:2412.00073. First, it ships a genuinely useful set of new benchmark data: seven CIFAKE variants spanning Stable Diffusion 2.1 and 3.0, blurred, prompt-varied, negative-prompt, and LoRA-finetuned images, with code and data links. Second, its main proposed-solution claim—that DenseNet121 fixes the CNN's fragility—rests on a table that contradicts the paper's own earlier results, so that claim is unverified as printed.\n\nWhat is actually new: the version-overfitting numbers (Table 3) are internally coherent, the blur result is stark—fake-image accuracy falls from 98.10% to 49.90%—and the LoRA drop (to 78.18% fake-image accuracy) is a real observation I have not seen in the cited literature. The prompt-robustness result (Tables 6 and 7) showing invariance to prompt phrasing is also a useful datapoint. The blur robustness finding overlaps Park et al. [13], but the specific CIFAKE-SD2.1 measurements and the LoRA experiments extend it.\n\nThe soft spots are proportionate. The stress-test note is correct: Table 9's CNN baseline lists 93.67, 95.23, 96.84, which is exactly the diagonal of Table 3(a)—three different models, each trained and tested on the same dataset—not one CIFAKE-SD2.1-trained model. The correct row from Table 3(a) for an SD2.1-trained CNN is 81.89, 95.23, 84.30. So the DenseNet deltas in Table 9 are inflated; the apparent 3.56- and 1.94-point gains would actually be 15.34 and 14.48 points against the valid baseline, and those need verification on identical splits. Table 10's blur comparison is consistent with Table 4 and less damaged, but it is not enough to carry the mitigation claim alone. There are also no error bars anywhere—every accuracy is a single run—and the LoRA training details (rank, epochs, learning rate, validation) are absent, so the LoRA result is a single data point without context. Minor formatting issues like 'Code and is available' are cosmetic.\n\nWho this is for: someone building or evaluating AI-image detectors who wants a failure catalog and fresh benchmark data. It deserves a serious referee, not a desk reject, because the dataset release and measurements have standalone value, but the paper needs heavy revision: fix Table 9, add variance statistics, and report the LoRA hyperparameters.\n\nRecommendation: engage, send it out, and require those fixes before acceptance.\n\nBest,\n\n[You]","headline":"A useful new benchmark with stark robustness failures, but the DenseNet mitigation claim is compromised by a table that contradicts the paper's own earlier numbers.","tokens_in":11777,"tokens_out":2049,"would_cite":true,"duration_ms":17447,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CNN-based AI-image detectors are robust to prompt changes but break under Gaussian blur and generator upgrades; DenseNet121 restores much of the lost accuracy.","keywords":["AI-generated image detection","Stable Diffusion","CNN classifier","DenseNet","Gaussian blur robustness","LoRA fine-tuning","CIFAKE dataset","prompt variability"],"falsifier":"Retrain the CNN and DenseNet on the same CIFAKE-SD2.1 split, evaluate both on the blur and LoRA datasets, and check the exact fake-image accuracies (CNN 49.90% blurred and 78.18% LoRA; DenseNet 75.18% blurred). If DenseNet does not beat CNN on blurred data, or the CNN does not fall to chance under blur, the central robustness-mitigation claim fails.","tokens_in":26,"feed_emoji":"🖼️","tokens_out":6333,"duration_ms":142419,"temperature":0.7,"pith_summary":"This paper tries to establish that the widely used CNN-based CIFAKE detector is not robust to the ways real attackers actually generate images. It reports that a CNN trained on Stable Diffusion 2.1 images identifies fake images with 98.10% accuracy on clean data, but that accuracy collapses to 49.90% when those images are Gaussian blurred and to 78.18% when the generator is fine-tuned with LoRA; prompt rewording and resolution changes, by contrast, barely matter. The proposed fix is to switch to a DenseNet121 architecture, which claims 86.88% overall accuracy and 75.18% fake-image accuracy on blurred data. If true, detector benchmarks need to include generator-evolution and post-processing perturbations, and architecture choice alone buys substantial robustness.","feed_headline":"Gaussian blur cuts AI-image detector to 49.9% fake accuracy","feed_subtitle":"A CNN trained on Stable Diffusion 2.1 collapses on blurred images; DenseNet keeps fake-image detection at 75.2%.","key_machinery":"The central object is DenseNet121's dense connectivity, where each layer receives the feature maps of all preceding layers, combined with transition layers that compress accumulated features. This architecture gives the detector access to both low-level texture cues and high-level semantics, and it regularizes training enough that blur does not wipe out the cues. The paper contrasts this with the original two-convolutional-layer CNN baseline, which learns a shallower, more brittle fingerprint. The argument is carried by head-to-head accuracy tables on the same CIFAKE-derived datasets.","core_discovery":"The central claim is a vulnerability map: CNN-based detectors trained on one Stable Diffusion version memorize version-specific fingerprints. The paper supports this with cross-dataset tests: a model trained on CIFAKE-SD2.1 scores 95.23% overall on its own data but 81.89% on CIFAKE (SD1.4) and 84.30% on CIFAKE-SD3.0; fake-image accuracy drops from 98.10% to 71.42% and 76.24%. Gaussian blur with radius 5 and σ = 1.1 destroys the fingerprint, reducing fake-image detection to 49.90%, chance. LoRA fine-tuning on MIT-Adobe FiveK reduces fake accuracy to 78.18%. In contrast, prompt variants (including GPT-4o and negative prompts) and native-resolution images cost only a few points. DenseNet121, tested under the same protocol, reaches 86.88% overall and 75.18% fake accuracy on blurred data and generally beats the CNN on clean cross-version data, supporting the paper's proposed mitigation.","pith_inferences":["Going beyond the paper: if version-specific fingerprints are the failure mechanism, training on mixed-generator data or adding generator-version augmentation should recover much of the lost accuracy; this is directly testable.","Going beyond the paper: LoRA is cheap and accessible, so a realistic threat model includes attacker-controlled fine-tuning; detectors trained with blur and LoRA augmentations may be substantially harder to fool.","Going beyond the paper: the same logic applies beyond Stable Diffusion; any rapidly evolving generator family, including GANs, should be evaluated with version-shift and post-processing tests."],"forward_implications":["Detector evaluation should treat generator-version shift, Gaussian blur, and LoRA fine-tuning as standard perturbations; the reported drops show that clean accuracy alone overstates real-world reliability.","Switching to DenseNet-style architectures is a low-cost robustness intervention that does not require retraining on adversarial examples.","Prompt variation is not the main attack surface: fixed-template training transfers to varied and negative prompts.","Model-update monitoring matters: a detector that works today can degrade substantially when the generator is upgraded, so deployed detectors need continuous re-evaluation."],"supporting_citations":[{"why":"Defines the CNN classifier and CIFAKE dataset that this paper replicates as the baseline.","marker":"[5]"},{"why":"Provides the original CIFAKE image repository used for training and evaluation.","marker":"[1]"},{"why":"Supplies the DenseNet121 architecture that the paper adapts as the proposed robust detector.","marker":"[8]"},{"why":"Defines Low-Rank Adaptation, the fine-tuning method used to create the LoRA-challenged dataset.","marker":"[7]"},{"why":"Describes the latent diffusion model underlying Stable Diffusion, the generator family under test.","marker":"[15]"},{"why":"Provides the Stable Diffusion 2.1 model used to generate CIFAKE-SD2.1 and its variants.","marker":"[2]"},{"why":"Provides the Stable Diffusion 3.0 model used to generate CIFAKE-SD3.0.","marker":"[3]"},{"why":"Supplies the MIT-Adobe FiveK image set used to train the photorealism LoRA.","marker":"[6]"}],"fun_headline_variants":["Blur attack drops AI image detector to coin flip","AI detectors fail on blurred images: 49.9% fake accuracy","DenseNet beats CNN: survives blur that blinds AI detectors","Gaussian blur exposes AI image detection vulnerabilities"],"cache_read_input_tokens":13824,"weakest_assumption_plain":"The conclusion that DenseNet is consistently more robust than the CNN rests on the assumption that the CNN baseline numbers in Table 9 were produced by a CIFAKE-SD2.1-trained model on the same splits as DenseNet; if that table is wrong, and it conflicts with the earlier Table 3, the central mitigation claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["Blur attack drops AI image detector to coin flip","AI detectors fail on blurred images: 49.9% fake accuracy","DenseNet beats CNN: survives blur that blinds AI detectors","Gaussian blur exposes AI image detection vulnerabilities"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000288,"raw_usage":{"total_tokens":1665,"prompt_tokens":893,"completion_tokens":772,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":705}},"tokens_in":509,"tokens_out":772,"duration_ms":7975,"temperature":1.0,"reasoning_tokens":705,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:27:01.995789+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the CNN and DenseNet on the same CIFAKE-SD2.1 split, evaluate both on the blur and LoRA datasets, and check the exact fake-image accuracies (CNN 49.90% blurred and 78.18% LoRA; DenseNet 75.18% blurred). If DenseNet does not beat CNN on blurred data, or the CNN does not fall to chance under blur, the central robustness-mitigation claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the CNN classifier and CIFAKE dataset that this paper replicates as the baseline."},{"cited_title":"https://huggingface.co/datasets/ dragonintelligence/CIFAKE-image-dataset/ viewer","cited_arxiv_id":null,"evidence_quote":"Provides the original CIFAKE image repository used for training and evaluation."},{"cited_title":"Huang, Z","cited_arxiv_id":null,"evidence_quote":"Supplies the DenseNet121 architecture that the paper adapts as the proposed robust detector."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Low-Rank Adaptation, the fine-tuning method used to create the LoRA-challenged dataset."},{"cited_title":"Rombach, A","cited_arxiv_id":null,"evidence_quote":"Describes the latent diffusion model underlying Stable Diffusion, the generator family under test."},{"cited_title":"https://huggingface.co/ stabilityai/stable-diffusion-2-1","cited_arxiv_id":null,"evidence_quote":"Provides the Stable Diffusion 2.1 model used to generate CIFAKE-SD2.1 and its variants."},{"cited_title":"https://huggingface.co/ stabilityai/stable-diffusion-3-medium","cited_arxiv_id":null,"evidence_quote":"Provides the Stable Diffusion 3.0 model used to generate CIFAKE-SD3.0."},{"cited_title":"Bychkovsky, S","cited_arxiv_id":null,"evidence_quote":"Supplies the MIT-Adobe FiveK image set used to train the photorealism LoRA."}],"review_version":1}