{"id":"6a720e7f-ffd9-4c58-aeb9-8c59deb8d01a","arxiv_id":"2504.13726","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MLEP uses multi-scale local entropy computed on shuffled 2x2 patches as input to a CNN, achieving 97.1 percent mean detection accuracy across 32 generative models.","lead":"This paper proposes MLEP, a feature extraction method that turns images into multi-scale local entropy maps after shuffling small patches, then trains a CNN to classify real photos versus AI-generated images. It reports 97.1 percent mean accuracy across 32 generative models, but the evaluation has caveats such as test-set hyperparameter selection and no released code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 97.1% mean accuracy is not an unbiased generalization estimate: all MLEP hyperparameters are selected by ablations on the same 32 test datasets (Tables 5-8), with no held-out validation split, so the margin over NPR may be inflated.","rationale":"The reader's weakest assumption is the same as the most load-bearing concern: hyperparameters are tuned on the same datasets used for evaluation, so the claimed generalization margin is optimistically biased. I considered stronger alternatives: the method may partly detect compression artifacts from social-platform images, and patch-shuffling randomness is unreported. However, the compression confound is speculative and the paper explicitly lists compression robustness as a limitation; the shuffle-seed issue affects reproducibility but is likely smaller than selection bias. The test-set tuning is directly documented in Tables 5-8 and affects the central quantitative claim, including the comparison with untuned baselines. The conditional verdict is appropriate: the method is plausible and the concern is testable, but until a held-out split or nested validation is reported, the 97.1% mean should not be taken at face value.","tokens_in":13944,"tokens_out":13190,"duration_ms":131323,"concrete_test":"Hold out 16 of the 32 generators (e.g., 8 GAN and 8 diffusion, including Midjourney and DALL-E 2) before any model selection. Use the remaining 16 only for the Table 5-8 ablations to pick l, S, interpolation, stride, and backbone; train once on ProGAN/LSUN with the chosen configuration; report mean Acc/AP on the held-out 16. Repeat with a complementary split that swaps validation and test. If the held-out mean drops by more than about 1.5 points or falls below NPR's 94.0 or FatFormer's 91.9, the headline number is not a reliable estimate of universal generalization. Additionally, rerun the chosen configuration with 5 different shuffle seeds and report mean plus/minus standard deviation to rule out seed luck.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 describes no held-out validation split. Tables 5-8 choose patch size l=2, scales S={1,1/2,1/4}, bilinear interpolation, stride 1, and ResNet-50 by maximizing mean Acc/AP over the same 16 GAN and 16 diffusion datasets that later produce the headline numbers in Tables 1-3. Using the test distribution for model selection introduces selection bias: the reported 97.1% is the maximum over a small grid, not the expected performance of the method with fixed hyperparameters. The central claim that MLEP is a source-invariant representation that outperforms NPR by 3.1 Acc points depends on the assumption that this tuning did not materially inflate the result. Since NPR and FatFormer were not equivalently tuned on these 32 datasets, the comparison may also be unfair. The absence of error bars and the unreported shuffle seed compound this: a single run with random patch shuffling can be lucky. The paper's own tables show per-dataset cases where MLEP trails FatFormer by more than 10 points (e.g., BigGAN 87.1 vs 99.5, GauGAN 82.0 vs 99.4), so the mean advantage is not uniformly robust. The conclusion that MLEP outperforms state-of-the-art by over 3% is therefore not yet established for unseen generators.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Multi-granularity Local Entropy Patterns (MLEP), a handcrafted feature representation for AI-generated image detection. The method computes Shannon entropy over 2x2 sliding windows on randomly shuffled small patches, repeats this at multiple resampling scales, and feeds the resulting multi-channel entropy maps to a ResNet classifier trained only on ProGAN images. The authors evaluate on 32 datasets from 16 GAN and 16 diffusion models and report a mean accuracy of 97.1%, surpassing the NPR baseline by 3.1 accuracy points. They also provide ablations on patch size, scale factors, interpolation method, stride, and backbone, plus qualitative visualizations of entropy maps and t-SNE features to support their semantic-suppression argument.","tokens_in":14202,"tokens_out":5532,"duration_ms":46981,"significance":"If the reported results are unbiased, MLEP is an interesting and practically simple contribution: it requires no pretrained generative model or vision-language backbone, it uses only a standard CNN on a derived entropy representation, and its cross-generator generalization (trained on ProGAN, tested on 32 generators) is strong. The paper also gives a clear qualitative story for why patch shuffling and local entropy should suppress semantic content. These strengths are real and make the method worth pursuing. However, the central quantitative claim — a 3.1-point mean accuracy advantage over NPR and a 5.2-point advantage over FatFormer — is currently supported by an evaluation protocol that selects hyperparameters on the same test datasets that produce the headline numbers, and it reports no error bars or significance tests. The claim of 'universal' detection is also weakened by per-dataset results where the method is far behind the baselines. The significance of the paper as written therefore rests on an evaluation that needs to be made statistically sound before the conclusions can be trusted.","major_comments":[{"comment":"The hyperparameters of MLEP — patch size l=2, scales S={1,1/2,1/4}, bilinear interpolation, stride 1, and the ResNet-50 backbone — are selected by maximizing mean accuracy/AP on the same 16 GAN and 16 diffusion test datasets that are later used for the headline results in Tables 1–3. The text does not describe any held-out validation split. This is selection on the test distribution and can inflate the reported 97.1% mean accuracy. It also makes the comparison to NPR and FatFormer unfair, since those baselines were not tuned on these 32 datasets and several were taken directly from the literature with their published settings. Please introduce a proper validation split (e.g., held-out categories of ForenSynths, or a separate set of generative models for validation), or report results for all hyperparameter combinations and discuss sensitivity explicitly.","section":"§4.1 and §4.2.2 (Tables 5–8)"},{"comment":"All reported numbers come from a single run, and the seed for the random patch shuffling is not reported. Because patch shuffling introduces randomness, the mean accuracy and AP could vary from run to run; the claimed advantage over NPR (97.1 vs 94.0) could be within run-to-run noise. Please report means and standard deviations over at least five runs for the key comparisons, and include a paired significance test (e.g., a paired t-test or Wilcoxon test over the 32 datasets) against the strongest baselines NPR and FatFormer.","section":"§4.2.1 (Tables 1–3)"},{"comment":"The paper concludes that MLEP 'outperforms state-of-the-art approaches by over 3% in accuracy' and calls the method 'universal', but this is based only on the mean over all 32 datasets. On the GAN subset, MLEP achieves 96.4 Acc versus FatFormer's 99.0 (Table 3), and on several individual GAN datasets the gap is much larger: BigGAN 87.1 vs 99.5 for FatFormer, GauGAN 82.0 vs 99.4, and S3GAN 83.4 vs 99.0 (Table 1). These are not isolated drops; they are on 3 of the 16 GAN datasets and contradict the impression of uniform superiority. The authors should report per-domain results, discuss the GAN-specific weakness, and temper the conclusion accordingly. A mean-only reporting is not sufficient to substantiate the 'universal' claim.","section":"§4.2.1, Tables 1 and 3"}],"minor_comments":[{"comment":"\"multiple image scaled\" should be \"multiple image scales\".","section":"Abstract"},{"comment":"The set V={0, 0.8, 1.0, 1.5, 2} suggests that entropy values are rounded to one decimal; please define the rounding rule explicitly and state how ties in the four pixel values are handled in the probability estimate.","section":"§3.3"},{"comment":"The text says \"The proof and an efficient computation algorithm for LEP on a 2×2 window are provided in the supplementary material.\" If the supplementary material is not part of this submission, the proof should be included in the main text or omitted; otherwise the statement is not verifiable.","section":"§3.3"},{"comment":"The caption lists percentages (e.g., 45.6%, 43.9%) but does not explain what they represent; please clarify whether they are the fraction of pixels with entropy 2.0, or some other quantity.","section":"Figure 1"},{"comment":"Reference [1] has a typo: \"Cramer Cistance\" should be \"Cramer Distance\", and reference [27] has \"Fiffusion\" which should be \"Diffusion\".","section":"References"},{"comment":"The t-SNE visualizations (Fig. 6) are qualitative; consider adding a quantitative separation metric (e.g., silhouette score or nearest-neighbor accuracy) so the claim of cleaner separation is testable.","section":"§4.2.3"}],"recommendation":"major_revision","confidential_remarks":"The main barrier is evaluation soundness. The central claim rests on hyperparameters selected on the test data itself, and the absence of error bars makes the reported margin over NPR and FatFormer fragile. I would urge the editor to request the authors to re-run with a proper validation split, report multi-run statistics, and provide code and seeds for reproducibility. The novelty is moderate but the idea is clear and the qualitative analysis is useful; with the evaluation fixed, the paper could be a solid contribution to the AIGI detection literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this paper reports a simple, cheap feature representation—local entropy over finely shuffled small patches at multiple scales—that beats NPR by about 3% mean accuracy across 32 generators. The idea is credible and the experiments are broad, but the headline number is not an unbiased estimate. All hyperparameters (patch size l=2, scales S={1,1/2,1/4}, stride 1, bilinear interpolation) were chosen by ablations on those same 32 test datasets (Tables 5–8), with no held-out split. That means 97.1% is the max over a small grid, not the expected performance of a fixed configuration. Single runs, an unreported shuffle seed, and no code make it hard to know how much luck is in the mean.\n\nWhat is genuinely new: using Shannon entropy instead of pixel differences as the local relationship feature, and pushing patch shuffling down to 2×2. The multi-scale resampling is a sensible extension of NPR's upsampling observation. The qualitative story—that entropy suppresses semantics while preserving generator artifacts—is well illustrated and plausible. Credit where due: the paper is clearly written, the LEP computation is well defined, and the evaluation covers a large and useful set of generators, including commercial models.\n\nThe soft spots are mostly evaluation-side but not fatal. Per-dataset results contradict the \"consistently top performance\" claim: MLEP trails FatFormer badly on BigGAN (87.1 vs 99.5), GauGAN (82.0 vs 99.4), and S3GAN (83.4 vs 99.0), and falls below NPR on Midjourney and DALL·E 2. The mean advantage is real, but the method is not uniformly better. Also missing: how the 9-channel MLEP input is fed to ResNet-50, the exact shuffle seed usage, and any repeated-run variance. These are fixable, not fundamental.\n\nThe central argument—that entropy on finely shuffled patches gives a source-invariant cue—holds up better than the headline numbers suggest. If the authors add a proper validation split, report repeated trials, and release code, the result would be worth taking seriously. As it stands, the paper deserves a serious referee but needs revision before the 3% margin over NPR is treated as established.\n\nBottom line: send it to review with instructions to demand held-out hyperparameter selection, error bars, and code. The idea is good enough to engage with seriously.","headline":"A plausible, cheap entropy-based feature for AIGI detection with a real but possibly inflated mean gain: the headline 97.1% is selected from ablations on the same test datasets, with no held-out validation or error bars.","tokens_in":14772,"tokens_out":1465,"would_cite":false,"duration_ms":16272,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Local entropy patterns spot AI images across 32 generators","keywords":["AI-generated image detection","local entropy","patch shuffling","multi-scale resampling","Shannon entropy","deepfake detection","cross-domain generalization","ProGAN"],"falsifier":"Fix hyperparameters ($l=2$, $S=\\{1,1/2,1/4\\}$, stride 1, bilinear interpolation) using a held-out validation set of a few generators, then test on a generator released after training that is not among the 32, or apply mild JPEG compression and Gaussian blur to the existing test sets. If mean accuracy drops to baseline levels, the claim of a universal entropy fingerprint fails.","tokens_in":13703,"feed_emoji":"🧩","tokens_out":6814,"duration_ms":58206,"temperature":0.7,"pith_summary":"Generative image models leave subtle statistical traces even when pixel-level artifacts are invisible. This paper proposes that the right trace is local information entropy: how randomly arranged pixel values are inside tiny 2x2 windows, measured after the image has been scrambled into small shuffled patches and resampled at several scales. It claims this Multi-granularity Local Entropy Pattern (MLEP) is content-agnostic, so a detector trained on images from one GAN, ProGAN, can recognize images from 31 other generators, including diffusion models. On its 32-dataset benchmark the detector reaches 97.1% mean accuracy and 98.9% average precision, outperforming prior pixel-difference and pretrained-model baselines. If the claim holds, entropy maps give a cheap, source-invariant cue for deepfake and misinformation defenses.","feed_headline":"Shuffled-patch entropy catches AI images from 32 generators","feed_subtitle":"Trained only on ProGAN, the detector reaches 97.1% mean accuracy across GAN and diffusion models.","key_machinery":"The load-bearing object is the MLEP feature map. Construction: split an image into $2\\times2$ patches, shuffle them randomly (per color channel), downsample and upsample the scrambled image at scales $S=\\{1,1/2,1/4\\}$ with bilinear interpolation, then slide a $2\\times2$ window with stride 1 over every resampled version and record the Shannon entropy of the four pixels inside each window. Because only four values enter the entropy formula, the LEP output takes values in $\\{0,0.8,1.0,1.5,2.0\\}$, and the overlapping windows capture intra-patch, inter-patch, and inter-scale randomness at once. Patch shuffling destroys the semantic layout, multi-scale resampling exposes upsampling artifacts common to generators, and the entropy statistic replaces pixel contrasts with a measure of local randomness. The resulting multi-channel maps are concatenated and fed to a standard ResNet-50 trained with binary cross-entropy.","core_discovery":"The paper's central discovery is that local entropy distributions over shuffled micro-patches separate real photographs from AI-generated images in a way that transfers across generator families. A real image consistently has a higher probability of maximum entropy (2.0) in 2x2 windows than a generated one, which the authors attribute to the glossy, over-smoothed texture generators produce. By computing entropy on randomly permuted $2\\times2$ patches across the resampling scales $S=\\{1,1/2,1/4\\}$, MLEP suppresses scene semantics while preserving and amplifying generator-specific randomness. Trained only on ProGAN images from ForenSynths and real LSUN images, a ResNet-50 classifier on MLEP maps achieves 97.1% mean accuracy over 32 generation models, with gains of 3.1% in accuracy and 1.6% in average precision over the nearest strong baseline NPR. The authors frame this as the first use of image entropy as a detection cue for AI-generated images.","pith_inferences":["One testable extension: evaluate MLEP under JPEG compression, blur, and noise; the paper's own limitation section says these are unstudied, and a drop there would bound its real-world use.","Because the feature is a five-value entropy pattern, a fast implementation could classify images in real time on edge devices; the paper does not measure latency or throughput.","If over-smoothed texture is the mechanism, MLEP should also catch partially generated edits such as face swaps and inpainting, a benchmark the paper does not include.","The claim of content agnosticism could be stress-tested by training on LSUN scenes and testing on face or document image datasets, where texture statistics differ."],"forward_implications":["A detector trained only on ProGAN images transfers to 16 GAN and 16 diffusion models, reaching 97.1% mean accuracy across all 32.","Patch size $l=2$ beats $l=4$ and $l=8$, supporting the claim that finer semantic scrambling improves content-agnostic detection.","Entropy alone, without shuffling or multi-scale resampling, already exceeds the NPR baseline, indicating the entropy statistic itself carries the signal.","Multi-scale fusion with scales $\\{1,1/2,1/4\\}$ is optimal; adding $1/8$ hurts, so moderate resampling introduces useful artifacts without excessive distortion.","Performance is stable across ResNet-18, ResNet-34, ResNet-50, and ResNet-101, so the representation rather than a particular classifier depth drives the result."],"supporting_citations":[{"why":"Supplies the NPR baseline, the upsampling-artifact hypothesis, and the cross-dataset evaluation protocol that MLEP is compared against.","marker":"[38]"},{"why":"Introduces patch shuffling for semantic disruption, which MLEP refines by using much smaller patches.","marker":"[45]"},{"why":"Defines the Shannon entropy used to compute local entropy patterns.","marker":"[35]"},{"why":"Provides the ForenSynths dataset of ProGAN-generated images used for training and the CNNDet baseline.","marker":"[41]"},{"why":"Provides the DiffusionForensics datasets for the first eight diffusion models in the evaluation.","marker":"[42]"},{"why":"Provides the UniversalFakeDetect datasets for the remaining diffusion models and serves as a CLIP-based baseline.","marker":"[30]"},{"why":"Defines ProGAN, the only generator used to create training images.","marker":"[14]"},{"why":"Supplies the LSUN real images used as the real class during training.","marker":"[44]"},{"why":"Provides the ResNet backbone that consumes the MLEP feature maps.","marker":"[10]"}],"fun_headline_variants":["Entropy patterns in shuffled patches unmask AI images from 32 generators","97.1% accuracy spotting AI images via shuffled-patch entropy","Trained on ProGAN alone, entropy patterns catch all 32 generator types","Shuffled-patch entropy: the universal fingerprint of AI-generated images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire reported advantage rests on the assumption that the hyperparameters were not effectively tuned to the 32 test datasets; the paper selects patch size, scales, stride, and interpolation through ablations on those same datasets without describing a held-out validation split for model selection.","fun_headline_variants_meta":{"raw":{"variants":["Entropy patterns in shuffled patches unmask AI images from 32 generators","97.1% accuracy spotting AI images via shuffled-patch entropy","Trained on ProGAN alone, entropy patterns catch all 32 generator types","Shuffled-patch entropy: the universal fingerprint of AI-generated images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000554,"raw_usage":{"total_tokens":2637,"prompt_tokens":938,"completion_tokens":1699,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":1620}},"tokens_in":554,"tokens_out":1699,"duration_ms":11322,"temperature":1.0,"reasoning_tokens":1620,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:01:47.276132+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fix hyperparameters ($l=2$, $S=\\{1,1/2,1/4\\}$, stride 1, bilinear interpolation) using a held-out validation set of a few generators, then test on a generator released after training that is not among the 32, or apply mild JPEG compression and Gaussian blur to the existing test sets. If mean accuracy drops to baseline levels, the claim of a universal entropy fingerprint fails.","supporting_citations":[{"cited_title":"Re- thinking the Up-Sampling Operations in CNN-Based Generative Network for Generalizable Deepfake Detection","cited_arxiv_id":null,"evidence_quote":"Supplies the NPR baseline, the upsampling-artifact hypothesis, and the cross-dataset evaluation protocol that MLEP is compared against."},{"cited_title":"Breaking Semantic Artifacts for Generalized AI-generated Image Detection","cited_arxiv_id":null,"evidence_quote":"Introduces patch shuffling for semantic disruption, which MLEP refines by using much smaller patches."},{"cited_title":"DIRE for Diffusion-Generated Image Detection","cited_arxiv_id":null,"evidence_quote":"Provides the DiffusionForensics datasets for the first eight diffusion models in the evaluation."},{"cited_title":"Towards Universal Fake Image Detectors that Generalize Across Generative Models","cited_arxiv_id":null,"evidence_quote":"Provides the UniversalFakeDetect datasets for the remaining diffusion models and serves as a CLIP-based baseline."},{"cited_title":"Progressive Growing of Gans for Improved Quality, Stability, and Variation","cited_arxiv_id":null,"evidence_quote":"Defines ProGAN, the only generator used to create training images."},{"cited_title":"Deep Residual Learning for Image Recognition","cited_arxiv_id":null,"evidence_quote":"Provides the ResNet backbone that consumes the MLEP feature maps."}],"review_version":1}