{"id":"b09bf105-895f-4af2-9659-6944ca98eadc","arxiv_id":"2505.16253","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A Swin Transformer classifier distinguishes computer-generated images from natural photos best in RGB color space, with reported accuracies from 90 to 98 percent on three public datasets.","lead":"This paper applies a Swin Transformer to distinguish computer-generated images from real photographs across three datasets and three color spaces, reporting 90 to 98 percent accuracy. It claims robust domain generalization, but the reported cross-dataset test is actually training on a mix of all datasets and does not measure generalization to unseen sources.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'inter-dataset testing' claim is not actually tested: the combined D1+D2+D3 random split in §4.1 confounds sample identity with domain, and Table 6's Columbia metrics contradict the 97-99% headline.","rationale":"The reader's strongest objection is that the inter-dataset evaluation named in the abstract is absent: Section 4.1 constructs a combined D1+D2+D3 pool and Section 4.3 reports its accuracy, but there is no train-on-one/test-on-another protocol. This is precisely the load-bearing weakness. The random split of the combined pool conflates domain shift with sample identity, so the reported 97-98% combined accuracy cannot establish robustness to unseen data sources. The Columbia table's 0.95 accuracy and 0.80 F1 also contradict the '97-99% across multiple datasets' contribution, further undermining the headline claim. I agree with the reader's assessment; no new concern moves the verdict, so it remains REJECT. The concrete leave-one-dataset-out test would settle whether the domain-generalization claim has any empirical basis, but as reported the paper does not provide that evidence.","tokens_in":10061,"tokens_out":2106,"duration_ms":17092,"concrete_test":"Run a leave-one-dataset-out evaluation: train the Swin Transformer on RGB images from any two of D1, D2, D3 and test on the held-out third dataset, then report accuracy, precision, recall, F1, and AUC for each held-out dataset. As a control, also train a lightweight classifier to predict the source dataset from validation features of the combined-split model; if held-out accuracy drops by more than a few points relative to Table 7, or if source-prediction accuracy is high (e.g., >90%), the inter-dataset generalization claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—robust inter-dataset generalization—rests on an evaluation that does not measure cross-dataset transfer. Section 4.1 defines the combined dataset D1+D2+D3 as 1500 images per class drawn from CiFAKE, Columbia, and JSSSTU, and Section 4.3 reports high accuracy (0.98 RGB) after a random train/test split of that pooled set. A random split of a pooled dataset lets the model exploit dataset-specific cues (e.g., resolution, compression, generator artifacts) that are consistent between its training and test partitions, so high accuracy can reflect source identification rather than domain-general CGI detection. No experiment trains on one dataset and tests on a held-out dataset. The word 'inter-dataset' is used only to describe the mixed-training pool, not a domain-shift protocol. A second, compounding inconsistency: the paper's contribution states accuracy 'consistently between 97-99% across multiple datasets,' but the Columbia results in Table 6 show RGB accuracy of 0.95, F1 of 0.80, and recall of 0.67, while Table 7 (combined) reports 0.98. Either the Columbia experiment is not part of 'multiple datasets' in the claim, or the claim overstates the reported numbers. The absence of a true held-out-dataset test means the 'domain generalization' and 'inter-dataset testing' statements in the abstract and contributions are not supported by the experiments as described.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Swin Transformer-based binary classifier to distinguish computer-generated imagery (CGI) from natural images, evaluated on three datasets (CiFAKE, JSSSTU, Columbia) across RGB, YCbCr, and HSV color spaces. The authors report per-dataset results (Tables 4–6), a combined-dataset experiment (Table 7), t-SNE visualizations, and a comparison against VGG-19 and ResNet-50 on the combined RGB data (Table 8). The central claim is that the proposed method achieves state-of-the-art accuracy, 'consistently between 97-99% across multiple datasets,' and that inter-dataset testing demonstrates robustness and domain generalization.","tokens_in":10378,"tokens_out":2636,"duration_ms":20004,"significance":"If the central claims were supported, the work would provide a useful benchmark for CGI detection with a modern transformer backbone and a systematic comparison of color spaces. The paper's strengths include the use of three publicly available datasets, consideration of multiple color spaces, and a straightforward comparison with two CNN baselines. However, the claimed inter-dataset generalization is not actually measured by the reported experimental protocol, and the headline accuracy range is inconsistent with the authors' own Columbia results. As presented, the contribution is a limited empirical study whose main conclusions are not established; the significance depends on whether proper cross-dataset experiments can be added.","major_comments":[{"comment":"The claim of 'inter-dataset testing' and 'domain generalization' is not supported by the experimental design. The combined dataset D1+D2+D3 in Section 4.1 is built by pooling images from all three sources and then Section 4.3 reports results on a random train/test split of that pooled set. A random split of a pooled dataset allows the model to exploit source-specific cues (resolution, compression, sensor or generator artifacts) that are shared between training and test partitions, so high accuracy does not demonstrate generalization to an unseen dataset. To support the stated claim, the authors need a leave-one-dataset-out protocol (train on two datasets, test on the third) or an explicit train-on-one/test-on-other experiment. Without such experiments, the abstract's 'inter-dataset testing' and 'domain generalization' statements are unsupported.","section":"Section 4.1, 4.3; Abstract"},{"comment":"The contribution list states that the approach achieves 'state-of-the-art accuracy, consistently between 97-99% across multiple datasets.' This is contradicted by Table 6, where the Columbia dataset yields RGB accuracy of 0.95, F1-score of 0.80, and recall of 0.67, and the HSV space yields F1-score of 0.50. If 'multiple datasets' includes Columbia, the claim is false; if Columbia is excluded, the claim should be rephrased and justified. The discrepancy also appears in the conclusion, which acknowledges the low Columbia performance, making the headline claim internally inconsistent.","section":"Contributions list; Table 6"},{"comment":"All reported results appear to come from a single training run per configuration. There are no standard deviations, no repeated runs, and no statistical significance tests. This is especially problematic for the Columbia dataset, which contains only 43 CGI and 150 real images after curation (Section 4.1); the metrics in Table 6 vary dramatically across color spaces (recall 0.33–0.67), indicating that the results are likely unstable. The paper should report multiple runs (or cross-validation) with mean and variance, and should clearly describe how the small Columbia set was split for training and evaluation.","section":"Section 4.2, Tables 4–8"},{"comment":"The 'RGB is best' conclusion is presented as a finding, but RGB was selected after observing that it scored highest, and the paper then uses RGB for the comparative analysis. This is a post-hoc selection without a held-out validation of the color-space choice. The same data are used to select the color space and to report the headline accuracy, which is circular for the comparison across color spaces. The authors should either define a pre-registered hypothesis, use a validation set for color-space selection, or clearly label the per-color-space comparison as exploratory rather than confirmatory.","section":"Section 4.2, Section 4.3, Table 8"}],"minor_comments":[{"comment":"The paragraph introducing JSSSTU results says 'The performance analysis of the CiFAKE dataset...' rather than referring to JSSSTU; this appears to be a copy-paste error.","section":"Section 4.3, JSSSTU subsection"},{"comment":"Table 1 reports CiFAKE as having 1,20,000 images, while Table 2 reports '1,000,000+'; these numbers should be reconciled.","section":"Table 1, Table 2"},{"comment":"The name of the Columbia dataset is spelled inconsistently ('columbia RCGI', 'Columbia PRCG', 'Columbia RCGI'); the canonical name should be used consistently.","section":"Throughout"},{"comment":"The paper does not describe how the combined dataset (D1+D2+D3) was exactly composed, e.g., how many images were taken from each dataset, whether the 1500-per-class count includes the augmented Columbia images, and what augmentation techniques were applied. This information is necessary for reproducibility.","section":"Section 4.2"},{"comment":"The 'state-of-the-art' claim in the contributions is not supported by a comparison with recent CGI detection methods; Table 8 compares only to VGG-19 and ResNet-50 on the combined RGB dataset, and no comparison is made on the individual datasets or against more recent transformer- or diffusion-aware detectors.","section":"Table 8 and Section 2"},{"comment":"The t-SNE visualizations are described qualitatively as showing RGB 'most distinct separation,' but no quantitative measure of cluster separability is provided; consider reporting a silhouette score or similar metric.","section":"Figures 3–6"}],"recommendation":"major_revision","confidential_remarks":"The core issue is that the paper's headline claim is about inter-dataset generalization, but the experimental protocol does not measure cross-dataset transfer. This is a fixable concern—adding leave-one-dataset-out experiments and revising the claims would substantially improve the manuscript. I also note the absence of code, data splits, or trained model release, which limits reproducibility; the authors should be encouraged to share these resources. The paper may be more appropriate for a workshop or short paper venue if the evaluation depth is not expanded."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a straightforward Swin Transformer application to CGI detection, and the most useful part is the systematic color-space comparison on three public datasets. But the paper's headline claims do not survive contact with its own experiments. The 'inter-dataset' and 'domain generalization' language describes a random split of a pooled dataset, not a transfer test, and the 'consistently between 97-99%' claim is contradicted by Table 6.\n\nWhat the paper does well: the authors are transparent about the Columbia dataset's broken URLs and the resulting small usable set (43 CGI, 150 real images). They report per-color-space metrics, apply augmentation to address imbalance, and compare against VGG-19 and ResNet-50 on the combined data. The finding that RGB outperforms YCbCr and HSV is plausible and is consistent with the t-SNE visualizations, though it is a post-hoc observation rather than a tested prediction.\n\nThe soft spots are load-bearing, not cosmetic. First, there is no held-out-dataset evaluation anywhere in the paper. Section 4.1 builds D1+D2+D3 as 1500 images per class and then does a random train/test split. That setup lets the model exploit dataset-specific cues (resolution, compression, generator artifacts) that are consistent between training and test partitions, so high accuracy can reflect source identification rather than domain-general CGI detection. No experiment trains on one dataset and tests on another. The abstract's 'inter-dataset testing' and 'domain generalization' claims are therefore unsupported by the reported methodology. Second, the contribution list says accuracy is 'consistently between 97-99% across multiple datasets,' but Table 6 shows Columbia RGB at 0.95 accuracy, 0.67 recall, and 0.80 F1; HSV drops to 0.33 recall. That is not a minor discrepancy. Third, the 'state-of-the-art' claim is unsupported: the comparison includes only VGG-19 and ResNet-50, and the proposed method matches ResNet-50 at 0.98 rather than beating it. Minor issues: single runs with no error bars, no code or data released, and modest training settings (20 epochs), which further weakens the robustness claims.\n\nWho is this for? A reader wanting a quick Swin baseline and a color-space sweep on CiFAKE, JSSSTU, and Columbia might find the tables useful as a data point. But as evidence for domain generalization, the paper should not be cited. The central evaluation gap and the internal contradiction with the headline accuracy make this, in current form, not worth referee time. I would desk-reject with an invitation to resubmit after adding real cross-dataset splits, correcting the accuracy claims, and releasing code and data.","headline":"Claims inter-dataset generalization but never actually tests it; the 97-99% headline also clashes with the paper's own Columbia numbers.","tokens_in":10862,"tokens_out":3309,"would_cite":false,"duration_ms":26951,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a Swin Transformer trained on raw RGB pixels separates computer-generated imagery from authentic photos with 95–98% accuracy across three datasets, and that this makes it a strong generic CGI detector.","keywords":["Swin Transformer","computer-generated image detection","digital image forensics","color space analysis","domain generalization","deep learning","image classification","CGI"],"falsifier":"Train the same Swin configuration on CiFAKE and JSSSTU only, then test on the un-augmented Columbia CGI and real images; if accuracy falls toward chance or well below the reported 95–98%, the inter-dataset generalization claim fails.","tokens_in":9877,"feed_emoji":"🖼️","tokens_out":8698,"duration_ms":64811,"temperature":0.7,"pith_summary":"This paper sets out to show that a Swin Transformer—a hierarchical vision transformer that processes images in shifted local windows—can reliably tell computer-generated imagery (CGI) from authentic photographs. The authors test the model on three datasets: CiFAKE, JSSSTU, and Columbia, across RGB, YCbCr, and HSV color encodings, and report that RGB gives the best accuracy: 98% on CiFAKE, 97% on JSSSTU, 95% on Columbia, and 98% on a pooled combination of all three. They position this as evidence of domain generalization and as a state-of-the-art result, since the Swin model matches ResNet-50 and beats VGG-19 on the pooled RGB test. If correct, the result matters because a single vision transformer could serve as a practical forensic tool for flagging synthetic images without handcrafted features.","feed_headline":"Swin Transformer flags CGI across 3 datasets at up to 98%","feed_subtitle":"RGB frames alone let the vision transformer match ResNet-50 and beat VGG-19, the paper reports.","key_machinery":"The load-bearing object is the Swin Transformer, a vision transformer whose self-attention operates on non-overlapping local windows that shift between consecutive layers, producing a hierarchical feature map that captures both fine texture and long-range structure. The authors feed it 224×224 images in each of three color spaces (RGB, YCbCr, HSV) and train a binary classifier with cross-entropy loss on the resulting features. Two supporting mechanisms carry the generalization argument: data augmentation to rebalance the severely depleted Columbia subset (43 CGI, 150 real images), and pooling of all three datasets into a balanced 1500-images-per-class set on which the headline accuracy is measured.","core_discovery":"The central claim, stated sympathetically, is that the inductive biases of the Swin Transformer—self-attention computed within shifted local windows, giving it both local texture sensitivity and global context—make it particularly suited to the CGI-versus-natural task. The paper reports that RGB frames alone separate the two classes in t-SNE projections more cleanly than YCbCr or HSV, and that the network achieves 0.98 accuracy/precision/recall/F1/AUC on the pooled dataset, matching ResNet-50 and exceeding VGG-19. The discovery is an extension result: an off-the-shelf vision transformer, trained on simple color frames, transfers across three different CGI sources at high accuracy, suggesting that synthetic-image detection may not need bespoke noise or frequency features.","pith_inferences":["The paper calls its pooled train/test split 'inter-dataset testing,' but a random split of the combined pool does not measure transfer to an unseen source; a true domain-shift test would train on, say, CiFAKE and JSSSTU and test on Columbia. That experiment is the real test of the generalization claim.","The Columbia results rest on only 43 positive and 150 negative images after data loss, so the 95% accuracy number should be read as testimony about a small augmented sample, not about the full 800-per-class Columbia dataset.","The claim that RGB is universally the most discriminative color space is likely source-dependent; a testable extension would run the same architecture on rendered images with unusual chromatic distributions (e.g., stylized game engines or path-traced animation) to see whether HSV or YCbCr becomes competitive.","If the Swin features are truly generic, the same model should also separate newer generative outputs (GANs, diffusion models) from real photos; the paper does not test that, but the CIFAKE dataset, which it uses, does contain AI-generated images, so the boundary between CGI and other synthetic imagery is already fuzzy."],"forward_implications":["If the claim holds, a forensic examiner could deploy a single Swin Transformer, pretrained on ordinary classification tasks and fine-tuned on RGB frames, as a first-pass CGI detector across heterogeneous image sources.","The reported RGB superiority implies that color-space engineering is unnecessary for this task; the raw pixel distribution in RGB already contains the discriminative signal, at least for these three sources.","The pooled-dataset result suggests that mixing CGI sources during training improves generalization, which would argue for building larger, multi-renderer training corpora rather than tuning per-source detectors.","Because the Swin model matches ResNet-50, the result indicates that attention-based architectures are at least as effective as CNNs for CGI detection, opening a straightforward architectural path for future forensics models.","The data-augmentation recovery of the tiny Columbia subset suggests that even severely imbalanced forensic collections can yield usable classifiers, though the confidence intervals on those numbers are not reported."],"supporting_citations":[{"why":"It supplies the CiFAKE dataset, the largest of the three test beds, containing real and AI-generated images used for D1 and part of the pooled set.","marker":"[1]"},{"why":"It supplies the Columbia PRCG dataset, the most challenging source; only a small usable subset remains after link rot, and it anchors D3.","marker":"[2]"},{"why":"It supplies the JSSSTU dataset of Blender/Maya-rendered CGI versus photographic images, used as D2.","marker":"[3]"},{"why":"It reports the prior 97.32% accuracy on CiFAKE with a lightweight CNN, the benchmark the paper implicitly claims to reach or exceed with 98%.","marker":"[27]"}],"fun_headline_variants":["Swin Transformer nails CGI detection across three datasets","Swin Transformer hits 98% on CGI with RGB alone","Swin Transformer outdoes VGG-19, matches ResNet-50 on CGI","Swin Transformer's window attention excels at spotting CGI","Swin Transformer shows robust CGI detection across color spaces"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The generalization claim rests on the assumption that a random train/test split of the pooled three-dataset collection predicts performance on an unseen image source, rather than just measuring the model's ability to fit a mixture of known sources.","fun_headline_variants_meta":{"raw":{"variants":["Swin Transformer nails CGI detection across three datasets","Swin Transformer hits 98% on CGI with RGB alone","Swin Transformer outdoes VGG-19, matches ResNet-50 on CGI","Swin Transformer's window attention excels at spotting CGI","Swin Transformer shows robust CGI detection across color spaces"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000487,"raw_usage":{"total_tokens":2438,"prompt_tokens":1020,"completion_tokens":1418,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":1334}},"tokens_in":636,"tokens_out":1418,"duration_ms":8981,"temperature":1.0,"reasoning_tokens":1334,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:04:33.274865+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same Swin configuration on CiFAKE and JSSSTU only, then test on the un-augmented Columbia CGI and real images; if accuracy falls toward chance or well below the reported 95–98%, the inter-dataset generalization claim fails.","supporting_citations":[{"cited_title":"Cifake: Image classification and explainable identification of AI-generated synthetic images,","cited_arxiv_id":null,"evidence_quote":"It supplies the CiFAKE dataset, the largest of the three test beds, containing real and AI-generated images used for D1 and part of the pooled set."},{"cited_title":"Columbia photographic images and photorealistic computer graphics dataset","cited_arxiv_id":null,"evidence_quote":"It supplies the Columbia PRCG dataset, the most challenging source; only a small usable subset remains after link rot, and it anchors D3."},{"cited_title":"Dataset for classification of computer graphic images and photographic images,","cited_arxiv_id":null,"evidence_quote":"It supplies the JSSSTU dataset of Blender/Maya-rendered CGI versus photographic images, used as D2."},{"cited_title":"Detection of AI-Generated Synthetic Images with a Lightweight CNN","cited_arxiv_id":null,"evidence_quote":"It reports the prior 97.32% accuracy on CiFAKE with a lightweight CNN, the benchmark the paper implicitly claims to reach or exceed with 98%."}],"review_version":1}