{"id":"3c80f0bb-ca8e-4fd3-835a-10993561641d","arxiv_id":"2501.11140","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"CLOFAI is a new domain-incremental benchmark where a classifier must detect fake images from five generative model types in sequence; replay methods beat the naive baseline, while EWC does not.","lead":"This paper introduces CLOFAI, a dataset that tests whether image classifiers can keep spotting AI-generated fakes while learning about new image generators one at a time. It runs three standard continual learning methods on the dataset and reports that replay-based methods retain old knowledge better than a regularization method.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-seed evaluation leaves the claimed superiority of replay methods over Naive statistically untested; observed differences are small enough to be plausible noise.","rationale":"We reviewed the paper and the reader's verdict in detail. The reader's weakest assumption—that all experiments use a single fixed seed and thus the reported accuracy differences may not be reliable—is the most load-bearing concern, because the central claim (Abstract) is the comparative statement that GEM and Experience Replay 'perform significantly better than a Naive baseline.' The effect sizes are modest: the final average accuracy after all tasks is about 61.0% for Naive, 62.2% for Experience Replay-100, 67.1% for Experience Replay-500, 65.0% for GEM-100, and 67.7% for GEM-500. Without repeated runs or statistical tests, a 1–2 point gap (Naive vs ER-100) is indistinguishable from seed noise. We also note that the Baseline accuracy matrix contains unexpectedly low diagonal entries (e.g., 54.40% on task 2 after joint training on tasks 1–2), which hints that the training budget is tight and results may be seed-sensitive. We do not see an internal inconsistency in the experimental logic; the issue is the absence of uncertainty quantification. Therefore, our read does not change the reader's conditional verdict. We would request multi-seed experiments and statistical testing before accepting the paper's headline claim.","tokens_in":9990,"tokens_out":12597,"duration_ms":104922,"concrete_test":"Rerun all benchmark methods (Naive, EWC, Experience Replay, GEM) with at least 10 different seeds while holding all other hyperparameters fixed; report mean and standard deviation for task-wise and average accuracies, and compute paired significance tests (e.g., Wilcoxon signed-rank) comparing each method against Naive on the final average accuracy across tasks. If the differences are not significant at the 0.05 level, the claim that replay methods significantly outperform Naive is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that GEM and Experience Replay 'perform significantly better than a Naive baseline' (Abstract) rests on a single random seed (Section 5: 'All stochastic parameters had seed set to 123'). With one seed there is no estimate of training noise. The effect sizes are modest: after all five tasks the average accuracy across tasks is roughly 61.0% for Naive, 62.2% for Experience Replay (100 samples), 67.1% for Experience Replay (500 samples), 65.0% for GEM (100 samples), and 67.7% for GEM (500 samples). A 1–2 point difference (e.g., Naive vs ER-100) is within the typical run-to-run variance for a network trained for only 3 epochs on 8,000 images per task, and the paper provides no confidence intervals or paired tests. Notably, the Baseline (joint training) itself shows an anomalously low diagonal entry (54.40% on task 2 after training on tasks 1 and 2, versus 79.85% for Naive on the same task), suggesting that the training budget is tight and results are highly sensitive to initialization and seed. Therefore the reported ordering of methods may not replicate.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces CLOFAI, a domain-incremental continual learning dataset for real versus fake image classification. The benchmark consists of five tasks in which a binary classifier distinguishes real CIFAR-10 images from images produced by five generators (VAE, VAEBM, GAN, Flow, Diffusion), with a different real-image set per task and an 80/20 train/test split. Using an ImageNet-pretrained EfficientNet-b0, the authors report accuracy matrices for a joint-training baseline, Naive sequential fine-tuning, EWC, Experience Replay (100 and 500 replayed samples), and GEM (100 and 500 replayed samples). The paper's central claim is that EWC performs poorly on this benchmark, while GEM and Experience Replay perform significantly better than a Naive baseline.","tokens_in":10225,"tokens_out":7674,"duration_ms":66733,"significance":"If the reported comparisons are robust, CLOFAI fills a clear gap: it provides a reusable domain-incremental benchmark that mimics the realistic scenario of updating a fake-image detector as new generative models appear, under storage or privacy constraints. The paper contributes a concrete dataset, a clean problem formulation, and accuracy matrices for five common continual learning methods, and it makes code and data publicly available. The benchmark itself and the reproducible baseline tables are the main strengths. The main limitations are that the headline ranking rests on a single random seed with no statistical validation, and that the task-difficulty ordering was determined using the same classifier architecture that is later benchmarked.","major_comments":[{"comment":"The claim that GEM and Experience Replay are 'significantly better' than Naive is not supported by the experimental protocol. Section 5 states 'All stochastic parameters had seed set to 123', so there are no repeated runs, confidence intervals, or significance tests. Averaging the final rows of Tables 4, 8, 9, 10, and 11 gives roughly 61.0% for Naive, 62.2% for ER-100, 67.1% for ER-500, 65.0% for GEM-100, and 67.7% for GEM-500; the 1.2-point difference between Naive and ER-100 is small relative to the run-to-run variance expected from a 3-epoch, 8,000-sample training budget. The Baseline matrix in Table 2 also shows an odd diagonal value (54.40% on task 2 after training on tasks 1 and 2, versus 79.85% for Naive at the same point), which suggests high sensitivity to initialization and training schedule. Please repeat the experiments over multiple seeds and report means, confidence intervals, or paired statistical tests before using the word 'significantly'.","section":"Section 5"},{"comment":"The task order is selected post hoc using the same classifier that is later evaluated: Section 3 says the authors tested the Classifier's performance on each task in isolation and ordered tasks from easiest (Task 1) to hardest (Task 5) according to those accuracies. Because the 'increasing difficulty' progression is defined by EfficientNet-b0's per-task accuracy, it is not an independent property of the generator sequence, and the central narrative of a real-world progression of generator realism is partly constructed by the benchmark itself. Please either derive the ordering from several architectures or an independent perceptual measure and show stability, or clearly disclose and discuss this circularity when presenting the task order as a simulation of real-world circumstances.","section":"Section 3, Table 1"},{"comment":"The conclusion that 'irrespective of the value of lambda, EWC cannot achieve good performance' is based on only two lambda values (100,000 and 100,000,000), and no lambda sweep is reported. The EWC implementation is also underspecified: the text does not state when the Fisher Information is computed, whether an online EWC variant is used, or how the quadratic penalty is normalized across layers. Without this information and a sweep over lambda, the reported EWC failure could be a tuning or implementation artifact rather than a property of the method on CLOFAI. Please report a lambda sweep and the full implementation details, or restrict the conclusion to the two tested values.","section":"Section 5.3"}],"minor_comments":[{"comment":"The section heading 'CLOF AI benchmark' appears to contain a typo and should read 'CLOFAI benchmark'.","section":"Section 3 heading"},{"comment":"The phrase 'relative difficultly' should be 'relative difficulty'.","section":"Section 5.2"},{"comment":"The description of Experience Replay does not state how the 100 or 500 replayed samples are selected from each past task (e.g., random sampling, mean-of-feature, or some other strategy); this choice should be documented because the comparison between buffer sizes depends on it.","section":"Section 4"},{"comment":"GEM is the only method taken from an external library (Avalanche) while the other methods are implemented from scratch; please clarify whether GEM shares exactly the same data pipeline, optimizer, epoch count, and seed handling as the other methods, since otherwise the method comparisons could be confounded by implementation differences.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The dataset contribution is timely and the benchmark setup is clear. The decision rests on whether the authors can turn the headline claim into a statistically supported one and address the circularity in the task ordering. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CLOFAI is a genuinely useful new benchmark artifact, and the paper is honest about much of its construction, but the headline result — that GEM and Experience Replay 'perform significantly better' than Naive — rests on a single fixed seed and no statistical test. That wording should be softened until there are repeated runs.\n\nWhat's actually new: the authors assemble five well-known generative model families (VAE, VAEBM, GAN, Flow, DDIM) into a domain-incremental real/fake classification benchmark with 5,000 real and 5,000 fake images per task, and they report full accuracy matrices for Naive, EWC, GEM, and Experience Replay. The dataset fills a real niche: most prior work looks at single-generator detection or generic continual learning benchmarks, while CLOFAI directly targets the scenario where new generators appear over time. The task ordering was chosen by measuring per-task difficulty with the same EfficientNet architecture used later — a mild selection effect, but it is disclosed and the resulting order is plausible.\n\nThe real soft spot is statistical. Section 5 states 'All stochastic parameters had seed set to 123.' With one seed, the 1–2 point gaps between Naive and Experience Replay (100 samples) are well within typical training noise for 3-epoch runs on 8,000 images. The paper's own Baseline matrix has an odd 54.40% diagonal on task 2, suggesting high sensitivity to initialization. So the qualitative ordering — replay helps somewhat, EWC does not — may replicate, but the word 'significantly' is not earned. The EWC section also overreaches: only two lambda values are tested, so the claim that no lambda works is not established. Minor points: no confidence intervals, no repeated runs, no dataset checksums or commit hash, though the code is public.\n\nOverall: the dataset is new and likely to be used; the baseline results are suggestive but not conclusive. This paper deserves serious peer review — it is a benchmark contribution with a public artifact — but the revision needs multi-seed runs with means and error bars, a proper EWC sweep, and language that matches the evidence.","headline":"A useful new continual-learning benchmark for fake-image detection, but the method comparisons are statistically underpowered and should be treated as preliminary.","tokens_in":10732,"tokens_out":2829,"would_cite":true,"duration_ms":25995,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces CLOFAI, a domain-incremental benchmark for real versus fake image classification, and claims that replay-based continual learning methods (GEM and Experience Replay) markedly outperform a naive baseline while the…","keywords":["continual learning","fake image detection","domain-incremental learning","catastrophic forgetting","experience replay","gradient episodic memory","generative models","EfficientNet"],"falsifier":"Re-run the full benchmark with ten random seeds per method and compare 95 percent confidence intervals on each task-accuracy cell; if the Naive and replay intervals overlap substantially, the claim that GEM and Experience Replay perform significantly better is refuted.","tokens_in":9764,"feed_emoji":"🖼️","tokens_out":5805,"duration_ms":52762,"temperature":0.7,"pith_summary":"This paper introduces CLOFAI, a benchmark that casts the problem of telling real from artificially generated images as domain-incremental continual learning. The dataset has five tasks, each pairing 5,000 real CIFAR-10 images with 5,000 fakes produced by a different generative model, ordered from easiest to hardest. The paper's central claim is that replay-based continual learning methods, specifically GEM and Experience Replay, markedly outperform a naive fine-tuning baseline at retaining accuracy on earlier tasks, while the regularization method EWC does not. If this holds, fake-image detectors could be updated on new generators without retraining on all past data.","feed_headline":"Replay methods beat forgetting in fake-image detectors","feed_subtitle":"CLOFAI orders five generative models by difficulty; GEM and Experience Replay stay well above a naive baseline.","key_machinery":"The central object is the CLOFAI task sequence: five binary real-versus-fake classification tasks whose input distribution changes with the generating model (VAE, VAEBM, GAN, flow, and DDIM diffusion) while the labels remain the same. Task order is set by a classifier's accuracy on each task in isolation, from easiest to hardest, so the sequence simulates the emergence of progressively more realistic generators. The benchmark's evaluation is a per-task accuracy matrix after each training stage, which makes forgetting visible as accuracy loss on earlier tasks.","core_discovery":"The paper's central claim is that CLOFAI works as a domain-incremental benchmark and that on it replay-based methods are the viable route: GEM and Experience Replay keep accuracy on earlier tasks far above the Naive fine-tuning baseline, whereas EWC performs roughly as poorly as Naive. The paper supports this with accuracy matrices showing replay methods holding earlier-task accuracy substantially above the Naive baseline across the full sequence, while EWC's trajectory tracks Naive closely. The proposed explanation for EWC's failure is that the same parameters are highly important across all tasks, so Fisher-information regularization either allows destructive updates or, at high strength, prevents learning the new task.","pith_inferences":["A held-out newer generator (for example, a transformer-based image model) could serve as a fresh task to test whether replay methods retain old knowledge while absorbing a genuinely new style; based on the dataset's difficulty ordering, performance should drop furthest on such a task.","Because all experiments use a single seed, the quantitative gaps between methods may be partly due to training noise; re-running with several seeds would tell whether Experience Replay's advantage over the Naive baseline is consistent.","The CLOFAI task layout -- one fixed label pair with shifting input distributions -- transfers directly to other deepfake domains such as audio or video, where new synthesis methods also arrive over time.","The accuracy matrices show near-chance performance on tasks the model has not yet seen, implying the model learns task-specific cues rather than a general real/fake invariant; if that is right, continual learning here is mainly about storing task-specific features."],"forward_implications":["A fake-image detector can be kept current on newly released generators by replaying a small buffer of earlier examples, instead of retraining on the full history of data.","Because labels stay the same across tasks, the trained model never needs to know which generative model produced an image at inference time.","GEM's larger gain from 100 to 500 replayed samples, compared with Experience Replay's small gain, suggests that enforcing constraints on past-task loss is what buys retention, not simply adding data.","Regularization-only protection, as implemented by EWC, fails on this benchmark even at extreme regularization strengths, signalling that weight-importance methods are not a safe default for this domain."],"supporting_citations":[{"why":"Supplies the EWC regularization method whose performance is compared and found poor.","marker":"[3]"},{"why":"Provides the GEM algorithm and the replay idea behind Experience Replay.","marker":"[6]"},{"why":"CIFAR-10 is the source of the real images in every task.","marker":"[15]"},{"why":"Defines the domain-incremental setting that CLOFAI instantiates.","marker":"[20]"},{"why":"Provides the VAE that generates fake images for Task 1.","marker":"[22]"},{"why":"Provides the VAEBM model that generates fake images for Task 2.","marker":"[23]"},{"why":"Provides the GAN that generates fake images for Task 3.","marker":"[24]"},{"why":"Provides the flow-based model that generates fake images for Task 4.","marker":"[25]"},{"why":"Provides the DDIM diffusion model that generates fake images for Task 5.","marker":"[26]"},{"why":"Provides the EfficientNet_b0 architecture and pretrained weights used as the classifier.","marker":"[27]"}],"fun_headline_variants":["Replay wins against fake-image forgetting","GEM and Replay outdo Naive on fake-image tasks","CLOFAI: replay methods keep fake-image accuracy","For fake images, replay beats EWC and naive","Replay methods top continual fake-image benchmark"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported method ranking assumes that one training run with a fixed random seed represents each method's typical behaviour; under heavy training noise, the differences between replay methods and the Naive baseline could disappear.","fun_headline_variants_meta":{"raw":{"variants":["Replay wins against fake-image forgetting","GEM and Replay outdo Naive on fake-image tasks","CLOFAI: replay methods keep fake-image accuracy","For fake images, replay beats EWC and naive","Replay methods top continual fake-image benchmark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000386,"raw_usage":{"total_tokens":2040,"prompt_tokens":946,"completion_tokens":1094,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":1019}},"tokens_in":562,"tokens_out":1094,"duration_ms":8184,"temperature":1.0,"reasoning_tokens":1019,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:35:42.556415+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full benchmark with ten random seeds per method and compare 95 percent confidence intervals on each task-accuracy cell; if the Naive and replay intervals overlap substantially, the claim that GEM and Experience Replay perform significantly better is refuted.","supporting_citations":[{"cited_title":"Overcomingcatastrophicforgettinginneuralnet- works","cited_arxiv_id":null,"evidence_quote":"Supplies the EWC regularization method whose performance is compared and found poor."},{"cited_title":"Gradient Episodic Memory for Continual Learning","cited_arxiv_id":null,"evidence_quote":"Provides the GEM algorithm and the replay idea behind Experience Replay."},{"cited_title":"Learning multiple layers of fea- tures from tiny images","cited_arxiv_id":null,"evidence_quote":"CIFAR-10 is the source of the real images in every task."},{"cited_title":"Three types of incremental learning","cited_arxiv_id":null,"evidence_quote":"Defines the domain-incremental setting that CLOFAI instantiates."},{"cited_title":"VAEBM: A Symbiosis between Variational Autoen- coders and Energy-based Models","cited_arxiv_id":null,"evidence_quote":"Provides the VAEBM model that generates fake images for Task 2."},{"cited_title":"Densely connected nor- malizing flows","cited_arxiv_id":null,"evidence_quote":"Provides the flow-based model that generates fake images for Task 4."},{"cited_title":"Diffusers: State-of-the-art diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the DDIM diffusion model that generates fake images for Task 5."},{"cited_title":"EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks","cited_arxiv_id":null,"evidence_quote":"Provides the EfficientNet_b0 architecture and pretrained weights used as the classifier."}],"review_version":1}