REVIEW 3 major objections 4 minor 32 references
CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper introduces CLOFAI, a domain-incremental benchmark for real versus fake image classification, and claims that replay-based continual learning methods (GEM and Experience Replay) markedly outperform a naive baseline while the…
desk verdict A useful new continual-learning benchmark for fake-image detection, but the method comparisons are statistically underpowered and should be treated as preliminary. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the CLOFAI task sequence: five binary real-versus-fake classification tasks whose input distribution changes with the generating model (VAE, VAEBM, GAN, flow, and DDIM diffusion) while the labels remain the same. Task order is set by a classifier's accuracy on each task in isolation, from easiest to hardest, so the sequence simulates the emergence of progressively more realistic generators. The benchmark's evaluation is a per-task accuracy matrix after each training stage, which makes forgetting visible as accuracy loss on earlier tasks.
What would settle it
Re-run the full benchmark with ten random seeds per method and compare 95 percent confidence intervals on each task-accuracy cell; if the Naive and replay intervals overlap substantially, the claim that GEM and Experience Replay perform significantly better is refuted.
Extended reading notes
Core claim
The paper's central claim is that CLOFAI works as a domain-incremental benchmark and that on it replay-based methods are the viable route: GEM and Experience Replay keep accuracy on earlier tasks far above the Naive fine-tuning baseline, whereas EWC performs roughly as poorly as Naive. The paper supports this with accuracy matrices showing replay methods holding earlier-task accuracy substantially above the Naive baseline across the full sequence, while EWC's trajectory tracks Naive closely. The proposed explanation for EWC's failure is that the same parameters are highly important across all tasks, so Fisher-information regularization either allows destructive updates or, at high strength, prevents learning the new task.
Load-bearing premise
The reported method ranking assumes that one training run with a fixed random seed represents each method's typical behaviour; under heavy training noise, the differences between replay methods and the Naive baseline could disappear.
Editorial extensions
If this is right
- A fake-image detector can be kept current on newly released generators by replaying a small buffer of earlier examples, instead of retraining on the full history of data.
- Because labels stay the same across tasks, the trained model never needs to know which generative model produced an image at inference time.
- GEM's larger gain from 100 to 500 replayed samples, compared with Experience Replay's small gain, suggests that enforcing constraints on past-task loss is what buys retention, not simply adding data.
- Regularization-only protection, as implemented by EWC, fails on this benchmark even at extreme regularization strengths, signalling that weight-importance methods are not a safe default for this domain.
Reading between the lines
- A held-out newer generator (for example, a transformer-based image model) could serve as a fresh task to test whether replay methods retain old knowledge while absorbing a genuinely new style; based on the dataset's difficulty ordering, performance should drop furthest on such a task.
- Because all experiments use a single seed, the quantitative gaps between methods may be partly due to training noise; re-running with several seeds would tell whether Experience Replay's advantage over the Naive baseline is consistent.
- The CLOFAI task layout -- one fixed label pair with shifting input distributions -- transfers directly to other deepfake domains such as audio or video, where new synthesis methods also arrive over time.
- The accuracy matrices show near-chance performance on tasks the model has not yet seen, implying the model learns task-specific cues rather than a general real/fake invariant; if that is right, continual learning here is mainly about storing task-specific features.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces CLOFAI, a domain-incremental continual learning dataset for real versus fake image classification. The benchmark consists of five tasks in which a binary classifier distinguishes real CIFAR-10 images from images produced by five generators (VAE, VAEBM, GAN, Flow, Diffusion), with a different real-image set per task and an 80/20 train/test split. Using an ImageNet-pretrained EfficientNet-b0, the authors report accuracy matrices for a joint-training baseline, Naive sequential fine-tuning, EWC, Experience Replay (100 and 500 replayed samples), and GEM (100 and 500 replayed samples). The paper's central claim is that EWC performs poorly on this benchmark, while GEM and Experience Replay perform significantly better than a Naive baseline.
Significance. If the reported comparisons are robust, CLOFAI fills a clear gap: it provides a reusable domain-incremental benchmark that mimics the realistic scenario of updating a fake-image detector as new generative models appear, under storage or privacy constraints. The paper contributes a concrete dataset, a clean problem formulation, and accuracy matrices for five common continual learning methods, and it makes code and data publicly available. The benchmark itself and the reproducible baseline tables are the main strengths. The main limitations are that the headline ranking rests on a single random seed with no statistical validation, and that the task-difficulty ordering was determined using the same classifier architecture that is later benchmarked.
major comments (3)
- [Section 5] The claim that GEM and Experience Replay are 'significantly better' than Naive is not supported by the experimental protocol. Section 5 states 'All stochastic parameters had seed set to 123', so there are no repeated runs, confidence intervals, or significance tests. Averaging the final rows of Tables 4, 8, 9, 10, and 11 gives roughly 61.0% for Naive, 62.2% for ER-100, 67.1% for ER-500, 65.0% for GEM-100, and 67.7% for GEM-500; the 1.2-point difference between Naive and ER-100 is small relative to the run-to-run variance expected from a 3-epoch, 8,000-sample training budget. The Baseline matrix in Table 2 also shows an odd diagonal value (54.40% on task 2 after training on tasks 1 and 2, versus 79.85% for Naive at the same point), which suggests high sensitivity to initialization and training schedule. Please repeat the experiments over multiple seeds and report means, confidence intervals, or paired statistical tests before using the word 'significantly'.
- [Section 3, Table 1] The task order is selected post hoc using the same classifier that is later evaluated: Section 3 says the authors tested the Classifier's performance on each task in isolation and ordered tasks from easiest (Task 1) to hardest (Task 5) according to those accuracies. Because the 'increasing difficulty' progression is defined by EfficientNet-b0's per-task accuracy, it is not an independent property of the generator sequence, and the central narrative of a real-world progression of generator realism is partly constructed by the benchmark itself. Please either derive the ordering from several architectures or an independent perceptual measure and show stability, or clearly disclose and discuss this circularity when presenting the task order as a simulation of real-world circumstances.
- [Section 5.3] The conclusion that 'irrespective of the value of lambda, EWC cannot achieve good performance' is based on only two lambda values (100,000 and 100,000,000), and no lambda sweep is reported. The EWC implementation is also underspecified: the text does not state when the Fisher Information is computed, whether an online EWC variant is used, or how the quadratic penalty is normalized across layers. Without this information and a sweep over lambda, the reported EWC failure could be a tuning or implementation artifact rather than a property of the method on CLOFAI. Please report a lambda sweep and the full implementation details, or restrict the conclusion to the two tested values.
minor comments (4)
- [Section 3 heading] The section heading 'CLOF AI benchmark' appears to contain a typo and should read 'CLOFAI benchmark'.
- [Section 5.2] The phrase 'relative difficultly' should be 'relative difficulty'.
- [Section 4] The description of Experience Replay does not state how the 100 or 500 replayed samples are selected from each past task (e.g., random sampling, mean-of-feature, or some other strategy); this choice should be documented because the comparison between buffer sizes depends on it.
- [Section 4] GEM is the only method taken from an external library (Avalanche) while the other methods are implemented from scratch; please clarify whether GEM shares exactly the same data pipeline, optimizer, epoch count, and seed handling as the other methods, since otherwise the method comparisons could be confounded by implementation differences.
Circularity Check
No significant circularity: the continual learning method comparison is an independent empirical benchmark; the task-difficulty ordering is an openly stated construction, not a derived prediction.
full rationale
The paper's central claims are (i) CLOFAI is a domain-incremental dataset for real/fake image classification and (ii) GEM and Experience Replay outperform EWC and a Naive baseline on it. The dataset construction is described concretely with five generative models, CIFAR-10 real images, and an 80/20 split. The task order was chosen by measuring the same EfficientNet_b0 classifier's isolated accuracy on each task and ordering tasks from easiest to hardest (Section 3, Table 1). This is an explicitly stated construction rule, not a hidden assumption or a prediction masquerading as a result; it defines the benchmark's difficulty ordering rather than deriving an independent fact from it. The method comparisons in Section 5 use implementations of Baseline, Naive, EWC, Experience Replay, and GEM evaluated on the same fixed task sequence. No parameter is fitted to the benchmark outcome and then renamed as a prediction; no result reduces by construction to the dataset definition; there are no self-citations or imported uniqueness theorems. The single-seed evaluation (Section 5) and the absence of confidence intervals are legitimate statistical-robustness concerns, but they are not circularity. Accordingly no circular step is identified.
Assumptions & free parameters
free parameters (7)
- Task order (VAE, VAEBM, GAN, Flow, Diffusion) =
ordered by decreasing isolated classifier accuracy (Table 1)
- EWC regularization lambda =
100,000 and 100,000,000
- Replay buffer size =
100 and 500 samples per task
- Training epochs per task =
3
- Batch size =
128
- Learning rate =
0.0001
- Random seed =
123
assumptions (5)
- domain assumption Generated images from the five model families are representative of fake images a detector must handle.
- ad hoc to paper The fixed task order simulates the real-world progression of generative model realism.
- domain assumption CIFAR-10 is a sufficient source of real images for evaluating fake image detectors.
- domain assumption A single random seed with deterministic training is sufficient to rank continual learning methods.
- domain assumption ImageNet pretraining transfers to the CIFAR-10 real/fake task.
Cite this review
Pith. "Pith review of CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning." pith.science (2026). https://pith.science/paper/UJEN37M5
@misc{pith2026250111140,
author = {Pith},
title = {Pith review of: CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/UJEN37M5}},
note = {Machine review of arXiv:2501.11140}
}
read the original abstract
The rapid advancement of generative AI models capable of creating realistic media has led to a need for classifiers that can accurately distinguish between genuine and artificially-generated images. A significant challenge for these classifiers emerges when they encounter images from generative models that are not represented in their training data, usually resulting in diminished performance. A typical approach is to periodically update the classifier's training data with images from the new generative models then retrain the classifier on the updated dataset. However, in some real-life scenarios, storage, computational, or privacy constraints render this approach impractical. Additionally, models used in security applications may be required to rapidly adapt. In these circumstances, continual learning provides a promising alternative, as the classifier can be updated without retraining on the entire dataset. In this paper, we introduce a new dataset called CLOFAI (Continual Learning On Fake and Authentic Images), which takes the form of a domain-incremental image classification problem. Moreover, we showcase the applicability of this dataset as a benchmark for evaluating continual learning methodologies. In doing this, we set a baseline on our novel dataset using three foundational continual learning methods -- EWC, GEM, and Experience Replay -- and find that EWC performs poorly, while GEM and Experience Replay show promise, performing significantly better than a Naive baseline. The dataset and code to run the experiments can be accessed from the following GitHub repository: https://github.com/Will-Doherty/CLOFAI.
Figures
Reference graph
Works this paper leans on
-
[1]
Robert M. French and Nick Chater. “Using Noise to Compute Error Sur- faces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting”. In:Neural Comput.14.7 (2002), pp. 1755–1769
work page 2002
-
[2]
A Continual Learning Survey: Defying Forget- ting in Classification Tasks
Matthias De Lange et al. “A Continual Learning Survey: Defying Forget- ting in Classification Tasks”. In:IEEE Trans. Pattern Anal. Mach. Intell. 44.7 (2022), pp. 3366–3385
work page 2022
-
[3]
Overcomingcatastrophicforgettinginneuralnet- works
JamesKirkpatricketal.“Overcomingcatastrophicforgettinginneuralnet- works”. In:Proceedings of the national academy of sciences114.13 (2017), pp. 3521–3526
work page 2017
-
[4]
Continual Learning Through Synaptic Intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli. “Continual Learning Through Synaptic Intelligence”. In:Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017. Ed. by Doina Precup and Yee Whye Teh. Vol. 70. Pro- ceedings of Machine Learning Research. PMLR, 2017, pp. 3987–3995
work page 2017
-
[5]
Memory Aware Synapses: Learning What (not) to Forget
Rahaf Aljundi et al. “Memory Aware Synapses: Learning What (not) to Forget”. In:Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part III. Ed. by Vittorio Ferrari et al. Vol. 11207. Lecture Notes in Computer Science. Springer, 2018, pp. 144–161. 14 William Doherty, Anton Lee, Heitor Murilo Gomes
work page 2018
-
[6]
Gradient Episodic Memory for Continual Learning
David Lopez-Paz and Marc’Aurelio Ranzato. “Gradient Episodic Memory for Continual Learning”. In:Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. Ed. by Isabelle Guyon et al. 2017, pp. 6467–6476
work page 2017
-
[7]
iCaRL: Incremental Classifier and Repre- sentation Learning
Sylvestre-Alvise Rebuffi et al. “iCaRL: Incremental Classifier and Repre- sentation Learning”. In:2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 2017, pp. 5533–5542
work page 2017
-
[8]
Online Continual Learning with Maximal Interfered Retrieval
Rahaf Aljundi et al. “Online Continual Learning with Maximal Interfered Retrieval”. In:Advances in Neural Information Processing Systems 32: An- nual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. Ed. by Hanna M. Wallach et al. 2019, pp. 11849–11860
work page 2019
Show all 32 references
-
[9]
Classification of real and fake human faces using deep learning
Fatima Maher Salman and Samy S Abu-Naser. “Classification of real and fake human faces using deep learning”. In: (2022)
2022
-
[10]
An improvised CNN model for fake image detection
Yasir Hamid et al. “An improvised CNN model for fake image detection”. In: International Journal of Information Technology15.1 (2023), pp. 5–15
2023
-
[11]
Fake Face Detection Methods: Can They Be Gen- eralized?
Ali Khodabakhsh et al. “Fake Face Detection Methods: Can They Be Gen- eralized?” In:2018 International Conference of the Biometrics Special In- terest Group, BIOSIG 2018, Darmstadt, Germany, September 26-28, 2018. Ed. by Arslan Brömme et al. Vol. P-282. LNI. GI / IEEE, 2018, pp. 1–6
2018
-
[12]
AI vs. AI: Can AI Detect AI-Generated Images?
Samah Saeed Baraheem and Tam V. Nguyen. “AI vs. AI: Can AI Detect AI-Generated Images?” In:J. Imaging 9.10 (2023), p. 199
2023
-
[13]
CIFAKE: Image Classification and Ex- plainable Identification of AI-Generated Synthetic Images
Jordan J. Bird and Ahmad Lotfi. “CIFAKE: Image Classification and Ex- plainable Identification of AI-Generated Synthetic Images”. In:IEEE Ac- cess 12 (2024), pp. 15642–15650
2024
-
[14]
High-Resolution Image Synthesis with Latent Dif- fusion Models
Robin Rombach et al. “High-Resolution Image Synthesis with Latent Dif- fusion Models”. In:IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 2022, pp. 10674–10685
2022
-
[15]
Learning multiple layers of fea- tures from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al. “Learning multiple layers of fea- tures from tiny images”. In: (2009)
2009
-
[16]
Level Up the Deepfake Detection: a Method to Effectively Discriminate Images Gener- atedbyGANArchitecturesandDiffusionModels
Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. “Level Up the Deepfake Detection: a Method to Effectively Discriminate Images Gener- atedbyGANArchitecturesandDiffusionModels”.In: CoRR abs/2303.00608 (2023)
2023 arXiv
-
[17]
Deep Residual Learning for Image Recognition
Kaiming He et al. “Deep Residual Learning for Image Recognition”. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 2016, pp. 770–778
2016
-
[18]
DIRE for Diffusion-Generated Image Detection
Zhendong Wang et al. “DIRE for Diffusion-Generated Image Detection”. In:IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023. IEEE, 2023, pp. 22388–22398. Title Suppressed Due to Excessive Length 15
2023
-
[19]
Diffusion Noise Feature: Accurate and Fast Generated Image Detection
Yichi Zhang and Xiaogang Xu. “Diffusion Noise Feature: Accurate and Fast Generated Image Detection”. In:CoRR abs/2312.02625 (2023)
2023
-
[20]
Three types of incremental learning
Gido M. van de Ven, Tinne Tuytelaars, and Andreas S. Tolias. “Three types of incremental learning”. In:Nat. Mac. Intell.4.12 (2022), pp. 1185– 1197
2022
-
[21]
Investigating the impact of data normalizationonclassificationperformance
Dalwinder Singh and Birmohan Singh. “Investigating the impact of data normalizationonclassificationperformance”.In: Appl. Soft Comput.97.Part B (2020), p. 105524
2020
-
[22]
Auto-encoding variational bayes
Diederik P Kingma and Max Welling. “Auto-encoding variational bayes”. In: arXiv preprint arXiv:1312.6114(2013)
2013 arXiv
-
[23]
VAEBM: A Symbiosis between Variational Autoen- coders and Energy-based Models
Zhisheng Xiao et al. “VAEBM: A Symbiosis between Variational Autoen- coders and Energy-based Models”. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7,
2021
-
[24]
Generative adversarial nets
Ian Goodfellow et al. “Generative adversarial nets”. In:Advances in neural information processing systems27 (2014)
2014
-
[25]
Densely connected nor- malizing flows
Matej Grcic, Ivan Grubisic, and Sinisa Segvic. “Densely connected nor- malizing flows”. In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual. Ed. by Marc’Aurelio Ra...
2021
-
[26]
Diffusers: State-of-the-art diffusion models
PatrickvonPlatenetal. Diffusers: State-of-the-art diffusion models. https: //github.com/huggingface/diffusers. 2022
2022
-
[27]
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Mingxing Tan and Quoc V. Le. “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”. In:Proceedings of the 36th Interna- tional Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA.Ed.byKamalikaChaudhuriandRuslanSalakhut- d...
2019
-
[28]
TorchVision: PyTorch’s Com- puter Vision library
TorchVision maintainers and contributors. TorchVision: PyTorch’s Com- puter Vision library. https://github.com/pytorch/vision. 2016
2016
-
[29]
ImageNet: A large-scale hierarchical image database
Jia Deng et al. “ImageNet: A large-scale hierarchical image database”. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE. 2009, pp. 248–255
2009
-
[30]
Avalanche: An End-to-End Library for Con- tinual Learning
Vincenzo Lomonaco et al. “Avalanche: An End-to-End Library for Con- tinual Learning”. In:IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19-25, 2021. Computer Vision Foundation / IEEE, 2021, pp. 3600–3610
2021
-
[31]
Adam: A Method for Stochastic Op- timization
Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Op- timization”. In:3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Pro- ceedings. Ed. by Yoshua Bengio and Yann LeCun. 2015
2015
-
[2021]
OpenReview.net, 2021
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.