Pith. sign in

REVIEW 3 major objections 4 minor 32 references

CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read The paper introduces CLOFAI, a domain-incremental benchmark for real versus fake image classification, and claims that replay-based continual learning methods (GEM and Experience Replay) markedly outperform a naive baseline while the…

desk verdict A useful new continual-learning benchmark for fake-image detection, but the method comparisons are statistically underpowered and should be treated as preliminary. read the letter →

arxiv 2501.11140 v1 pith:UJEN37M5 submitted 2025-01-19 cs.CV cs.AI

classification cs.CVcs.AI
keywords continuallearningfakeimagedetectiondomain-incrementalcatastrophicforgettingexperiencereplaygradientepisodicmemorygenerativemodelsEfficientNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CLOFAI, a benchmark that casts the problem of telling real from artificially generated images as domain-incremental continual learning. The dataset has five tasks, each pairing 5,000 real CIFAR-10 images with 5,000 fakes produced by a different generative model, ordered from easiest to hardest. The paper's central claim is that replay-based continual learning methods, specifically GEM and Experience Replay, markedly outperform a naive fine-tuning baseline at retaining accuracy on earlier tasks, while the regularization method EWC does not. If this holds, fake-image detectors could be updated on new generators without retraining on all past data.

What carries the argument

The central object is the CLOFAI task sequence: five binary real-versus-fake classification tasks whose input distribution changes with the generating model (VAE, VAEBM, GAN, flow, and DDIM diffusion) while the labels remain the same. Task order is set by a classifier's accuracy on each task in isolation, from easiest to hardest, so the sequence simulates the emergence of progressively more realistic generators. The benchmark's evaluation is a per-task accuracy matrix after each training stage, which makes forgetting visible as accuracy loss on earlier tasks.

What would settle it

Re-run the full benchmark with ten random seeds per method and compare 95 percent confidence intervals on each task-accuracy cell; if the Naive and replay intervals overlap substantially, the claim that GEM and Experience Replay perform significantly better is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that CLOFAI works as a domain-incremental benchmark and that on it replay-based methods are the viable route: GEM and Experience Replay keep accuracy on earlier tasks far above the Naive fine-tuning baseline, whereas EWC performs roughly as poorly as Naive. The paper supports this with accuracy matrices showing replay methods holding earlier-task accuracy substantially above the Naive baseline across the full sequence, while EWC's trajectory tracks Naive closely. The proposed explanation for EWC's failure is that the same parameters are highly important across all tasks, so Fisher-information regularization either allows destructive updates or, at high strength, prevents learning the new task.

Load-bearing premise

The reported method ranking assumes that one training run with a fixed random seed represents each method's typical behaviour; under heavy training noise, the differences between replay methods and the Naive baseline could disappear.

Editorial extensions

If this is right

  • A fake-image detector can be kept current on newly released generators by replaying a small buffer of earlier examples, instead of retraining on the full history of data.
  • Because labels stay the same across tasks, the trained model never needs to know which generative model produced an image at inference time.
  • GEM's larger gain from 100 to 500 replayed samples, compared with Experience Replay's small gain, suggests that enforcing constraints on past-task loss is what buys retention, not simply adding data.
  • Regularization-only protection, as implemented by EWC, fails on this benchmark even at extreme regularization strengths, signalling that weight-importance methods are not a safe default for this domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A held-out newer generator (for example, a transformer-based image model) could serve as a fresh task to test whether replay methods retain old knowledge while absorbing a genuinely new style; based on the dataset's difficulty ordering, performance should drop furthest on such a task.
  • Because all experiments use a single seed, the quantitative gaps between methods may be partly due to training noise; re-running with several seeds would tell whether Experience Replay's advantage over the Naive baseline is consistent.
  • The CLOFAI task layout -- one fixed label pair with shifting input distributions -- transfers directly to other deepfake domains such as audio or video, where new synthesis methods also arrive over time.
  • The accuracy matrices show near-chance performance on tasks the model has not yet seen, implying the model learns task-specific cues rather than a general real/fake invariant; if that is right, continual learning here is mainly about storing task-specific features.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper introduces CLOFAI, a domain-incremental continual learning dataset for real versus fake image classification. The benchmark consists of five tasks in which a binary classifier distinguishes real CIFAR-10 images from images produced by five generators (VAE, VAEBM, GAN, Flow, Diffusion), with a different real-image set per task and an 80/20 train/test split. Using an ImageNet-pretrained EfficientNet-b0, the authors report accuracy matrices for a joint-training baseline, Naive sequential fine-tuning, EWC, Experience Replay (100 and 500 replayed samples), and GEM (100 and 500 replayed samples). The paper's central claim is that EWC performs poorly on this benchmark, while GEM and Experience Replay perform significantly better than a Naive baseline.

Significance. If the reported comparisons are robust, CLOFAI fills a clear gap: it provides a reusable domain-incremental benchmark that mimics the realistic scenario of updating a fake-image detector as new generative models appear, under storage or privacy constraints. The paper contributes a concrete dataset, a clean problem formulation, and accuracy matrices for five common continual learning methods, and it makes code and data publicly available. The benchmark itself and the reproducible baseline tables are the main strengths. The main limitations are that the headline ranking rests on a single random seed with no statistical validation, and that the task-difficulty ordering was determined using the same classifier architecture that is later benchmarked.

major comments (3)
  1. [Section 5] The claim that GEM and Experience Replay are 'significantly better' than Naive is not supported by the experimental protocol. Section 5 states 'All stochastic parameters had seed set to 123', so there are no repeated runs, confidence intervals, or significance tests. Averaging the final rows of Tables 4, 8, 9, 10, and 11 gives roughly 61.0% for Naive, 62.2% for ER-100, 67.1% for ER-500, 65.0% for GEM-100, and 67.7% for GEM-500; the 1.2-point difference between Naive and ER-100 is small relative to the run-to-run variance expected from a 3-epoch, 8,000-sample training budget. The Baseline matrix in Table 2 also shows an odd diagonal value (54.40% on task 2 after training on tasks 1 and 2, versus 79.85% for Naive at the same point), which suggests high sensitivity to initialization and training schedule. Please repeat the experiments over multiple seeds and report means, confidence intervals, or paired statistical tests before using the word 'significantly'.
  2. [Section 3, Table 1] The task order is selected post hoc using the same classifier that is later evaluated: Section 3 says the authors tested the Classifier's performance on each task in isolation and ordered tasks from easiest (Task 1) to hardest (Task 5) according to those accuracies. Because the 'increasing difficulty' progression is defined by EfficientNet-b0's per-task accuracy, it is not an independent property of the generator sequence, and the central narrative of a real-world progression of generator realism is partly constructed by the benchmark itself. Please either derive the ordering from several architectures or an independent perceptual measure and show stability, or clearly disclose and discuss this circularity when presenting the task order as a simulation of real-world circumstances.
  3. [Section 5.3] The conclusion that 'irrespective of the value of lambda, EWC cannot achieve good performance' is based on only two lambda values (100,000 and 100,000,000), and no lambda sweep is reported. The EWC implementation is also underspecified: the text does not state when the Fisher Information is computed, whether an online EWC variant is used, or how the quadratic penalty is normalized across layers. Without this information and a sweep over lambda, the reported EWC failure could be a tuning or implementation artifact rather than a property of the method on CLOFAI. Please report a lambda sweep and the full implementation details, or restrict the conclusion to the two tested values.
minor comments (4)
  1. [Section 3 heading] The section heading 'CLOF AI benchmark' appears to contain a typo and should read 'CLOFAI benchmark'.
  2. [Section 5.2] The phrase 'relative difficultly' should be 'relative difficulty'.
  3. [Section 4] The description of Experience Replay does not state how the 100 or 500 replayed samples are selected from each past task (e.g., random sampling, mean-of-feature, or some other strategy); this choice should be documented because the comparison between buffer sizes depends on it.
  4. [Section 4] GEM is the only method taken from an external library (Avalanche) while the other methods are implemented from scratch; please clarify whether GEM shares exactly the same data pipeline, optimizer, epoch count, and seed handling as the other methods, since otherwise the method comparisons could be confounded by implementation differences.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the continual learning method comparison is an independent empirical benchmark; the task-difficulty ordering is an openly stated construction, not a derived prediction.

full rationale

The paper's central claims are (i) CLOFAI is a domain-incremental dataset for real/fake image classification and (ii) GEM and Experience Replay outperform EWC and a Naive baseline on it. The dataset construction is described concretely with five generative models, CIFAR-10 real images, and an 80/20 split. The task order was chosen by measuring the same EfficientNet_b0 classifier's isolated accuracy on each task and ordering tasks from easiest to hardest (Section 3, Table 1). This is an explicitly stated construction rule, not a hidden assumption or a prediction masquerading as a result; it defines the benchmark's difficulty ordering rather than deriving an independent fact from it. The method comparisons in Section 5 use implementations of Baseline, Naive, EWC, Experience Replay, and GEM evaluated on the same fixed task sequence. No parameter is fitted to the benchmark outcome and then renamed as a prediction; no result reduces by construction to the dataset definition; there are no self-citations or imported uniqueness theorems. The single-seed evaluation (Section 5) and the absence of confidence intervals are legitimate statistical-robustness concerns, but they are not circularity. Accordingly no circular step is identified.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The benchmark's conclusions rest on the choice of five generator families, the chosen task order, and the experimental hyperparameters. The task order was selected using the same classifier family used in the evaluation, which is the main circularity signal. The empirical method ranking additionally rests on a single seed and on two EWC lambda values.

free parameters (7)
  • Task order (VAE, VAEBM, GAN, Flow, Diffusion) = ordered by decreasing isolated classifier accuracy (Table 1)
    The order was selected after measuring the same classifier's accuracy, so the increasing-difficulty narrative is enforced by construction rather than discovered.
  • EWC regularization lambda = 100,000 and 100,000,000
    Only two values were tested; the claim that EWC cannot work irrespective of lambda is not supported by a sweep.
  • Replay buffer size = 100 and 500 samples per task
    Chosen by hand to probe the effect of memory size; no tuning criterion is given.
  • Training epochs per task = 3
    Stated as avoiding overfitting; no sensitivity analysis is provided.
  • Batch size = 128
    Standard choice, but not varied or justified beyond the experimental setup.
  • Learning rate = 0.0001
    Fixed for all methods; no learning rate tuning is reported.
  • Random seed = 123
    Single seed used for all stochastic parameters, giving no variance estimate.
assumptions (5)
  • domain assumption Generated images from the five model families are representative of fake images a detector must handle.
    The benchmark's relevance rests on this; no external validation against other fake image sources is provided (Section 3).
  • ad hoc to paper The fixed task order simulates the real-world progression of generative model realism.
    The order was chosen by the authors to reflect increasing difficulty (Section 3); it is not an observed property of the deployed ecosystem.
  • domain assumption CIFAR-10 is a sufficient source of real images for evaluating fake image detectors.
    Low-resolution object images may not represent realistic photos in security applications; Section 3.
  • domain assumption A single random seed with deterministic training is sufficient to rank continual learning methods.
    Section 5 uses one seed, so the assumption that results are not noise-driven is unverified.
  • domain assumption ImageNet pretraining transfers to the CIFAR-10 real/fake task.
    Section 4 uses pretrained EfficientNet; the benefit is shown for the joint baseline but not for continual learning methods specifically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning." pith.science (2026). https://pith.science/paper/UJEN37M5

@misc{pith2026250111140,
  author       = {Pith},
  title        = {Pith review of: CLOFAI: A Dataset of Real And Fake Image Classification Tasks for Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJEN37M5}},
  note         = {Machine review of arXiv:2501.11140}
}
read the original abstract

The rapid advancement of generative AI models capable of creating realistic media has led to a need for classifiers that can accurately distinguish between genuine and artificially-generated images. A significant challenge for these classifiers emerges when they encounter images from generative models that are not represented in their training data, usually resulting in diminished performance. A typical approach is to periodically update the classifier's training data with images from the new generative models then retrain the classifier on the updated dataset. However, in some real-life scenarios, storage, computational, or privacy constraints render this approach impractical. Additionally, models used in security applications may be required to rapidly adapt. In these circumstances, continual learning provides a promising alternative, as the classifier can be updated without retraining on the entire dataset. In this paper, we introduce a new dataset called CLOFAI (Continual Learning On Fake and Authentic Images), which takes the form of a domain-incremental image classification problem. Moreover, we showcase the applicability of this dataset as a benchmark for evaluating continual learning methodologies. In doing this, we set a baseline on our novel dataset using three foundational continual learning methods -- EWC, GEM, and Experience Replay -- and find that EWC performs poorly, while GEM and Experience Replay show promise, performing significantly better than a Naive baseline. The dataset and code to run the experiments can be accessed from the following GitHub repository: https://github.com/Will-Doherty/CLOFAI.

Figures

Figures reproduced from arXiv: 2501.11140 by the authors.

Figure 1
Figure 1. Problem Setup to have mean (0, 0, 0) and standard deviation of (1, 1, 1), as normalisation typ￾ically improves classification performance [21]. The three elements of the tuple correspond to the three image channels - red, green, and blue. The generative models used for each task are listed below. An exemplar image from each of the models is shown in [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Example of a Fake Image for Each Task The task order was chosen to reflect real-world circumstances where fake im￾ages become more realistic and harder to classify over time. To achieve this, we tested the Classifier’s performance on each task in isolation and then ordered [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 28 canonical work pages

  1. [1]

    Using Noise to Compute Error Sur- faces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting

    Robert M. French and Nick Chater. “Using Noise to Compute Error Sur- faces in Connectionist Networks: A Novel Means of Reducing Catastrophic Forgetting”. In:Neural Comput.14.7 (2002), pp. 1755–1769

  2. [2]

    A Continual Learning Survey: Defying Forget- ting in Classification Tasks

    Matthias De Lange et al. “A Continual Learning Survey: Defying Forget- ting in Classification Tasks”. In:IEEE Trans. Pattern Anal. Mach. Intell. 44.7 (2022), pp. 3366–3385

  3. [3]

    Overcomingcatastrophicforgettinginneuralnet- works

    JamesKirkpatricketal.“Overcomingcatastrophicforgettinginneuralnet- works”. In:Proceedings of the national academy of sciences114.13 (2017), pp. 3521–3526

  4. [4]

    Continual Learning Through Synaptic Intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. “Continual Learning Through Synaptic Intelligence”. In:Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017. Ed. by Doina Precup and Yee Whye Teh. Vol. 70. Pro- ceedings of Machine Learning Research. PMLR, 2017, pp. 3987–3995

  5. [5]

    Memory Aware Synapses: Learning What (not) to Forget

    Rahaf Aljundi et al. “Memory Aware Synapses: Learning What (not) to Forget”. In:Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part III. Ed. by Vittorio Ferrari et al. Vol. 11207. Lecture Notes in Computer Science. Springer, 2018, pp. 144–161. 14 William Doherty, Anton Lee, Heitor Murilo Gomes

  6. [6]

    Gradient Episodic Memory for Continual Learning

    David Lopez-Paz and Marc’Aurelio Ranzato. “Gradient Episodic Memory for Continual Learning”. In:Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. Ed. by Isabelle Guyon et al. 2017, pp. 6467–6476

  7. [7]

    iCaRL: Incremental Classifier and Repre- sentation Learning

    Sylvestre-Alvise Rebuffi et al. “iCaRL: Incremental Classifier and Repre- sentation Learning”. In:2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 2017, pp. 5533–5542

  8. [8]

    Online Continual Learning with Maximal Interfered Retrieval

    Rahaf Aljundi et al. “Online Continual Learning with Maximal Interfered Retrieval”. In:Advances in Neural Information Processing Systems 32: An- nual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. Ed. by Hanna M. Wallach et al. 2019, pp. 11849–11860

Show all 32 references
  1. [9]

    Classification of real and fake human faces using deep learning

    Fatima Maher Salman and Samy S Abu-Naser. “Classification of real and fake human faces using deep learning”. In: (2022)

  2. [10]

    An improvised CNN model for fake image detection

    Yasir Hamid et al. “An improvised CNN model for fake image detection”. In: International Journal of Information Technology15.1 (2023), pp. 5–15

  3. [11]

    Fake Face Detection Methods: Can They Be Gen- eralized?

    Ali Khodabakhsh et al. “Fake Face Detection Methods: Can They Be Gen- eralized?” In:2018 International Conference of the Biometrics Special In- terest Group, BIOSIG 2018, Darmstadt, Germany, September 26-28, 2018. Ed. by Arslan Brömme et al. Vol. P-282. LNI. GI / IEEE, 2018, pp. 1–6

  4. [12]

    AI vs. AI: Can AI Detect AI-Generated Images?

    Samah Saeed Baraheem and Tam V. Nguyen. “AI vs. AI: Can AI Detect AI-Generated Images?” In:J. Imaging 9.10 (2023), p. 199

  5. [13]

    CIFAKE: Image Classification and Ex- plainable Identification of AI-Generated Synthetic Images

    Jordan J. Bird and Ahmad Lotfi. “CIFAKE: Image Classification and Ex- plainable Identification of AI-Generated Synthetic Images”. In:IEEE Ac- cess 12 (2024), pp. 15642–15650

  6. [14]

    High-Resolution Image Synthesis with Latent Dif- fusion Models

    Robin Rombach et al. “High-Resolution Image Synthesis with Latent Dif- fusion Models”. In:IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 2022, pp. 10674–10685

  7. [15]

    Learning multiple layers of fea- tures from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. “Learning multiple layers of fea- tures from tiny images”. In: (2009)

  8. [16]

    Level Up the Deepfake Detection: a Method to Effectively Discriminate Images Gener- atedbyGANArchitecturesandDiffusionModels

    Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. “Level Up the Deepfake Detection: a Method to Effectively Discriminate Images Gener- atedbyGANArchitecturesandDiffusionModels”.In: CoRR abs/2303.00608 (2023)

  9. [17]

    Deep Residual Learning for Image Recognition

    Kaiming He et al. “Deep Residual Learning for Image Recognition”. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 2016, pp. 770–778

  10. [18]

    DIRE for Diffusion-Generated Image Detection

    Zhendong Wang et al. “DIRE for Diffusion-Generated Image Detection”. In:IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023. IEEE, 2023, pp. 22388–22398. Title Suppressed Due to Excessive Length 15

  11. [19]

    Diffusion Noise Feature: Accurate and Fast Generated Image Detection

    Yichi Zhang and Xiaogang Xu. “Diffusion Noise Feature: Accurate and Fast Generated Image Detection”. In:CoRR abs/2312.02625 (2023)

  12. [20]

    Three types of incremental learning

    Gido M. van de Ven, Tinne Tuytelaars, and Andreas S. Tolias. “Three types of incremental learning”. In:Nat. Mac. Intell.4.12 (2022), pp. 1185– 1197

  13. [21]

    Investigating the impact of data normalizationonclassificationperformance

    Dalwinder Singh and Birmohan Singh. “Investigating the impact of data normalizationonclassificationperformance”.In: Appl. Soft Comput.97.Part B (2020), p. 105524

  14. [22]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. “Auto-encoding variational bayes”. In: arXiv preprint arXiv:1312.6114(2013)

  15. [23]

    VAEBM: A Symbiosis between Variational Autoen- coders and Energy-based Models

    Zhisheng Xiao et al. “VAEBM: A Symbiosis between Variational Autoen- coders and Energy-based Models”. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7,

  16. [24]

    Generative adversarial nets

    Ian Goodfellow et al. “Generative adversarial nets”. In:Advances in neural information processing systems27 (2014)

  17. [25]

    Densely connected nor- malizing flows

    Matej Grcic, Ivan Grubisic, and Sinisa Segvic. “Densely connected nor- malizing flows”. In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual. Ed. by Marc’Aurelio Ra...

  18. [26]

    Diffusers: State-of-the-art diffusion models

    PatrickvonPlatenetal. Diffusers: State-of-the-art diffusion models. https: //github.com/huggingface/diffusers. 2022

  19. [27]

    EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

    Mingxing Tan and Quoc V. Le. “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”. In:Proceedings of the 36th Interna- tional Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA.Ed.byKamalikaChaudhuriandRuslanSalakhut- d...

  20. [28]

    TorchVision: PyTorch’s Com- puter Vision library

    TorchVision maintainers and contributors. TorchVision: PyTorch’s Com- puter Vision library. https://github.com/pytorch/vision. 2016

  21. [29]

    ImageNet: A large-scale hierarchical image database

    Jia Deng et al. “ImageNet: A large-scale hierarchical image database”. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE. 2009, pp. 248–255

  22. [30]

    Avalanche: An End-to-End Library for Con- tinual Learning

    Vincenzo Lomonaco et al. “Avalanche: An End-to-End Library for Con- tinual Learning”. In:IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19-25, 2021. Computer Vision Foundation / IEEE, 2021, pp. 3600–3610

  23. [31]

    Adam: A Method for Stochastic Op- timization

    Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Op- timization”. In:3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Pro- ceedings. Ed. by Yoshua Bengio and Yann LeCun. 2015

  24. [2021]

    OpenReview.net, 2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.