Pith. sign in

REVIEW 3 major objections 4 minor 63 references

Dealing with Synthetic Data Contamination in Online Continual Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Entropy-based replay buffer selection plus a contrastive alignment loss keeps online continual learners accurate even when 95% of training images are synthetic.

desk verdict Solid empirical paper on synthetic contamination in online continual learning; ESRM works on the tested distributions, but the entropy discriminator's generalization is the main open question. read the letter →

arxiv 2411.13852 v1 pith:BA37UQSO submitted 2024-11-21 cs.CV cs.LG

Maorong Wang , Nicolas Michel , Jiafeng Mao , Toshihiko Yamasaki This is my paper · ORCID
classification cs.CVcs.LG
keywords onlinecontinuallearningsyntheticdatacontaminationdiffusionmodelsreplaybufferentropy-basedsampleselectioncontrastiveclass-incrementalcatastrophicforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that synthetic images from diffusion models, when mixed into the training stream of an online continual learner, cause measurable accuracy loss that grows with the contamination ratio, and that this loss can be largely undone without knowing which images are synthetic. The proposed method, Entropy Selection with Real-synthetic similarity Maximization (ESRM), keeps a replay buffer stocked with high-entropy (real) samples and uses a contrastive loss to pull real and synthetic feature embeddings closer together. On four benchmarks, ESRM's accuracy drop stays small even at 70-80% contamination, while six existing replay-based methods lose up to roughly twenty points. The reason it works is an observed regularity: across the five tested diffusion generators, synthetic images receive lower prediction entropy from the continual learner and cluster more tightly in embedding space.

What carries the argument

Two mechanisms carry the method. Entropy Selection (ES) is a buffer-management rule: each incoming batch drops its lower-entropy half, applies reservoir sampling to the rest, and when replacement is needed selects the buffer sample of the same class with the lowest prediction entropy; all buffer entropies are refreshed at task ends. Real-synthetic similarity Maximization (RM) is a contrastive loss that maximizes cosine similarity between the high-entropy half of the stream and the low-entropy half, and between stream and memory, pulling embedded synthetic features toward real ones. A self-distillation term from a fixed copy of the augmented classifier prevents overconfidence. Together they exploit Observation 3 and Observation 4: entropy separates synthetic from real, and aligning embeddings removes the observed feature misalignment.

What would settle it

Construct a contamination stream with a newer generator and diverse per-class prompts, train ESRM and a random-replay baseline, then measure the entropy separation (ROC AUC) between synthetic and real samples at the end of training; if AUC approaches 0.5 or if ESRM no longer outperforms random reservoir sampling, the method's grounding is refuted.

Watch

Extended reading notes

Core claim

The paper establishes that AI-generated images are a distinct corruption source for online class-incremental learning and that a replay method can be made robust to them using only learner-internal signals. Four observations support the claim: contamination degrades all tested baselines; keeping real images in the buffer nearly restores clean accuracy; synthetic images have lower prediction entropy; and synthetic features are better clustered, misaligning with real features. ESRM combines Entropy Selection, which discards low-entropy incoming samples and replaces same-class lowest-entropy buffer entries, with a Real-synthetic similarity Maximization loss that aligns high- and low-entropy groups and stream with memory. In experiments, ESRM is the least affected method under contamination, often improving on its own clean baseline, and remains usable at P=95% where baselines collapse.

Load-bearing premise

The method depends on the empirical regularity that synthetic images receive lower prediction entropy than real images in the online continual learner, with the gap large enough that entropy-based buffer choice beats random choice.

Editorial extensions

If this is right

  • At contamination ratios above 70%, existing replay methods lose accuracy by 5 to more than 20 points; ESRM keeps the drop under about 7 points on most benchmarks.
  • A learner does not need a separate synthetic-image detector or ground-truth synthetic labels; entropy and embedding alignment suffice.
  • Keeping the buffer mostly real is the dominant protective mechanism: with oracle real-only memory, ER matches clean accuracy even at 80% contamination, and ES approaches this without labels.
  • ESRM also transfers to domain-incremental learning, holding degradation below baselines on a 20-superclass split.
  • Because ESRM's plasticity is higher, it does not trade away new-task learning to gain stability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the entropy gap persists across future generators, ES could double as an unsupervised data-quality filter for any replay-based learner, not just the specific architecture here.
  • The paper's limitation that prompts were single-template 'an image of a class' suggests a natural stress test: with diverse user-style prompts, synthetic entropy may rise, and the method's advantage should shrink; measuring that would bound the method's validity.
  • A detector-free buffer policy that increases the real-to-synthetic ratio may also reduce bias amplification and diversity loss reported in non-continual settings, though the paper does not evaluate those outcomes.
  • The near-flat accuracy of ESRM across contamination ratios suggests online continual learning could tolerate streams that are mostly synthetic if a small anchor set of real data is stored, an implicit argument for curating small clean seeds rather than entire clean datasets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies the effect of AI-generated (synthetic) images on online continual learning (CL). It simulates contaminated training streams by replacing a fraction P of four benchmark datasets (CIFAR-10, CIFAR-100, TinyImageNet, ImageNet-100) with synthetic twins produced by Stable Diffusion XL, Stable Diffusion v1.4/v2.1, VQDM, and GLIDE, using simple class-name prompts. It reports that six existing online CL methods degrade as P increases, and it proposes ESRM, a replay-based method with two components: Entropy Selection (ES), which preferentially stores high-entropy (more likely real) samples in the buffer, and Real-synthetic similarity Maximization (RM), a contrastive loss that pulls features of high- and low-entropy groups together. Experiments over four datasets, two contamination settings (SDXL-only and a five-model mix), contamination ratios from 50% to 95%, DIL results, ablations, and buffer-size variants show that ESRM generally suffers smaller contamination-induced degradation and often achieves higher absolute accuracy than the baselines. Source code and detailed hyperparameter settings are provided.

Significance. If the results hold, the paper identifies a realistic and understudied threat to online CL—synthetic data on the Internet—and offers a simple, replay-compatible mitigation. The empirical protocol is extensive: four datasets, six baselines, multiple contamination ratios, five generators, ablations, and error bars over 5–10 runs. The paper also contributes four qualitative observations about synthetic data in online CL, and releases code, which supports reproducibility. The main limitation is that the mechanism of both ESRM components rests on a single moderate quantitative estimate of the entropy gap (ROC AUC 0.7098 on one dataset/model), and the paper's own limitation section acknowledges the narrow generator and prompt coverage. The central claim that contamination degrades online CL and that ESRM alleviates the degradation is therefore empirically supported on the tested settings, but the generality of the entropy-based mechanism is not yet established.

major comments (3)
  1. [4.3, 5.1, 5.2, Fig. 7] Both ES and RM are explicitly grounded in Observation 3, that synthetic images have lower predictive entropy than real images. The only quantitative support is the ROC curve in Fig. 7, which reports AUC = 0.7098 on In-100/SDXL (P = 50%) for an ESRM-trained model. This is a single, moderate discriminability estimate: at an operating point near the ROC curve, roughly one third of samples would be misclassified. Moreover, the AUC is measured after ESRM training, so it may partly reflect the method's own effect rather than a pre-existing property that ES and RM exploit. Since ES's filtering and RM's group splitting both become arbitrary if the entropy ordering weakens or reverses, the robustness claim is conditional on an empirical regularity that is not yet quantified across the test datasets, training stages, or unseen generators. I request additional evidence: entropy-gap statistics or AUC values for all datasets and baselines, before and after training, and for at least one held-out generator or more diverse prompt distribution; alternatively, the authors should explicitly restrict the claim to the tested generator set.
  2. [Table 2 (C10/SDXL rows)] On CIFAR-10, the headline claim of 'alleviating performance deterioration' is less clear in absolute terms. At P = 70%, ESRM achieves 67.47 ± 1.43 while OnPro achieves 68.79 ± 1.17, and at P = 80% the two are close (66.81 vs. 66.07). ESRM's smaller degradation is measured relative to its own clean accuracy of 67.35, which is substantially below OnPro's clean accuracy of 74.87. A method with lower clean accuracy can show smaller absolute drops even if its contaminated accuracy is not the best. The paper should discuss this trade-off explicitly and, ideally, report a paired analysis of contamination-induced drops or normalized degradation for all methods, not only the raw accuracy tables.
  3. [6.2, Tables 5 and 6] The paper uses the word 'significant' repeatedly (e.g., in the abstract, Section 4.1, and Section 6.2) but reports only means and standard deviations. Some of the key comparisons that support the method—ES vs. random in Table 5 (45.92 ± 0.42 vs. 44.84 ± 0.80 at P = 70%) and the loss-term ablation in Table 6 (Baseline 42.61 ± 0.84 vs. ESRM 45.92 ± 0.42)—are close enough that a formal paired test or confidence interval would materially strengthen the claim. I recommend adding paired significance tests or bootstrap confidence intervals for the main comparisons, especially the ES/Random comparison and the ESRM-vs-best-baseline comparisons at severe contamination levels.
minor comments (4)
  1. [5.3] In the sentence introducing the total loss, 'balancing hyperparamteres' is a typo for 'balancing hyperparameters.'
  2. [Eq. (4)] The phrase 'the fixed copy of ϕ(f(aug(X)))' is ambiguous; it should be clarified that this is the same network with gradients stopped, not a separate or fixed network copy.
  3. [Fig. 7] The ROC curve would be more informative if the paper also reported the entropy distributions of real and synthetic samples for the same model, and ideally for a baseline model before ESRM training, so readers can see how the separation evolves.
  4. [Tables 3 and 4] For ESRM, Relative Forgetting in Table 4 is lower at high contamination (e.g., 35.40 at P = 95% vs. 49.79 at P = 0%), which seems counterintuitive; the text says ESRM 'solves the problem of stability degradation' but does not explain why forgetting would decrease as contamination increases. A brief explanation would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ESRM's components are empirical and evaluated against external benchmarks; self-citations are non-load-bearing.

full rationale

The paper is an empirical study rather than a formal derivation chain, and no load-bearing claim reduces to its own inputs by construction. Observation 3 (synthetic entropy is lower) and Observation 4 (real/synthetic feature gap) are measured on baseline models such as ER and OnPro before ESRM is introduced (Figures 2 and 3), so the motivating premises are not manufactured by the method. ES uses these observations as a heuristic for buffer bias, and its benefit is evaluated against random, real-only, and synthetic-only memory strategies (Table 5) and against six external baselines on standard benchmarks (Table 2), not by reinserting the entropy rule as the outcome. RM is a contrastive objective whose grouping uses the same entropy proxy, but the claimed outcome, final average accuracy on real test sets and reduced forgetting, is measured on held-out real test data and is not identical to the training loss. Hyperparameters lambda1 and lambda2 are tuned on clean CIFAR-100 (Appendix D.6), not fitted to contaminated test results. The only self-citations, Ref. [42] for self-distillation and the relative-forgetting metric, are not load-bearing: LSDC is ablated in Table 6, and removing it still leaves a method that outperforms baselines. The paper's acknowledged limitations (Appendix A: five generators and simple prompts; Section 7: a single AUC of 0.71 for the entropy proxy) are robustness and generalization concerns about an empirical assumption, not evidence of circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim is empirical; the method introduces no new entities. The two loss weights are fitted on CIFAR-100 and reused everywhere, and the entropy-gap observations are assumptions rather than derived facts.

free parameters (2)
  • lambda_1 (weight of self-distillation loss LSDC) = 1
    Set after a hyperparameter search on clean CIFAR-100 (Appendix D.6); applied to all datasets and contamination levels.
  • lambda_2 (weight of real-synthetic similarity loss LRM) = 0.5
    Set after a hyperparameter search on clean CIFAR-100 (Appendix D.6); applied to all datasets and contamination levels.
assumptions (3)
  • domain assumption In online CL, synthetic images tend to have lower prediction entropy and tighter feature clustering than real images (Obs. 3 and Obs. 4).
    Empirically observed on five diffusion models and four datasets; the method assumes the gap persists across generators and contamination levels.
  • domain assumption Storing real images in the replay buffer is more beneficial than storing synthetic images (Obs. 2).
    Shown in Table 1 and Table 5 for ER and ESRM; assumed to hold for other replay-based methods.
  • domain assumption Prediction entropy is a usable proxy for real-versus-synthetic status in the training stream.
    The reported ROC AUC is 0.71 (Fig. 7), meaning the proxy is only moderately reliable; both ES and RM depend on it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dealing with Synthetic Data Contamination in Online Continual Learning." pith.science (2026). https://pith.science/paper/BA37UQSO

@misc{pith2026241113852,
  author       = {Pith},
  title        = {Pith review of: Dealing with Synthetic Data Contamination in Online Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BA37UQSO}},
  note         = {Machine review of arXiv:2411.13852}
}
read the original abstract

Image generation has shown remarkable results in generating high-fidelity realistic images, in particular with the advancement of diffusion-based models. However, the prevalence of AI-generated images may have side effects for the machine learning community that are not clearly identified. Meanwhile, the success of deep learning in computer vision is driven by the massive dataset collected on the Internet. The extensive quantity of synthetic data being added to the Internet would become an obstacle for future researchers to collect "clean" datasets without AI-generated content. Prior research has shown that using datasets contaminated by synthetic images may result in performance degradation when used for training. In this paper, we investigate the potential impact of contaminated datasets on Online Continual Learning (CL) research. We experimentally show that contaminated datasets might hinder the training of existing online CL methods. Also, we propose Entropy Selection with Real-synthetic similarity Maximization (ESRM), a method to alleviate the performance deterioration caused by synthetic images when training online CL models. Experiments show that our method can significantly alleviate performance deterioration, especially when the contamination is severe. For reproducibility, the source code of our work is available at https://github.com/maorong-wang/ESRM.

Figures

Figures reproduced from arXiv: 2411.13852 by the authors.

Figure 1
Figure 1. Overview of proposed ESRM framework for online CL. The proposed ESRM framework [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. T-SNE visualization of the mem￾ory data at the end of training on In￾100/SDXL (P = 50%). For clarity, only the first 10 classes are visualized. the following sections, we will investigate the effect of synthetic contamination with these simulated datasets. 4 Synthetic Data Contamination in Online CL In this section, we explore the potential impact of synthetic data contamination on the existing online CL methods and… view at source ↗
Figure 4
Figure 4. Overview of the proposed En￾tropy Selection strategy. The color of the samples indicates the class, and the num￾ber in the samples represents the entropy predicted by the learner. The introduced ES aims to select more representative sam￾ples in the memory buffer. For replay-based methods, having high-quality samples in the memory buffer helps alleviate forgetting and achieve better overall performance. As per Obs. 2… view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: The percentage of synthetic data in the memory buffer throughout the training of ESRM on the In-100/SDXL dataset with different contam￾ination ratios (P). The average value of 5 runs is plotted. CIFAR-100 dataset. Since the 100 classes in CIFAR-100 are grouped into 20 …
Figure 7
Figure 7. Figure 7: The ROC curve of the model trained with ESRM on the In-100/SDXL dataset (P = 50%) in predicting the synthetic status of samples in the training dataset. Real sam￾ples are regarded as positives and synthetic samples as nega￾tives. The alleviation of feature misalignment…
Figure 8
Figure 8. Figure 8: The entropy distribution of the training set produced by all methods on In-100/SDXL [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: T-SNE visualization of the memory data at the end of training on In-100/SDXL ( [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Random sampled images from class “n01558993” (Robin) in SDXL-In100 and original [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: The average training time of each method trained on CIFAR-100 (M=5k), IN-100 (M=5k), [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 48 canonical work pages

  1. [42]

    Improving Plasticity in Online Continual Learning via Collaborative Learning

    Maorong Wang, Nicolas Michel, Ling Xiao, and Toshihiko Yamasaki. Improving plasticity in online continual learning via collaborative learning. arXiv preprint arXiv:2312.00600, 2023

    work page Pith review arXiv 2023

  2. [1]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018

    work page 2018

  3. [2]

    Online continual learning with maximal interfered retrieval

    Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In NeurIPS, 2019

    work page 2019

  4. [3]

    Expert gate: Lifelong learning with a network of experts

    Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with a network of experts. In CVPR, 2017

    work page 2017

  5. [4]

    Gradient based sample selection for online continual learning

    Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In NeurIPS, 2019

    work page 2019

  6. [5]

    Dark experience for general continual learning: a strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In NeurIPS, 2020. 10

    work page 2020

  7. [6]

    New insights on reducing abrupt representation change in online continual learning

    Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. arXiv preprint arXiv:2104.05025, 2021

    arXiv 2021

  8. [7]

    Riemannian walk for incremental learning: Understanding forgetting and intransigence

    Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, 2018

    work page 2018

Show all 63 references
  1. [8]

    Would deep generative models amplify bias in future models? arXiv preprint arXiv:2404.03242, 2024

    Tianwei Chen, Yusuke Hirota, Mayu Otani, Noa Garcia, and Yuta Nakashima. Would deep generative models amplify bias in future models? arXiv preprint arXiv:2404.03242, 2024

    2024 arXiv

  2. [9]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597–1607, 2020

    2020

  3. [10]

    Lifelong machine learning

    Zhiyuan Chen and Bing Liu. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12(3):1–207, 2018

    2018

  4. [11]

    IN100pytorch: Pytorch implementation: Training resnets on imagenet-100

    Yubei Chen Chun-Hsiao Yeh. IN100pytorch: Pytorch implementation: Training resnets on imagenet-100. https://github.com/danielchyeh/ImageNet-100-Pytorch , 2022

    2022

  5. [12]

    A continual learning survey: Defying forgetting in classification tasks

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. TPAMI, 44(7):3366–3385, 2021

    2021

  6. [13]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009

    2009

  7. [14]

    Pathnet: Evolution channels gradient descent in super neural networks

    Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734, 2017

    2017 arXiv

  8. [15]

    Vector quantized diffusion model for text-to-image synthesis

    Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In CVPR, pages 10696–10706, 2022

    2022

  9. [16]

    Online continual learning through mutual information maximization

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In ICML, pages 8109–8126, 2022

    2022

  10. [17]

    Dealing with cross-task class discrimination in online continual learning

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task class discrimination in online continual learning. In CVPR, pages 11878–11887, 2023

    2023

  11. [18]

    Will large-scale generative models corrupt future datasets? In ICCV, pages 20555–20565, 2023

    Ryuichiro Hataya, Han Bao, and Hiromi Arai. Will large-scale generative models corrupt future datasets? In ICCV, pages 20555–20565, 2023

    2023

  12. [19]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, volume 33, pages 6840–6851, 2020

    2020

  13. [20]

    Learning a unified classifier incrementally via rebalancing

    Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, pages 831–839, 2019

    2019

  14. [21]

    Selective experience replay for lifelong learning

    David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. In AAAI, volume 32, 2018

    2018

  15. [22]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In NeurIPS, 2020

    2020

  16. [23]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. PNAS, 114(13):3521–3526, 2017

    2017

  17. [24]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009

    2009

  18. [25]

    Tiny imagenet visual recognition challenge

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. 11

    2015

  19. [26]

    Overcoming catastrophic forgetting by incremental moment matching

    Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In NeurIPS, 2017

    2017

  20. [27]

    Forgery-aware adaptive transformer for generalizable synthetic image detection

    Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Jingdong Wang, and Yao Zhao. Forgery-aware adaptive transformer for generalizable synthetic image detection. In CVPR, pages 10770–10780, 2024

    2024

  21. [28]

    Towards understanding the interplay of generative artificial intelligence and the internet

    Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Marc Juarez, and Rik Sarkar. Towards understanding the interplay of generative artificial intelligence and the internet. In International Workshop on Epistemic Uncertainty in Artificial Intelligence, pag...

    2023

  22. [29]

    GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In ICML, volume 162, pages 16784–16804, 2022

    2022

  23. [30]

    Towards universal fake image detectors that generalize across generative models

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In CVPR, pages 24480–24489, 2023

    2023

  24. [31]

    Continual lifelong learning with neural networks: A review

    German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019

    2019

  25. [32]

    Learning to learn without forgetting by maximizing transfer and minimizing interference

    Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Ger- ald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11910, 2018

    arXiv

  26. [33]

    Experi- ence replay for continual learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experi- ence replay for continual learning. In NeurIPS, 2019

    2019

  27. [34]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, pages 10684–10695, 2022

    2022

  28. [35]

    Incremental learning through deep adaptation

    Amir Rosenfeld and John K Tsotsos. Incremental learning through deep adaptation. TPAMI, 42(3):651–663, 2018

    2018

  29. [36]

    Progressive neural networks

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016

    2016 arXiv

  30. [37]

    Overcoming catastrophic forgetting with hard attention to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML, 2018

    2018

  31. [38]

    Three scenarios for continual learning.arXiv preprint arXiv:1904.07734, 2019

    Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning.arXiv preprint arXiv:1904.07734, 2019

    1904 arXiv

  32. [39]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 9(11), 2008

    2008

  33. [40]

    Random sampling with a reservoir

    Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37–57, 1985

    1985

  34. [41]

    A comprehensive survey of continual learning: Theory, method and application

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00487, 2023

    2023 arXiv

  35. [43]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In ECCV, pages 631–648, 2022

    2022

  36. [44]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, pages 139–149, 2022. 12

    2022

  37. [45]

    Online prototype learning for online continual learning

    Yujie Wei, Jiaxin Ye, Zhizhong Huang, Junping Zhang, and Hongming Shan. Online prototype learning for online continual learning. In ICCV, pages 18764–18774, 2023

    2023

  38. [46]

    an image of a <class_name>

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, 2017. 13 Algorithm 1 PyTorch-like pseudo-code of ES. # model: continual learning model. # criteria(): loss function as in Eq. 5. # n_seen_so_far: count of images seen by ...

    2017

  39. [47]

    The partial augmentation is a weaker version of augmentation, consisting of random cropping with p = 0.5, followed by random horizontal flip with p = 0.5

    Partial strategy. The partial augmentation is a weaker version of augmentation, consisting of random cropping with p = 0.5, followed by random horizontal flip with p = 0.5

  40. [48]

    The full augmentation strategy is a stronger version of augmentation

    Full strategy. The full augmentation strategy is a stronger version of augmentation. The full augmentation strategy is a superset of its partial counterpart, which consists of random cropping, random horizontal flip, color jitter, and random grayscale. The parameters for color...

  41. [49]

    22 Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: All the main claims made in both the abstract and introduction can accurately reflect the paper’s contribution. 22 Gui...

  42. [50]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The limitation section is included in the appendix. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that...

  43. [51]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory Assumptions and Proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: This paper does not include theoretical results. Guidelines: • The answer NA means that the p...

  44. [52]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental Result Reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  45. [53]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: The code is included in the...

  46. [54]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental Setting/Details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: The main experimental setting...

  47. [55]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment Statistical Significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Error bars are reported in the experiments. Guidel...

  48. [56]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments Compute Resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: Detailed information is...

  49. [57]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code Of Ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The Code of Ethics is fully respected and obeyed in our research. Guidelines: •...

  50. [58]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: There is no societal impact of the work performed. Guidelines: • The answer NA means that there is no soci...

  51. [59]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper does ...

  52. [60]

    The licenses of existing assets are properly respected

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We have cited th...

  53. [61]

    Guidelines: • The answer NA means that the paper does not release new assets

    New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] 27 Justification: We have a Readme file along with our source code. Guidelines: • The answer NA means that the paper does not relea...

  54. [62]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and Research with Human Subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  55. [63]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.