Pith. sign in

REVIEW 5 major objections 6 minor 49 references

CCD: Continual Consistency Diffusion for Lifelong Generative Modeling

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that generative catastrophic forgetting in diffusion models has a principled fix: bounding inter-task disagreement by three consistency losses, then minimizing them.

desk verdict Useful empirical package and benchmark, but Theorem 1's proof is invalid: the central theoretical claim should be reframed as a heuristic. read the letter →

arxiv 2505.11936 v3 pith:2E3RCJDW submitted 2025-05-17 cs.LG cs.AI

Jingren Liu , Shuning Xu , Yun Wang , Zhong Ji , Xiangyu Chen This is my paper · ORCID
classification cs.LGcs.AI
keywords continualdiffusiongenerationgenerativecatastrophicforgettingmodelsrehearsalbufferscore-basedSDEconsistencyregularizationlifelongmodelingknowledgeretention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that generative catastrophic forgetting in diffusion models can be understood as a misalignment between task-specific reverse-time diffusion processes, and that it can be controlled by enforcing exactly three consistency constraints across tasks. The authors prove an upper bound stating that the discrepancy between two tasks' generative dynamics is bounded by a weighted sum of inter-task knowledge consistency, unconditional knowledge consistency, and prior knowledge consistency. They then build the Continual Consistency Diffusion (CCD) framework, which turns these three terms into training losses applied alongside a rehearsal buffer of real images. If the bound holds, continual diffusion training becomes a matter of minimizing these consistency objectives rather than borrowing heuristics from continual classification, and the result is a standardized way to evaluate and mitigate forgetting in lifelong generative modeling.

What carries the argument

The load-bearing object is the cross-task diffusion evolution bound of Theorem 1, derived by comparing two task-specific DDPM/SDE processes through a shared noisy state $x_t$. Its three terms are realized as losses: $L_{\mathrm{IKC}}$ uses a Bregman divergence with a curvature (Fisher-information) preconditioner that keeps the student noise estimator close to the frozen teacher; $L_{\mathrm{UKC}}$ aligns unconditional reverse-process means with a time-dependent weight $\bar{\alpha}_t^2/\bar{\beta}_t^2$ that emphasizes mid-diffusion phases; $L_{\mathrm{PKC}}$ aligns label-conditioned priors on original samples through a task-adaptive label regressor. The paper also introduces the CDG pipeline, a standardized rehearsal-buffer protocol with two metrics, mean fidelity and incremental mean fidelity, that make continual diffusion evaluation systematic.

What would settle it

On a standard DDPM trained on MNIST, estimate $p_t(x_0|x_t,y)$ and $p_t(x_0|y)$ at intermediate noise levels such as $t\approx 500$ and show that they differ materially; that difference invalidates the simplification in Equation 32 on which Theorem 1 rests.

Watch

Extended reading notes

Core claim

The paper's central discovery is Theorem 1: for two tasks with diffusion processes equivalent when conditioned on clean samples and labels, there exist positive constants $\kappa$, $\lambda$, $\eta$ such that the inter-task discrepancy is uniformly bounded by $$L_{\mathrm{UB}} = \kappa L_{\mathrm{IKC}} + \$\lambda$ L_{\mathrm{UKC}} + \eta L_{\mathrm{PKC}},$$ where $L_{\mathrm{IKC}}$ is the difference between the tasks' noise estimators, $L_{\mathrm{UKC}}$ penalizes differences between unconditional reverse-process means, and $L_{\mathrm{PKC}}$ penalizes divergence between label-conditional priors. The paper argues that minimizing this bound aligns the reverse-time diffusion gradients between tasks, preserving shared generative knowledge in a continual sequence. This turns the empirical goal of mitigating generative forgetting into a concrete optimization objective and directly motivates the three hierarchical losses in the CCD framework.

Load-bearing premise

The proof requires that, given a clean image and its label, adding noise to that image makes the label provide no further information about the clean image: the posterior of the clean image given the noisy image and the label must equal the posterior given only the label.

Editorial extensions

If this is right

  • In task sequences where tasks share visual structure, CCD should keep early-task FID close to the non-continual level, and its advantage should grow as the rehearsal buffer grows.
  • Parameter-regularization baselines and generative-replay approaches should underperform CCD because they do not align the score geometry captured by the three consistency terms.
  • CCD provides a theoretical justification for using direct sample replay rather than generative replay, since the consistency losses are defined on real buffered samples.
  • On tasks with minimal cross-task semantic overlap, the bound offers weak protection, so CCD's effectiveness should diminish on datasets like CIFAR100-10T, matching the paper's reported limitation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to measure, on real diffusion trajectories, how strongly the simplification $p_t(x_0|x_t,y)=p_t(x_0|y)$ is violated at mid-diffusion noise levels; if it is violated exactly where the loss weights peak, the theorem's derivation would need a corrected posterior term.
  • The same three consistency axes could be ported to latent or text-to-image diffusion models by replacing the label regressor with a text-embedding similarity measure, so that prior consistency constrains semantic drift of the prompt representation rather than class labels.
  • The bound also suggests a principled buffer-selection rule: choose rehearsal samples that minimize the prior-consistency divergence term rather than relying on diversity heuristics, which could improve behavior on coarse-grained datasets where buffered-sample quality matters most.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper addresses generative catastrophic forgetting in continual learning of diffusion models. It first proposes a standardized pipeline, CDG, for evaluating continual diffusion generation, then introduces a method called CCD that combines three consistency losses: inter-task knowledge consistency (LIKC), unconditional knowledge consistency (LUKC), and prior knowledge consistency (LPKC). The central claim is Theorem 1, which states that there exist positive constants kappa, lambda, eta such that an inter-task discrepancy LUB is uniformly bounded by kappa LIKC + lambda LUKC + eta LPKC; the authors use this theorem as the theoretical foundation for CCD. Experiments on MNIST, OxfordPets, CIFAR-100, Flowers102, and CUB-200 report improvements in MF and IMF metrics over several continual learning baselines, with ablations and qualitative visualizations.

Significance. If Theorem 1 were valid, CCD would be the first principled, theoretically grounded method for continual diffusion generation, and the CDG pipeline could serve as a useful standardized benchmark. The empirical study is extensive: five datasets, multiple buffer sizes, several baselines, ablations, hyperparameter sensitivity, and qualitative results. The paper also provides detailed hyperparameter configurations and an explicit description of the benchmark pipeline, which are strengths. However, the theoretical contribution is the centerpiece of the submission, and it is not sound as written; the proof relies on a false posterior-independence assumption and several unquantified simplifications. The empirical results may be useful independently, but they do not compensate for the failure of the main theoretical claim.

major comments (5)
  1. [Appendix, Eq. (32)] The proof of Theorem 1 rests on the assumption p_t(x0|xt,y)=p_t(x0|y) and q_t(x0|xt,y)=q_t(x0|y). This is false for diffusion posteriors: xt = alpha_bar_t x0 + beta_bar_t epsilon makes x0 and xt dependent, and p(x0|xt,y) is proportional to p(y|x0)p(x0)N(xt; alpha_bar_t x0, beta_bar_t^2 I), which depends on xt for every finite beta_bar_t. Under this assumption the expectation in Eq. (31) becomes independent of xt, so its xt-gradient is exactly zero and cannot generate the nonzero LUKC and LPKC terms claimed in Eqs. (37) and (44). Eq. (32) is the bridge between the cross-task score difference and the three losses, so Theorem 1 is not established.
  2. [Theorem 1 / Eq. (1)] As stated, Theorem 1 is tautological: LUB is defined as the right-hand side kappa LIKC + lambda LUKC + eta LPKC, so 'there exist positive constants' is true for any choice once the three losses are finite. No independent notion of inter-task discrepancy is bounded, no constants are specified, and no rate or uniformity in t is given. The theorem therefore provides no quantitative bound on generative forgetting, contrary to the abstract's claim of a theoretical foundation.
  3. [Appendix Eqs. (36)-(37)] The approximation gradient of mu minus gradient of nu approximately delta is introduced with delta called a 'small perturbation determined based on the shared knowledge,' but delta is never defined and the approximation is not derived from the preceding assumptions. Since this step is what converts the KL gradient into the time-weighted mean difference in Eq. (37), the LUKC loss is not a justified consequence of the analysis.
  4. [Appendix Eqs. (43)-(44)] The term ||beta_t epsilon^p_theta + epsilon||^2 is 'non-optimizable' and is declared a learnable scaling factor, but it depends on t and on the frozen previous-task model and therefore cannot be absorbed into a constant factor without changing the objective. The proportionality in Eq. (44) silently drops this term, so the LPKC objective is not equivalent to the prior-knowledge KL gradient derived above.
  5. [Appendix Eq. (34)] The 'Simplifiable Aspect' term gradient of log[q_t(y|xt)/p_t(y|xt)] is omitted on the informal ground that xt and y become independent under noise; no bound on this term is supplied. Because this term appears inside the claimed bound, dropping it without a quantitative estimate leaves Eq. (45) an inequality only under an additional unverified assumption.
minor comments (6)
  1. [Theorem 1, Eq. (2)] Eq. (2) in Theorem 1 defines LIKC as the vector difference epsilon^q_theta minus epsilon^p_theta, while the implemented loss in Eq. (7) is an expectation of a Bregman divergence; the theorem should use a norm or divergence to make LIKC a scalar loss.
  2. [Introduction] The paper alternates between CDG and CCD; the relationship between the 'Continual Diffusion Generation pipeline' and the 'Continual Consistency Diffusion' method should be stated once in a notation table.
  3. [Table 1] In Table 1 the 'Weighted Average' is said to be a normalized mean of MF and IMF using Non-CL scores, but the Non-CL row reports values 1.00 for both metrics while the other rows report values around 4-14; the normalization rule is not described precisely enough to reproduce.
  4. [Ablation Studies, Figure 2] Figure 2 reports hyperparameter sensitivity for kappa, lambda, eta but does not state which metric is on the y-axis, whether the other two hyperparameters are held at their optimal values, or how many seeds were averaged.
  5. [Visualization and Ablation Studies] There are several cross-reference errors: the visualization paragraph refers to 'Figure. 3' when the displayed images are in Figure 4, and the ablation subsection also refers to 'Figure. 3' for both the sensitivity and per-task ablation panels.
  6. [Perceptual Metrics, Table 4] The LPIPS results in Table 4 show essentially no improvement over ER, and the text attributes this to the UNet architecture; this should be discussed in the main paper rather than only in the appendix, since it tempers the claim of consistent generative-quality gains.

Circularity Check

3 steps flagged · score 7.0 of 10

Theorem 1 defines its own bound: LUB is the weighted sum of the losses, with constants fitted post hoc; the appendix's posterior-independence ansatz is the only bridge to the three-term decomposition.

  1. self definitional [Theorem 1 (Eq. 1); Appendix Eq. 45]
    "There exist constants {κ,λ,η} ⊂ R>0 such that the inter-task discrepancy is uniformly bounded: LUB = κLIKC + λLUKC + ηLPKC, (1) ... As shown above, the upper bound in Eq. 33 can be expressed as: LUB = κ(ϵqθ−ϵpθ)+λLUKC+ηLPKC."

    LUB is never defined as an independent inter-task discrepancy. Eq. 45 defines it as exactly the weighted sum of the three losses that the paper proposes (Eq. 46). The proof therefore does not bound a pre-existing quantity by the losses; it defines the 'bound' to be the losses. The existential constants make the statement vacuous: for any positive weights the equation holds as a definition. The claimed theoretical foundation is the loss formulation restated as a theorem.

  2. fitted input called prediction [Appendix Eq. 36 and Eq. 44; Eq. 45-46; Table 3; Fig. 2]
    "where κ,λ, and η are weighting hyperparameters that balance the contributions of three knowledge consistency components in the total upper bound loss LUB. ... δ represents a small perturbation determined based on the shared knowledge observed in Figure. 5. ... we consider this term as a learnable scaling factor that the network can adjust on its own."

    The constants κ, λ, η are not derived; Table 3 tunes them per dataset in the range 1e-5 to 1e-7 (Fig. 2). In the proof, an undefined δ (Eq. 36) and a 'non-optimizable' factor ∥βtϵp+ε∥² (Eq. 44) are absorbed into these tunable weights. Thus Eq. 1 is made true by fitting the weights, not by a predictive bound: the 'prediction' that the three losses bound cross-task discrepancy is reverse-engineered from the loss definition.

1 more flagged steps
  1. other [Appendix Eq. 32 with Eq. 18]
    "To achieve a simplified formulation, we employ Jensen’s inequality, assuming equivalence between the conditional distributions of the original and synthesized imagery, i.e., pt(x0|xt,y) = pt(x0|y) and qt(x0|xt,y) = qt(x0|y). This alignment facilitates the derivation of a computationally amenable lower bound through the interchange of the logarithmic operation with the expectation: ... xt = ¯αtx0 + ¯βtε, ε ∼ N(0,I)."

    The assumed equality is the bridge that converts the cross-task score gap into the three losses. But under the paper's own forward process (Eq. 18), xt is a deterministic linear function of x0 plus noise, so p(x0|xt,y) depends on xt for every finite β̄t; the equality is false. The derivation is thus not a consequence of diffusion dynamics but an assumption imposed to obtain the desired three-term decomposition. This is an unproved ansatz at the center of Theorem 1, not a derived bound.

full rationale

The central theoretical result is not an independent derivation. In Theorem 1, LUB is introduced as the 'inter-task discrepancy' to be bounded, but the appendix defines LUB as exactly κLIKC + λLUKC + ηLPKC, i.e., the weighted sum of the three losses the paper then implements. No quantity such as a score gap or divergence is shown to be bounded by that combination; the 'bound' is the loss by construction. The proof's bridge (Appendix Eq. 32) assumes p(x0|xt,y)=p(x0|y), which contradicts the paper's own forward process xt=ᾱt x0+β̄tε and is the step that lets the cross-task score difference be replaced by the three consistency terms. The constants κ, λ, and η are not specified by the theory; they are tuned per dataset over 1e-5 to 1e-7, and the derivation explicitly absorbs an undefined δ and a non-optimizable factor into these weights. Thus the 'there exist constants' claim is vacuous and the theoretical foundation reduces to the loss design rather than a first-principles prediction. The empirical results are self-contained and may show genuine improvement; this is partial circularity in the theory, not a fraud or a self-citation chain.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central derivation rests on several unproved or approximate assumptions. Three loss weights are fitted per dataset, and the theory absorbs an undefined perturbation and a non-optimizable factor. No new physical entities are introduced.

free parameters (5)
  • kappa (inter-task knowledge consistency weight) = 1e-5 to 1e-7, tuned per dataset
    Weight of LIKC in LUB; chosen via sensitivity analysis (Figure 2), not derived.
  • lambda (unconditional knowledge consistency weight) = 1e-5 to 1e-7, tuned per dataset
    Weight of LUKC in LUB; chosen via sensitivity analysis (Figure 2), not derived.
  • eta (prior knowledge consistency weight) = 1e-5 to 1e-7, tuned per dataset
    Weight of LPKC in LUB; chosen via sensitivity analysis (Figure 2), not derived.
  • delta (gradient alignment perturbation) = unspecified, effectively absorbed into lambda
    Introduced in Eq 36 to bridge the KL gradient and the mean difference; never specified, so it operates as a hidden fitted constant.
  • Time-dependent masking threshold for LUKC = t < 700
    Hand-selected cutoff in the appendix implementation details to emphasize mid-diffusion phases.
assumptions (6)
  • domain assumption Forward diffusion kernels are task-independent: pt(xt|x0,y)=qt(xt|x0,y) for all x0,y,t.
    Stated in Theorem 1. Reasonable for a shared noising schedule, but it assumes both tasks use identical dynamics.
  • domain assumption Gradient alignment of reverse-time mean functions: grad_xt mu(xt,t) approx grad_xt nu(xt,t).
    Borrowed from Yu et al. (2020) multi-task gradient surgery; used in Eq 36 to replace the KL gradient with a linear mean difference. Not guaranteed for streaming tasks with different labels.
  • ad hoc to paper Posterior independence of the noisy state: pt(x0|xt,y)=pt(x0|y) and qt(x0|xt,y)=qt(x0|y).
    Explicitly assumed in Eq 32 to simplify the derivation; generally false because x0 and xt are strongly dependent in diffusion. Load-bearing for the loss decomposition.
  • domain assumption Label y is independent of the diffused state xt.
    Used to drop grad_xt log qt(y|xt)/pt(y|xt) and to factor expectations; only approximately true at high noise levels.
  • domain assumption Reverse posterior p(x0|xt) is Gaussian with shared variance sigma_t^2.
    Used in Eq 36 and Eq 39 to reduce KL terms to mean differences; a standard approximation, but the exact posterior is not Gaussian.
  • ad hoc to paper The non-optimizable term ||beta_t eps_p_theta + eps||^2 can be treated as a learnable scaling factor.
    Declared in Eq 44 rather than derived, effectively absorbing an unmodeled factor into the loss weights.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CCD: Continual Consistency Diffusion for Lifelong Generative Modeling." pith.science (2026). https://pith.science/paper/2E3RCJDW

@misc{pith2026250511936,
  author       = {Pith},
  title        = {Pith review of: CCD: Continual Consistency Diffusion for Lifelong Generative Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2E3RCJDW}},
  note         = {Machine review of arXiv:2505.11936}
}
abstract

While diffusion-based models have shown remarkable generative capabilities in static settings, their extension to continual learning (CL) scenarios remains fundamentally constrained by Generative Catastrophic Forgetting (GCF). We observe that even with a rehearsal buffer, new generative skills often overwrite previous ones, degrading performance on earlier tasks. Although some initial efforts have explored this space, most rely on heuristics borrowed from continual classification methods or use trained diffusion models as ad hoc replay generators, lacking a principled, unified solution to mitigating GCF and often conducting experiments under fragmented and inconsistent settings. To address this gap, we introduce the Continual Diffusion Generation (CDG), a structured pipeline that redefines how diffusion models are implemented under CL and enables systematic evaluation of GCF. Beyond the empirical pipeline, we propose the first theoretical foundation for CDG, grounded in a cross-task analysis of diffusion-specific generative dynamics. Our theoretical investigation identifies three fundamental consistency principles essential for preserving knowledge in the rehearsal buffer over time: inter-task knowledge consistency, unconditional knowledge consistency, and prior knowledge consistency. These criteria expose the latent mechanisms through which generative forgetting manifests across sequential tasks. Motivated by these insights, we further propose \textit{Continual Consistency Diffusion} (CCD), a principled training framework that enforces these consistency objectives via hierarchical loss functions: $\mathcal{L}_{IKC}$, $\mathcal{L}_{UKC}$, and $\mathcal{L}_{PKC}$. Extensive experiments show that CCD achieves SOTA performance across various benchmarks, especially improving generative metrics in overlapping-task scenarios.

Figures

Figures reproduced from arXiv: 2505.11936 by the authors.

Figure 1
Figure 1. An overview of CDG pipeline and its challenges, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Hyperparameter sensitivity analysis on MNIST, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Ablation studies on MNIST-5T (ER buffer 5120). [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Borrowing from (Yu et al. 2020), during multi-task optimization, the gradients will eventually converge. Therefore, [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Overall architecture of the CCD framework. The backbone follows a U-Net-style encoder–decoder structure with [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Comparison of generated results in 0-th task of [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Comparison of generated results in the 0-th task of OxfordPets-5T. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Comparison of generated results in the 0-th task of CIFAR100-10T. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Comparison of generated results in the 0-th task of Flowers102-10T. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Comparison of generated results in the 0-th task of CUB200-10T. [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 23 canonical work pages

  1. [1]

    Cfts-gan: Continual few-shot teacher student for generative adversarial networks

    Munsif Ali, Leonardo Rossi, and Massimo Bertozzi. Cfts-gan: Continual few-shot teacher student for generative adversarial networks. In International Conference on Pattern Recognition, pages 249--262. Springer, 2025

    work page 2025

  2. [2]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision, pages 139--154, 2018

    work page 2018

  3. [3]

    Reverse-time diffusion equation models

    Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 0 (3): 0 313--326, 1982

    1982

  4. [4]

    Efficient lifelong learning with a-gem

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. In International Conference on Learning Representations, 2019

    work page 2019

  5. [5]

    Visual--semantic graph matching net for zero-shot learning

    Bowen Duan, Shiming Chen, Yufei Guo, Guo-Sen Xie, Weiping Ding, and Yisong Wang. Visual--semantic graph matching net for zero-shot learning. IEEE Transactions on Neural Networks and Learning Systems, 2024

    work page 2024

  6. [6]

    Association: Remind Your GAN not to Forget

    Yi Gu, Jie Li, Yuting Gao, Ruoxin Chen, Chentao Wu, Feiyang Cai, Chao Wang, and Zirui Zhang. Association: Remind your gan not to forget. arXiv preprint arXiv:2011.13553, 2020

    work page Pith review arXiv 2011

  7. [7]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  8. [8]

    Distilling the knowledge in a neural network

    Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    arXiv 2015

Show all 49 references
  1. [9]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33: 0 6840--6851, 2020

    2020

  2. [10]

    A unified semantic embedding: Relating taxonomies and attributes

    Sung Ju Hwang and Leonid Sigal. A unified semantic embedding: Relating taxonomies and attributes. Advances in Neural Information Processing Systems, 27, 2014

    2014

  3. [11]

    Generating instance-level prompts for rehearsal-free continual learning

    Dahuin Jung, Dongyoon Han, Jihwan Bang, and Hwanjun Song. Generating instance-level prompts for rehearsal-free continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11847--11857, 2023

    2023

  4. [12]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Scien...

    2017

  5. [13]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

    2009

  6. [14]

    Gradient-based learning applied to document recognition

    Yann LeCun, L \'e on Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998

    1998

  7. [15]

    Vs-boost: Boosting visual-semantic association for generalized zero-shot learning

    Xiaofan Li, Yachao Zhang, Shiran Bian, Yanyun Qu, Yuan Xie, Zhongchao Shi, and Jianping Fan. Vs-boost: Boosting visual-semantic association for generalized zero-shot learning. In IJCAI, pages 1107--1115, 2023

    2023

  8. [16]

    Learning without forgetting

    Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40 0 (12): 0 2935--2947, 2017

    2017

  9. [17]

    Parameter-efficient fine-tuning for continual learning: A neural tangent kernel perspective

    Jingren Liu, Zhong Ji, YunLong Yu, Jiale Cao, Yanwei Pang, Jungong Han, and Xuelong Li. Parameter-efficient fine-tuning for continual learning: A neural tangent kernel perspective. arXiv preprint arXiv:2407.17120, 2024

    2024

  10. [18]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in Neural Information Processing Systems, 30, 2017

    2017

  11. [19]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35: 0 5775--5787, 2022

    2022

  12. [20]

    Continual learning of diffusion models with generative distillation

    Sergi Masip, Pau Rodriguez, Tinne Tuytelaars, and Gido M van de Ven. Continual learning of diffusion models with generative distillation. arXiv preprint arXiv:2311.14028, 2023

    2023 arXiv

  13. [21]

    Dual adversarial semantics-consistent network for generalized zero-shot learning

    Jian Ni, Shanghang Zhang, and Haiyong Xie. Dual adversarial semantics-consistent network for generalized zero-shot learning. Advances in neural information processing systems, 32, 2019

    2019

  14. [22]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162--8171. PMLR, 2021

    2021

  15. [23]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722--729. IEEE, 2008

    2008

  16. [24]

    Continual lifelong learning with neural networks: A review

    German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113: 0 54--71, 2019

    2019

  17. [25]

    Continual learning with fully probabilistic models

    Benedikt Pf \"u lb, Alexander Gepperth, and Benedikt Bagus. Continual learning with fully probabilistic models. arXiv preprint arXiv:2104.09240, 2021

    2021 arXiv

  18. [26]

    Experience replay for continual learning

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  19. [27]

    Learning to approximate a bregman divergence

    Ali Siahkamari, Xide Xia, Venkatesh Saligrama, David Casta \ n \'o n, and Brian Kulis. Learning to approximate a bregman divergence. Advances in Neural Information Processing Systems, 33: 0 3603--3612, 2020

    2020

  20. [28]

    Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning

    James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEE...

    2023

  21. [29]

    A closer look at rehearsal-free continual learning

    James Seale Smith, Junjiao Tian, Shaunak Halbe, Yen-Chang Hsu, and Zsolt Kira. A closer look at rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2410--2420, 2023 b

    2023

  22. [30]

    Continual diffusion: Continual customization of text-to-image diffusion with c-lora

    James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual diffusion: Continual customization of text-to-image diffusion with c-lora. arXiv preprint arXiv:2304.06027, 2024

    2024 arXiv

  23. [31]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  24. [32]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021

    2021

  25. [33]

    Create your world: Lifelong text-to-image diffusion

    Gan Sun, Wenqi Liang, Jiahua Dong, Jun Li, Zhengming Ding, and Yang Cong. Create your world: Lifelong text-to-image diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  26. [34]

    Cam-gan: Continual adaptation modules for generative adversarial networks

    Sakshi Varshney, Vinay Kumar Verma, PK Srijith, Lawrence Carin, and Piyush Rai. Cam-gan: Continual adaptation modules for generative adversarial networks. Advances in Neural Information Processing Systems, 34: 0 15175--15187, 2021

    2021

  27. [35]

    A connection between score matching and denoising autoencoders

    Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23 0 (7): 0 1661--1674, 2011

    2011

  28. [36]

    The Caltech-UCSD Birds-200-2011 Dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 Dataset . Technical Report CNS-TR-2011-001, California Institute of Technology, 2011

    2011

  29. [37]

    Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality

    Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, and Jun Zhu. Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality. Advances in Neural Information Processing Systems, 36, 2024

    2024

  30. [38]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning

    Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. Advances in Neural Information Processing Systems, 35: 0 5682--5695, 2022 a

    2022

  31. [39]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pages 631--648. Spring...

    2022

  32. [40]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 139--149, 2022 c

    2022

  33. [41]

    Lifelong twin generative adversarial networks

    Fei Ye and Adrian G Bors. Lifelong twin generative adversarial networks. In IEEE International Conference on Image Processing, pages 1289--1293. IEEE, 2021 a

    2021

  34. [42]

    Lifelong teacher-student network learning

    Fei Ye and Adrian G Bors. Lifelong teacher-student network learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44 0 (10): 0 6280--6296, 2021 b

    2021

  35. [43]

    Gradient surgery for multi-task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33: 0 5824--5836, 2020

    2020

  36. [44]

    Continual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987--3995. PMLR, 2017

    2017

  37. [45]

    Clog: Benchmarking continual learning of image generation models

    Haotian Zhang, Junting Zhou, Haowei Lin, Hang Ye, Jianhua Zhu, Zihao Wang, Liangcai Gao, Yizhou Wang, and Yitao Liang. Clog: Benchmarking continual learning of image generation models. arXiv preprint arXiv:2406.04584, 2024 a

    2024 arXiv

  38. [46]

    S3a: Towards realistic zero-shot classification via self structural semantic alignment

    Sheng Zhang, Muzammal Naseer, Guangyi Chen, Zhiqiang Shen, Salman Khan, Kun Zhang, and Fahad Shahbaz Khan. S3a: Towards realistic zero-shot classification via self structural semantic alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages...

    2024

  39. [47]

    Invertavatar: Incremental gan inversion for generalized head avatars

    Xiaochen Zhao, Jingxiang Sun, Lizhen Wang, Jinli Suo, and Yebin Liu. Invertavatar: Incremental gan inversion for generalized head avatars. In ACM SIGGRAPH 2024 Conference Papers, pages 1--10, 2024

    2024

  40. [48]

    Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics

    Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. Advances in Neural Information Processing Systems, 36: 0 55502--55542, 2023

    2023

  41. [49]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23554--23564, 2024

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.