Pith. sign in

REVIEW 3 major objections 5 minor 66 references

Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Images

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that teacher-guided pseudo-replay plus feature- and covariance-distillation lets medical image classifiers add new tasks without storing old data, reporting final-task accuracy of 64.43% on OCT and 67.23% on CIFAR-10.

desk verdict A privacy-preserving continual learning recipe with a novel component mix, but two load-bearing inconsistencies--the covariance loss is not an alignment term and the OCT protocol makes triplet loss undefined--make the results unreproducible as stated. read the letter →

arxiv 2504.20033 v1 pith:ZCIVQPE3 submitted 2025-04-28 cs.CV

Sara Yavari , Jacob Furst This is my paper · ORCID
classification cs.CV
keywords catastrophicforgettingincrementallearningknowledgedistillationmedicalimageclassificationgenerativereplayvariationalautoencodertripletlosscovarianceregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that an image classifier can keep learning new medical-imaging tasks without forgetting earlier ones, even when no real examples from past tasks are stored or replayed. The recipe is to train a shallow variational autoencoder against the fixed previous model so it generates synthetic images that mimic the old task's distribution, then to train the new model on the current task while distilling the previous model's normalized feature maps and embedding covariance structure. The authors report that this keeps accuracy high across tasks on prostate MRI, OCT, and histopathology data and on CIFAR-10, with final-task average accuracy of 64.43% on OCT and 67.23% on CIFAR-10, above the compared continual-learning baselines on those two datasets. If the claim holds, it offers a privacy-compatible way for medical centers to update models incrementally without exchanging or storing patient images.

What carries the argument

The central object is the distillation loss $L_{KD}=L_{FAM}+L_{Cov}$. $L_{FAM}$ is an $L^2$-norm feature-attention matching term that compares normalized feature maps at every convolutional layer except the embedding layer, so the Student keeps the Teacher's spatial and structural patterns. $L_{Cov}$ is a Barlow-Twins-style covariance regularizer: it penalizes the squared off-diagonal coefficients of the embedding covariance matrices, written as $L_{Cov}=C(Z_k)+C(Z_{k-1})$, where $C(Z)=\frac{1}{n-1}\sum_i (z_i-m)(z_i-m)^t$. A shallow VAE supplies the pseudo-replay data; its generator is trained adversarially to maximize $D_E(M_k(x_g),M_{k-1}(x_g))$, pushing the synthetic images toward the previous task's distribution. The triplet loss on real current-task data, together with a nearest-class-mean classifier, supplies the metric-learning backbone that keeps new classes separable and discriminative.

What would settle it

Track the cross-covariance between Teacher and Student embeddings, $\frac{1}{n-1}\sum_i (z_i^k-m_k)(z_i^{k-1}-m_{k-1})^\top$, during training on a fixed task sequence. If this quantity does not respond while $L_{Cov}=C(Z_k)+C(Z_{k-1})$ decreases, the loss is not aligning the two models' covariances; equivalently, replace $L_{Cov}$ by $C(Z_k)$ alone and check whether final-task accuracy changes. If replacing it leaves accuracy essentially unchanged, the claimed teacher-student covariance mechanism is not the active ingredient.

Watch

Extended reading notes

Core claim

The paper's central claim is that catastrophic forgetting in zero-shot class-incremental learning can be mitigated without using any real data from past tasks by combining two mechanisms: (1) a lightweight VAE generator trained adversarially against the fixed previous model produces synthetic images $\hat{D}_k$ that approximate the previous task's distribution; (2) a knowledge-distillation loss $L_{KD}=L_{FAM}+L_{Cov}$ transfers the previous model's representation to the current one. The feature-attention term $L_{FAM}$ aligns $L^2$-normalized feature maps across intermediate layers, and the covariance term $L_{Cov}$ penalizes off-diagonal entries of the embedding covariance matrices of the Teacher and the Student. Training uses the total loss $L_{M_k}=L_{tri}(x_k)+\lambda L_{KD}(x_g)+D_E(M_k,M_{k-1})$. On the PI-CAI prostate MRI dataset the method reaches 68.73% average accuracy versus a non-incremental upper bound of 83.21% and a fine-tuning lower bound of 26.25%; on OCT and CIFAR-10 it reports the best final-task average accuracy among the baselines (64.43% and 67.23%). On PathMNIST it reaches 53.75%, below the EFT baseline's 66.82%, a gap the paper attributes to that baseline's dynamic architecture. If these results hold, the method provides a privacy-compatible route to continual medical image analysis.

Load-bearing premise

The load-bearing assumption is that the covariance term, written as the sum of each model's own covariance penalty, actually transfers the previous model's feature-correlation structure to the current model; if it does not couple the two models' correlations, the reported forgetting protection may come from elsewhere in the training objective.

Editorial extensions

If this is right

  • Medical-image models could be updated across new sites or disease categories without retaining patient scans, lowering storage and privacy barriers.
  • On the PI-CAI prostate MRI benchmark, the reported 68.73% average accuracy sits between the 83.21% joint-training upper bound and the 26.25% fine-tuning lower bound, indicating that most old-task knowledge can survive the transition.
  • Ablation results imply both distillation terms matter: using either feature-attention matching or covariance regularization alone drops OCT accuracy from 64.43% to roughly 47–50% and CIFAR-10 accuracy from 67.23% to roughly 44–46%.
  • Because the same framework improves a natural-image benchmark (CIFAR-10), the mechanism is not tied to medical-image statistics and may transfer to other class-incremental learning settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This goes beyond the paper: as written, $L_{Cov}$ is a sum of two within-model penalties with no explicit cross-model term; an explicit cross-covariance alignment term would be a sharper test and a plausible refinement of the stated mechanism.
  • This goes beyond the paper: the pseudo-replay design could be carried over to segmentation or detection tasks, where preserving past distributions without storing original annotations is even more valuable.
  • This goes beyond the paper: the PathMNIST result suggests the advantage of synthetic replay may shrink when tasks have many classes or strong visual similarity; varying class counts and task order would map where the method helps.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a class-incremental learning method for medical image classification. For each new task, the student model is trained with triplet loss on the current real data and a knowledge-distillation loss applied to synthetic images of previous tasks generated by a shallow VAE. The distillation loss combines feature-attention matching (Eq. 4) and a covariance term (Eq. 7) that the text describes as aligning teacher and student embedding statistics. The method is evaluated on PI-CAI, OCT, PathMNIST, and CIFAR-10, and the authors report that it outperforms baselines on OCT and CIFAR-10 while avoiding storage of real past samples.

Significance. The target problem is relevant: privacy-preserving continual learning for medical imaging has practical value, and the proposed pipeline (VAE-based pseudo-replay plus distillation) is a plausible design direction. The paper evaluates on multiple datasets, includes a comparison with several baselines, and provides an ablation of the two distillation terms. However, the central mechanism as written is not what is claimed: Eq. (7) is not a teacher-student covariance alignment term, and the OCT protocol contains single-class tasks for which the only current-task loss (triplet loss) is undefined. These are load-bearing inconsistencies, not presentation issues, so the empirical claims are not currently supported.

major comments (3)
  1. [Section 3.2.2, Eq. (7)] The covariance loss as written is LCov = C(Zk) + C(Zk-1), a sum of two d x d covariance matrices, yet LKD and LMk in Eq. (1) are scalars. If the intended scalar is c(Zk) + c(Zk-1) using Eq. (6), then the loss only pushes the off-diagonal entries of each model's embedding covariance toward zero independently; there is no cross-model term comparing C(Zk) with C(Zk-1). The text and Figure 2 state that the loss implements covariance alignment between Teacher and Student, but the equations express no such alignment. The claimed mechanism for preserving previous knowledge through aligned feature correlations is therefore not implemented by the given loss.
  2. [Section 4.1, OCT benchmark] The OCT protocol assigns task 2 to the single class DME and task 3 to the single class Drusen. The triplet loss Ltri in Eq. (1) and Algorithm 1 requires an anchor, a positive, and a negative from a different class, so with a one-class task it is undefined or identically zero for those tasks. The student would receive no gradient from the current real images, yet Table 2 reports 64.43% average OCT accuracy. This is an internal contradiction in the central experimental evidence and makes the OCT results unreproducible from the described algorithm.
  3. [Section 3.2.1, Eq. (2) and Algorithm 1] The generator objective is described inconsistently. The text says the generator minimizes LG in ng iterations, with LG = -DE(Mk(xg), Mk-1(xg)); gradient descent on this LG maximizes the teacher-student distance, which is an adversarial objective, not one that 'adheres closely' to the teacher's distribution. It is also unclear whether the generator update is performed once per epoch over all tasks or per mini-batch. Clarify the exact optimization roles of the generator, student, and teacher, because this determines whether the pseudo-replay mechanism is actually as claimed.
minor comments (5)
  1. [Abstract] The abstract describes the work solely in terms of PI-CAI prostate MRI detection, but the experiments also cover OCT, PathMNIST, and CIFAR-10; please align the abstract with the actual scope.
  2. [Eq. (5) and Section 4.1] The symbol n is used both for the number of samples in the covariance estimate in Eq. (5) and for the KD mini-batch size in the implementation details; this makes the formulas ambiguous.
  3. [Eq. (6)] 'The adding of the squared-off diagonal coefficients' should read 'the sum of the squared off-diagonal coefficients'; the current phrasing is a typo.
  4. [Tables 2 and 3] The tables report averages over ten trials but no standard deviations or statistical significance tests, which makes it hard to judge whether the differences from the nearest baselines are reliable.
  5. [Section 4.1, PI-CAI] The PI-CAI experiment is reported only as a single number in Table 1 without comparison to other IL baselines, so the claim that the method is effective on MRI data is not supported by the same evidence as the other datasets.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports empirical comparisons against external baselines and does not derive its results from a self-referential or definitionally circular chain.

full rationale

The paper's central evidence is a set of empirical accuracy comparisons (Tables 1-3) against external baselines (LwF, GR, RWalk, OWM, EFT, BIR) on public datasets. The proposed method is described by explicit loss equations (Eqs. 1-7), and the reported results are not obtained by substituting the target accuracies into those equations. There is no fitted parameter that is later renamed as a prediction: the hyperparameters (lambda = 0.8, n_g = 3, n_s = 20) are stated as implementation choices rather than fit to the reported outcome, and no subset of the accuracy numbers is used to construct the loss. The paper does not invoke a uniqueness theorem or any load-bearing self-citation; the related work is contextual, and the only in-paper citation used for the method's inspiration, Micaelli and Storkey [32], is external. The noted inconsistency in Eq. (7), where LCov = C(Z_k) + C(Z_{k-1}) is described as covariance alignment between Teacher and Student but contains no cross-model comparison term, is a correctness or reproducibility concern about whether the loss does what the text claims, not a circularity concern: the result is not equivalent to its input by definition. Similarly, the difficulty of forming triplets in the one-class OCT tasks would affect whether the stated loss can be evaluated, but this is an internal consistency issue rather than a circular derivation. Because the paper is self-contained against external benchmarks and no central claim reduces to a fit or to a self-citation chain, the appropriate circularity score is 0.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters, an unvalidated adversarial generator assumption, an asserted but undemonstrated covariance-alignment behavior, and the implicit assumption that triplet loss is applicable to all benchmark tasks. No new physical entities are introduced.

free parameters (7)
  • lambda (KD loss weight) = 0.8
    Hand-chosen in Eq. 1; no sensitivity analysis is reported.
  • Generator and student iteration counts (n_g, n_s) = 3 and 20
    Set manually in Section 4.1; controls how much synthetic data is used and is not varied.
  • Triplet margin m
    Introduced in the triplet loss definition in Section 3.1.1; the value is never stated, so the loss is under-specified.
  • Embedding dimension d = 512
    Set by the ResNet-18 embedding and used in the covariance scaling of Eq. 6.
  • Learning rates eta_g and eta_s = 0.001 and 0.00001
    Hand-chosen optimizer settings reported in Section 4.1.
  • VAE latent dimension z = 100
    Generator architecture choice reported in Section 4.1.
  • Batch sizes b and n = 64 and 16
    Hand-chosen in Section 4.1; determines the ratio of synthetic to real images used in each update.
assumptions (4)
  • ad hoc to paper The adversarial generator loop in Eq. 2, with n_g generator updates against n_s student updates, converges to synthetic images that match the past-task distribution closely enough for distillation.
    Section 3.2.1 and Algorithm 1 assume this minimax procedure works without any convergence analysis or validation of the generated images.
  • ad hoc to paper Applying Barlow's off-diagonal penalty c(Z) to each model's embedding covariance separately keeps the student's feature correlations aligned with the teacher's.
    Section 3.2.2, Eqs. 6-7: the text asserts this preserves previous knowledge, but Eq. 7 sums two independent covariance penalties and contains no teacher-student cross term.
  • domain assumption Every task has at least two classes so that triplet loss can be formed.
    Section 4.1 defines OCT tasks 2 and 3 as one-class (only DME, only Drusen), while L_tri requires anchor, positive, and negative samples; the paper does not explain how the loss is computed.
  • standard math Standard neural network optimization with Euclidean distances in the embedding space is sufficient for the proposed losses.
    Used throughout the method; not controversial but relies on standard training assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Images." pith.science (2026). https://pith.science/paper/ZCIVQPE3

@misc{pith2026250420033,
  author       = {Pith},
  title        = {Pith review of: Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZCIVQPE3}},
  note         = {Machine review of arXiv:2504.20033}
}
read the original abstract

This paper proposes an Incremental Learning (IL) approach to enhance the accuracy and efficiency of deep learning models in analyzing T2-weighted (T2w) MRI medical images prostate cancer detection using the PI-CAI dataset. We used multiple health centers' artificial intelligence and radiology data, focused on different tasks that looked at prostate cancer detection using MRI (PI-CAI). We utilized Knowledge Distillation (KD), as it employs generated images from past tasks to guide the training of models for subsequent tasks. The approach yielded improved performance and faster convergence of the models. To demonstrate the versatility and robustness of our approach, we evaluated it on the PI-CAI dataset, a diverse set of medical imaging modalities including OCT and PathMNIST, and the benchmark continual learning dataset CIFAR-10. Our results indicate that KD can be a promising technique for IL in medical image analysis in which data is sourced from individual health centers and the storage of large datasets is not feasible. By using generated images from prior tasks, our method enables the model to retain and apply previously acquired knowledge without direct access to the original data.

Figures

Figures reproduced from arXiv: 2504.20033 by the authors.

Figure 1
Figure 1. a) Separable features. b) Discriminative features. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview diagram of the proposed framework: The triplet loss [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Ak (%) for the ablation study conducted on two datasets. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 57 canonical work pages

  1. [1]

    Aljundi, E

    R. Aljundi, E. Belilovsky, T. Tuytelaars, L. Charlin, M. Caccia, M. Lin, and L. Page-Caccia. Online continual learning with maximal inter-task interference. InAdvances in Neural Information Processing Systems (NeurIPS), 2019

    work page 2019

  2. [2]

    Bayasi, G

    N. Bayasi, G. Hamareh, and R. Garbi. Gc2: Generalizable continual classification of medical images. IEEE Transactions on Medical Imaging, 2024

    work page 2024

  3. [3]

    Celin and K

    S. Celin and K. Vasanth. Ecg signal classification using various machine learning techniques. Journal of Medical Systems, 42(12):1–11, 2018

    work page 2018

  4. [4]

    Chaudhry, N

    A. Chaudhry, N. Khan, P. K. Dokania, and P. H. Torr. Continual learning in low-rank orthogonal subspaces. In Advances in Neural Information Processing Systems (NeurIPS) , 2020

    work page 2020

  5. [5]

    Chaudhry, M

    A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with a-gem. In Proceedings of the International Conference on Learning Representations (ICLR) , 2018

    work page 2018

  6. [6]

    Chaudhry, M

    A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with a-gem. In Proceedings of the International Conference on Learning Representations (ICLR) , 2019

    work page 2019

  7. [7]

    Chaudhry, M

    A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. K. Dokania, P. H. Torr, and M. Ranzato. Continual learning with tiny episodic memories. arXiv preprint arXiv:1902.10486, 2019

    arXiv 1902

  8. [8]

    Riemannian walk for incremental learning: Understanding forgetting and intransigence

    Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pages 532–547, 2018

    2018

Show all 66 references
  1. [9]

    Cheraghian, S

    A. Cheraghian, S. Rahman, P. Fang, S. K. Roy, L. Petersson, and M. Harandi. Semantic-aware knowl- edge distillation for few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2534–2543. IEEE, 2021

    2021

  2. [10]

    D. Deng, G. Chen, J. Hao, Q. Wang, and Q. Hao. Flattening sharpness for dynamic gradient projection memory benefits continual learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  3. [11]

    P. Dhar, R. Singh, K.-C. Peng, Z. Wu, and R. Chellappa. Learning without memorizing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5138–5146, 2019

    2019

  4. [12]

    Robert M. French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences , 3(4):128–135, 1999

    1999

  5. [13]

    J. Gou, B. Yu, S. J. Maybank, and D. Tao. Knowledge distillation: A survey. arXiv preprint arXiv:2006.05525v7, May 2021

    2006 arXiv

  6. [14]

    Softmax dissection: Towards understand- ing intra- and inter-class objective for embedding learning

    Lanqing He, Zhongdao Wang, Yali Li, and Shengjin Wang. Softmax dissection: Towards understand- ing intra- and inter-class objective for embedding learning. arXiv preprint arXiv:1908.01281, 2019

    1908 arXiv

  7. [15]

    Joseph, S

    K. Joseph, S. Khan, F. S. Khan, R. M. Anwer, and V . N. Balasubramanian. Energy-based latent aligner for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7452–7461, 2022. 10

    2022

  8. [16]

    S. Jung, H. Ahn, S. Cha, and T. Moon. Continual learning with node-importance based adaptive group sparse regularization. In Advances in Neural Information Processing Systems (NeurIPS) , 2020

    2020

  9. [17]

    H. Kang, R. J. L. Mina, S. R. H. Madjid, J. Yoon, M. Hasegawa-Johnson, S. J. Hwang, and C. D. Yoo. Forget-free continual learning with winning subnetworks. In Proceedings of the International Conference on Machine Learning (ICML) , pages 10734–10750. PMLR, 2022

    2022

  10. [18]

    H. Kang, J. Yoon, S. R. Madjid, S. J. Hwang, and C. D. Yoo. Forget-free continual learning with soft-winning subnetworks. arXiv preprint arXiv:2303.14962, 2023

    2023 arXiv

  11. [19]

    H. Kang, J. Yoon, S. R. H. Madjid, S. J. Hwang, and C. D. Yoo. On the soft-subnetwork for few-shot class incremental learning. arXiv preprint arXiv:2209.07529, 2022

    2022 arXiv

  12. [20]

    Class-incremental learning by knowledge distillation with adaptive feature consolidation

    Minsoo Kang, Jaeyoo Park, and Bohyung Han. Class-incremental learning by knowledge distillation with adaptive feature consolidation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022

    2022

  13. [21]

    Large dataset of labeled optical coherence tomography (oct) and chest x-ray images, 2018

    Daniel Kermany, Kang Zhang, and Michael Goldbaum. Large dataset of labeled optical coherence tomography (oct) and chest x-ray images, 2018

    2018

  14. [22]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting i...

    2017

  15. [23]

    Krizhevsky

    A. Krizhevsky. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto, 2009

    2009

  16. [24]

    Kumar and H

    A. Kumar and H. Daum´e III. Learning task grouping and overlap in multi-task learning. InProceedings of the International Conference on Machine Learning (ICML) , 2012

    2012

  17. [25]

    A continual learning survey: Defying forgetting in classification tasks

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale ˇs Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(7):3366–...

    2022

  18. [26]

    Continual learning with knowledge distillation: A survey

    Songze Li, Tonghua Su, Xu-Yao Zhang, and Zhongjie Wang. Continual learning with knowledge distillation: A survey. IEEE Transactions on Neural Networks and Learning Systems , 2024

    2024

  19. [27]

    Li and D

    Z. Li and D. Hoiem. Learning without forgetting. In Proceedings of the European Conference on Computer Vision (ECCV), 2016

    2016

  20. [29]

    Li and D

    Z. Li and D. Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935–2947, 2018

    2018

  21. [30]

    Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A. W. M. van der Laak, Bram van Ginneken, and Clara I. S´anchez. A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60–88, 2017

    2017

  22. [31]

    McCloskey and N

    M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation , 24:109–165, 1989. 11

    1989

  23. [32]

    Zero-shot knowledge transfer via adversarial belief matching

    Paul Micaelli and Amos J Storkey. Zero-shot knowledge transfer via adversarial belief matching. Advances in Neural Information Processing Systems , 32, 2019

    2019

  24. [33]

    S. I. Mirzadeh, M. Farajtabar, D. Gorur, R. Pascanu, and H. Ghasemzadeh. Linear mode connectivity in multitask and continual learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  25. [34]

    X. Ning, W. Tian, Z. Yu, W. Li, X. Bai, and Y . Wang. Hcfnn: High-order coverage function neural network for image classification. Pattern Recognition, 60, 2022

    2022

  26. [35]

    G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019

    2019

  27. [36]

    Parisi, Ronald Kemker, Jose L

    German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Lifelong learning of human actions with deep neural network self-organization. Neural Networks, 110:1–30, 2019

    2019

  28. [37]

    Perkonigg, J

    M. Perkonigg, J. Hofmanninger, C. Herold, H. Prosch, and G. Langs. Continual active learning using pseudo-domains for limited labelling resources and changing acquisition characteristics. In Machine Learning for Biomedical Imaging , 2022

    2022

  29. [38]

    Patrikakis, and Petros Daras

    Athanasios Psaltis, Christos Chatzikonstantinou, Charalampos Z. Patrikakis, and Petros Daras. Fe- drcil: Federated knowledge distillation for representation-based contrastive incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision Works...

    2021

  30. [39]

    Romero, N

    A. Romero, N. Ballas, S. Ebrahimi Kahou, A. Chassang, C. Gatta, and Y . Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  31. [40]

    K. Roy, P. Moghadam, and M. Harandi. L3dmc: Lifelong learning using distillation via mixed- curvature space. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 123–133, 2023

    2023

  32. [41]

    A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016

    2016 arXiv

  33. [42]

    Bosma, Jasper J

    Anindo Saha, Joeran S. Bosma, Jasper J. Twilt, Bram van Ginneken, Anders Bjartell, Anwar R. Pad- hani, et al. Artificial intelligence and radiologists in prostate cancer detection on mri (pi-cai): An international, paired, non-inferiority, confirmatory study. The Lancet Oncolo...

    2024

  34. [43]

    G. Saha, I. Garg, and R. Roy. Gradient projection memory for continual learning. In Proceedings of the International Conference on Learning Representations (ICLR) , 2021

    2021

  35. [44]

    Facenet: A unified embedding for face recognition and clustering

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 815–823. IEEE, 2015

    2015

  36. [45]

    H. Shin, J. K. Lee, J. Kim, and J. Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems (NeurIPS) , 2017

    2017

  37. [46]

    Continual learning with deep generative replay

    Hanul Shin, Jung Kwon Lee, Jaehon Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems (NeurIPS) , 2017. 12

    2017

  38. [47]

    J. Si, W. Jia, and H. Jiang. Breast cancer histopathology images classification through multi-view augmented contrastive learning and pre-learning knowledge distillation. IEEE Access, 2024

    2024

  39. [48]

    Ilcoc: An incremental learn- ing framework based on contrastive one-class classifiers

    Wenju Sun, Jing Zhang, Danyu Wang, Yangli-ao Geng, and Qingyong Li. Ilcoc: An incremental learn- ing framework based on contrastive one-class classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3580–3588. IEEE, 2021

    2021

  40. [49]

    Deep learning face representation by joint identification-verification

    Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation by joint identification-verification. In Advances in Neural Information Processing Systems (NeurIPS) , vol- ume 27. NeurIPS, 2014

    2014

  41. [50]

    R. Tang, Y . Lu, L. Liu, L. Mou, O. Vechtomova, and J. Lin. Distilling task-specific knowledge from bert into simple neural networks. arXiv preprint arXiv:1903.12136, 2019

    1903 arXiv

  42. [51]

    M. K. Titsias, J. Schwarz, A. G. de Matthews, R. Pascanu, and Y . W. Teh. Functional regularisation for continual learning with gaussian processes. In Proceedings of the International Conference on Learning Representations (ICLR), 2020

    2020

  43. [52]

    Van De Ven, Zhe Li, and Andreas S

    Gido M. Van De Ven, Zhe Li, and Andreas S. Tolias. Class-incremental learning with generative classifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3611–3620. IEEE, 2021

    2021

  44. [53]

    Brain-inspired replay for continual learning with artificial neural networks

    Gido M Van de Ven, Hava T Siegelmann, and Andreas S Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature Communications, 11(1):4069, 2020

    2020

  45. [54]

    Cam-gan: Con- tinual adaptation modules for generative adversarial networks

    Sakshi Varshney, Vinay Kumar Verma, Srijith P K, Lawrence Carin, and Piyush Rai. Cam-gan: Con- tinual adaptation modules for generative adversarial networks. Department of Computer Science and Engineering, IIT Hyderabad, Duke University, IIT Kanpur , 2023

    2023

  46. [55]

    Ting, et al

    Tanvi Verma, Liyuan Jin, Jun Zhou, Jia Huang, Mingrui Tan, Benjamin Chen Ming Choong, Ting Fang Tan, Fei Gao, Xinxing Xu, Daniel S. Ting, et al. Privacy-preserving continual learning methods for medical image classification: A comparative analysis. Frontiers in Medicine, 10:12...

    2023

  47. [56]

    Efficient feature transformations for discriminative and generative continual learning

    Vinay Kumar Verma, Kevin J Liang, Nikhil Mehta, Piyush Rai, and Lawrence Carin. Efficient feature transformations for discriminative and generative continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 13865–13875, 2021

    2021

  48. [57]

    L. Wang, M. Zhang, Z. Jia, Q. Li, C. Bao, K. Ma, J. Zhu, and Y . Zhong. Afec: Active forgetting of negative transfer in continual learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pages 22379–22391, 2021

    2021

  49. [58]

    A discriminative feature learning approach for deep face recognition

    Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A discriminative feature learning approach for deep face recognition. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII , pages 499–515. Springer, 2016

    2016

  50. [59]

    Wortsman, V

    M. Wortsman, V . Ramanujan, R. Liu, A. Kembhavi, M. Rastegari, J. Yosinski, and A. Farhadi. Super- masks in superposition. In Advances in Neural Information Processing Systems (NeurIPS) , 2020

    2020

  51. [60]

    H. Wu, Z. Wang, Z. Zhao, C. Chen, and J. Qin. Continual nuclei segmentation via prototype-wise relation distillation and contrastive learning. IEEE Transactions on Medical Imaging, 2023

    2023

  52. [61]

    Xu and Z

    J. Xu and Z. Zhu. Reinforced continual learning. In Advances in Neural Information Processing Systems (NeurIPS), 2018. 13

    2018

  53. [62]

    J. Yang, R. Shi, and B. Ni. Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) , pages 191–195. IEEE, 2021

    2021

  54. [63]

    J. Yim, D. Joo, J. Bae, and J. Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    2017

  55. [64]

    D. Yu, H. Tan, S. Chen, and K. Huang. Learning face representation from scratch. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017

    2017

  56. [65]

    K. Yue, J. Deng, and F. Zhou. Matching guided distillation. In Proceedings of the 16th European Conference on Computer Vision (ECCV), pages 312–328. Springer, 2020

    2020

  57. [66]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St ´ephane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In Proceedings of the International Conference on Machine Learn- ing (ICML), pages 12310–12320. PMLR, 2021

    2021

  58. [67]

    Continual learning of context-dependent process- ing in neural networks

    Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent process- ing in neural networks. Nature Machine Intelligence, 1(8):364–372, 2019. 14 Appendix A. Discriminative-separable Feature Space and Task Confusion: Task confusion is due to classify...

    2019

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.