REVIEW 4 major objections 5 minor 1 cited by
Foundation Models as Class-Incremental Learners for Dermatological Image Classification
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Frozen foundation models outperform specialized continual-learning methods for dermatology image classification, achieving zero forgetting.
desk verdict Useful baseline study, but the derm-FM results are compromised by a plausible pretraining/test overlap that the authors never address. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is the frozen foundation model as a fixed feature extractor: images are mapped to pre-trained embeddings $z = F_\theta(x)$ that are never fine-tuned. On top, per-task lightweight MLP heads are trained independently and concatenated at inference, or class-mean prototypes $\mu_c$ are stored in a memory bank and matched by nearest-neighbor distance. The paper also uses data transformations on the embedding space, $\ell^2$ normalization and a learnable hyperbolic projection, to improve the prototype classifier.
What would settle it
Compare the pretraining image sets of Google Derm and PanDerm against the three evaluation datasets; if any evaluation image or near-duplicate appears in pretraining, the claimed advantage over methods that never saw those images is inflated. A cleaner check is to rerun the identical CIL protocol on a skin-lesion dataset released after the models' pretraining cutoff and see whether the frozen-MLP result still stands.
Extended reading notes
Core claim
The paper's central discovery is that the continual-learning problem in dermatology largely disappears when the feature extractor is a frozen foundation model trained on large-scale skin imagery. With PanDerm embeddings and a two-hidden-layer MLP head trained only on current-task data, the model reaches a balanced accuracy of 92.25% on HAM10000, surpassing the SINGLE reference model (88.35%), and reports zero forgetting on all three datasets; the same pattern holds on Dermofit (93.11%) and Derm7pt (77.80%). A prototype-based nearest-mean classifier with no trained parameters also produces competitive results on some benchmarks, and a learnable hyperbolic projection boosts it substantially (e.g., 64.75% to 81.41% on HAM with Google Derm). The authors interpret these results as evidence that rich frozen features, rather than specialized continual-learning machinery, are what drive stability in class-incremental settings.
Load-bearing premise
The foundation models were pretrained on massive dermatology image collections that could include the exact HAM10000, Dermofit, and Derm7pt images used for evaluation, and the paper does not rule out that overlap.
Editorial extensions
If this is right
- On the three benchmarks tested, training a small MLP on frozen embeddings eliminates catastrophic forgetting while outperforming every compared continual-learning method, so replay and regularization appear unnecessary for this setting.
- A general-purpose CLIP ViT-L/14 backbone with an MLP head also beats all previous continual-learning methods on these datasets, suggesting the benefit is not unique to dermatology-specific pretraining.
- Prototype-based zero-training classifiers depend heavily on the alignment between pretraining and target domain: they lag with CLIP but improve markedly with dermatology-specific embeddings and embedding-space projections.
- The paper reports that the frozen-MLP approach exceeds the SINGLE upper bound on HAM (92.25% vs 88.35%), which would mean incremental training on frozen features costs nothing relative to task-specialized models.
Reading between the lines
- Inference: If frozen-embedding classifiers truly achieve zero forgetting and beat the single-task upper bound, then a large share of continual-learning research for medical imaging may be attacking a problem that pretrained representations already solve; the remaining open question is representation freshness as new disease classes appear over time.
- Inference: Because the method stores only class-mean prototypes or lightweight MLP weights rather than raw patient images, it offers a natural fit for clinical privacy constraints, though the same property means the memory bank cannot be corrected if the embedding space is biased.
- Inference: A decisive test the paper leaves implicit is whether its evaluation datasets overlap with the foundation models' pretraining data; running the same protocol on a dermatology dataset released after the models' training cutoff would settle whether the advantage is generalization or memorization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates frozen dermatology foundation models (Google Derm and PanDerm) as feature extractors for class-incremental learning on three public skin-lesion datasets: HAM10000, Dermofit, and Derm7pt. It proposes two baselines: a per-task MLP classifier trained on frozen embeddings, and a prototype-based nearest-mean classifier that requires no training. The authors report that the MLP baseline achieves balanced accuracy of 92.25% on HAM10000 with zero forgetting, outperforming all compared continual learning methods and also the SINGLE upper-bound reference. Ablation studies explore NMC variants (normalization, random projection, hyperbolic projection, PCA, LDA) and replace the dermatology-specific models with CLIP ViT-L/14. The paper concludes that frozen foundation models should be the default starting point for continual learning in dermatology.
Significance. If the empirical claims hold, the paper would provide a practically important result for medical continual learning: a frozen foundation model with a lightweight per-task classifier can outperform specialized regularization, replay, and architecture-based methods while avoiding the privacy and storage issues of replay. The paper is among the first to evaluate dermatology-specific foundation models in a class-incremental setting, and it includes ablations across multiple feature extractors and classifier variants. The central claims, however, rest entirely on the validity of the evaluation protocol, and the manuscript currently does not rule out pretraining overlap between the foundation models and the evaluation datasets, nor does it provide enough protocol detail to verify the comparisons against prior work.
major comments (4)
- [Section 4.1, Implementation Details] The pretraining corpora of Google Derm and PanDerm are not analyzed for overlap with HAM10000, Dermofit, or Derm7pt. PanDerm is described as pretrained on millions of clinical and dermoscopic dermatology images, and these three benchmarks are standard public skin-lesion collections; if any evaluation images or near-duplicates appeared in pretraining, the Table 1 results are inflated and the comparison with methods that did not use those features is unfair. This is load-bearing because the paper's headline result of surpassing the SINGLE upper bound depends on the evaluation being out-of-distribution for the frozen encoders. Please provide an overlap analysis, cite pretraining data documentation, or use models with explicit exclusion guarantees.
- [Section 4.1, Reference Methods and Competitors; Table 1] All non-FM baseline numbers in Table 1 are taken from Continual-Zoo [8], but the manuscript does not state whether the train/test splits, task order, image preprocessing, and evaluation code are exactly those used in [8] or were re-implemented. If any protocol detail differs, the reported gains over Continual-Zoo may reflect protocol differences rather than the frozen-FM effect. Please provide the exact task partitions, the number of tasks T, and the version of [8]'s code used; ideally, re-run the baselines with the same evaluation harness. Also clarify the backbone used for the SINGLE upper bound, since a SINGLE model trained on a weaker backbone is not an upper bound for a method that uses a stronger frozen feature extractor.
- [Section 3.2, Inference Phase; Table 1] The zero forgetting (F=0) reported for the MLP baseline is guaranteed by construction, because each task receives a newly initialized head and old heads are never updated or regularized; the same holds for the NMC prototypes. Reporting F=0 as an empirical finding therefore overstates the result. The meaningful comparison is the final balanced accuracy under the task-agnostic inference rule (arg max over concatenated heads), and this should be stated explicitly in the main text.
- [Section 4.1, Implementation Details; abstract] The MLP hidden layer sizes, the number of tasks T, the class partition for each dataset, and the validation split are not reported, and the promised code link is absent. These details are necessary to reproduce the central results, especially because the method's advantage is empirical and protocol-dependent. Please specify these items or provide a public repository with the exact configuration.
minor comments (5)
- [Section 4.2, Ablation 1] The hyperbolic projection is described as having a significant positive impact, but on DMF it decreases accuracy relative to the base NMC for both models (Google Derm 63.79 vs 67.56; PanDerm 43.21 vs 49.27); the sentence should be qualified to specific datasets.
- [Section 4.1, Implementation Details] The sentence 'Our code and datasets are available here' contains no link or reference; please add a URL or footnote.
- [References] Reference [10] appears unrelated to continual learning (it is a survey of glucose monitoring systems); please verify the citation and replace it with the intended work.
- [Table 2 and Table 3] The column header 'Derm' should be written as 'Google Derm' for consistency with the main text.
- [Section 4.2, Results and Analysis] The statement that NMC-based baselines 'achieve comparable, and sometimes superior, results' is too strong: on DMF and D7P with PanDerm, the base NMC achieves 49.27% and 44.51%, respectively, which is below most competing methods; please calibrate this sentence.
Circularity Check
No significant circularity: the central result is an empirical benchmark comparison, and the only self-referential element is a non-load-bearing adoption of the Continual-Zoo experimental protocol.
full rationale
The paper's central claim is an empirical comparison: frozen dermatology foundation models with a lightweight MLP or prototype classifier outperform existing continual learning methods on three public benchmarks. This claim is established through external baselines and measured accuracy, not through a derivation that reduces to its own inputs. The only self-referential element is the adoption of dataset splits and protocol from Continual-Zoo [8], whose author list overlaps with the present paper; however, that protocol is an independent experimental setting, and the reported accuracies do not reduce to it or depend on it for their validity. The zero-forgetting values (F = 0) are a direct consequence of the frozen-backbone and independent-head design, but the paper presents them as measured metrics rather than as a prediction derived from a fitted parameter. The learnable hyperbolic projection is an ablation component with parameters optimized on training data, not a fitted input renamed as a prediction. The concern that PanDerm or Google Derm may have been pretrained on evaluation images is a data-leakage validity risk, not a logical circularity, and would affect correctness rather than the derivation chain. No equation, definition, or self-citation chain makes the central result true by construction.
Assumptions & free parameters
free parameters (3)
- MLP hidden layer sizes =
not specified
- MLP optimization hyperparameters =
learning rate 0.001, batch size 200, up to 200 epochs with early stopping
- Hyperbolic projection parameters =
optimized during training
assumptions (4)
- domain assumption The three public datasets are representative of the clinical deployment distribution and the task partitioning is meaningful for CIL.
- domain assumption The Google Derm and PanDerm foundation models were not pretrained on the evaluation images.
- domain assumption The numbers reported for prior methods from Continual-Zoo [8] are comparable to the authors' numbers.
- domain assumption Balanced accuracy (BAAC) is the appropriate evaluation metric.
Cite this review
Pith. "Pith review of Foundation Models as Class-Incremental Learners for Dermatological Image Classification." pith.science (2026). https://pith.science/paper/ONP4CGGN
@misc{pith2026250714050,
author = {Pith},
title = {Pith review of: Foundation Models as Class-Incremental Learners for Dermatological Image Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/ONP4CGGN}},
note = {Machine review of arXiv:2507.14050}
}
read the original abstract
Class-Incremental Learning (CIL) aims to learn new classes over time without forgetting previously acquired knowledge. The emergence of foundation models (FM) pretrained on large datasets presents new opportunities for CIL by offering rich, transferable representations. However, their potential for enabling incremental learning in dermatology remains largely unexplored. In this paper, we systematically evaluate frozen FMs pretrained on large-scale skin lesion datasets for CIL in dermatological disease classification. We propose a simple yet effective approach where the backbone remains frozen, and a lightweight MLP is trained incrementally for each task. This setup achieves state-of-the-art performance without forgetting, outperforming regularization, replay, and architecture based methods. To further explore the capabilities of frozen FMs, we examine zero training scenarios using nearest mean classifiers with prototypes derived from their embeddings. Through extensive ablation studies, we demonstrate that this prototype based variant can also achieve competitive results. Our findings highlight the strength of frozen FMs for continual learning in dermatology and support their broader adoption in real world medical applications. Our code and datasets are available here.
Forward citations
Cited by 1 Pith paper
-
LesionGen: A Concept-Guided Diffusion Model for Dermatology Image Synthesis
Concept-guided captions and prompt balancing improve synthetic skin lesion images enough that augmenting real data with them lifts rare-class classification, though synthetic-only training remains clearly worse than r...
Reference graph
Works this paper leans on
-
[8]
In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition
Bayasi, N., Hamarneh, G., Garbi, R.: Continual-zoo: Leveraging zoo models for continual classification of medical images. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition. pp. 4128–4138 (2024)
2024
-
[1]
Ayromlou, S., Tsang, T., Abolmaesumi, P., Li, X.: CCSI: Continual class-specific impression for data-free class incremental learning. Medical Image Analysis (2024)
work page 2024
-
[2]
In: Color Medical Image Analysis, pp
Ballerini, L., Fisher, R.B., Aldridge, B., Rees, J.: A color and texture based hierar- chical k-nn approach to the classification of non-melanoma skin lesions. In: Color Medical Image Analysis, pp. 63–86. Springer (2013)
work page 2013
-
[3]
Bang, J., Kim, H., Yoo, Y., Ha, J.W., Choi, J.: Rainbow memory: Continual learn- ing with a memory of diverse samples. In: IEEE/CVF CVPR. pp. 8218–8227 (2021)
work page 2021
-
[4]
Bayasi, N.: Beyond catastrophic forgetting: advancing continual learning for robust andfairmedicalimageanalysis.Ph.D.thesis,UniversityofBritishColumbia(2025)
work page 2025
-
[5]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
Bayasi, N., Du, S., Hamarneh, G., Garbi, R.: Continual-gen: Continual group en- sembling for domain-agnostic skin lesion classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 3–13 (2023)
2023
-
[6]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
Bayasi, N., Fayyad, J., Bissoto, A., Hamarneh, G., Garbi, R.: Biaspruner: Debiased continual learning for medical image classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 90–101. Springer (2024)
2024
-
[7]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)
Bayasi, N., Hamarneh, G., Garbi, R.: Culprit-prune-net: Efficient continual se- quential multi-domain learning with application to skin lesion classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). pp. 165–175. Springer (2021)
2021
Show all 36 references
-
[9]
IEEE Transactions on Medical Imaging (2024)
Bayasi, N., Hamarneh, G., Garbi, R.: GC2: Generalizable continual classification of medical images. IEEE Transactions on Medical Imaging (2024)
2024
-
[10]
In: 2013 IEEE 20th International Confer- ence on Electronics, Circuits, and Systems (ICECS)
Bayasi, N., Saleh, H., Mohammad, B., Ismail, M.: The revolution of glucose mon- itoring methods and systems: A survey. In: 2013 IEEE 20th International Confer- ence on Electronics, Circuits, and Systems (ICECS). pp. 92–93 (2013)
2013
-
[11]
In: (MICCAI)
Bera, S., Ummadi, V., Sen, D., Mandal, S., Biswas, P.K.: Memory replay for contin- ual medical image segmentation through atypical sample selection. In: (MICCAI). pp. 513–522 (2023)
2023
-
[12]
arXiv preprint arXiv:2108.07258 (2021)
Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)
2021 arXiv
-
[13]
In: Proceedings of the AAAI conference on artificial intelligence
Chee, E., Lee, M.L., Hsu, W.: Leveraging old knowledge to continually learn new classes in medical images. In: Proceedings of the AAAI conference on artificial intelligence. vol. 37, pp. 14178–14186 (2023)
2023
-
[14]
arXiv preprint arXiv:2405.16328 (2024)
Chen, X., Zheng, H., Xie, Y., Ma, Y., Li, T.: A classifier-free incremental learning framework for scalable medical image segmentation. arXiv preprint arXiv:2405.16328 (2024)
2024 arXiv
-
[15]
Fayyad, J.: Out-of-distribution detection using inter-level features of deep neural networks. Ph.D. thesis, University of British Columbia (2023)
2023
-
[16]
Computer Methods and Programs in Biomedicine 253, 108231 (2024)
Fayyad, J., Alijani, S., Najjaran, H.: Empirical validation of conformal prediction for trustworthy skin lesions classification. Computer Methods and Programs in Biomedicine 253, 108231 (2024)
2024
-
[17]
Gonzalez-Jimenez, A., Lionetti, S., Amruthalingam, L., Gottfrois, P., Gröger, F., Pouly, M., Navarini, A.A.: Is hyperbolic space all you need for medical anomaly detection? (2025), provisionally accepted at MICCAI 2025 10 Elkhayat & Mahmoud et al
2025
-
[18]
https://developers.google.com/ health-ai-developer-foundations/derm-foundation (2025)
Google Health AI: Derm foundation model. https://developers.google.com/ health-ai-developer-foundations/derm-foundation (2025)
2025
-
[19]
In: NeurIPS (2023)
Janson, P., Zhang, W., Aljundi, R., Elhoseiny, M.: A simple baseline that questions the use of pretrained models in continual learning. In: NeurIPS (2023)
2023
-
[20]
In: CVPR (2023)
Javed, K., Wang, Y., Xu, Q., Zhang, Y.: Parametric prompt tuning for vision- language models. In: CVPR (2023)
2023
-
[21]
The IEEE Journal of Biomedical and Health Informatics23(2), 538–546 (2018)
Kawahara, J., Daneshvar, S., Argenziano, G., Hamarneh, G.: Seven-point checklist and skin lesion classification using multitask multimodal neural nets. The IEEE Journal of Biomedical and Health Informatics23(2), 538–546 (2018)
2018
-
[22]
Proceedings of the National Academy of Sciences 114(13), 3521–3526 (2017)
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114(13), 3521–3526 (2017)
2017
-
[23]
IEEE Transactions on Pattern Analysis and Machine Intelligence40(12), 2935–2947 (2017)
Li, Z., Hoiem, D.: Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence40(12), 2935–2947 (2017)
2017
- [24]
-
[25]
In: CVPR
Ostapenko, O., Puscas, M., Klein, T., Jahnichen, P., Nabi, M.: Learning to remem- ber: A synaptic plasticity driven framework for continual learning. In: CVPR. pp. 11321–11329 (2019)
2019
-
[26]
Psychological Review97(2), 285 (1990)
Ratcliff, R.: Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological Review97(2), 285 (1990)
1990
-
[27]
In: IEEE/CVF CVPR
Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: iCaRL: Incremental clas- sifier and representation learning. In: IEEE/CVF CVPR. pp. 2001–2010 (2017)
2017
-
[28]
Scientific data 5(1), 1–9 (2018)
Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data 5(1), 1–9 (2018)
2018
-
[29]
Nature communications11(1), 1–14 (2020)
van de Ven, G.M., Siegelmann, H.T., Tolias, A.S.: Brain-inspired replay for con- tinual learning with artificial neural networks. Nature communications11(1), 1–14 (2020)
2020
-
[30]
In: ECCV
Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.Y., Ren, X., Su, G., Perot, V., Dy, J., et al.: Dualprompt: Complementary prompting for rehearsal-free continual learning. In: ECCV. pp. 631–648 (2022)
2022
-
[31]
In: IEEE/CVF CVPR
Wang, Z., Zhang, Z., Lee, C.Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., Pfister, T.: Learning to prompt for continual learning. In: IEEE/CVF CVPR. pp. 139–149 (2022)
2022
-
[32]
Signal Processing (2024)
Wu, Z., Zhu, F., Guo, K., Sheng, R., Chao, L., Fang, H.: Modal adaptive super- resolution for medical images via continual learning. Signal Processing (2024)
2024
-
[33]
arXiv preprint arXiv:2410.15038 (2024)
Yan, S., Yu, Z., Primiero, C., Vico-Alonso, C., Wang, Z., Yang, L., Tschandl, P., Hu, M., Tan, G., Tang, V., Ng, A.B., Powell, D., Bonnington, P., See, S., Janda, M., Mar, V., Kittler, H., Soyer, H.P., Ge, Z.: A general-purpose multimodal foundation model for dermatology. arXi...
2024 arXiv
-
[34]
Visual Intelli- gence 1(1), 5 (2023)
Yang, Y., Cui, Z., Xu, J., Zhong, C., Zheng, W.S., Wang, R.: Continual learning with bayesian model based on a fixed pre-trained feature extractor. Visual Intelli- gence 1(1), 5 (2023)
2023
-
[35]
In: MICCAI
Zhang, W., Huang, Y., Zhang, T., Zou, Q., Zheng, W.S., Wang, R.: Adapter learn- ing in pretrained feature extractor for continual learning of diseases. In: MICCAI. pp. 68–78 (2023)
2023
-
[36]
In: MICCAI
Zhang, Y., Li, X., Chen, H., Yuille, A.L., Liu, Y., Zhou, Z.: Continual learning for abdominal multi-organ and tumor segmentation. In: MICCAI
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.