Pith. sign in

REVIEW 4 major objections 4 minor 30 references

Automated Measurement of Eczema Severity with Self-Supervised Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a two-stage pipeline combining few-shot segmentation with self-supervised features measures eczema severity from ordinary photos more accurately than finetuned deep networks when labeled data is scarce.

desk verdict A sensible pipeline and a plausible direction, but the headline F1 gap likely reflects the baselines' fixed, under-tuned training budget rather than a genuine superiority of self-supervised features. read the letter →

arxiv 2504.15193 v1 pith:4PHRD6O6 submitted 2025-04-21 cs.CV cs.LG

Neelesh Kumar , Oya Aran This is my paper · ORCID
classification cs.CVcs.LG
keywords eczemaseverityclassificationself-supervisedlearningDINOSegGPTfew-shotsegmentationlimitedlabeleddatain-the-wildskinimagestransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Eczema severity is usually assessed by training deep networks on large labeled image sets, but labels are costly because eczema varies across skin tone, body site, and severity. This paper proposes a two-stage pipeline that avoids large labeled training sets: SegGPT segments the eczema region using only two labeled examples, and features from a frozen self-supervised DINO vision transformer are fed into a small MLP that classifies severity into four levels. On a dataset of 528 in-the-wild images, the pipeline reaches a weighted F1 of 0.67, compared with 0.44 for finetuned ResNet-18 and 0.40 for finetuned ViT-B. The paper argues this makes self-supervised learning a viable route for automated skin diagnosis where annotated data is scarce.

What carries the argument

The pipeline rests on two pretrained models. SegGPT casts segmentation as image inpainting and, at inference, retrieves two nearest-neighbor training images in DINO feature space to build the prompt; this supplies the eczema mask. DINO, a self-supervised ViT trained by teacher-student distillation without labels, provides the feature vectors of the masked region. A one-hidden-layer MLP with 128 units and dropout 0.3 classifies those features into severity classes 0 through 3. The mechanism is the combination: in-context segmentation removes background, and frozen DINO features avoid finetuning a large network on a small dataset.

What would settle it

Re-run the same 5-fold evaluation with a proper hyperparameter search for the finetuned baselines, varying learning rate, epochs, weight decay, and early stopping on the validation fold; if ResNet-18 or ViT-B reaches or exceeds a weighted F1 of 0.67, the claimed advantage of the self-supervised pipeline over finetuning collapses.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a pretrained self-supervised ViT (DINO) used as a fixed feature extractor, combined with a SegGPT-based few-shot segmentation stage, outperforms standard finetuned ResNet-18 and ViT-B on four-class eczema severity classification, with weighted F1 scores of 0.67 versus 0.44 and 0.40. The advantage holds when training data is cut to 20%, where performance drops only slightly. Segmentation contributes a small but positive gain (0.67 with segmentation versus 0.65 without). The authors interpret these results as evidence that SSL features encode task-relevant local and semantic information for skin images, reducing dependence on large labeled datasets.

Load-bearing premise

The central result assumes that the fixed finetuning protocol used for the baselines (50 epochs, learning rate 1e-4, batch size 16, Adam, no hyperparameter search or early stopping) lets ResNet-18 and ViT-B show roughly their achievable performance; if they are undertrained, the F1 gap reflects training budget rather than the pipeline's superiority.

Editorial extensions

If this is right

  • A person could photograph their own skin and receive a four-level severity score using a model initialized from only two annotated examples, enabling self-monitoring of recovery.
  • Because the feature extractor is frozen, deployment on a phone or low-resource clinic needs only a small MLP rather than a large finetuned network.
  • The method holds up when training data is cut to 20%, so collecting small, diverse datasets may be enough to build usable classifiers for variable or rare skin conditions.
  • The small but consistent gain from segmentation suggests that focusing the classifier on the lesion region matters more as labeled data shrinks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the fixed finetuning protocol (50 epochs, learning rate 1e-4, batch size 16, no hyperparameter search) undertrains the baselines, the reported gap could narrow under a fair tuned comparison.
  • Editorial inference: SegGPT's neighbor retrieval uses DINO features, so the same representation carries both stages; improving segmentation or using a dermatology-specialized SSL model could yield larger gains than the 0.02 segmentation effect observed here.
  • Editorial inference: the two-stage recipe is not eczema-specific; the same frozen-feature-plus-few-shot-segmentation design could be tested on other skin diseases where labeled images are scarce.
  • Editorial inference: each image was labeled by a single annotator, so label noise sets a ceiling on the 0.67 F1; multiple-expert relabeling and inter-rater agreement would clarify how much headroom remains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This manuscript proposes a two-stage pipeline for automated eczema severity classification from in-the-wild photographs. The first stage uses the pretrained SegGPT model with in-context learning (K=2 prompt images) to segment eczema regions; the second stage extracts features from the segmented region using a pretrained DINO ViT-B model and trains a small MLP with one hidden layer of 128 units, dropout 0.3, Adam, learning rate 1e-4, for 50 epochs to classify severity into four classes. On a dataset of 528 images with five random 80/20 splits, the method achieves weighted F1 0.67±0.01, compared with 0.44±0.16 for finetuned ResNet-18 and 0.40±0.22 for finetuned ViT-B. An ablation shows a small drop to 0.65±0.03 when segmentation is removed, and a limited-data experiment (Fig. 2) shows mild degradation with 20% of training data. The paper concludes that self-supervised features are a viable data-efficient alternative to full finetuning for this task.

Significance. If the reported advantage is reproducible, the pipeline would be a practically useful, low-annotation approach to eczema severity estimation, and the two-stage design (in-context segmentation plus frozen SSL features) is a sensible way to reduce labeled-data requirements. The strengths of the paper are its clear description of the two-stage architecture, the use of five random splits, and the inclusion of an ablation for the segmentation stage. The main weakness is that the baseline comparison is not controlled: the finetuning protocol for ResNet-18/ViT-B is fixed and unaugmented, with no early stopping, no hyperparameter tuning, and no paired significance testing, while the proposed method's frozen feature extractor receives no similar handicap. The absence of a linear-probe control using the same MLP on frozen features from the same backbones means the claimed superiority is not attributable to the self-supervised features per se. The manuscript is therefore a reasonable proof-of-concept but does not currently support its central claim.

major comments (4)
  1. [Section III-B, Table I] The baseline comparison is not controlled: ResNet-18 and ViT-B are finetuned with a single fixed protocol (50 epochs, learning rate 1e-4, batch size 16, Adam, no early stopping or hyperparameter search), while the proposed MLP on frozen DINO features uses the same optimizer setting but has far fewer parameters. Given only about 422 training images per split, ViT-B receives roughly 1,300 optimizer updates, which is often insufficient for convergence on a small imbalanced dataset. The reported fold-level standard deviations (±0.16 and ±0.22) are large relative to the 0.23–0.27 gap, and no paired statistical test is provided. Please report per-split results, learning curves or convergence checks, and results with a tuned or early-stopped baseline; without this, the central claim that the SSL pipeline outperforms finetuning is not supported.
  2. [Section III-C, Table I] To attribute the performance difference to DINO feature quality rather than to the training protocol, add a control where the same small MLP trained on DINO features is also trained on features extracted from the frozen ImageNet-pretrained ResNet-18 and ViT-B that are used as finetuning baselines. This probe-based comparison would isolate the contribution of the self-supervised pretraining from the finetuning schedule, and would make the comparison symmetric with respect to the classifier.
  3. [Section III-A, Section III-C] The segmentation stage is a load-bearing component of the pipeline, but its quality is never evaluated. The paper does not report any segmentation metric (e.g., Dice or IoU) on the eczema masks, nor does it show example segmentation outputs or analyze how segmentation errors propagate to the severity classifier. Additionally, the severity labels come from a single human annotator per image with no reported inter-annotator agreement or class distribution; this limits the reliability of the target variable. Please include segmentation evaluation and label quality statistics.
  4. [Section III-D, Section III-E] The ablation in Table II reports 0.67±0.01 with segmentation versus 0.65±0.03 without; the overlapping standard deviations and the absence of a paired test mean the claimed benefit of segmentation is not statistically established. The limited-data experiment in Figure 2 provides no numerical table, no error-bar values in the text, and no comparison with the finetuned baselines at the same reduced training fractions, so the robustness claim is not quantified. Please report these results with per-split numbers and appropriate statistical tests.
minor comments (4)
  1. [Section III-A] The preprocessing description 'z-score normalization using the ImageNet dataset statistics' is ambiguous; specify whether per-channel mean and standard deviation are used and whether the segmentation masks are normalized in the same way as the input images.
  2. [Section III-C] The MLP architecture is described as one hidden layer of 128 dimensions with dropout 0.3, but the input dimension (the DINO feature vector size) is not stated, and the procedure for cropping or masking the segmented region before feature extraction is not described.
  3. [Abstract, Section III-B] The abstract refers to finetuned ResNet-18 and ViT-B as 'state-of-the-art deep learning methods,' but no comparison with eczema-specific prior work (e.g., EczemaNet or other dermatology classifiers) is provided, so the baselines are not situated in the existing literature.
  4. [Section III-E] Figure 2 reports only a limited-data curve; please provide the numerical mean and standard deviation values in the text or caption for each fraction of training data, and indicate the number of splits used.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the severity-classification claim is an empirical comparison against externally labeled held-out data using fixed pretrained feature extractors.

full rationale

The paper's central claim is an empirical performance comparison on held-out validation folds with external human severity labels. The DINO and SegGPT models are fixed pretrained models; the only trained component on the target labels is the MLP, whose weights are fit to training folds and evaluated on validation folds. No equation in the paper defines the reported F1 in terms of the inputs, and no fitted parameter is renamed as a prediction. The only self-citation is ref. [19], a prior work by the same authors for in-context-learning eczema segmentation, but the current segmentation stage is actually implemented with SegGPT (ref. [28]) plus KNN retrieval in DINO feature space, and the paper's severity-classification comparison does not reduce to [19]'s content. Baseline training budgets and hyperparameter choices are a fairness/correctness concern, not a circularity concern. Therefore no circular step can be exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard transfer-learning assumptions: the pretrained models are appropriate for the domain and the label quality is adequate. No new physical or mathematical entities are introduced.

free parameters (6)
  • K (number of SegGPT prompt examples) = 2
    Set to 2 for the K-nearest-neighbor retrieval in the segmentation stage; no sensitivity analysis is provided.
  • MLP hidden layer size = 128
    Chosen by hand for the classification head on top of DINO features.
  • MLP dropout fraction = 0.3
    Used for regularization; no ablation on this value is reported.
  • Learning rate = 1e-4
    Used for both baseline and MLP training; no tuning reported.
  • Batch size = 16
    Used for all training; no tuning reported.
  • Number of training epochs = 50
    Fixed for all models; no early stopping or convergence checks reported.
assumptions (4)
  • domain assumption Each image's severity label from a single human annotator is an accurate ground truth.
    Section III-A states each image was labeled by one annotator and the labels were inspected by domain experts, but no inter-annotator agreement is reported.
  • domain assumption DINO features pretrained on ImageNet transfer to eczema severity classification.
    The method relies on frozen DINO ViT-B features without domain-specific finetuning.
  • domain assumption SegGPT produces accurate enough segmentation masks from two prompt examples for severity classification.
    No segmentation metrics are reported; the ablation shows only a 0.02 F1 difference with and without segmentation.
  • domain assumption The random 80/20 splits do not leak identity information across train and validation.
    Section III-A describes only random splits, not stratification by participant or source.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automated Measurement of Eczema Severity with Self-Supervised Learning." pith.science (2026). https://pith.science/paper/4PHRD6O6

@misc{pith2026250415193,
  author       = {Pith},
  title        = {Pith review of: Automated Measurement of Eczema Severity with Self-Supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4PHRD6O6}},
  note         = {Machine review of arXiv:2504.15193}
}
abstract

Automated diagnosis of eczema using images acquired from digital camera can enable individuals to self-monitor their recovery. The process entails first segmenting out the eczema region from the image and then measuring the severity of eczema in the segmented region. The state-of-the-art methods for automated eczema diagnosis rely on deep neural networks such as convolutional neural network (CNN) and have shown impressive performance in accurately measuring the severity of eczema. However, these methods require massive volume of annotated data to train which can be hard to obtain. In this paper, we propose a self-supervised learning framework for automated eczema diagnosis under limited training data regime. Our framework consists of two stages: i) Segmentation, where we use an in-context learning based algorithm called SegGPT for few-shot segmentation of eczema region from the image; ii) Feature extraction and classification, where we extract DINO features from the segmented regions and feed it to a multi-layered perceptron (MLP) for 4-class classification of eczema severity. When evaluated on a dataset of annotated "in-the-wild" eczema images, we show that our method outperforms (Weighted F1: 0.67 $\pm$ 0.01) the state-of-the-art deep learning methods such as finetuned Resnet-18 (Weighted F1: 0.44 $\pm$ 0.16) and Vision Transformer (Weighted F1: 0.40 $\pm$ 0.22). Our results show that self-supervised learning can be a viable solution for automated skin diagnosis where labeled data is scarce.

Figures

Figures reproduced from arXiv: 2504.15193 by the authors.

Figure 1
Figure 1. Overview. Our framework consists of two stages: In the first stage, we use a pretrained Seg-GPT model to perform segmentation of eczema region using just 2 labeled examples in the prompt. In the second stage, we use a pretrained DINO model to extract features from the segmented region. The features are then used to train an MLP to perform classification of eczema severity levels. on a dataset of annotated ”in-the-wi… view at source ↗
Figure 2
Figure 2. Training with limited data. Shown here are the means and standard [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 24 canonical work pages

  1. [1]

    Dermnet dataset, https://www.dermnetnz.org/

    work page

  2. [2]

    Journal of the European Academy of Dermatology and Venereology 29(12), 2417– 2422 (2015)

    Agner, T., Aalto-Korte, K., Andersen, K., Foti, C., Gimen ´ez-Arnau, A., Goncalo, M., Goossens, A., Le Coz, C., Diepgen, T., Environmental, E., Group, C.D.R.: Classification of hand eczema. Journal of the European Academy of Dermatology and Venereology 29(12), 2417– 2422 (2015)

    work page 2015

  3. [3]

    arXiv preprint arXiv:2112.05814 2(3), 4 (2021)

    Amir, S., Gandelsman, Y ., Bagon, S., Dekel, T.: Deep vit features as dense visual descriptors. arXiv preprint arXiv:2112.05814 2(3), 4 (2021)

    arXiv 2021

  4. [4]

    Expert Systems with Applications 213, 119230 (2023)

    Anand, V ., Gupta, S., Koundal, D., Singh, K.: Fusion of u-net and cnn model for segmentation and classification of skin lesion from dermoscopy images. Expert Systems with Applications 213, 119230 (2023)

    work page 2023

  5. [5]

    Advances in neural information processing systems 33, 9912–9924 (2020)

    Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems 33, 9912–9924 (2020)

    work page 2020

  6. [6]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Caron, M., Touvron, H., Misra, I., J ´egou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision trans- formers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)

    work page 2021

  7. [7]

    In: International conference on machine learning

    Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PMLR (2020)

    2020

  8. [8]

    British Journal of Dermatology 160(2), 353–358 (2009)

    Diepgen, T., Andersen, K., Brandao, F., Bruze, M., Bruynzeel, D., Frosch, P., Goncalo, M., Goossens, A., Le Coz, C., Rustemeyer, T., et al.: Hand eczema classification: a cross-sectional, multicentre study of the aetiology and morphology of hand eczema. British Journal of Dermatology 160(2), 353–358 (2009)

    work page 2009

Show all 30 references
  1. [9]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    2020 arXiv

  2. [10]

    Journal of consulting and clinical psychology 63(4), 624 (1995)

    Ehlers, A., Stangier, U., Gieler, U.: Treatment of atopic dermatitis: a comparison of psychological and dermatological approaches to relapse prevention. Journal of consulting and clinical psychology 63(4), 624 (1995)

    1995

  3. [11]

    Advances in Data Science and Information Engineering: Proceedings from ICDATA 2020 and IKE 2020 pp

    Farahani, A., V oghoei, S., Rasheed, K., Arabnia, H.R.: A brief review of domain adaptation. Advances in Data Science and Information Engineering: Proceedings from ICDATA 2020 and IKE 2020 pp. 877– 894 (2021)

    2021

  4. [12]

    In: Machine learning for health workshop

    Ghorbani, A., Natarajan, V ., Coz, D., Liu, Y .: Dermgan: Synthetic generation of clinical skin images with pathology. In: Machine learning for health workshop. pp. 155–170. PMLR (2020)

    2020

  5. [13]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

    2016

  6. [14]

    British Journal of Medical Psychology 62(3), 241–248 (1989)

    Horne, D.J.d.L., White, A.E., Varigos, G.A.: A preliminary study of psychological therapy in the management of atopic eczema. British Journal of Medical Psychology 62(3), 241–248 (1989)

    1989

  7. [15]

    Hurault, G., Pan, K., Mokhtari, R., Olabi, B., Earp, E., Steele, L., Williams, H.C., Tanaka, R.J.: Detecting eczema areas in digital images: an impossible task? JID Innovations 2(5), 100133 (2022)

    2022

  8. [16]

    Technologies 9(1), 2 (2020)

    Jaiswal, A., Babu, A.R., Zadeh, M.Z., Banerjee, D., Makedon, F.: A survey on contrastive self-supervised learning. Technologies 9(1), 2 (2020)

    2020

  9. [17]

    In: 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS)

    Junayed, M.S., Sakib, A.N.M., Anjum, N., Islam, M.B., Jeny, A.A.: Eczemanet: A deep cnn-based eczema diseases classification. In: 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS). pp. 174–179. IEEE (2020)

    2020

  10. [18]

    Nature Biomedical Engineering 6(12), 1346– 1352 (2022)

    Krishnan, R., Rajpurkar, P., Topol, E.J.: Self-supervised learning in medicine and healthcare. Nature Biomedical Engineering 6(12), 1346– 1352 (2022)

    2022

  11. [19]

    In: 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

    Kumar, N., Aran, O., Vasudevan, V .: Visual in-context learning for few-shot eczema segmentation. In: 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). pp. 1–5. IEEE (2024)

    2024

  12. [20]

    nature 521(7553), 436–444 (2015)

    LeCun, Y ., Bengio, Y ., Hinton, G.: Deep learning. nature 521(7553), 436–444 (2015)

    2015

  13. [21]

    IEEE transactions on knowledge and data engineering 35(1), 857–876 (2021)

    Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., Tang, J.: Self-supervised learning: Generative or contrastive. IEEE transactions on knowledge and data engineering 35(1), 857–876 (2021)

    2021

  14. [22]

    In: Advances in Non-Invasive Biomedical Signal Sensing and Processing with Machine Learning, pp

    Nisar, H., Tan, Y .R., Yeap, K.H.: Eczema skin lesions segmentation using deep neural network (u-net). In: Advances in Non-Invasive Biomedical Signal Sensing and Processing with Machine Learning, pp. 229–262. Springer (2023)

    2023

  15. [23]

    arXiv preprint arXiv:2308.12453 (2023)

    Sagers, L.W., Diao, J.A., Melas-Kyriazi, L., Groh, M., Rajpurkar, P., Adamson, A.S., Rotemberg, V ., Daneshjou, R., Manrai, A.K.: Augmenting medical image classifiers with synthetic data from latent diffusion models. arXiv preprint arXiv:2308.12453 (2023)

    2023 arXiv

  16. [24]

    Current dermatology reports 6, 204–210 (2017)

    Seth, D., Cheldize, K., Brown, D., Freeman, E.E.: Global burden of skin disease: inequities and innovations. Current dermatology reports 6, 204–210 (2017)

    2017

  17. [25]

    In: Machine Learning in Medical Imaging: 10th International Workshop, MLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Proceedings 10

    Shi, X., Dou, Q., Xue, C., Qin, J., Chen, H., Heng, P.A.: An active learning approach for reducing annotation cost in skin lesion analysis. In: Machine Learning in Medical Imaging: 10th International Workshop, MLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, O...

    2019

  18. [26]

    Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C.: A survey on deep transfer learning. In: Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part III 2...

    2018

  19. [27]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, X., Wang, W., Cao, Y ., Shen, C., Huang, T.: Images speak in images: A generalist painter for in-context visual learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6830–6839 (2023)

    2023

  20. [28]

    arXiv preprint arXiv:2304.03284 (2023)

    Wang, X., Zhang, X., Cao, Y ., Wang, W., Shen, C., Huang, T.: Seggpt: Segmenting everything in context. arXiv preprint arXiv:2304.03284 (2023)

    2023 arXiv

  21. [29]

    Psychology, health & medicine 12(4), 445–449 (2007)

    Wittkowski, A., Richards, H.L.: How beneficial is cognitive behaviour therapy in the treatment of atopic dermatitis? a single-case study. Psychology, health & medicine 12(4), 445–449 (2007)

    2007

  22. [30]

    Nature Machine Intelligence pp

    Zheng, H., Shen, L., Tang, A., Luo, Y ., Hu, H., Du, B., Wen, Y ., Tao, D.: Learning from models beyond fine-tuning. Nature Machine Intelligence pp. 1–12 (2025)

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.