Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

PSO-Net: Development of an automated psoriasis assessment system using attention-based interpretable deep neural networks

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PSO-Net directly converts patient-taken photos of four body regions into absolute PASI scores, with ICCs of 82.2% and 87.8% against two clinician raters—close to the 88.1% agreement between the two clinicians.

desk verdict A solid feasibility result for remote photo-based PASI scoring, but the headline ICCs measure agreement with photo-based raters, not with in-person clinical PASI. read the letter →

arxiv 2501.18782 v1 pith:COM4DHR3 submitted 2025-01-30 eess.IV cs.CV

Sharif A. Kamran , Molly V. Lucas , Brendon Lutnick , Chaitanya Parmar , Basudha Pal , Asha Patel Shah , David Apfel , Steven Fakharzadeh
show 4 more authors
Lloyd Miller Stephen Yip Kristopher Standish Gabriela Oana Cula
This is my paper · ORCID
classification eess.IVcs.CV
keywords psoriasisPASIdeeplearningattentionmechanismregressionactivationmapinterpretabilityremotepatientmonitoringtele-dermatology
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a deep-learning system, PSO-Net, can estimate the Psoriasis Area and Severity Index (PASI) directly from patient-captured images of four body regions, without manual cropping or masking. On a held-out test set, the model's scores reached intra-class correlation coefficients of 82.2% and 87.8% against two clinician raters, close to the two raters' agreement with each other (88.1%). If true, this would let clinical trials and routine monitoring measure psoriasis severity remotely, saving patients trips to the clinic and reducing the human labor and variability of manual scoring. The authors also introduce a regression activation map that highlights which lesions most influenced each score.

What carries the argument

The attention block is a MLP-TanH-MLP-softmax module that assigns a weight to each image in a set, then aggregates weighted feature vectors before regression. It does two jobs: it lets the network focus on the most informative lesions for PASI estimation, and it enables Grad-RAM, a back-propagated regression activation map that ranks attention scores to produce 224x224 saliency heatmaps over images, showing which areas drove each regional score.

What would settle it

Compare PSO-Net scores directly to in-person PASI evaluations on a new cohort where both are collected at the same visit; if the ICC between PSO-Net and the in-person dermatologist falls well below the human inter-rater ICC of 88.1%, the claim of near-human agreement is refuted.

Watch

Extended reading notes

Core claim

PSO-Net maps a variable-number set of images per body region through a pretrained encoder, pools the features with an attention block, and regresses a regional PASI sub-score; the four weighted regional scores produce the absolute PASI. The central claim is that this attention-based pooling, trained with mean-absolute-error loss on 28,060 images from a contracted research organization's ratings, achieves ICCs of 82.2% [77-87%] with Rater A and 87.8% [84-91%] with Rater B on the test set, while Rater A versus Rater B agreement is 88.1% [84-91%]. Because the confidence intervals overlap, the authors argue the model's agreement with clinicians is statistically indistinguishable from the agreement between two clinicians.

Load-bearing premise

The load-bearing premise is that the photo-based PASI labels produced by the contracted research organization are reliable enough to serve as ground truth, and that the eighth dermatologist's rescoring of the test set is an unbiased benchmark; if those labels carry systematic bias, the reported agreement may overstate real-world clinical validity.

Editorial extensions

If this is right

  • Patients in clinical trials could submit photos from home and receive a PASI estimate in seconds, eliminating the need for in-person scoring visits.
  • Automated scoring would remove the inter-rater variability that plagues manual PASI, yielding consistent endpoints across sites.
  • Grad-RAM heatmaps give clinicians a visual check of which lesions the model used, aiding trust and quality control.
  • The approach covers anatomical regions including the head and neck, and Fitzpatrick skin types I through VI, addressing gaps in earlier automated psoriasis systems.
  • If deployed widely, it could expand access to specialist-level severity assessment for patients in underserved areas.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The same attention-based architecture could be repurposed for other ordinal dermatology severity scales, such as the Eczema Area and Severity Index, if a labeled dataset with regional photos existed.
  • Inference: Because the model is trained and evaluated on photo-based labels, deployment in a new trial with different photography instructions or lighting conditions would likely require a calibration or domain-adaptation step before the reported ICCs transfer.
  • Inference: A practical test the authors did not run is measuring the consistency of PSO-Net scores across repeated uploads of the same patient's photos; low variance would strengthen the case for using the model as an objective anchor in trials.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PSO-Net, an attention-based deep learning system that takes sets of patient-captured photographs from four body regions and predicts regional and absolute Psoriasis Area and Severity Index (PASI) scores. The architecture combines a pretrained encoder (ConvNeXt, ViT, or NextViT) with an attention block and a regression head, and the authors introduce Grad-RAM, a regression activation map that ranks attention scores to produce interpretability heatmaps. The model is trained and evaluated on a dataset of remote patient photos with photo-based PASI labels provided by a CRO ('RaterA') and a separate dermatologist ('RaterB'). The main quantitative claims are ICCs of 82.2% and 87.8% against the two raters and MAEs around 1.4-1.7, with an inter-rater ICC of 88.1% between the two human raters. The paper also compares its MAE with several previously published AI-based PASI estimation methods.

Significance. If the central claims hold, PSO-Net would be a practically valuable tool for remote psoriasis monitoring, with agreement to clinician photo-based raters close to human inter-rater variability, and it would be among the first systems to provide interpretable, attention-based regional scoring without manual masking. The study has notable strengths: a patient-level train/validation/test split, 95% confidence intervals for the ICC estimates, comparison of several encoder architectures, and an independent RaterB rescoring of the test set. These design choices support the internal reliability of the reported comparisons. However, the clinical significance of the result depends on an assumption that is not validated in the manuscript: that photo-based PASI scores, from both the training labels and the comparison raters, are a faithful proxy for in-person clinical PASI. If that assumption fails, the high ICCs may reflect agreement on photographic rating conventions rather than true clinical severity. The paper also contains inconsistencies in the reported MAE values and an under-specified ICC methodology, and its cross-study comparison with prior work is not controlled.

major comments (4)
  1. [§3.1, §4] The headline claim that PSO-Net can replace in-person assessment is not supported by the evaluation design. Both the training labels (pooled CRO/RaterA photo scores) and the independent comparison (RaterB photo rescoring) are ratings of the same remote photographs, not in-person PASI examinations. Agreement between model and RaterB could therefore reflect shared photographic rating conventions (e.g., how area is estimated from 2D images, lighting, pose) rather than accurate measurement of true clinical PASI. The 'human inter-rater variability' of 88.1% is also photo-vs-photo. Please add an external validation against in-person PASI (even a small subset), or explicitly restrict the claims to photo-based PASI and discuss the direction and magnitude of the likely bias.
  2. [§3.3, Table 1, Table 2] The reported MAE values are internally inconsistent. The text in Section 3.3 states that the high-resolution ConvNeXt model achieved MAEs of 1.68 and 1.59 against RaterA and RaterB, respectively, but Table 1 lists the high-resolution ConvNeXt MAE against RaterB as 1.43±1.41, and Table 2 uses 1.43 for 'Ours'. Please reconcile these numbers and specify which value corresponds to which comparison; the discrepancy matters because the MAE is a central quantitative claim.
  3. [§3.3, Table 1] The ICC calculation is under-specified. The manuscript does not state which ICC form was used (e.g., one-way random, two-way random, consistency vs. absolute agreement), whether single or average measures are reported, or why the term 'inter-class' is used for a measure that is normally an intra-class correlation. Without this information the 95% confidence intervals cannot be reproduced or compared with the RaterA-vs-RaterB ICC of 88.1%, and the conclusion that the model's agreement is 'comparable' to human inter-rater variability is not fully interpretable.
  4. [§3.3, Table 2] The comparison with previously published AI-based PASI approaches in Table 2 is not a controlled comparison. Each prior method was evaluated on a different dataset, with different inclusion criteria, image acquisition protocols, and possibly different label conventions, and the table reports only MAE (without confidence intervals or patient-level splits). The statement that PSO-Net 'outperformed all prior approaches, except for Raj et al.' is therefore not supported by the evidence as presented. Please reframe this as a descriptive comparison or provide a common evaluation protocol.
minor comments (5)
  1. [Abstract, §3.1] The abstract states that the model was trained on 28,060 images, while Section 3.1 reports 38,824 photos over 844 patient visits; please clarify the relationship (e.g., number of images used after preprocessing or patient exclusion).
  2. [§3.3] The term 'inter-class correlation coefficient' should be 'intra-class correlation coefficient' (ICC), since the measure assesses agreement within the same class of absolute PASI scores.
  3. [§2.2] The per-region image counts (NHN=12, NUE=18, NLE=13, NTR=10) are introduced without explaining whether these are fixed per patient or selected by a rule; please specify how images are chosen when a patient provides more or fewer than the stated count.
  4. [§3.4] The Figure 3 quartile ranges are reported only in the text; moving them into the figure caption would improve readability.
  5. [General] The paper does not include a data availability or code availability statement; given the reproducibility claims, please state whether code or models can be shared or explain any restrictions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the ICC results are held-out empirical evaluations against external clinician labels, not fits renamed as predictions.

full rationale

PSO-Net's central claim is an empirical accuracy measurement, not a derived prediction. The model is trained on CRO photo-based PASI labels (Section 3.1) and evaluated against held-out clinician raters using ICC and MAE on a patient-disjoint test set; the reported 82.2%/87.8% ICCs are not constructed from the attention weights or from any parameter fitted to RaterB, so the evaluation does not reduce to its inputs by definition. The PASI target is defined externally by Eqs. 1-2 and clinical ordinal scales (Section 2.1), and the model's output is a regression on those labels. The one author self-citation [7] is used for context on PASI as an endpoint and remote AI assessment, not as the source of the central result. The concern that photo-based labels may deviate from in-person PASI is a question of external validity and ground-truth bias, not circular derivation; it does not make the ICC a tautology. Therefore no significant circularity is present.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a trained model, so all hyperparameters and preprocessing choices are free parameters fit to the development data. The main unvalidated domain assumptions are that photo-based labels are reliable ground truth and that ordinal PASI can be learned as a continuous scalar. No new physical entities are introduced.

free parameters (7)
  • Learning rate = 1e-6
    Adam optimizer hyperparameter chosen for training; all reported results depend on it.
  • Weight decay = 1e-4
    Regularization hyperparameter for the optimizer.
  • Batch size = 4
    Training batch size used for optimization.
  • Training epochs = 100
    Number of epochs used to train the model.
  • Per-region image counts = HN=12, UE=18, LE=13, TR=10
    The number of images fed per region was chosen by the authors and affects feature concatenation in the attention block.
  • Encoder architecture = ConvNeXt
    Selected over ViT, NextViT, and ConvNeXt-V2 based on development performance.
  • 4-crop preprocessing = high-resolution 4 crops
    Selected as the best preprocessing setting among no-crop low-resolution and 4-crop high-resolution variants.
assumptions (4)
  • domain assumption CRO photo-based PASI ratings are valid ground truth for training.
    Section 3.1 states labels come from one of seven CRO dermatologists rating image sets; no validation against in-person PASI is provided.
  • domain assumption Home-captured mobile photos preserve the clinical information needed for PASI scoring.
    The entire approach depends on remote images being sufficient proxies for clinical examination.
  • domain assumption ImageNet-pretrained encoders transfer useful features to dermatology images.
    Section 2.2 states ImageNet-pretrained encoders are used for transfer learning.
  • ad hoc to paper Ordinal PASI components can be treated as a continuous regression target.
    Section 2.4 uses MAE regression on continuous regional scores despite PASI components being ordinal (0-4 and 0-6), without discussing this modeling choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PSO-Net: Development of an automated psoriasis assessment system using attention-based interpretable deep neural networks." pith.science (2026). https://pith.science/paper/COM4DHR3

@misc{pith2026250118782,
  author       = {Pith},
  title        = {Pith review of: PSO-Net: Development of an automated psoriasis assessment system using attention-based interpretable deep neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/COM4DHR3}},
  note         = {Machine review of arXiv:2501.18782}
}
read the original abstract

Psoriasis is a chronic skin condition that requires long-term treatment and monitoring. Although, the Psoriasis Area and Severity Index (PASI) is utilized as a standard measurement to assess psoriasis severity in clinical trials, it has many drawbacks such as (1) patient burden for in-person clinic visits for assessment of psoriasis, (2) time required for investigator scoring and (3) variability of inter- and intra-rater scoring. To address these drawbacks, we propose a novel and interpretable deep learning architecture called PSO-Net, which maps digital images from different anatomical regions to derive attention-based scores. Regional scores are further combined to estimate an absolute PASI score. Moreover, we devise a novel regression activation map for interpretability through ranking attention scores. Using this approach, we achieved inter-class correlation scores of 82.2% [95% CI: 77- 87%] and 87.8% [95% CI: 84-91%] with two different clinician raters, respectively.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GRASP-PsONet: Gradient-based Removal of Spurious Patterns for PsOriasis Severity Classification

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A gradient-tracing data pruning method removes spurious images from a weakly supervised psoriasis severity classifier, raising test AUC by 5 points and catching most inter-rater label disagreements.

Reference graph

Works this paper leans on

22 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    With appropriate and timely treatment, in- dividuals can maintain a high quality of life

    INTRODUCTION Psoriasis (PsO) is a chronic inflammatory skin disease that af- fects approximately 2%-3% of the general population world- wide, and its pathogenesis derives from dysregulation of the immune system. With appropriate and timely treatment, in- dividuals can maintain a high quality of life. In recent years, advanced therapies, including biologic...

    arXiv 2025

  2. [2]

    METHODOLOGY 2.1. PASI Calculation The PASI is a standard scoring method for assessing PsO severity and response to treatment and is typically used as a primary endpoint measure in clinical trials [7,8]. The PASI evaluates both qualitative features of plaques (erythema [red- ness], induration [thickness], and desquamation [scaling]) as well as quantitative...

    work page

  3. [3]

    Dataset Of the 533 patients who provided images, 344 met criteria for inclusion and were advanced through screening

    EXPERIMENTA TION 3.1. Dataset Of the 533 patients who provided images, 344 met criteria for inclusion and were advanced through screening. Of these pa- tients, 220 patients were female and 124 were male. In total, 38,824 photos were captured over 844 patient visits (base- line and weeks 2, 4 and 8). Patients completed between 1 and 4 visits. The data coll...

    work page

  4. [4]

    Moreover, we also devise a novel regression activation map for inter- pretability by ranking attention scores

    CONCLUSION In this paper, we propose a novel and interpretable architec- ture, PSO-Net, which uses the latest advances in AI and deep learning to automatically estimate PASI scores. Moreover, we also devise a novel regression activation map for inter- pretability by ranking attention scores. Our computer-based model surpasses existing architectures in abs...

    work page

  5. [5]

    ACKNOWLEDGMENTS This work was sponsored by Johnson and Johnson

    work page

  6. [6]

    COMPLIANCE WITH ETHICAL STANDARDS This study was performed in accordance with the principles of the Declaration of Helsinki and was IRB approved

    work page

  7. [7]

    J Berth-Jones, K Grotzinger, C Rainville, B Pham, J Huang, S Daly, M Herdman, P Firth, and K Hotchkiss, “A study examining inter-and intrarater reliability of three scales for measuring severity of psoriasis: Psori- asis area and severity index, physician’s global assess- ment and lattice system physician’s global assessment,” British Journal of Dermatolo...

    work page 2006

  8. [8]

    Dermatology in china,

    Yi Zhou, Yujun Sheng, Jinping Gao, and Xuejun Zhang, “Dermatology in china,” in Journal of Investigative Dermatology Symposium Proceedings . Elsevier, 2015, vol. 17, pp. 12–14

    work page 2015

Show all 22 references
  1. [9]

    Negative impact of comorbidities on all-cause mortality of patients with psoriasis is partially alleviated by biologic treatment: A real-world case-control study,

    Saba Riaz, Sepideh Emam, Ting Wang, and Robert Gni- adecki, “Negative impact of comorbidities on all-cause mortality of patients with psoriasis is partially alleviated by biologic treatment: A real-world case-control study,” Journal of the American Academy of Dermatology, vol....

    2024

  2. [10]

    Psenet: Psoriasis severity evaluation net- work,

    Yi Li, Zhe Wu, Shuang Zhao, Xian Wu, Yehong Kuang, Yangtian Yan, Shen Ge, Kai Wang, Wei Fan, Xiang Chen, et al., “Psenet: Psoriasis severity evaluation net- work,” in Proceedings of the AAAI conference on artifi- cial intelligence, 2020, vol. 34, pp. 800–807

    2020

  3. [11]

    Artificial intelligence–based psoria- sis severity assessment: Real-world study and applica- tion,

    Kai Huang, Xian Wu, Yixin Li, Chengzhi Lv, Yangtian Yan, Zhe Wu, Mi Zhang, Weihong Huang, Zixi Jiang, Kun Hu, et al., “Artificial intelligence–based psoria- sis severity assessment: Real-world study and applica- tion,” Journal of Medical Internet Research, vol. 25, pp. e44932, 2023

    2023

  4. [12]

    Image-based automated psoria- sis area severity index scoring by convolutional neural networks,

    Mirjam J Schaap, Nolan J Cardozo, Ajay Pa- tel, Elke MGJ de Jong, Bram van Ginneken, and Marieke MB Seyger, “Image-based automated psoria- sis area severity index scoring by convolutional neural networks,” Journal of the European Academy of Der- matology and Venereology, vol. ...

    2022

  5. [13]

    078 remote assessment of psoriasis sever- ity with ai-based automated classification,

    MV Lucas, S Kamran, S Yip, B Lutnick, AP Shah, C Parmar, S Fakharzadeh, K Standish, L Miller, and G Cula, “078 remote assessment of psoriasis sever- ity with ai-based automated classification,” Journal of Investigative Dermatology, vol. 144, no. 12, pp. S241, 2024

    2024

  6. [14]

    Deep learning-based psoriasis assess- ment: Harnessing clinical trial imaging for accurate pso- riasis area severity index prediction,

    Yunzhao Xing, Sheng Zhong, Samuel L Aronson, Fran- cisco M Rausa, Dan E Webster, Michelle H Crouthamel, and Li Wang, “Deep learning-based psoriasis assess- ment: Harnessing clinical trial imaging for accurate pso- riasis area severity index prediction,” Digital Biomark- ers, v...

    2024

  7. [15]

    Imagenet: A large-scale hierarchical image database,

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255

    2009

  8. [16]

    Automatic psoriasis lesion segmentation in two- dimensional skin images using multiscale superpixel clustering,

    Yasmeen George, Mohammad Aldeen, and Rahil Gar- navi, “Automatic psoriasis lesion segmentation in two- dimensional skin images using multiscale superpixel clustering,” Journal of Medical Imaging , vol. 4, no. 4, pp. 044004–044004, 2017

    2017

  9. [17]

    Evaluating psoriasis with psoriasis area and severity index, psori- asis global assessment, and lattice system physician’s global assessment,

    Richard G Langley and Charles N Ellis, “Evaluating psoriasis with psoriasis area and severity index, psori- asis global assessment, and lattice system physician’s global assessment,” Journal of the American Academy of Dermatology, vol. 51, no. 4, pp. 563–569, 2004

    2004

  10. [18]

    Intra-and interobserver variability of image- based pasi assessments in 120 patients suffering from plaque-type psoriasis,

    Christine Fink, Christina Alt, Lorenz Uhlmann, Christina Klose, Alexander Enk, and Holger A Haenssle, “Intra-and interobserver variability of image- based pasi assessments in 120 patients suffering from plaque-type psoriasis,” Journal of the European Academy of Dermatology and...

    2018

  11. [19]

    Imagenet classification with deep convolutional neural networks,

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hin- ton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information pro- cessing systems, vol. 25, 2012

    2012

  12. [20]

    Pytorch: An imperative style, high-performance deep learning library,

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, ...

    2019

  13. [21]

    Adam: A method for stochastic optimization,

    Diederik P Kingma, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  14. [22]

    Objective scoring of psoriasis area and severity index in 2d rgb images using deep learning,

    Ritesh Raj, Narendra D Londhe, and Rajendra S Son- awane, “Objective scoring of psoriasis area and severity index in 2d rgb images using deep learning,” Multime- dia Tools and Applications, pp. 1–27, 2024

    2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.