REVIEW 3 major objections 5 minor 32 references
Predict Patient Self-reported Race from Skin Histological Images
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Skin pathology slides carry a detectable signal for patient self-reported race, with the epidermis as the main morphological cue the model learns.
desk verdict A carefully done, honestly limited study showing skin histology can predict self-reported race, with an epidermal shortcut that is plausible but rests on an unvalidated segmentation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is an attention-based multiple instance learning (AB-MIL) aggregator that assigns each tissue tile an attention score per race class, combined with a UMAP embedding and a Random Forest classifier trained on a few pathologist-annotated landmark tiles to segment tissue compartments such as epidermis. The AB-MIL attention scores identify which tiles drive each race prediction, and the tissue segmenter lets the authors compare attention in epidermis versus non-epidermis regions. The same segmenter powers the ablation that removes or keeps epidermal tiles to show their causal role. The epidermis is the named histological structure that carries the predictive signal.
What would settle it
Hold out slides with pathologist-traced epidermis regions, then rerun the Exp3 model with exactly those tiles masked and with only those tiles kept. If masking genuine epidermis does not reduce race-prediction AUC, or if the automated segmenter's epidermis masks disagree with the manual outlines on more than a small fraction of tiles, the epidermis-shortcut claim would be refuted.
Extended reading notes
Core claim
The central claim is that self-reported race is predictively encoded in routine skin histology independent of the clinical task. In the strict experiment (Exp3), an attention-based multiple instance learning model over four pretrained pathology foundation-model embeddings achieves one-vs-rest AUC 0.663 overall and 0.799/0.762 for White and Black patients, and this remains after disease balancing and ICD-code restriction. Attention maps concentrate on the epidermis, and removing epidermal tiles from validation slides lowers AUC by about 0.05, while keeping only epidermal tiles retains comparable performance. The paper interprets this as evidence that melanin-related epidermal morphology is a learned shortcut for race.
Load-bearing premise
The claim that the epidermis is the decisive morphological cue depends on an automated tissue-type segmentation whose accuracy is never measured, so if that segmenter mislabels tissue, the attention and ablation results could be artifacts.
Editorial extensions
If this is right
- If the claim holds, any skin-pathology model trained on routine slides can silently learn patient race even when race is not the task being diagnosed.
- Dataset curation that balances disease distribution reduces but does not eliminate race prediction, so demographic shortcuts can survive standard confounder control.
- Epidermis-focused morphology is a candidate shortcut in dermatopathology; models trained for skin-cancer or inflammatory-skin-disease tasks may rely on it and misclassify across demographic groups.
- Attention-based interpretability combined with tissue segmentation provides a practical way to audit pathology models for demographic shortcuts.
- White and Black patients being the most separable groups suggests fairness metrics should be reported per demographic subgroup, not only as an overall average.
Reading between the lines
- Beyond the paper, I infer that masking or suppressing epidermal-tile features during training could serve as a testable debiasing strategy: if the epidermis is the shortcut, models trained with those tiles down-weighted should show smaller race separability.
- The same attention-plus-segmentation audit could be applied to other organs: if non-skin tissues also show race-associated attention, the shortcut is not unique to melanin biology.
- Because self-reported race is a noisy social label, the true biological correlate may be genetic ancestry; a direct comparison of self-reported race versus ancestry-informative markers would sharpen the finding.
- The approximately 0.05 AUC drop from removing epidermis likely understates the shortcut if segmentation is imperfect and some epidermal tiles remain; a pathologist-level manual mask experiment would quantify this.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper asks whether self-reported race can be predicted from digitized dermatopathology whole-slide images using foundation-model tile embeddings followed by attention-based multiple instance learning. It reports three dataset curation strategies (uncurated, disease-balanced, and strict ICD-code) and finds that overall one-vs-rest AUC drops from 0.702 to 0.663, with White and Black groups retaining higher AUCs (0.799 and 0.762) in the strict experiment. Using a UMAP/Random-Forest tissue segmentation and attention analysis, the authors conclude that the epidermis is the dominant predictive morphological structure; removing epidermal tiles in validation reduces AUC by about 0.05. The paper includes code, multisite data, four foundation models, bootstrap-averaged AUCs, and a discussion of limitations.
Significance. If the findings hold, the paper is a valuable contribution to the emerging literature on demographic attribute leakage in medical imaging and specifically in computational pathology. The study is well-motivated, uses a clinically realistic multi-site dataset, evaluates multiple foundation models, performs patient-level splitting, and explicitly attempts to control for disease-distribution confounding through three curation schemes. The release of code and the use of pathologist-reviewed landmark annotations are also strengths. However, the central empirical claims currently rest on validation-set performance with no held-out test set, a post hoc choice of the final experiment, and an unvalidated tissue-segmentation model that drives the main morphological conclusion. These issues make the results interesting but not yet conclusive.
major comments (3)
- [§3.2, Table 2] The paper reports all AUCs from the validation set and states that 'no separate test set [was] allocated since generalization was not the focus.' This is load-bearing for the central claim that race is predictable from skin histology. In addition, Exp3 was selected only after examining the results of Exp1 and Exp2, so the reported AUCs for Exp3 inherit selection bias. I request a held-out test set that is used exactly once after all curation decisions are frozen, along with confidence intervals for the reported AUCs. Without this, the numerical values in Table 2 are not reliable estimates of predictive performance.
- [§3.3, Figure 3] The epidermis-as-shortcut conclusion depends entirely on the Random Forest tissue segmentation trained in UMAP space on 'a few pathologist-annotated landmark tiles,' yet no segmentation accuracy is reported: no Dice, F1, pixel-level agreement, or validation against held-out annotations. This is especially concerning because UMAP is stochastic and the fitted transform is applied to out-of-sample tiles, so the ROI boundaries may not be reliable. Since both the attention comparison in Figure 3A and the ablation in Figure 3B remove or retain tiles based on this segmentation, an inaccurate segmentation could make the epidermal attention and ablation findings artifacts. Please report segmentation performance on held-out annotated tiles and, ideally, repeat the attention/ablation analysis with a second segmentation approach.
- [§4.2, Figure 3B] The 'epidermis-only' condition is difficult to interpret. The paper notes that 85% of validation slides contain less than 20% epidermis tiles, so evaluating AUC on such slides is not a like-for-like comparison with the full-slide model. The number of slides used for each epidermis-only AUC, the distribution of epidermis tile counts, and the chance-level AUC should be reported. A more informative analysis would stratify by the fraction of epidermal tiles or evaluate tile-level predictions within matched slides.
minor comments (5)
- [Table 2] The table reports means over 1000 bootstrap iterations but provides no confidence intervals or measures of variability. Adding 95% CIs (or at least standard deviations across seeds) would help the reader assess the stability of the differences between experiments and racial groups.
- [§4.2, Figure 3A] One-sided paired t-tests are performed across multiple racial groups and experiments without correction for multiple comparisons. Please state whether any correction was applied, or interpret the uncorrected p-values as exploratory.
- [§3.3] The list of morphological classes includes 'gastrointestinal (GI) tissue' and 'smooth muscle' in what is described as a dermatopathology dataset. This is surprising and should be clarified: were such structures truly present in the skin slides, or do they reflect a broader training set used for the UMAP atlas?
- [§3.1, Table 1] The sentence 'Although the White group is slightly overrepresented (39.3%)' is confusing because 39.3% is the health-system percentage, not the skin-cohort percentage (40.8%). Please clarify whether the comparison is to the health system, the city population, or to TCGA.
- [§3.2] The number of patients in the training and validation splits is not stated. Please report the patient counts per split and per racial group, as the effective sample size for the Asian and Other groups in Exp3 may be small.
Circularity Check
No significant circularity: the race-prediction AUCs are supervised empirical measurements on a held-out patient split, and the epidermis finding rests on an independently trained tissue segmentation plus ablations, not on a by-construction reduction.
full rationale
The paper's derivation chain does not contain a step in which a prediction or first-principles result is equivalent to its inputs by construction. The central numerical claims are the validation AUCs in Table 2. These are obtained by training an AB-MIL classifier on tile embeddings from four different foundation models, with self-reported race as the label, and evaluating on an 80/20 patient-level split. That is a standard supervised empirical measurement, not a self-definitional or fitted-parameter-then-predicted construction. The epidermis claim is supported by two independent lines of evidence: attention-score comparisons and ablations. The tissue segmentation used to define epidermis ROIs (section 3.3) is built from UMAP projections of SP22M features and a Random Forest trained on pathologist-annotated landmark tiles. It is not defined in terms of the race labels or the attention scores, so comparing attention between epidermis and non-epidermis regions is an empirical post hoc analysis rather than a tautology. The ablation in Figure 3B removes epidermis tiles from the validation data and measures the resulting AUC change; this is a physical manipulation of the input, not a re-statement of the training objective. The paper does contain limitations that are correctness risks, including the absence of a separate test set, the lack of reported segmentation accuracy metrics, and the spatially unaware AB-MIL aggregator; however, these affect validity and generalizability, not circularity. Self-citations to SP22M and related benchmark papers are present, but they are not load-bearing for the central claim because the main AUC results are reported across four encoders, including the externally developed UNI, GigaPath, and Virchow, and the findings are consistent across them. No uniqueness theorem, ansatz smuggled in via citation, or renaming of a known result is present. Therefore, no circular step can be exhibited with the required specificity, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- ROI pixel threshold =
20%
- Minimum epidermis tiles per slide =
15
- High-attention contour threshold =
top 10%
assumptions (5)
- domain assumption Pretrained foundation models provide meaningful tile-level embeddings for skin histology.
- domain assumption Self-reported race from patient records is a valid target label.
- domain assumption The UMAP/Random Forest tissue segmentation accurately identifies epidermal regions.
- domain assumption Central laboratory staining and digitization eliminates site-specific digital signatures.
- domain assumption ICD-10 codes reflect clinically meaningful skin disease categories.
Cite this review
Pith. "Pith review of Predict Patient Self-reported Race from Skin Histological Images." pith.science (2026). https://pith.science/paper/KZQCHWVW
@misc{pith2026250721912,
author = {Pith},
title = {Pith review of: Predict Patient Self-reported Race from Skin Histological Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/KZQCHWVW}},
note = {Machine review of arXiv:2507.21912}
}
read the original abstract
Artificial Intelligence (AI) has demonstrated success in computational pathology (CPath) for disease detection, biomarker classification, and prognosis prediction. However, its potential to learn unintended demographic biases, particularly those related to social determinants of health, remains understudied. This study investigates whether deep learning models can predict self-reported race from digitized dermatopathology slides and identifies potential morphological shortcuts. Using a multisite dataset with a racially diverse population, we apply an attention-based mechanism to uncover race-associated morphological features. After evaluating three dataset curation strategies to control for confounding factors, the final experiment showed that White and Black demographic groups retained high prediction performance (AUC: 0.799, 0.762), while overall performance dropped to 0.663. Attention analysis revealed the epidermis as a key predictive feature, with significant performance declines when these regions were removed. These findings highlight the need for careful data curation and bias mitigation to ensure equitable AI deployment in pathology. Code available at: https://github.com/sinai-computational-pathology/CPath_SAIF.
Figures
Reference graph
Works this paper leans on
-
[1]
Laleh Seyyed-Kalantari, Haoran Zhang, Matthew BA McDermott, Irene Y Chen, and Marzyeh Ghassemi. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature medicine, 27(12):2176–2182, 2021
work page 2021
-
[2]
Bias in artificial intelligence algorithms and recommendations for mitigation
Lama H Nazer, Razan Zatarah, Shai Waldrip, Janny Xue Chen Ke, Mira Moukheiber, Ashish K Khanna, Rachel S Hicklen, Lama Moukheiber, Dana Moukheiber, Haobo Ma, et al. Bias in artificial intelligence algorithms and recommendations for mitigation. PLOS Digital Health, 2(6):e0000278, 2023
work page 2023
-
[3]
A causal perspective on dataset bias in machine learning for medical imaging
Charles Jones, Daniel C Castro, Fabio De Sousa Ribeiro, Ozan Oktay, Melissa McCradden, and Ben Glocker. A causal perspective on dataset bias in machine learning for medical imaging. Nature Machine Intelligence, 6(2):138–146, 2024
work page 2024
-
[4]
An adversarial training framework for mitigating algorithmic biases in clinical machine learning
Jenny Yang, Andrew AS Soltan, David W Eyre, Yang Yang, and David A Clifton. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ digital medicine, 6(1):55, 2023
work page 2023
-
[5]
Hidden stratification causes clinically meaningful failures in machine learning for medical imaging
Luke Oakden-Rayner, Jared Dunnmon, Gustavo Carneiro, and Christopher Ré. Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. In Proceedings of the ACM conference on health, inference, and learning, pages 151–159, 2020
work page 2020
-
[6]
The impact of site-specific digital histology signatures on deep learning model accuracy and bias
Frederick M Howard, James Dolezal, Sara Kochanny, Jefree Schulte, Heather Chen, Lara Heij, Dezheng Huo, Rita Nanda, Olufunmilayo I Olopade, Jakob N Kather, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nature communications, 12(1):4423, 2021
work page 2021
-
[7]
The risk of shortcutting in deep learning algorithms for medical imaging research
Brandon G Hill, Frances L Koback, and Peter L Schilling. The risk of shortcutting in deep learning algorithms for medical imaging research. Scientific reports, 14(1):29224, 2024
work page 2024
-
[8]
Computational pathology: a survey review and the way forward
Mahdi S Hosseini, Babak Ehteshami Bejnordi, Vincent Quoc-Huy Trinh, Lyndon Chan, Danial Hasan, Xingwen Li, Stephen Yang, Taehyo Kim, Haochen Zhang, Theodore Wu, et al. Computational pathology: a survey review and the way forward. Journal of Pathology Informatics, page 100357, 2024
work page 2024
Show all 32 references
-
[9]
From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology
Omar SM El Nahhas, Marko van Treeck, Georg Wölflein, Michaela Unger, Marta Ligero, Tim Lenz, Sophia J Wagner, Katherine J Hewitt, Firas Khader, Sebastian Foersch, et al. From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational...
2025
-
[10]
Artificial intelligence for digital and computational pathology
Andrew H Song, Guillaume Jaume, Drew FK Williamson, Ming Y Lu, Anurag Vaidya, Tiffany R Miller, and Faisal Mahmood. Artificial intelligence for digital and computational pathology. Nature Reviews Bioengineering, 1(12):930–949, 2023
2023
-
[11]
Demographic bias in misdiagnosis by computational pathology models
Anurag Vaidya, Richard J Chen, Drew FK Williamson, Andrew H Song, Guillaume Jaume, Yuzhe Yang, Thomas Hartvigsen, Emma C Dyer, Ming Y Lu, Jana Lipkova, et al. Demographic bias in misdiagnosis by computational pathology models. Nature Medicine, 30(4):1174–1190, 2024
2024
-
[12]
The limits of fair medical imaging ai in real-world generalization
Yuzhe Yang, Haoran Zhang, Judy W Gichoya, Dina Katabi, and Marzyeh Ghassemi. The limits of fair medical imaging ai in real-world generalization. Nature Medicine, 30(10):2838–2848, 2024
2024
-
[13]
Ai recognition of patient race in medical imaging: a modelling study
Judy Wawira Gichoya, Imon Banerjee, Ananth Reddy Bhimireddy, John L Burns, Leo Anthony Celi, Li-Ching Chen, Ramon Correa, Natalie Dullerud, Marzyeh Ghassemi, Shih-Cheng Huang, et al. Ai recognition of patient race in medical imaging: a modelling study. The Lancet Digital Healt...
2022
-
[14]
Predicting patient demographics from chest radiographs with deep learning
Jason Adleberg, Amr Wardeh, Florence X Doo, Brett Marinelli, Tessa S Cook, David S Mendelson, and Alexander Kagen. Predicting patient demographics from chest radiographs with deep learning. Journal of the American College of Radiology, 19(10):1151–1161, 2022
2022
-
[15]
Skin of color in dermatopathology: does color matter? Human Pathology, 140:240–266, 2023
Kyle A Williams, Bitania Wondimu, Ayodeji M Ajayi, and Olayemi Sokumbi. Skin of color in dermatopathology: does color matter? Human Pathology, 140:240–266, 2023
2023
-
[16]
Integrating skin color assessments into clinical practice and research: a review of current approaches
Valerie M Harvey, Andrew Alexis, Chidubem A V Okeke, Lynn McKinley-Grant, Susan C Taylor, Seemal R Desai, Tarannum Jaleel, Candrice R Heath, Sewon Kang, Neelam Vashi, et al. Integrating skin color assessments into clinical practice and research: a review of current approaches....
2024
-
[17]
A clinical benchmark of public self-supervised pathology foundation models
Gabriele Campanella, Shengjia Chen, Ruchika Verma, Jennifer Zeng, Aryeh Stock, Matt Croken, Brandon Veremis, Abdulkadir Elmas, Kuan-lin Huang, Ricky Kwan, et al. A clinical benchmark of public self-supervised pathology foundation models. arXiv preprint arXiv:2407.06508, 2024
2024 arXiv
-
[18]
Attention-based deep multiple instance learning
Maximilian Ilse, Jakub Tomczak, and Max Welling. Attention-based deep multiple instance learning. In International conference on machine learning, pages 2127–2136. PMLR, 2018. 8 Predict Patient Self-reported Race from Skin Histological Images Chen et al
2018
-
[19]
Benchmarking embedding aggregation methods in computational pathology: A clinical data perspective
Shengjia Chen, Gabriele Campanella, Abdulkadir Elmas, Aryeh Stock, Jennifer Zeng, Alexandros D Polydorides, Adam J Schoenfeld, Kuan-lin Huang, Jane Houldsworth, Chad Vanderbilt, et al. Benchmarking embedding aggregation methods in computational pathology: A clinical data persp...
2024 arXiv
-
[20]
Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated pathology slides
Adalberto Claudio Quiros, Nicolas Coudray, Anna Yeaton, Xinyu Yang, Bojing Liu, Hortense Le, Luis Chiriboga, Afreen Karimkhan, Navneet Narula, David A Moore, et al. Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unannotated path...
2024
-
[21]
Implications of predicting race variables from medical images
James Zou, Judy Wawira Gichoya, Daniel E Ho, and Ziad Obermeyer. Implications of predicting race variables from medical images. Science, 381(6654):149–150, 2023
2023
-
[22]
Risk of bias in chest radiography deep learning foundation models
Ben Glocker, Charles Jones, Mélanie Roschewitz, and Stefan Winzeck. Risk of bias in chest radiography deep learning foundation models. Radiology: Artificial Intelligence, 5(6):e230060, 2023
2023
-
[23]
Analysing race and sex bias in brain age prediction
Carolina Piçarra and Ben Glocker. Analysing race and sex bias in brain age prediction. In Workshop on Clinical Image-Based Procedures, pages 194–204. Springer, 2023
2023
-
[24]
Bias in histopathology datasets: A comprehensive investigation on possible factors
F Kheiri, S Rahnamayan, M Makrehchi, and AA Bidgoli. Bias in histopathology datasets: A comprehensive investigation on possible factors. 2024
2024
-
[25]
Computational pathology at health system scale–self-supervised foundation models from three billion images
Gabriele Campanella, Ricky Kwan, Eugene Fluder, Jennifer Zeng, Aryeh Stock, Brandon Veremis, Alexandros D Polydorides, Cyrus Hedvat, Adam Schoenfeld, Chad Vanderbilt, et al. Computational pathology at health system scale–self-supervised foundation models from three billion ima...
-
[26]
Towards a general-purpose foundation model for computational pathology
Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Andrew H Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30(3):850–862, 2024
2024
-
[27]
A whole-slide foundation model for digital pathology from real-world data
Hanwen Xu, Naoto Usuyama, Jaspreet Bagga, Sheng Zhang, Rajesh Rao, Tristan Naumann, Cliff Wong, Zelalem Gero, Javier González, Yu Gu, et al. A whole-slide foundation model for digital pathology from real-world data. Nature, pages 1–8, 2024
2024
-
[28]
A foundation model for clinical-grade computational pathology and rare cancers detection
Eugene V orontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Kristen Severson, Eric Zimmermann, James Hall, Neil Tenenholtz, Nicolo Fusi, et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nature medicine,...
2024
-
[29]
Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5, 2017
Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5, 2017
2017 arXiv
-
[30]
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010
2010
-
[31]
Umap: Uniform manifold approximation and projection for dimension reduction
Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018
2018 arXiv
-
[32]
Data- efficient and weakly supervised computational pathology on whole-slide images
Ming Y Lu, Drew FK Williamson, Tiffany Y Chen, Richard J Chen, Matteo Barbieri, and Faisal Mahmood. Data- efficient and weakly supervised computational pathology on whole-slide images. Nature biomedical engineering, 5(6):555–570, 2021. 9
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.