REVIEW 4 major objections 5 minor 28 references
Cross-Modality Learning for Predicting IHC Biomarkers from H&E-Stained Whole-Slide Images
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read HistoStainAlign predicts P53, PD-L1, and Ki-67 IHC staining patterns directly from H&E whole-slide images by aligning paired slide embeddings during training.
desk verdict Sensible contrastive training framework, but the headline IHC-prediction claim is undermined by missing majority baselines and the authors' own significance tests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanical core is HistoStainAlign's three-loss training objective over slide-level embeddings produced by the Prov-GigaPath foundation model, a ViT tile encoder plus a LongNet-based slide encoder. The inter-modality loss is a symmetric contrastive loss adapted from the TANGLE framework: within a batch of paired H&E and IHC slides, each H&E embedding is matched to its paired IHC embedding against all others in the batch, and symmetrically each IHC embedding is matched back to its H&E partner. The intra-modality loss feeds the same H&E slide twice, once with 50% of patches randomly masked, and penalizes the mean squared error between the two slide embeddings, promoting local consistency. The class-based loss supervises the H&E slide embedding with the slide-level biomarker label. At inference, only the H&E slide is used; the IHC slide has disappeared.
What would settle it
Train HistoStainAlign with the same H&E slides but with IHC partners randomly shuffled across cases; if F1 scores on P53 and Ki-67 do not drop materially, then the paired-slide alignment is not the source of the gain and the method's central premise is unsupported.
Extended reading notes
Core claim
The central claim is that paired but unregistered whole-slide images of the same case can serve as a training signal that teaches a slide encoder to read molecular staining patterns from H&E morphology alone. HistoStainAlign takes the GigaPath tile and slide encoders, embeds both the H&E and the IHC slide of a case, and optimizes three losses together: a symmetric contrastive loss pulling paired H&E and IHC embeddings together while pushing unpaired pairs apart, a mean-squared-error loss forcing the slide embedding of an H&E slide to match its own embedding computed from 50% masked patches, and a supervised classification loss on H&E embeddings. Measured by nested five-fold cross-validation, the result is weighted F1 of 0.735 for P53, 0.830 for PD-L1, and 0.723 for Ki-67, with the PD-L1 improvement over GigaPath-Finetuned statistically significant and the others positive but not significant. The authors conclude that the contrastive alignment is responsible for the gain, supported by cosine-similarity analyses showing a larger gap between paired and shuffled cross-modality similarities than the fine-tuned baseline.
Load-bearing premise
The load-bearing premise is that an unregistered IHC slide cut from a different physical section of the same case carries the molecular ground truth for the H&E slide, and that the one slide-level biomarker label applies uniformly across the entire slide.
Editorial extensions
If this is right
- A model trained with HistoStainAlign can be run on H&E slides alone at test time, so the clinical workflow needs no additional staining, registration, or patch-level annotations.
- For PD-L1 in lung resections, the framework's improvement over GigaPath-Finetuned is statistically significant, indicating that paired IHC slides during training can add real predictive signal beyond foundation-model fine-tuning.
- The self-supervised variant pretrained on the public ACROBAT breast cancer Ki-67 dataset improves Ki-67 classification on gastrointestinal samples, suggesting the alignment objective transfers across tissue types.
- Cosine-similarity measurements show HistoStainAlign embeddings separate matched from unmatched cross-modality pairs more strongly than the baseline, supporting the claim that the contrastive loss changes the embedding geometry as intended.
- These results position the framework as a potential pre-screening tool that could prioritize cases for actual IHC staining rather than replace it.
Reading between the lines
- Extending beyond the paper, the unregistered-pair assumption could be stress-tested by using H&E and IHC sections cut from different blocks of the same tumor; if performance drops sharply when the tissue sections diverge, the model is relying on global case-level correlation rather than a stable morphology-to-molecule mapping.
- Extending beyond the paper, the method's value for clinical triage would be clearer if the model produced spatial prediction maps from H&E and these were compared with the actual IHC stain distribution, since the current slide-level label cannot show whether the model learned localized correspondences or only aggregate case statistics.
- A further extension would be to apply the same contrastive alignment at finer label granularity, for example regressing the continuous PD-L1 TPS or Ki-67 index instead of binarizing at 50% or 3%; the 1% PD-L1 positivity threshold used in clinical trials is a natural next target.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. HistoStainAlign is a contrastive learning framework that predicts IHC biomarker patterns (P53, PD-L1, Ki-67) from H&E-stained whole-slide images (WSIs). The method fine-tunes Prov-GigaPath using paired but unregistered H&E and IHC slides during training, optimizing inter-modality, intra-modality, and classification losses; at inference only H&E images are used. The authors evaluate on three in-house datasets (180, 181, and 61 WSIs) and report weighted F1 scores of 0.735, 0.830, and 0.723 for P53, PD-L1, and Ki-67, respectively, alongside comparisons with CONCH, UNI, and GigaPath baselines. They claim the proposed framework consistently outperforms baseline and fine-tuned GigaPath variants across all tasks.
Significance. The paper addresses a clinically relevant problem: predicting IHC status from H&E slides to reduce cost and turnaround time. The method leverages a modern foundation model (Prov-GigaPath), avoids patch-level annotations and tissue registration, and uses patient-level data splits with bootstrapped confidence intervals. The inclusion of a self-supervised variant and use of the public ACROBAT dataset for Ki-67 are positive aspects. Code and model are promised to be released. However, the reported evidence is substantially weaker than the claims: for two of the three tasks the improvement over GigaPath-Finetuned is not statistically significant, the absolute F1 gains over a trivial majority-class predictor are negligible (P53 and PD-L1), and the statistical test used is questionable. The framework is potentially useful, but the current manuscript overstates the results and needs a rigorous evaluation of baselines and significance.
major comments (4)
- [Datasets / Table 1 and Model Performance / Table 2] The paper does not compare against a majority-class baseline. From Table 1, the class distributions yield majority-class weighted F1 scores of 0.733 for P53 (132/180 wild-type), 0.823 for PD-L1 (149/181 low), and 0.672 for Ki-67 (41/61 low). The reported HistoStainAlign F1 scores are 0.735, 0.830, and 0.723, i.e., margins of +0.002, +0.007, and +0.051. For P53 and PD-L1, these margins are within the reported confidence intervals and essentially indistinguishable from guessing the majority class. The authors must report a majority-class baseline and perform a significance test against it before claiming that the model has predictive value.
- [Statistical Comparison of Performance of Models (p.17)] The Wilcoxon signed-rank test is applied only to 'prediction scores of true positive samples.' This is not a valid test for overall classifier comparison because it ignores true negatives, false positives, and false negatives. For example, a classifier that always predicts the majority class would have no true-positive scores for the minority class and could artifactually appear to perform well under this test. The authors should compare the full classification outputs (e.g., McNemar's test, DeLong test for AUC, or bootstrap over the full test set). The current test does not support the claim of significant improvement, including for PD-L1.
- [Discussion (p.20)] The statement 'the model consistently outperformed baseline and fine-tuned GigaPath variants across all classification tasks' is directly contradicted by the authors' own results. The Statistical Comparison section reports p=0.32 for P53 and p=0.34 for Ki-67, and Table 2 shows HistoStainAlign AUC is lower than GigaPath-Finetuned for both P53 (0.726 vs 0.750) and Ki-67 (0.663 vs 0.672). The claims in the Abstract and Discussion must be revised to reflect that only PD-L1 shows a statistically significant improvement, and even that result rests on the questionable test described above.
- [Evaluation (p.15)] The evaluation protocol is ambiguous: after training the model with the class-based loss, the authors state that 'a logistic regression model was fitted to the training set of embeddings.' It is unclear whether the reported metrics come from the trained classification head or from a separately fitted logistic regression on the slide embeddings. If the latter, the class-based loss and the classification head appear redundant and the end-to-end training procedure is not what is evaluated. The authors should clarify the exact evaluation pipeline, including how the validation and test sets are used with the logistic regression, and whether the logistic regression is trained on the same training fold used for the deep model.
minor comments (5)
- [Loss Functions (Eq. 1)] The inter-modality loss formula uses symbols K+ and Q without defining them; the text defines q_i and k_i^+ but the cross-entropy arguments use matrix notation. Please define all variables, including the temperature τ and the batch composition.
- [Evaluation (p.15)] The text says '5-fold nested cross-validation,' but the described procedure (one 60/20/20 split per fold) is a standard 5-fold cross-validation, not nested cross-validation. Nested cross-validation requires an inner loop for hyperparameter tuning; if no such tuning was performed, please use the correct term.
- [Cosine Similarity Analysis / Table 3] Table 3 shows that the paired cosine similarity of HistoStainAlign is lower than that of GigaPath-Finetuned for P53 (0.755 vs 0.901) and Ki-67 (0.753 vs 0.774), although the paired-versus-shuffled difference is larger. The manuscript emphasizes only the difference, which may mislead readers. The absolute values should be discussed, as they indicate that HistoStainAlign embeddings are less aligned in absolute terms, even if more discriminative.
- [Throughout] There are several typos and formatting issues, e.g., 'staining patten samples' on page 8, 'ACROBAT23' in the datasets section, and inconsistent use of hyphens in 'HistoStainAlign-SSL.' A careful proofreading pass is recommended.
- [Hyperparameters] The paper does not report the values of key hyperparameters (temperature τ, masking ratio, loss weights, batch size, learning rate, number of epochs). Providing these would improve reproducibility and allow readers to gauge the sensitivity of the method.
Circularity Check
No significant circularity: supervised cross-modal training evaluated on held-out H&E slides; paired IHC is only a training signal.
full rationale
HistoStainAlign is an empirical supervised learning benchmark rather than a derivation chain that reduces to its inputs. At inference, the model consumes only H&E WSIs; paired IHC embeddings appear exclusively in the training losses (L_inter, L_intra), alongside slide-level labels for L_class. The central claim that contrastive training improves prediction is tested against GigaPath-Finetuned (trained with only L_class), GigaPath-Base, CONCH, and UNI, so the comparison is not manufactured by the method's own construction. The fact that both the training signal and the ground-truth labels originate from IHC slides is ordinary supervised learning, not circularity: no parameter is fitted to a subset and then reported as a prediction of that same subset, and no predicted quantity is defined in terms of the input quantity by construction. The cosine-similarity analysis verifies that the inter-modality loss changes the embedding geometry, which is a sanity check of the training objective rather than a claim derived from it. Self-citations (e.g., refs. 13 and 24) are related methodology and are not load-bearing for the performance claims. Concerns about the small margins over majority-class baselines and the non-significant Wilcoxon results for P53 and Ki-67 are correctness/evidence issues, not circularity. Therefore the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (3)
- Contrastive temperature tau =
not reported
- Masking ratio for intra-modality loss =
50%
- Loss weights =
1.0 each
assumptions (4)
- domain assumption Paired H&E and IHC whole slides from the same case are meaningfully aligned at slide level despite no pixel registration.
- domain assumption IHC-derived slide-level labels (P53 pattern, TPS score, Ki-67 index) are uniform across the whole slide and transfer to the H&E slide.
- domain assumption Prov-GigaPath tile encoder, pretrained on H&E and IHC patches, embeds both modalities in a shared space suitable for contrastive learning.
- standard math Classical machine learning losses (cross-entropy, MSE, InfoNCE-style contrastive loss) behave as expected in this setting.
Cite this review
Pith. "Pith review of Cross-Modality Learning for Predicting IHC Biomarkers from H&E-Stained Whole-Slide Images." pith.science (2026). https://pith.science/paper/VLKKNUSU
@misc{pith2026250615853,
author = {Pith},
title = {Pith review of: Cross-Modality Learning for Predicting IHC Biomarkers from H&E-Stained Whole-Slide Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/VLKKNUSU}},
note = {Machine review of arXiv:2506.15853}
}
read the original abstract
Hematoxylin and Eosin (H&E) staining is a cornerstone of pathological analysis, offering reliable visualization of cellular morphology and tissue architecture for cancer diagnosis, subtyping, and grading. Immunohistochemistry (IHC) staining provides molecular insights by detecting specific proteins within tissues, enhancing diagnostic accuracy, and improving treatment planning. However, IHC staining is costly, time-consuming, and resource-intensive, requiring specialized expertise. To address these limitations, this study proposes HistoStainAlign, a novel deep learning framework that predicts IHC staining patterns directly from H&E whole-slide images (WSIs) by learning joint representations of morphological and molecular features. The framework integrates paired H&E and IHC embeddings through a contrastive training strategy, capturing complementary features across staining modalities without patch-level annotations or tissue registration. The model was evaluated on gastrointestinal and lung tissue WSIs with three commonly used IHC stains: P53, PD-L1, and Ki-67. HistoStainAlign achieved weighted F1 scores of 0.735 [95% Confidence Interval (CI): 0.670-0.799], 0.830 [95% CI: 0.772-0.886], and 0.723 [95% CI: 0.607-0.836], respectively for these three IHC stains. Embedding analyses demonstrated the robustness of the contrastive alignment in capturing meaningful cross-stain relationships. Comparisons with a baseline model further highlight the advantage of incorporating contrastive learning for improved stain pattern prediction. This study demonstrates the potential of computational approaches to serve as a pre-screening tool, helping prioritize cases for IHC staining and improving workflow efficiency.
Figures
Reference graph
Works this paper leans on
-
[1]
Chan, J. K. C. The wonderful colors of the hematoxylin-eosin stain in diagnostic surgical pathology. Int J Surg Pathol 22, 12–32 (2014)
work page 2014
-
[2]
Magaki, S., Hojat, S. A., Wei, B., So, A. & Yong, W. H. An Introduction to the Performance of Immunohistochemistry. Methods Mol Biol 1897, 289–298 (2019)
work page 2019
-
[3]
Liu, Y., Su, Z., Tavana, O. & Gu, W. Understanding the complexity of p53 in a new era of tumor suppression. Cancer Cell 42, 946–967 (2024)
work page 2024
-
[4]
Shiravand, Y., Khodadadi, F., Kashani, S. M. A., Hosseini-Fard, S. R., Hosseini, S., Sadeghirad, H. et al. Immune Checkpoint Inhibitors in Cancer Therapy. Curr Oncol 29, 3044–3060 (2022)
work page 2022
-
[5]
Andrés-Sánchez, N., Fisher, D. & Krasinska, L. Physiological functions and roles in cancer of the proliferation marker Ki-67. J Cell Sci 135, jcs258932 (2022)
work page 2022
-
[6]
van der Laak, J., Litjens, G. & Ciompi, F. Deep learning in histopathology: the path to the clinic. Nat Med 27, 775–784 (2021)
work page 2021
-
[7]
Su, A., Lee, H., Tan, X., Suarez, C. J., Andor, N., Nguyen, Q. et al. A deep learning model for molecular label transfer that enables cancer cell identification from histopathology images. npj Precis. Onc. 6, 1–11 (2022)
work page 2022
-
[8]
Frascarelli, C., Venetis, K., Marra, A., Mane, E., Ivanova, M., Cursano, G. et al. Deep learning algorithm on H&E whole slide images to characterize TP53 alterations frequency and spatial distribution in breast cancer. Comput Struct Biotechnol J 23, 4252–4259 (2024)
work page 2024
Show all 28 references
-
[9]
Shamai, G., Livne, A., Polónia, A., Sabo, E., Cretu, A., Bar-Sela, G. et al. Deep learning-based image analysis predicts PD-L1 status from H&E-stained histopathology images in breast cancer. Nat Commun 13, 6753 (2022). 25
2022
-
[10]
Liu, Y., Li, X., Zheng, A., Zhu, X., Liu, S., Hu, M. et al. Predict Ki-67 Positive Cells in H&E-Stained Images Using Deep Learning Independently From IHC-Stained Images. Front. Mol. Biosci. 7, (2020)
2020
-
[11]
C., Nie, J., Liu, H., Song, Q., Yan, L
Atabansi, C. C., Nie, J., Liu, H., Song, Q., Yan, L. & Zhou, X. A survey of Transformer applications for histopathological image analysis: New developments and future directions. BioMedical Engineering OnLine 22, 96 (2023)
2023
- [12]
-
[13]
S., Workman, A
McMahon, J., Tomita, N., Tatishev, E. S., Workman, A. A., Costales, C. R., Banaei, N. et al. A novel framework for the automated characterization of Gram-stained blood culture slides using a large-scale vision transformer. Journal of Clinical Microbiology 63, e01514-24 (2025)
2025
-
[14]
https://www.nature.com/articles/s41586-024-07441-w
A whole-slide foundation model for digital pathology from real-world data | Nature. https://www.nature.com/articles/s41586-024-07441-w
-
[15]
Nguyen, A.-T., Nguyen, D. M. H., Diep, N. T., Nguyen, T. Q., Ho, N., Metsch, J. M. et al. MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classification. (2025) doi:10.48550/arXiv.2502.07409
2025 doi
-
[16]
PD-L1 IHC 22C3 pharmDx, Interpretation Manual, NSCLC 1% 50%
Krohn, D. PD-L1 IHC 22C3 pharmDx, Interpretation Manual, NSCLC 1% 50%
-
[17]
G., Hui, R., Csőszi, T., Fülöp, A
Reck, M., Rodríguez-Abreu, D., Robinson, A. G., Hui, R., Csőszi, T., Fülöp, A. et al. Pembrolizumab versus Chemotherapy for PD-L1–Positive Non–Small-Cell Lung Cancer. New England Journal of Medicine 375, 1823–1833 (2016)
2016
-
[18]
B., Wei, W., Gupta, S., Zugazagoitia, J., Robbins, C., Adamson, B
Doroshow, D. B., Wei, W., Gupta, S., Zugazagoitia, J., Robbins, C., Adamson, B. et al. Programmed Death-Ligand 1 Tumor Proportion Score and Overall Survival From First-Line 26 Pembrolizumab in Patients With Nonsquamous Versus Squamous NSCLC. Journal of Thoracic Oncology 16, 21...
2021
-
[19]
Morita, M., Tamiya, M., Fujimoto, D., Tamiya, A., Suzuki, H., Hirano, K. et al. Prediction of patients with a tumor proportion score > 50% who do not respond to first-line monotherapy with pembrolizumab. BMC Cancer 20, 93 (2020)
2020
-
[20]
Kabir, S., Chowdhury, M. E. H., Sarmun, R., Vranić, S., Saady, R. M. A., Rose, I. et al. A novel deep learning framework for automatic scoring of PD-L1 expression in non-small cell lung cancer. Biomolecules and Biomedicine (2025) doi:10.17305/bb.2025.12056
2025
-
[21]
Huang, Z., Chen, L., Lv, L., Fu, C.-C., Jin, Y., Zheng, Q. et al. A new AI-assisted scoring system for PD-L1 expression in NSCLC. Computer Methods and Programs in Biomedicine 221, 106829 (2022)
2022
-
[22]
Wang, C., Ma, J., Shao, J., Zhang, S., Li, J., Yan, J. et al. Non-Invasive Measurement Using Deep Learning Algorithm Based on Multi-Source Features Fusion to Predict PD-L1 Expression and Survival in NSCLC. Front Immunol 13, 828560 (2022)
2022
-
[23]
Weitz, P., Valkonen, M., Solorzano, L., Carr, C., Kartasalo, K., Boissin, C. et al. A Multi-Stain Breast Cancer Histological Whole-Slide-Image Data Set from Routine Diagnostics. Sci Data 10, 562 (2023)
2023
-
[24]
Jiang, S., Hondelink, L., Suriawinata, A. A. & Hassanpour, S. Masked pre-training of transformers for histology image analysis. Journal of Pathology Informatics 15, 100386 (2024)
2024
- [25]
-
[26]
Y., Chen, B., Williamson, D
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T. et al. A visual- language foundation model for computational pathology. Nat Med 30, 863–874 (2024). 27
2024
-
[27]
J., Ding, T., Lu, M
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H. et al. Towards a general-purpose foundation model for computational pathology. Nat Med 30, 850–862 (2024)
2024
-
[28]
Fundytus, A., Booth, C. M. & Tannock, I. F. How low can you go? PD-L1 expression as a biomarker in trials of cancer immunotherapy. Annals of Oncology 32, 833–836 (2021)
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.