REVIEW 4 major objections 4 minor 32 references
This paper claims that a pathology foundation model can recover cell-level morphological signal by attention-pooling local patch tokens with the global class token, and that this yields more accurate classification than reading only the cla
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
2026-08-03 23:29 UTC pith:X52NMCYV
load-bearing objection The paper's real result is a modest but plausible patch-level classification gain from attention pooling over DINOv3 tokens; the advertised biomarker claim is untested and the manuscript front matter doesn't match the body. the 4 major comments →
Towards Cellular-Scale Interpretability in Pathology Foundation Models for Biomarker Assessment
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is the Joint-Weighted Token Hierarchy: a frozen vision-transformer encoder's local patch tokens, once stabilized by Gram-anchored post-training, can be aggregated by attention pooling whose query is the global class token. This yields a representation that carries both global tissue context and fine-grained local morphology. The authors show this beats linear probing on the class token alone across five classification benchmarks, with the ablation attributing 2.3 points of average balanced accuracy to staining augmentation and 3.3 points to attention pooling. They interpret this as evidence that cell-level cues—nuclear morphology, tissue microarchitecture—are latent in
What carries the argument
The key machinery is the attention-pooling head over the encoder's final layer, combined with two training choices that make local tokens worth pooling. Gram-anchored post-training adds a regularizer that encourages diversity among local token embeddings, counteracting the collapse that the authors say occurs in large-scale pretraining. Random staining augmentation during pretraining perturbs LAB and HSV color channels to make representations robust to staining differences. Together they let a lightweight attention head fuse the global class token with local patch tokens to produce the prediction.
Load-bearing premise
The claim rests on local patch tokens, after Gram-anchored post-training, carrying genuine cell-level morphological signal that attention pooling can turn into better predictions; no cellular annotations or true biomarker outcomes are used in the paper's experiments.
What would settle it
Run the exact JWTH pipeline—Gram-anchored post-training plus attention pooling—on a true biomarker task with patient-level labels, such as MSI or HER2 status from H&E slides. If linear probing of the class token matches or outperforms attention pooling on that task, the central claim that local tokens carry cell-level biomarker signal collapses. A cheaper check is to ablate attention pooling to mean pooling; equal performance would show the attention mechanism is not doing the work.
If this is right
- Linear-probing evaluations of frozen pathology encoders may under-report what the models already know: attention pooling over local tokens improves balanced accuracy by up to 8.3 points and 1.2 points on average across the tested tasks.
- The two components contribute additively: staining augmentation adds about 2.3 points and attention pooling adds about 3.3 points, so both are needed for the full gain.
- Because the encoder stays frozen and only the attention head is trained, the same upgrade can be applied to existing pathology foundation models without expensive fine-tuning.
- If the cell-level claim holds, H&E-based biomarker screening could become more interpretable: predictions would point to specific local morphology rather than a single global score.
Where Pith is reading between the lines
- Beyond the paper's evidence: the reported benchmarks are tissue-type classification, not molecular or genetic biomarker tasks; the 'biomarker' framing is an extrapolation that remains untested.
- The abstract describes a different model, Hireca/CytoMap, with pathologist evaluation on 10 biomarker tasks; the main text instead validates JWTH on patch-level tissue classification, so the two result sets should not be conflated.
- A cheap ablated experiment comparing attention pooling with mean pooling of local tokens would tell whether the attention mechanism itself, rather than simply using local tokens, produces the gain.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript (full-text version) proposes JWTH, a ViT-based pathology foundation model built on DINOv3 with staining augmentation, Gram-anchored post-tuning, and an attention-pooling head that fuses the class token with local patch tokens. The abstract and introduction claim that JWTH enables accurate and interpretable biomarker detection across four biomarkers and eight cohorts, achieving up to 8.3% higher balanced accuracy and 1.2% average improvement over prior pathology foundation models. However, the experiments reported in Section 3 consist of five patch-level tissue classification tasks (CRC-Norm, CRC-Unnorm, MHIST, BACH, CCRCC); no biomarker outcome, molecular label, or cellular ground truth is used. The supplied metadata abstract describes a different model ('Hireca' and 'CytoMap') with ten biomarker tasks and pathologist evaluation, which does not match the body text. As submitted, the paper's central biomarker and interpretability claims are unsupported by the reported evaluation.
Significance. If the claims were substantiated, adding attention pooling over local tokens to a modern pathology foundation model and demonstrating improved, interpretable biomarker prediction would be a useful contribution to computational pathology. The paper compares against seven recent foundation models on public benchmarks, which is a reasonable evaluation design for patch-level classification. However, the actual contribution as evidenced is limited: a 1.2% average balanced-accuracy gain on five tissue-classification tasks, reported without error bars or significance tests, and no evidence for the 'cell-level morphological detail' or 'interpretable biomarker prediction' claims. The mismatch between the abstract, introduction, and experimental sections prevents the reader from knowing what is actually being proposed and evaluated.
major comments (4)
- [Abstract; §3.1; Conclusion] The paper claims biomarker detection across 'four biomarkers and eight cohorts' (Abstract; Introduction; Conclusion), but Section 3.1 lists only patch-level tissue classification tasks: NCT-CRC-HE-100K, MHIST, BACH, and CCRCC. No MSI, HER2, mutation, IHC, or other biomarker labels appear in the evaluation. Therefore the reported 8.3% BACC gain and 1.2% average improvement (§3.2.2, Fig. 2) support only tissue classification, not biomarker prediction. This is a load-bearing mismatch between the central claim and the evidence.
- [Abstract (metadata) vs. full text] The abstract supplied with the submission introduces 'Hireca' and 'CytoMap', reports ten biomarker tasks, and describes evaluation by eight pathologists, while the full text presents 'JWTH' and evaluates five patch-level classification tasks with no pathologist study. These appear to describe different models and different experiments. The manuscript must be harmonized so that the abstract, title, and body refer to the same system and the same empirical claims; as written, the internal inconsistency prevents a coherent technical assessment.
- [§2.1.3, §2.2.3] The paper asserts that Gram-anchored post-tuning makes local tokens 'informative cell-level representations' and that attention pooling produces 'interpretable biomarker predictions.' No experiment connects attention weights, token content, or model predictions to cells, nuclei, or any biomarker ground truth. There is no cellular annotation, no interpretability metric, and no human evaluation in the body. The cell-centric interpretability claim is therefore asserted but unverified.
- [§3.2.2, Table 1] No error bars, confidence intervals, or statistical significance tests are reported for any comparison, and the average gain over the next-best model is only 1.2%. The ablation table does not specify which tasks are included in the average, what each checkmark denotes beyond the footnotes, or whether results are from a single run. With such small margins, the superiority claim is not established. Per-task numbers with variability and significance tests are needed.
minor comments (4)
- [Introduction vs. §2.1.1] The Introduction states 'six tasks' and pretraining over '20 organs,' while Section 2.1.1 says 'more than ten tissue types' and Section 3.1 lists five tasks. These numbers should be reconciled.
- [Fig. 2] Figure 2 is difficult to read: the x-axis labels are partially obscured, the legend is unclear, and no numerical values are given. A table with per-task balanced accuracy and standard deviations would be much more informative.
- [Eq. (5)] The attention-pooling formula uses a single class token as the query and local tokens as keys/values. Please clarify whether multi-head attention is used, how the output dimension is set, and whether this is equivalent to a standard cross-attention layer or a learned weighted average.
- [References] Reference [23] has a malformed identifier ('arXiv:2508.101043(4)') and the notation 'DinoV3-L' is used inconsistently with 'DINOv3' in the text.
Circularity Check
No circular derivation: JWTH's reported gains come from external benchmarks and are not constructed from the target labels; unsupported biomarker/cell-level claims are validation gaps, not circularity.
full rationale
JWTH's derivation is not circular in the sense that would require flagging. The pretraining objectives (Eq. 1: L_DINO + L_iBOT + L_Koleo; Eq. 2 adds L_Gram) are trained on unlabeled TUM pathology patches, and Gram-anchoring is explicitly adopted from the external DINOv3 paper ([23]), not from the authors' own prior work. No downstream biomarker label is used to define the pretraining objective or the Gram-anchoring regularizer. The attention-pooling head (Eq. 5: h = Attn(Q=z_cls, K=Z_patch, V=Z_patch); Eq. 6: yhat = sigma(W_attn h + b)) is trained on labeled public patch-classification datasets and compared against frozen-backbone linear-probing baselines. None of the reported balanced-accuracy numbers is a restatement of a training loss or a fitted parameter; the benchmarks (NCT-CRC-Norm/Unnorm, MHIST, BACH, CCRCC) are external and the comparison protocol is standard. The abstract/conclusion claims that the evaluation covers 'four biomarkers and eight cohorts', whereas Section 3 describes five tissue-type patch-level classification tasks; this is a claims-versus-evaluation mismatch, not a derivation reducing to its inputs. Similarly, the 'cell-level morphological details' and 'interpretable biomarker predictions' statements in Sections 2.1.3 and 2.2.3 are not validated with cellular annotations or biomarker outcomes, but that is an evidence gap, not circularity. Self-citations (refs [2], [10], [17]) are contextual and do not carry the central comparison; the load-bearing citation [23] is external and openly identified as the source of Gram-anchoring. We therefore find no circular step, with only minor self-citation usage that is not load-bearing.
Axiom & Free-Parameter Ledger
free parameters (3)
- Loss weights λ_DINO, λ_iBOT, λ_KoLeo, λ_Gram
- Staining augmentation perturbation strength
- Attention pooling classifier head =
trained per task
axioms (3)
- domain assumption DINOv3, pretrained on 1.7B natural images, transfers effectively to H&E pathology patches.
- ad hoc to paper ViT local tokens, after Gram-anchoring, correspond to cell-level morphology.
- domain assumption Public benchmark labels (NCT-CRC, MHIST, BACH, CCRCC) are accurate and representative of clinical biomarker status.
Cite this review
Pith. "Pith review of Towards Cellular-Scale Interpretability in Pathology Foundation Models for Biomarker Assessment." pith.science (2026). https://pith.science/paper/X52NMCYV
@misc{pith2026251105150,
author = {Pith},
title = {Pith review of: Towards Cellular-Scale Interpretability in Pathology Foundation Models for Biomarker Assessment},
year = {2026},
howpublished = {\url{https://pith.science/paper/X52NMCYV}},
note = {Machine review of arXiv:2511.05150}
}
read the original abstract
Molecular biomarker testing in pathology is often costly and tissue-consuming, limiting scalable clinical deployment. Artificial intelligence applied to hematoxylin and eosin (HE)-stained histology could enable rapid biomarker screening, but clinical translation requires models that are both accurate and interpretable. Here we introduce Hireca, a biomarker-focused pathology foundation model pretrained on more than 80,000 whole-slide images spanning 38 organ types from three medical centers, together with CytoMap, an interpretability module that localizes cellular-scale evidence underlying predictions. Across 10 biomarker tasks encompassing morphological, molecular, genetic, and spatial-transcriptomic-proxy readouts, Hireca ranked first in five tasks and outperformed comparable models overall. In evaluation by eight pathologists from two countries, CytoMap was consistently preferred over alternative visualization approaches and revealed error patterns in difficult cases. These results position Hireca and CytoMap as a transparent framework for clinically reviewable biomarker assessment directly from routine HE histology.
Reference graph
Works this paper leans on
-
[1]
The Journal of Pathology: Clinical Research11(5), 70049 (2025)
Tan-Garcia, A., Chua, T.H., Leow, W.-Q.: Computational pathology in the age of artificial intelligence–embrace not fear. The Journal of Pathology: Clinical Research11(5), 70049 (2025)
2025
-
[2]
Nature Medicine, 1–9 (2025)
Campanella, G., Kumar, N., Nanda, S., Singi, S., Fluder, E., Kwan, R., Muehlst- edt, S., Pfarr, N., Sch¨ uffler, P.J., H¨ aggstr¨ om, I., et al.: Real-world deployment of a fine-tuned pathology foundation model for lung cancer biomarker detection. Nature Medicine, 1–9 (2025)
2025
-
[3]
NPJ Digital Medicine7(1), 300 (2024)
Matthews, G.A., McGenity, C., Bansal, D., Treanor, D.: Public evidence on ai products for digital pathology. NPJ Digital Medicine7(1), 300 (2024)
2024
-
[4]
Computers in Biology and Medicine173, 108306 (2024)
Kazemi, A., Rasouli-Saravani, A., Gharib, M., Albuquerque, T., Eslami, S., Sch¨ uffler, P.J.: A systematic review of machine learning-based tumor-infiltrating lymphocytes analysis in colorectal cancer: Overview of techniques, performance metrics, and clinical outcomes. Computers in Biology and Medicine173, 108306 (2024)
2024
-
[5]
NPJ digital medicine7(1), 114 (2024)
McGenity, C., Clarke, E.L., Jennings, C., Matthews, G., Cartlidge, C., Stocken, D.D., Treanor, D.: Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy. NPJ digital medicine7(1), 114 (2024)
2024
-
[6]
arXiv preprint arXiv:2401.04079 (2024)
Dippel, J., Feulner, B., Winterhoff, T., Milbich, T., Tietz, S., Schallenberg, S., Dernbach, G., Kunft, A., Heinke, S., Eich, M.-L., et al.: Rudolfv: a foundation model by pathologists for pathologists. arXiv preprint arXiv:2401.04079 (2024)
Pith/arXiv arXiv 2024
-
[7]
Nature medicine30(3), 850–862 (2024)
Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M.,et al.: Towards a general-purpose foundation model for computational pathology. Nature medicine30(3), 850–862 (2024)
2024
-
[8]
arXiv preprint arXiv:2408.00738 (2024)
Zimmermann, E., Vorontsov, E., Viret, J., Casson, A., Zelechowski, M., Shaikovski, G., Tenenholtz, N., Hall, J., Klimstra, D., Yousfi, R., et al.: Virchow2: Scaling self-supervised mixed magnification models in pathology. arXiv preprint arXiv:2408.00738 (2024)
Pith/arXiv arXiv 2024
-
[9]
Nature634(8035), 970–978 (2024)
Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., Zhang, J., Li, R., Tang, H., Wang, K., Li, Y.,et al.: A pathology foundation model for cancer diagnosis and prognosis prediction. Nature634(8035), 970–978 (2024)
2024
-
[10]
arXiv preprint arXiv:2508.02528 (2025)
Liu, J., Deng, X., Li, H., Kazemi, A., Grashei, C., Wilkens, G., You, X., Groll, T., Navab, N., Mogler, C., et al.: From pixels to pathology: Restoration diffusion 9 for diagnostic-consistent virtual ihc. arXiv preprint arXiv:2508.02528 (2025)
Pith/arXiv arXiv 2025
-
[11]
IEEE Transactions on Medical Imaging (2024)
Wu, K., Jiang, Z., Tang, K., Shi, J., Xie, F., Wang, W., Wu, H., Zheng, Y.: Pan- cancer histopathology wsi pre-training with position-aware masked autoencoder. IEEE Transactions on Medical Imaging (2024)
2024
-
[12]
The Cancer Imaging Archive) (2022)
Farahmand, S., Fernandez, A., Ahmed, F., Rimm, D., Chuang, J., Reisenbichler, E., Zarringhalam, K.: Her2 and trastuzumab treatment response h&e slides with tumor roi annotations. The Cancer Imaging Archive) (2022)
2022
-
[13]
arXiv preprint arXiv:2501.15724 (2025)
Li, D., Wan, G., Wu, X., Wu, X., Nirmal, A.J., Lian, C.G., Sorger, P.K., Semenov, Y.R., Zhao, C.: A survey on computational pathology foundation models: Datasets, adaptation strategies, and evaluation tasks. arXiv preprint arXiv:2501.15724 (2025)
Pith/arXiv arXiv 2025
-
[14]
NPJ breast cancer4(1), 30 (2018)
Couture, H.D., Williams, L.A., Geradts, J., Nyante, S.J., Butler, E.N., Marron, J., Perou, C.M., Troester, M.A., Niethammer, M.: Image analysis with deep learning to predict breast cancer grade, er status, histologic subtype, and intrinsic subtype. NPJ breast cancer4(1), 30 (2018)
2018
-
[15]
IEEE Transactions on Medical Imaging41(4), 757–770 (2020)
Chen, R.J., Lu, M.Y., Wang, J., Williamson, D.F., Rodig, S.J., Lindeman, N.I., Mahmood, F.: Pathomic fusion: an integrated framework for fusing histopathol- ogy and genomic features for cancer diagnosis and prognosis. IEEE Transactions on Medical Imaging41(4), 757–770 (2020)
2020
-
[16]
In: 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp
Xu, Z., Du, X., Kang, Y., Lv, H., Li, M., Yang, W., Cui, L., Li, H., Xing, Y., Feng, J.,et al.: Addressing sparse annotation: a novel semantic energy loss for tumor cell detection from histopathologic images. In: 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1588–1595 (2023). IEEE
2023
-
[17]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention, pp
Liu, J., Li, H., Yang, C., Deutges, M., Sadafi, A., You, X., Breininger, K., Navab, N., Sch¨ uffler, P.J.: Hasd: Hierarchical adaption for pathology slide-level domain- shift. In: International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 332–342 (2025). Springer
2025
-
[18]
Medical image analysis75, 102264 (2022)
Pati, P., Jaume, G., Foncubierta-Rodriguez, A., Feroce, F., Anniciello, A.M., Scognamiglio, G., Brancati, N., Fiche, M., Dubruc, E., Riccio, D.,et al.: Hier- archical graph representations in digital pathology. Medical image analysis75, 102264 (2022)
2022
-
[19]
arXiv preprint arXiv:2509.06105 (2025)
Huang, Y., Huang, Z., Xiang, L., Yang, Q., Yin, H.: Pathohr: Hierarchical rea- soning for vision-language models in pathology. arXiv preprint arXiv:2509.06105 (2025)
arXiv 2025
-
[20]
arXiv preprint arXiv:2304.07193 (2023)
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust 10 visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)
Pith/arXiv arXiv 2023
-
[21]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp
He, K., Chen, X., Xie, S., Li, Y., Doll´ ar, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)
2022
-
[22]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp
Shen, Y., Luo, Y., Shen, D., Ke, J.: Randstainna: Learning stain-agnostic fea- tures from histology slides by bridging stain augmentation and normalization. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 212–221 (2022). Springer
2022
-
[23]
arXiv preprint arXiv:2508.101043(4) (2025)
Sim´ eoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M.: Dinov3. arXiv preprint arXiv:2508.101043(4) (2025)
Pith/arXiv arXiv 2025
-
[24]
arXiv preprint arXiv:2110.09348 (2021)
Jing, L., Vincent, P., LeCun, Y., Tian, Y.: Understanding dimensional collapse in contrastive self-supervised learning. arXiv preprint arXiv:2110.09348 (2021)
Pith/arXiv arXiv 2021
-
[25]
arXiv preprint arXiv:2504.01017 (2025)
Fan, D., Tong, S., Zhu, J., Sinha, K., Liu, Z., Chen, X., Rabbat, M., Ballas, N., LeCun, Y., Bar, A., et al.: Scaling language-free visual representation learning. arXiv preprint arXiv:2504.01017 (2025)
Pith/arXiv arXiv 2025
-
[26]
arXiv preprint arXiv:2106.10270 (2021)
Steiner, A., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., Beyer, L.: How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2106.10270 (2021)
Pith/arXiv arXiv 2021
-
[27]
Kather, J.N., Halama, N., Marx, A.: 100,000 histological images of human colorec- tal cancer and healthy tissue, april 2018. URL https://doi. org/10.5281/zenodo 1214456(2024)
doi:10.5281/zenodo 2018
-
[28]
In: 2009 IEEE International Symposium on Biomedical Imaging: from Nano to Macro, pp
Macenko, M., Niethammer, M., Marron, J.S., Borland, D., Woosley, J.T., Guan, X., Schmitt, C., Thomas, N.E.: A method for normalizing histology slides for quantitative analysis. In: 2009 IEEE International Symposium on Biomedical Imaging: from Nano to Macro, pp. 1107–1110 (2009). IEEE
2009
-
[29]
In: International Conference on Artificial Intelligence in Medicine, pp
Wei, J., Suriawinata, A., Ren, B., Liu, X., Lisovsky, M., Vaickus, L., Brown, C., Baker, M., Tomita, N., Torresani, L.,et al.: A petri dish for histopathology image analysis. In: International Conference on Artificial Intelligence in Medicine, pp. 11–24 (2021). Springer
2021
-
[30]
PloS one12(6), 0177544 (2017)
Ara´ ujo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Pol´ onia, A., Campilho, A.: Classification of breast cancer histology images using convolutional neural networks. PloS one12(6), 0177544 (2017)
2017
-
[31]
British Journal of Cancer129(4), 683–695 (2023) 11
Brummer, O., P¨ ol¨ onen, P., Mustjoki, S., Br¨ uck, O.: Computational textural mapping harmonises sampling variation and reveals multidimensional histopatho- logical fingerprints. British Journal of Cancer129(4), 683–695 (2023) 11
2023
-
[32]
Nature medicine30(3), 863–874 (2024) 12
Lu, M.Y., Chen, B., Williamson, D.F., Chen, R.J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L.P., Gerber, G.,et al.: A visual-language foundation model for computational pathology. Nature medicine30(3), 863–874 (2024) 12
2024
This paper was first reviewed by deepseek-v4-flash on August 3, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.