REVIEW 4 major objections 4 minor 25 references
This paper shows that combining a patient's tumor pathology slide with their preoperative CT scan predicts kidney cancer recurrence better than either image type alone, and nearly as well as the standard clinical risk score.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Multimodal fusion of CT and pathology images improves recurrence risk prediction in kidney cancer, with the best model approaching the clinical Leibovich score.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection Careful feasibility benchmark for CT+WSI fusion in ccRCC recurrence, but the fusion-advantage claim rests on differences within one standard deviation without significance testing. the 4 major comments →
Integrating Pathology and CT Imaging for Personalized Recurrence Risk Prediction in Renal Cancer
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The central claim is that whole-slide histopathology and preoperative CT carry complementary prognostic information for clear cell renal cell carcinoma recurrence, and that a simple fusion of their learned representations extracts it. In the authors' experiments, frozen pathology foundation-model embeddings consistently produced higher C-index values than fine-tuned CT encoders when used alone, so pathology dominates the signal; but every WSI–CT combination that concatenated the two feature vectors before the survival head matched or exceeded its WSI-only baseline, with the best result from TITAN–CONCH plus ResNet-18 (C-index 0.775 ± 0.044). The adjusted Leibovich score remained the highest
What carries the argument
The pipeline's engine is intermediate fusion by embedding concatenation inside a Cox-based survival model. Whole-slide images are compressed by frozen pathology foundation models (TITAN–CONCH or CHIEF–CTransPath) into patient-level vectors; CT volumes are cropped to the kidneys, downsampled, and encoded by fine-tuned 3D encoders (MedicalNet ResNet variants or SwinUNETR) into vectors of the same dimension. Concatenating the two vectors and feeding them through a small multilayer perceptron trained with the Cox partial-likelihood loss lets the model learn cross-modality feature interactions before risk scoring. The same setup is also run unimodally and with late fusion (a weighted average of s
Load-bearing premise
The 156-patient cohort used for all comparisons is the subset of patients whose CT scans passed a manual quality review; if the excluded 31 patients differ systematically in recurrence risk, the relative performance of the models may not generalize.
What would settle it
Take the same intermediate-fusion pipeline to an external ccRCC cohort with more recurrence events and compare, under identical nested cross-validation, fused WSI+CT against WSI-only: if the fused model does not exceed the WSI-only C-index, the paper's central claim that CT adds complementary value through fusion is refuted. Separately, if the adjusted Leibovich score keeps a large lead over learned models after randomized tie-breaking in a larger sample, the claim that discretization overstates its individualized performance would be weakened.
If this is right
- If the reported ordering holds, future recurrence-risk tools could be built from routine imaging alone, with no extra tests beyond slides and CT scans already acquired in standard care.
- CT's prognostic value in this setting is conditional on fusion: the same CT encoders that scored near chance alone contributed to the best fused model, so radiology should be evaluated in combination, not in isolation.
- The adjusted Leibovich score's lead narrows substantially under random tie-breaking, implying that discrete clinical scores may overstate their ability to rank individual patients; continuous learned risk scores may be fairer comparators.
- Because simple concatenation already improved every WSI–CT combination, more expressive fusion (cross-attention or co-learning) is a plausible next step the authors explicitly leave open.
Where Pith is reading between the lines
- If the fusion gain replicates in a larger cohort, it would imply that current CT encoders, pretrained mainly for segmentation, are leaving prognostic information on the table; a CT foundation model trained for survival tasks might close the gap with pathology.
- The paper's case analysis hints that slide sampling can miss aggressive regions; a natural extension is to test multiple slides per tumor or attention-based slide selection, which the authors did not do.
- A practical translation would be a continuous imaging-based risk score that could be thresholded flexibly for surveillance intensity, rather than locked into three clinical risk groups; this follows from the paper's tie-breaking argument but is not tested here.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a modular deep learning framework that integrates preoperative CT and postoperative whole-slide images (WSIs) for recurrence risk prediction in clear cell renal cell carcinoma (ccRCC). Using a TCGA-KIRC subset of 156 patients, it compares unimodal WSI and CT models, late fusion, and intermediate fusion, all based on frozen foundation-model encoders plus a Cox-based MLP prediction head. The adjusted Leibovich score is used as a clinical baseline. The paper reports that WSI-based models outperform CT-only models, that intermediate fusion gives the best learned model (TITAN-CONCH + ResNet-18, C-index 0.775 ± 0.044), approaching the adjusted Leibovich score (0.805 ± 0.035), and that random tie-breaking lowers the baseline to 0.749 ± 0.043.
Significance. If the claims hold, the study is a useful feasibility demonstration of multimodal imaging integration for a clinical task where structured scores remain the standard. Strengths include the use of a public dataset, nested cross-validation with fixed outer folds, and honest reporting of fold-level standard deviations. The main limitations are the small cohort (156 patients, 40 events), manual cohort filtering, and the absence of statistical testing for model comparisons, which makes the central fusion-advantage claim suggestive rather than established. The paper's methodological framework is sound in structure but needs additional statistical rigor before the conclusions can be accepted at face value.
major comments (4)
- [Section 4, Table 1] The central claim that intermediate fusion improves over unimodal WSI models is not statistically supported. The best intermediate model (TITAN-CONCH + ResNet-18) achieves C-index 0.775 ± 0.044 versus 0.745 ± 0.046 for TITAN-CONCH unimodal. With n=156 and 40 events, a 0.030 difference is well within the reported fold-level standard deviations. No paired significance test (e.g., permutation over patients or outer folds, bootstrap confidence intervals) is provided. Please add such tests for all pairwise comparisons of interest, report p-values/confidence intervals, and adjust the abstract/conclusion wording accordingly if the difference is not significant.
- [Section 3, Experimental Protocol] The 'best model' is identified from 18 learned configurations (2 WSI encoders × 3 CT encoders × 3 fusion strategies) using the same outer folds. No multiple-comparison correction or independent model-selection procedure is described. The reported best result may partly reflect selection bias. Please report results for all configurations (already in Table 1) and additionally provide a model-selection rule, e.g., choosing the fusion strategy on inner-fold performance only, or correct for the number of comparisons when making inferential claims.
- [Section 3, Datasets] The manual exclusion of 31 out of 187 patients based on CT quality, contrast phase, and kidney visibility is a potential source of selection bias. No comparison of excluded versus included patients' clinical characteristics or outcomes is provided. If the exclusions correlate with outcome, the relative model performance may not generalize. Please report baseline demographics, stage, grade, event rates, and follow-up for the excluded patients, and discuss or quantitatively assess the impact of this filtering.
- [Sections 3 and 4 (Adjusted Leibovich score; Case A)] There is an internal inconsistency regarding necrosis. Section 3 states that necrosis was omitted from the adjusted Leibovich score 'due to its absence in the dataset,' but Section 4 Case A states that 'the pathology report noted necrosis and high Fuhrman grade.' Clarify whether structured necrosis data were unavailable despite pathology reports containing the information, and discuss how this affects the validity of the adjusted baseline. Additionally, random tie-breaking is applied only to the Leibovich baseline; if the claim is that discretization overstates baseline performance, provide the same analysis for any tied risk scores produced by the learned models (or justify their absence).
minor comments (4)
- [Section 4, 'Performance Analysis of Unimodal and Multimodal Strategies'] The text states that 'all WSI-CT combinations matching or exceeding their WSI-only baselines' in intermediate fusion, but Table 1 shows TITAN-CONCH + ResNet-10 intermediate fusion at 0.742 versus 0.745 unimodal TITAN-CONCH. Please correct this overstatement.
- [Section 2/Table 1] The naming of CT encoders is unclear: Table 1 lists 'ResNet-10' and 'ResNet-18,' but Section 2 mentions MedicalNet and SwinUNETR. Please clarify which architecture corresponds to each named model and whether ResNet-10/18 are MedicalNet variants.
- [Section 4, Leibovich (RT)] The 'random tie-breaking' result is reported as a single mean C-index. Please specify the number of random repetitions and the seed or variance across repetitions to make the result reproducible.
- [General] The paper does not mention code or feature-extraction pipeline availability. Given the public dataset, providing code would strengthen reproducibility, even if only for preprocessing and evaluation.
Circularity Check
No circularity: the pipeline is a standard nested cross-validation benchmark against an external clinical score, with no fitted parameter renamed as a prediction and no load-bearing self-citation.
full rationale
The paper's derivation chain is empirical rather than definitional. Patient-level CT and WSI features are produced by externally pretrained encoders (TITAN, CONCH, CHIEF, CTransPath, MedicalNet, SwinUNETR) and then fed into a Cox-based MLP. Training uses five outer folds and three inner folds: hyperparameters, including the late-fusion weight alpha, are tuned only on inner folds, and final evaluation is on held-out outer folds. The reported C-indices are therefore not forced by construction. The adjusted Leibovich baseline is computed from structured clinicopathologic variables (with necrosis omitted) and is an independent clinical comparator, not an input to the learned models. The random tie-breaking analysis only re-scores the same baseline under a different tie convention; it does not transform a fitted quantity into a prediction. The paper invokes no uniqueness theorem and no self-citation chain: all referenced foundation models are external prior work, and no load-bearing claim depends on a citation by the present authors. The lack of significance testing and the variability across CT encoders are correctness/robustness concerns, not circularity. No self-definitional, fitted-input-as-prediction, or citation-smuggling pattern is present.
Axiom & Free-Parameter Ledger
free parameters (2)
- Late fusion weight alpha =
0.8-0.9 (per outer fold)
- MLP hyperparameters (learning rate, weight decay, L1, hidden size, dropout, scheduler floor) =
Selected via Optuna on inner folds
axioms (4)
- domain assumption Cox proportional hazards assumption holds for recurrence risk
- domain assumption Pretrained foundation models transfer to ccRCC recurrence prediction
- domain assumption Manual CT quality selection is unbiased
- domain assumption Adjusted Leibovich score without necrosis is a valid clinical comparator
Cite this review
Pith. "Pith review of Integrating Pathology and CT Imaging for Personalized Recurrence Risk Prediction in Renal Cancer." pith.science (2026). https://pith.science/paper/HTA2FIEU
@misc{pith2026250821581,
author = {Pith},
title = {Pith review of: Integrating Pathology and CT Imaging for Personalized Recurrence Risk Prediction in Renal Cancer},
year = {2026},
howpublished = {\url{https://pith.science/paper/HTA2FIEU}},
note = {Machine review of arXiv:2508.21581}
}
read the original abstract
Recurrence risk estimation in clear cell renal cell carcinoma (ccRCC) is essential for guiding postoperative surveillance and treatment. The Leibovich score remains widely used for stratifying distant recurrence risk but offers limited patient-level resolution and excludes imaging information. This study evaluates multimodal recurrence prediction by integrating preoperative computed tomography (CT) and postoperative histopathology whole-slide images (WSIs). A modular deep learning framework with pretrained encoders and Cox-based survival modeling was tested across unimodal, late fusion, and intermediate fusion setups. In a real-world ccRCC cohort, WSI-based models consistently outperformed CT-only models, underscoring the prognostic strength of pathology. Intermediate fusion further improved performance, with the best model (TITAN-CONCH with ResNet-18) approaching the adjusted Leibovich score. Random tie-breaking narrowed the gap between the clinical baseline and learned models, suggesting discretization may overstate individualized performance. Using simple embedding concatenation, radiology added value primarily through fusion. These findings demonstrate the feasibility of foundation model-based multimodal integration for personalized ccRCC risk prediction. Future work should explore more expressive fusion strategies, larger multimodal datasets, and general-purpose CT encoders to better match pathology modeling capacity.
Figures
Reference graph
Works this paper leans on
-
[1]
Koyama, M.: Optuna: A Next-generation Hyperparameter Optimization Framework (2019)
Akiba, T., Sano, S., Yanase, T., Ohta, T. Koyama, M.: Optuna: A Next-generation Hyperparameter Optimization Framework (2019). https://doi.org/10.48550/arXiv.1907.10902
-
[2]
https://doi.org/10.7937/K9/TCIA.2016.V6PBVTDR
Akin, O., et al.: The Cancer Genome Atlas Kidney Re- nal Clear Cell Carcinoma Collection (TCGA-KIRC) (2016). https://doi.org/10.7937/K9/TCIA.2016.V6PBVTDR
-
[3]
https://doi.org/10.7937/JC8X-9874
Baid, U., et al.: RSNA-ASNR-MICCAI-BraTS-2021 (2023). https://doi.org/10.7937/JC8X-9874
-
[4]
Uro- logic Oncology: Seminars and Original Investigations 39(7), 438.e11–438.e21 (Jul 2021)
Blackmur, J.P., et al.: Leibovich score is the optimal clinico-pathological system associated with recurrence of non-metastatic clear cell renal cell carcinoma. Uro- logic Oncology: Seminars and Original Investigations 39(7), 438.e11–438.e21 (Jul 2021). https://doi.org/10.1016/j.urolonc.2021.04.007
-
[5]
Capitanio, U. Montorsi, F.: Renal cancer. The Lancet 387(10021), 894–906 (Feb 2016). https://doi.org/10.1016/S0140-6736(15)00046-X
-
[6]
Zheng, Y.: Med3D: Transfer Learning for 3D Medical Image Analysis (2019)
Chen, S., Ma, K. Zheng, Y.: Med3D: Transfer Learning for 3D Medical Image Analysis (2019). https://doi.org/10.48550/arXiv.1904.00625
-
[7]
Cox, D.R.: Regression Models and Life-Tables. Journal of the Royal Statis- tical Society Series B: Statistical Methodology 34(2), 187–202 (Jan 1972). https://doi.org/10.1111/j.2517-6161.1972.tb00899.x
arXiv 1972
-
[8]
Journal of Open Source Software 4(40), 1317 (Aug 2019)
Davidson-Pilon, C.: Lifelines: Survival analysis in Python. Journal of Open Source Software 4(40), 1317 (Aug 2019). https://doi.org/10.21105/joss.01317
-
[9]
https://doi.org/10.48550/arXiv.2411.19666
Ding, T., et al.: Multimodal Whole Slide Foundation Model for Pathology (Nov 2024). https://doi.org/10.48550/arXiv.2411.19666
-
[10]
The Lancet Digital Health 5(8), e515–e524 (Aug 2023)
Gui, C.P., et al.: Multimodal recurrence scoring system for prediction of clear cell renal cell carcinoma outcome: A discovery and validation study. The Lancet Digital Health 5(8), e515–e524 (Aug 2023). https://doi.org/10.1016/S2589- 7500(23)00095-X
doi:10.1016/s2589- 2023
-
[11]
McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (ROC) curve
Hanley, J.A. McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1), 29–36 (Apr 1982). https://doi.org/10.1148/radiology.143.1.7063747
-
[12]
JAMA: The Jour- nal of the American Medical Association 247(18), 2543 (May 1982)
Harrell, F.E.: Evaluating the Yield of Medical Tests. JAMA: The Jour- nal of the American Medical Association 247(18), 2543 (May 1982). https://doi.org/10.1001/jama.1982.03320430047030
-
[13]
Xu, D.: Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images (2022)
Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. Xu, D.: Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images (2022). https://doi.org/10.48550/arXiv.2201.01266
-
[14]
https://doi.org/10.48550/arXiv.2307.01984
Heller, N., et al.: The KiTS21 Challenge: Automatic segmentation of kid- neys, renal tumors, and renal cysts in corticomedullary-phase CT (2023). https://doi.org/10.48550/arXiv.2307.01984
-
[15]
https://doi.org/10.48550/arXiv.1809.10486
Isensee, F., et al.: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (2018). https://doi.org/10.48550/arXiv.1809.10486
-
[16]
Katzman, J.L., Shaham, U., Cloninger, A., Bates, J., Jiang, T. Kluger, Y.: Deep- Surv: Personalized treatment recommender system using a Cox proportional haz- ards deep neural network. BMC Medical Research Methodology 18(1), 24 (Dec 2018). https://doi.org/10.1186/s12874-018-0482-1 12 D. Boeke et al
-
[17]
Cancer 97(7), 1663–1671 (Apr 2003)
Leibovich, B.C., et al.: Prediction of progression after radical nephrec- tomy for patients with clear cell renal cell carcinoma: A stratification tool for prospective clinical trials. Cancer 97(7), 1663–1671 (Apr 2003). https://doi.org/10.1002/cncr.11234
-
[18]
European Urology 73(5), 772–780 (May 2018)
Leibovich, B.C., et al.: Predicting Oncologic Outcomes in Renal Cell Carcinoma After Surgery. European Urology 73(5), 772–780 (May 2018). https://doi.org/10.1016/j.eururo.2018.01.005
-
[19]
Nature Medicine 30(3), 863–874 (Mar 2024)
Lu, M.Y., et al.: A visual-language foundation model for computational pathology. Nature Medicine 30(3), 863–874 (Mar 2024). https://doi.org/10.1038/s41591-024- 02856-4
-
[20]
npj Precision Oncology 8(1), 45 (Feb 2024)
Margue, G., et al.: UroPredict: Machine learning model on real-world data for pre- diction of kidney cancer recurrence (UroCCR-120). npj Precision Oncology 8(1), 45 (Feb 2024). https://doi.org/10.1038/s41698-024-00532-x
-
[21]
Nature Genetics 45(10), 1113–1120 (Oct 2013)
The Cancer Genome Atlas Research Network, et al.: The Cancer Genome At- las Pan-Cancer analysis project. Nature Genetics 45(10), 1113–1120 (Oct 2013). https://doi.org/10.1038/ng.2764
doi:10.1038/ng.2764 2013
-
[22]
https://doi.org/10.48550/arXiv.2501.16652
Vaidya, A., et al.: Molecular-driven Foundation Model for Oncologic Pathology (Jan 2025). https://doi.org/10.48550/arXiv.2501.16652
-
[23]
Medical Image Analysis 81, 102559 (Oct 2022)
Wang, X., et al.: Transformer-based unsupervised contrastive learning for histopathological image classification. Medical Image Analysis 81, 102559 (Oct 2022). https://doi.org/10.1016/j.media.2022.102559
-
[24]
Nature 634(8035), 970–978 (Oct 2024)
Wang, X., et al.: A pathology foundation model for cancer diagno- sis and prognosis prediction. Nature 634(8035), 970–978 (Oct 2024). https://doi.org/10.1038/s41586-024-07894-z
-
[25]
Mahmood, F.: Accelerating Data Processing and Benchmarking of AI Models for Pathology (Feb 2025)
Zhang, A., Jaume, G., Vaidya, A., Ding, T. Mahmood, F.: Accelerating Data Processing and Benchmarking of AI Models for Pathology (Feb 2025). https://doi.org/10.48550/arXiv.2502.06750
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.