Pith. sign in

REVIEW 3 major objections 5 minor 23 references

The uncertainty that best tracks cardiac segmentation error is not the one that most improves disease classification.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 12:21 UTC pith:U7FSKHIN

load-bearing objection Clean empirical observation that error-localization quality ≠ disease-signal quality, but the +1.7 AUROC claim is statistically fragile on N=40 with per-variant retuning. the 3 major comments →

arxiv 2607.10357 v1 pith:U7FSKHIN submitted 2026-07-11 cs.CV cs.AI

GRC-ProbNet: Uncertainty-aware Feature Extraction for Cardiovascular Disease Classification

Yash Shah , Omar Todd , Philipp Seeb\"ock , Georg Langs , Ben Glocker , Raghav Mehta This is my paper
classification cs.CV cs.AI
keywords cardiovascular disease classificationuncertainty quantificationcardiac CTdeep ensemblesradiomicsgeometric featuressegmentation errorhybrid pipelines
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Cardiac CT analysis pipelines usually produce a single deterministic segmentation of the heart and then extract radiomic and geometric features for disease classification. This paper extends that approach by feeding an ensemble of segmentations through the same registration and feature pipeline, then adding simple summaries of how the models disagree. On a small public disease cohort the best uncertainty-augmented feature set raises classification AUROC from 91.25 percent to 92.92 percent. More importantly, the uncertainty map that most closely matches actual segmentation mistakes (predictive entropy) is not the one that most consistently helps the downstream classifier; inter-model disagreement (KL divergence) does. The result shows that, when intermediate hand-crafted features sit between deep networks, the uncertainty representation that is most useful for diagnosis need not be the one that best localises segmentation error.

Core claim

In a modular hybrid pipeline that extracts radiomic and geometric features from cardiac CT segmentations, appending structure-wise uncertainty summaries from a deep ensemble improves cardiovascular disease classification, and the uncertainty measure most correlated with segmentation error (predictive entropy) is not the one that yields the strongest classification gain (KL / epistemic disagreement).

What carries the argument

GRC-ProbNet: a five-member deep ensemble of Anatomix segmentors whose outputs are registered by a frozen Atlas-ISTN; the resulting mean probability maps yield three uncertainty maps (total entropy, Jensen-Shannon/KL epistemic, residual aleatoric) that are reduced either globally or per cardiac structure and concatenated with the original radiomic-geometric vector for an MLP classifier.

Load-bearing premise

That disagreement among five models fine-tuned on only thirteen labelled volumes yields stable, disease-informative uncertainty features that transfer to an independent forty-scan disease cohort.

What would settle it

Re-run the identical ASOCA five-fold protocol after replacing the five-member ensemble with a single deterministic Anatomix model (or with ten members) and check whether the AUROC gap between the best uncertainty-augmented variant and the deterministic baseline disappears or reverses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes GRC-ProbNet, an extension of the authors’ prior GRC-Net hybrid pipeline for CVD classification from cardiac CT. A five-member deep ensemble of fine-tuned Anatomix models produces multiple segmentations that are registered via a shared Atlas-ISTN; from the resulting probability maps the authors extract total entropy, KL/JSD (epistemic), and aleatoric uncertainty features, either as a single global scalar or as per-structure averages. These features are concatenated with the original radiomic and geometric descriptors and fed to an MLP. On MM-WHS the authors show that total entropy correlates most strongly with segmentation error (NCC 0.521), while on the 40-scan ASOCA cohort the best uncertainty-augmented variant (per-structure KL) reaches 92.92% AUROC versus a 91.25% deterministic baseline. The central scientific claim is that the uncertainty measure best aligned with segmentation error is not necessarily the one that most improves downstream disease classification.

Significance. If the comparative finding holds under tighter statistical controls, the work is a useful contribution to hybrid medical-imaging pipelines: it shows that explicit uncertainty features can carry disease-relevant signal beyond deterministic segmentations, and that error-localisation quality is an incomplete proxy for downstream utility. The systematic comparison of total/epistemic/aleatoric maps and global versus per-structure encodings is cleanly designed, the code is released, and the modular framing (uncertainty extracted before non-learned radiomic/geometric stages) is relevant to many cascaded clinical pipelines. The absolute AUROC lift is modest and currently under-powered, so the primary value lies in the methodological analysis rather than in a new state-of-the-art classifier.

major comments (3)
  1. Table 2: the headline claim that uncertainty features “substantially improve” AUROC (92.92% vs 91.25%) is not supported by the reported numbers. The absolute gain is +1.67 points while the standard deviations are 8.5–9.4 points; the intervals heavily overlap. No paired statistical test (e.g., Wilcoxon or permutation test across the same folds/seeds) is provided. With N=40 the observed lift is consistent with chance. The abstract, §3.3 and §4 should either supply a rigorous significance analysis or substantially temper the language of improvement.
  2. §3.1 and Table 2: Bayesian Optimization retunes the full MLP hyper-parameter set (L, H, p, η, E) independently for every uncertainty encoding on the same 40-scan ASOCA cohort. This confounds attribution: a 1–2-point AUROC difference can arise from the optimizer finding a better fit for a particular feature dimension rather than from a genuine disease-informative uncertainty signal. A fixed-hyperparameter ablation (or at least a shared search space with the baseline) is needed to isolate the contribution of the uncertainty features themselves.
  3. §2.1 and §3.1: the five Anatomix models are fine-tuned on only 13 MM-WHS volumes, and the resulting structure-wise uncertainty averages are treated as stable, transferable features on the independent ASOCA cohort. With such a small labelled set the ensemble may largely capture training-seed noise rather than anatomically meaningful epistemic disagreement. The paper should quantify ensemble diversity/stability (e.g., pairwise Dice across seeds, sensitivity of the Table 2 ranking to K) and discuss the risk that the KL advantage is an artefact of under-trained members rather than a disease-specific signal.
minor comments (5)
  1. Abstract: typographical error “92.92\)” (stray backslash).
  2. §3.1: the MM-WHS split is given as 13/1/6; with a single validation volume the early-stopping / model-selection procedure for Anatomix fine-tuning is under-specified and should be clarified.
  3. Table 1 reports NCC only; adding a second localisation metric (e.g., AUPRC of uncertainty vs. error voxels) would strengthen the error-correlation analysis.
  4. Figure 2 is informative but the multi-class vs. per-structure panels are hard to compare at print size; consider a clearer layout or an additional quantitative summary.
  5. §4: future work on registration-stage uncertainty is noted; a brief statement on whether the current frozen Atlas-ISTN could itself inject systematic bias into the uncertainty maps would help readers assess residual limitations.

Circularity Check

0 steps flagged

No significant circularity: the AUROC improvement is an empirical measurement on held-out ASOCA data, not forced by construction or by a load-bearing self-citation chain.

full rationale

The paper's central claims are empirical comparisons (Table 1 NCC of uncertainty maps vs. segmentation error on MM-WHS; Table 2 AUROC of uncertainty-augmented feature vectors vs. deterministic GRC-Net baseline on ASOCA 5-fold CV). Uncertainty maps are produced by a multi-seed deep ensemble of independently fine-tuned Anatomix models; the maps are then summarized into scalar or per-structure features that are simply concatenated to the existing radiomic+geometric vector and fed to an MLP. No equation equates the classification metric to the ensemble variance by definition, and no parameter is fitted on the target labels and then re-presented as a prediction of those same labels. The sole self-citation of note is the reuse of the authors' prior GRC-Net pipeline as the deterministic baseline; that baseline is an independent modular system whose radiomic/geometric features are held fixed while the new uncertainty features are ablated. Because the reported lift is measured (not derived) and the uncertainty computation does not presuppose the disease labels, the derivation chain is self-contained against external public benchmarks. Minor self-citation of the baseline does not raise the score above 1.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The central claim rests on standard deep-ensemble uncertainty formulas, the assumption that five seeds on 13 volumes suffice, and the modeling choice that structure-wise averages of those maps are useful features. No new physical entities are postulated; free parameters are the usual training and architecture choices plus the Bayesian-optimized MLP hyper-parameters that differ per feature set.

free parameters (3)
  • ensemble size K = 5
    Fixed at 5 by computational trade-off (Sec. 2.1); not cross-validated.
  • MLP hyper-parameters (L, H, p, η, E) = per-variant Optuna search
    Re-tuned by Bayesian optimization separately for every uncertainty encoding on the same small ASOCA folds (Sec. 3.1), allowing the optimizer to favor the best-looking encoding.
  • PCA components for geometric features = ≤3
    Up to three eigenvalues retained per structure (Sec. 2); choice not justified by explained-variance threshold.
axioms (3)
  • domain assumption Deep-ensemble disagreement approximates epistemic uncertainty and total entropy minus JSD isolates aleatoric uncertainty.
    Invoked via citations [12,22,23] in Sec. 2.1.1; standard but still an approximation.
  • domain assumption A population atlas built from 13 healthy MM-WHS volumes is a valid normative reference for both healthy and diseased ASOCA hearts.
    Atlas-ISTN trained only on the MM-WHS split (Sec. 3.1); disease-related shape variation is assumed to appear as deformation relative to this healthy atlas.
  • ad hoc to paper Structure-wise average of a voxel-wise uncertainty map is a sufficient summary statistic for downstream classification.
    Encoding choice introduced in Sec. 2.1.1 without theoretical justification beyond empirical comparison.

pith-pipeline@v1.1.0-grok45 · 13414 in / 2854 out tokens · 23400 ms · 2026-07-14T12:21:09.483373+00:00 · methodology

0 comments
read the original abstract

The automatic detection and classification of cardiovascular disease (CVD) from computed tomography (CT) images plays an important role in clinical practice. Recently, a hybrid pipeline (GRC-Net) for CVD classification was proposed, which leverages a deep-learning-based segmentation and registration method to extract radiomic and geometric features. However, GRC-Net relies on a deterministic segmentation mask, without considering the inherent ambiguity associated with cardiac anatomy. In this paper, we propose GRC-ProbNet, which takes advantage of a deep ensemble to produce multiple segmentation masks for a given input. From these masks, we extract multiple uncertainty features. We analyze these uncertainty features for both their correlation with segmentation error and their propagation effects on downstream CVD classification performance. Our experiments on the publicly available MM-WHS and ASOCA datasets show that the uncertainty measure that best reflects segmentation quality is not necessarily the one that provides the strongest signal for downstream CVD classification. Overall, our results demonstrate that GRC-ProbNet utilizing uncertainty features substantially improves CVD classification AUROC (92.92\) compared to the baseline GRC-Net model (91.25%). Our code is publicly available: https://github.com/biomedia-mira/GRC-ProbNet.

Figures

Figures reproduced from arXiv: 2607.10357 by Ben Glocker, Georg Langs, Omar Todd, Philipp Seeb\"ock, Raghav Mehta, Yash Shah.

Figure 1
Figure 1. Figure 1: Overview of the proposed framework. N (ex. 5) independently fine-tuned Anatomix models segment each input CT; outputs pass through an Atlas-ISTN, produc￾ing deformation fields and warped atlas probability maps. These are aggregated into a mean probability map, from which the consensus segmentation and the uncertainty maps are derived. From these, the radiomic, geometric, and uncertainty features are extrac… view at source ↗
Figure 2
Figure 2. Figure 2: Representative sample showing the original CT image, Anatomix segmenta [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 5 canonical work pages

  1. [1]

    Chong, B., Jayabaskaran, J., Jauhari, S.M., Chan, S.P., Goh, R., Kueh, M.T.W., et al.: Global burden of cardiovascular diseases: projections from 2025 to 2050.Euro- pean Journal of Preventive Cardiology32(11), 1001–1015 (2025).10.1093/eurjpc/ zwae281

    work page doi:10.1093/eurjpc/ 2025

  2. [2]

    10.1007/s10554-009-9501-y

    Luijnenburg, S.E., Robbers-Visser, D., Moelker, A., Vliegen, H.W., Mulder, B.J., Helbing, W.A.: Intra-observer and interobserver variability of biventricular func- tion, volumes and mass in patients with congenital heart disease measured by CMRimaging.International Journal of Cardiovascular Imaging26(1), 57–64(2010). 10.1007/s10554-009-9501-y

    work page doi:10.1007/s10554-009-9501-y 2010

  3. [3]

    media.2017.07.005

    Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., Sánchez, C.I.: A survey on deep learning 8 in medical image analysis.Medical Image Analysis42, 60–88 (2017).10.1016/j. media.2017.07.005

    doi:10.1016/j 2017

  4. [4]

    Topol, E.J.: High-performance medicine: the convergence of human and artifi- cial intelligence.Nature Medicine25(1), 44–56 (2019).https://www.nature.com/ articles/s41591-018-0300-7

    2019

  5. [5]

    ieee.org/document/9541203

    Mehta, R., Christinck, T., Nair, T., Bussy, A., Premasiri, S., Costantino, M., Chakravarthy, M.M., Arnold, D.L., Gal, Y., Arbel, T.: Propagating uncertainty across cascaded medical imaging tasks for improved deep learning inference.IEEE Transactions on Medical Imaging41(2), 360–373 (2021).https://ieeexplore. ieee.org/document/9541203

    arXiv 2021

  6. [6]

    Lin, A., Manral, N., McElhinney, P., Killekar, A., Matsumoto, H., Kwiecinski, J., Pieszko, K., Razipour, A., Grodecki, K., Park, C., et al.: Deep learning- enabled coronary CT angiography for plaque and stenosis quantification and cardiac risk prediction: an international multicentre study.The Lancet Digital Health4(4), e256–e265 (2022).https://www.thelan...

    2022

  7. [7]

    1007/s00330-025-11410-w

    Alvén, J., Petersen, R., Hagerman, D., Sandstedt, M., Kitslaar, P., Bergström, G., Fagman, E., Hjelmgren, O.: PlaqueVit: a vision transformer model for fully auto- matic vessel and plaque segmentation in coronary computed tomography angiogra- phy.European Radiology, 1–11 (2025).https://link.springer.com/article/10. 1007/s00330-025-11410-w

    2025

  8. [8]

    Mittal, A., Mehta, R., Todd, O., Seeböck, P., Langs, G., Glocker, B.: Cardiovascular disease classification using radiomics and geometric features from cardiac CT.arXiv preprint arXiv:2506.22226(2025).https://arxiv.org/abs/2506.22226

    Pith/arXiv arXiv 2025

  9. [9]

    Wundram, A.M., Fischer, P., Wunderlich, S., Faber, H., Koch, L.M., Berens, P. and Baumgartner, C.F.: Leveraging probabilistic segmentation models for improved glaucoma diagnosis: A clinical pipeline approach.In Medical Imaging with Deep Learning(2024).https://openreview.net/forum?id=kcbAZwxCFV

    2024

  10. [10]

    Fischer, P., Thomas, K., Baumgartner, C. F.: Uncertainty estimation and prop- agation in accelerated MRI reconstruction.In International Workshop on Uncer- tainty for Safe Utilization of Machine Learning in Medical Imaging(2023).https: //link.springer.com/chapter/10.1007/978-3-031-44336-7_9

    work page doi:10.1007/978-3-031-44336-7_9 2023

  11. [11]

    https://arxiv.org/abs/1506.02142

    Gal, Y., Ghahramani, Z.: Dropout as a Bayesian Approximation: Represent- ing Model Uncertainty in Deep Learning.arXiv preprint arXiv:1506.02142(2016). https://arxiv.org/abs/1506.02142

    Pith/arXiv arXiv 2016

  12. [12]

    https://arxiv.org/abs/1612.01474

    Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive un- certainty estimation using deep ensembles.arXiv preprint arXiv:1612.01474(2017). https://arxiv.org/abs/1612.01474

    Pith/arXiv arXiv 2017

  13. [13]

    https://arxiv.org/abs/2006.06015 9

    Monteiro, M., Le Folgoc, L., de Castro, D.C., Pawlowski, N., Marques, B., Kamnit- sas, K., van der Wilk, M., Glocker, B.: Stochastic segmentation networks: Modelling spatially correlated aleatoric uncertainty.arXiv preprint arXiv:2006.06015(2020). https://arxiv.org/abs/2006.06015 9

    Pith/arXiv arXiv 2006

  14. [14]

    In:International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, (2023).https://link.springer.com/ chapter/10.1007/978-3-031-44336-7_1

    Feiner, L.F., Menten, M.J., Hammernik, K., Hager, P., Huang, W., Rueckert, D., Braren, R.F., Kaissis, G.: Propagation and attribution of uncertainty in medical imaging pipelines. In:International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, (2023).https://link.springer.com/ chapter/10.1007/978-3-031-44336-7_1

    work page doi:10.1007/978-3-031-44336-7_1 2023

  15. [15]

    Wasserthal, J., Breit, H.C., Meyer, M.T., Pradella, M., Hinck, D., Sauter, A.W., Heye, T., Boll, D.T., Cyriac, J., Yang, S., Bach, M., Segeroth, M.: TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images.Radiology: Artificial Intelligence5(5), e230024 (2023).10.1148/ryai.230024

    doi:10.1148/ryai.230024 2023

  16. [16]

    Zhuang, X., Li, L., Payer, C., Štern, D., Urschler, M., Heinrich, M.P., Oster, J., Wang, C., Smedby, Ö., Bian, C., Yang, X., Heng, P.-A., Mortazi, A., Bagci, U., Yang, G., Sun, C., Galisot, G., Ramel, J.-Y., Brouard, T., Tong, Q., Si, W., Liao, X., Zeng, G., Shi, Z., Zheng, G., Wang, C., MacGillivray, T., Newby, D., Rhode, K., Ourselin, S., Mohiaddin, R.,...

    work page doi:10.1016/j.media.2019.101537 2019

  17. [17]

    Gharleghi, R., et al.: Automated segmentation of normal and diseased coronary arteries – The ASOCA challenge.Computerized Medical Imaging and Graphics97, 102049 (2022).10.1016/j.compmedimag.2022.102049

    doi:10.1016/j.compmedimag.2022.102049 2022

  18. [18]

    Project page (2025).https://www.neeldey.com/anatomix/

    Dey, N., Billot, B., Wong, H.E., Wang, C.J., Ren, M., Grant, P.E., Dalca, A.V., Golland, P.: Anatomix: Stable representations for any biomedical image. Project page (2025).https://www.neeldey.com/anatomix/. Accessed 19 Jan 2026

    2025

  19. [19]

    Sinclair, M., Schuh, A., Hahn, K., Petersen, K., Bai, Y., Batten, J., Schaap, M., Glocker, B.: Atlas-ISTN: Joint segmentation, registration and atlas construc- tion with image-and-spatial transformer networks.arXiv preprint arXiv:2012.10533 (2020).https://arxiv.org/abs/2012.10533

    Pith/arXiv arXiv 2012

  20. [20]

    van Griethuysen, J.J.M., Fedorov, A., Parmar, C., Hosny, A., Aucoin, N., Narayan, V., Beets-Tan, R.G.H., Fillon-Robin, J.C., Pieper, S., Aerts, H.J.W.L.: Computa- tional Radiomics System to Decode the Radiographic Phenotype.Cancer Research 77(21), e104–e107 (2017).10.1158/0008-5472.CAN-17-0339

    doi:10.1158/0008-5472.can-17-0339 2017

  21. [21]

    https://arxiv.org/abs/1907.10902

    Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A next-generation hyperparameter optimization framework.arXiv preprint arXiv:1907.10902(2019). https://arxiv.org/abs/1907.10902

    Pith/arXiv arXiv 1907

  22. [22]

    https://arxiv.org/abs/1703.04977

    Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision?.Advances in neural information processing systems(2017). https://arxiv.org/abs/1703.04977

    Pith/arXiv arXiv 2017

  23. [23]

    Kahl, K.C., Lüth, C., Zenk, M., Maier-Hein, K., Jaeger, P.F.: Val- ues: A framework for systematic validation of uncertainty estimation in semantic segmentation.In International Conference on Learning Represen- tations(2024).https://proceedings.iclr.cc/paper_files/paper/2024/hash/ 1548d98b62d3a4382a31ba77d89186cd-Abstract-Conference.html 10

    2024