Improving PET/CT-Based Whole-Body Lesion Segmentation Using Prediction Uncertainty-Augmented Models
Pith reviewed 2026-06-27 16:52 UTC · model grok-4.3
The pith
Bayesian ensembling and uncertainty-augmented training improve lesion segmentation robustness and recovery over deterministic nnU-Net models in whole-body PET/CT scans.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Bayesian ensembling reduces training stochasticity and raises performance on the unseen AutoPET-III test set; voxel-wise epistemic and aleatoric uncertainty maps correlate with misclassifications; epistemic uncertainty-augmented training improves lesion recovery while increasing false-positive volume; and a case-adaptive routing strategy that selects between the two models yields higher Dice scores. The study uses two public datasets covering FDG and PSMA tracers across multiple cancer types and claims to be the first systematic investigation of uncertainty quantification in multi-tracer, pan-cancer whole-body PET/CT segmentation.
What carries the argument
Bayesian ensembling of nnU-Net models combined with voxel-wise epistemic uncertainty quantification and uncertainty-augmented training that feeds uncertainty estimates back into model optimization.
If this is right
- Bayesian ensembling yields more robust predictions than single deterministic nnU-Net runs on unseen AutoPET-III test data.
- Epistemic uncertainty maps identify regions of model disagreement that correspond to false-positive and missed-lesion errors.
- Uncertainty-augmented training increases lesion recovery at the expense of higher false-positive volume.
- Case-adaptive routing between the base and augmented models raises overall Dice performance.
Where Pith is reading between the lines
- The uncertainty maps could serve as a practical signal for radiologists to review borderline regions rather than re-reading entire scans.
- The observed precision-recall trade-off suggests the framework may be tuned differently depending on whether missing a lesion or over-calling one is more clinically costly.
- Because the method operates on already-trained ensembles, it could be applied post hoc to existing nnU-Net deployments in other PET/CT tasks without full retraining.
Load-bearing premise
The performance gains observed on the AutoPET-III and Deep-PSMA public datasets will hold for scans acquired on new scanners, different patient populations, and varied clinical workflows without retraining or recalibration.
What would settle it
Applying the trained models to a held-out external cohort from a different scanner vendor or institution and finding no improvement in lesion detection rate or Dice score compared with the deterministic baseline.
Figures
read the original abstract
Accurate lesion segmentation from whole-body Positron Emission Tomography (PET)/Computed Tomography (CT) scans is essential for cancer staging and treatment planning. PET provides functional metabolic information with different radiotracers, while CT offers anatomical localization. Lesion delineation from PET/CT imaging is clinically challenging due to subtle imaging features, confounders, and inter-reader variability. Existing deep learning approaches suffer from training-related stochasticity, inconsistent predictions, missed lesions in high tumor-burden cases, and lack uncertainty quantification, limiting their clinical reliability. Using nnU-Net as a baseline, we propose an uncertainty-aware framework for whole-body PET/CT lesion segmentation that integrates (1) Bayesian ensembling to reduce training stochasticity, (2) voxel-wise uncertainty quantification with epistemic and aleatoric decomposition, and (3) epistemic uncertainty-augmented training to improve lesion detection. Two public datasets, AutoPET-III (1,611 scans) and Deep-PSMA (200 scans), comprising FDG and PSMA studies across multiple cancer types, are used for training and evaluation. Bayesian ensembling improves robustness and performance over deterministic nnU-Net models on the unseen AutoPET-III test set. Uncertainty maps highlight regions of model disagreement and correlate with misclassifications, particularly false positives. Uncertainty-augmented training improves lesion recovery at the cost of increased FPVol, reflecting a precision-recall trade-off. A case-adaptive routing strategy further improves Dice by selecting between the base and augmented models. To our knowledge, this is the first study to systematically investigate uncertainty quantification in multi-tracer, pan-cancer PET/CT segmentation and to combine Bayesian ensembling with uncertainty-aware modeling for this task.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes an uncertainty-aware framework for whole-body PET/CT lesion segmentation extending nnU-Net with (1) Bayesian ensembling to mitigate training stochasticity, (2) voxel-wise epistemic/aleatoric uncertainty maps, and (3) epistemic uncertainty-augmented training. Experiments on AutoPET-III (1,611 scans) and Deep-PSMA (200 scans) across FDG and PSMA tracers report improved robustness and Dice on held-out AutoPET-III test data, correlation between uncertainty and misclassifications, a precision-recall trade-off from augmented training, and further Dice gains via case-adaptive model routing. The work positions itself as the first systematic study of uncertainty quantification in multi-tracer, pan-cancer PET/CT segmentation.
Significance. If the empirical gains hold under external scrutiny, the combination of Bayesian ensembling and uncertainty-augmented training could meaningfully increase the clinical reliability of automated lesion delineation by flagging uncertain regions and recovering missed lesions, addressing documented limitations of deterministic models in high tumor-burden cases. The multi-tracer, multi-cancer scope on two sizable public datasets is a positive contribution; however, the absence of independent external cohorts limits the strength of the robustness and generalization assertions.
major comments (2)
- [Abstract (evaluation description) and Results] The central robustness and clinical-reliability claims rest on performance lifts observed only on the held-out AutoPET-III test set and Deep-PSMA splits; no additional external validation cohort acquired on a different scanner vendor, reconstruction kernel, or demographic is reported, leaving the generalization step required by the abstract unsupported by direct evidence.
- [Abstract] The abstract states that Bayesian ensembling “improves robustness and performance” and that uncertainty-augmented training “improves lesion recovery,” yet supplies no numerical Dice, sensitivity, FPVol, or statistical-test values, ablation tables, or error bars; without these quantitative details the magnitude and reliability of the reported lifts cannot be assessed.
minor comments (2)
- [Methods] Notation for epistemic versus aleatoric uncertainty decomposition should be introduced with explicit equations or pseudocode in the Methods section to avoid ambiguity when readers interpret the uncertainty maps.
- [Methods / Results] The case-adaptive routing strategy is mentioned only briefly; a dedicated paragraph or small table clarifying the selection criterion (e.g., uncertainty threshold) would improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive review. We address each major comment below with honest responses and indicate planned revisions to the manuscript.
read point-by-point responses
-
Referee: [Abstract (evaluation description) and Results] The central robustness and clinical-reliability claims rest on performance lifts observed only on the held-out AutoPET-III test set and Deep-PSMA splits; no additional external validation cohort acquired on a different scanner vendor, reconstruction kernel, or demographic is reported, leaving the generalization step required by the abstract unsupported by direct evidence.
Authors: We acknowledge that the evaluation relies on held-out splits from AutoPET-III (1,611 scans) and Deep-PSMA (200 scans), which include multi-tracer (FDG/PSMA) and multi-cancer diversity but do not constitute fully independent external cohorts from different vendors or demographics. This limits the strength of broad generalization assertions in the abstract. We will revise the abstract to temper claims regarding generalization and add a limitations section explicitly discussing the need for future multi-center validation on diverse scanner protocols and populations. revision: partial
-
Referee: [Abstract] The abstract states that Bayesian ensembling “improves robustness and performance” and that uncertainty-augmented training “improves lesion recovery,” yet supplies no numerical Dice, sensitivity, FPVol, or statistical-test values, ablation tables, or error bars; without these quantitative details the magnitude and reliability of the reported lifts cannot be assessed.
Authors: We agree that the abstract should provide quantitative support for the stated improvements. In the revised manuscript, we will update the abstract to include specific numerical results (e.g., Dice scores, sensitivity, FPVol changes, and references to statistical tests) drawn from the results section, along with mentions of the ablation studies and uncertainty correlation findings. revision: yes
- Absence of an independent external validation cohort from different scanner vendors, reconstruction kernels, or demographics, which cannot be addressed without new data acquisition or access.
Circularity Check
No circularity; empirical results on public datasets are self-contained
full rationale
The paper describes an uncertainty-aware segmentation framework built on nnU-Net with Bayesian ensembling, epistemic/aleatoric uncertainty maps, and uncertainty-augmented training. All reported gains (Dice, lesion recovery, FPVol) are obtained via direct empirical comparison against deterministic baselines on the held-out AutoPET-III test split and Deep-PSMA dataset. No equations, parameter-fitting steps, or self-citations are invoked that would make any performance metric equivalent to a quantity defined by the model itself. The evaluation therefore rests on external public benchmarks rather than any self-referential reduction.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Trotter, A
J. Trotter, A. R. Pantel, B.-K. K. Teo, F. E. Escorcia, T. Li, D. A. Pryma, and N. K. Taunk, Positron emission tomography (PET)/computed tomography (CT) imaging in radiation therapy treatment planning: a review of PET imaging tracers and methods to incorporate PET/CT, Advances in radiation oncology8, 101212 (2023)
2023
-
[2]
D. A. Silver, I. Pellicer, W. R. Fair, W. Heston, and C. Cordon-Cardo, Prostate-specific membrane antigen expression in normal and malignant human tissues., Clinical cancer research: an official journal of the American Association for Cancer Research3, 81–85 (1997)
1997
-
[3]
Som et al., A fluorinated glucose analog, 2-fluoro-2-deoxy-D-glucose (F-18): nontoxic tracer for rapid tumor detection, Journal of Nuclear Medicine21, 670–675 (1980)
P. Som et al., A fluorinated glucose analog, 2-fluoro-2-deoxy-D-glucose (F-18): nontoxic tracer for rapid tumor detection, Journal of Nuclear Medicine21, 670–675 (1980)
1980
-
[4]
M. Blau, W. Nagler, and M. Bender, Fluorine-18: a new isotope for bone scanning, J. nuclear Med.3(1962)
1962
-
[5]
M. S. Hofman and R. J. Hicks, How we read oncologic FDG PET/CT, Cancer Imaging16, 35 (2016). 27 Improving PET/CT-based WBLS Using Uncertainy-Augmented ModelsA PREPRINT
2016
-
[6]
H. Yu, C. Caldwell, K. Mah, and D. Mozeg, Coregistered FDG PET/CT-based textural characterization of head and neck cancer for radiation treatment planning, IEEE transactions on medical imaging28, 374–383 (2008)
2008
-
[7]
J. Yang, B. M. Beadle, A. S. Garden, D. L. Schwartz, and M. Aristophanous, A multimodality segmentation framework for automatic target delineation in head and neck radiotherapy, Medical physics42, 5310–5320 (2015)
2015
-
[8]
Q. Song, J. Bai, D. Han, S. Bhatia, W. Sun, W. Rockey, J. E. Bayouth, J. M. Buatti, and X. Wu, Optimal co- segmentation of tumor in PET-CT images with context information, IEEE transactions on medical imaging32, 1685–1697 (2013)
2013
-
[9]
Bagci, J
U. Bagci, J. K. Udupa, N. Mendhiratta, B. Foster, Z. Xu, J. Yao, X. Chen, and D. J. Mollura, Joint segmentation of anatomical and functional images: Applications in quantification of lesions from PET, PET-CT, MRI-PET, and MRI-PET-CT images, Medical image analysis17, 929–945 (2013)
2013
-
[10]
B. Huang et al., Fully automated delineation of gross tumor volume for head and neck cancer on PET-CT using deep learning: A dual-center study, Contrast media & molecular imaging2018, 8923028 (2018)
2018
-
[11]
Oreiller et al., Head and neck tumor segmentation in PET/CT: the HECKTOR challenge, Medical image analysis77, 102336 (2022)
V . Oreiller et al., Head and neck tumor segmentation in PET/CT: the HECKTOR challenge, Medical image analysis77, 102336 (2022)
2022
-
[12]
X. Zhao, L. Li, W. Lu, and S. Tan, Tumor co-segmentation in PET/CT using multi-modality fully convolutional neural network, Physics in Medicine & Biology64, 015011 (2018)
2018
-
[13]
D. Jin, D. Guo, T.-Y . Ho, A. P. Harrison, J. Xiao, C.-K. Tseng, and L. Lu, Accurate esophageal gross tumor volume segmentation in PET/CT using two-stream chained 3D deep network fusion, inInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 182–191, Springer, 2019
2019
-
[14]
P. Blanc-Durand et al., Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG- PET/CT for total metabolic tumour volume prediction using a convolutional neural network., European Journal of Nuclear Medicine and Molecular Imaging48, 1362–1370 (2021)
2021
-
[15]
L. Xu, G. Tetteh, J. Lipkova, Y . Zhao, H. Li, P. Christ, M. Piraud, A. Buck, K. Shi, and B. H. Menze, Auto- mated whole-body bone lesion detection for multiple myeloma on 68Ga-pentixafor PET/CT imaging using deep learning methods, Contrast media & molecular imaging2018, 2391925 (2018)
2018
-
[16]
Jemaa, J
S. Jemaa, J. Fredrickson, R. A. Carano, T. Nielsen, A. de Crespigny, and T. Bengtsson, Tumor segmentation and feature extraction from whole-body FDG-PET/CT using cascaded 2D and 3D convolutional neural networks, Journal of digital imaging33, 888–894 (2020). 28 Improving PET/CT-based WBLS Using Uncertainy-Augmented ModelsA PREPRINT
2020
-
[17]
J. He, Y . Zhang, M. Chung, M. Wang, K. Wang, Y . Ma, X. Ding, Q. Li, and Y . Pu, Whole-body tumor seg- mentation from PET/CT images using a two-stage cascaded neural network with camouflaged object detection mechanisms, Medical Physics50, 6151–6162 (2023)
2023
-
[18]
Gatidis et al., Results from the autoPET challenge on fully automated lesion segmentation in oncologic PET/CT imaging, Nature Machine Intelligence6, 1396–1405 (2024)
S. Gatidis et al., Results from the autoPET challenge on fully automated lesion segmentation in oncologic PET/CT imaging, Nature Machine Intelligence6, 1396–1405 (2024)
2024
-
[19]
A. Liu, D. Jia, K. Sun, R. Meng, M. Zhao, Y . Jiang, Z. Dong, Y . Gao, and D. Shen, LM-UNet: Whole- Body PET-CT Lesion Segmentation with Dual-Modality-Based Annotations Driven by Latent Mamba U-Net, inInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 405–414, Springer, 2024
2024
-
[20]
Y . Xu, I. Klyuzhin, S. Harsini, A. Ortiz, S. Zhang, F. B ´enard, R. Dodhia, C. F. Uribe, A. Rahmim, and J. L. Ferres, Automatic segmentation of prostate cancer metastases in PSMA PET/CT images using deep neural networks with weighted batch-wise dice loss, Computers in Biology and Medicine158, 106882 (2023)
2023
-
[21]
Y . Li et al., An automated deep learning-based framework for uptake segmentation and classification on PSMA PET/CT imaging of patients with prostate cancer, Journal of Imaging Informatics in Medicine37, 2206–2215 (2024)
2024
-
[22]
Milletari, N
F. Milletari, N. Navab, and S.-A. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in2016 fourth international conference on 3D vision (3DV), pages 565–571, Ieee, 2016
2016
-
[23]
M. Ingrisch, J. Dexl, K. Jeblick, C. Cyran, S. Gatidis, and T. Kuestner, Automated Lesion Segmentation in Whole-Body PET/CT - Multitracer Multicenter Generalization, Zenodo, Version 1, 2024, Available athttps: //zenodo.org/records/10990932
-
[24]
H. Kalisch, F. H ¨orst, K. Herrmann, J. Kleesiek, and C. Seibold, Autopet III challenge: Incorporating anatomical knowledge into nnUNet for lesion segmentation in PET/CT, arXiv preprint arXiv:2409.12155 (2024)
-
[25]
J. Song, X. Yang, X. Liang, J. Huang, J. Ma, Y . Sun, W. Luo, S. Mok, Y . Wang, and T. Tan, DuEU-Net: Dual Encoder UNet with Modality-Agnostic Training for PET-CT Multi-modal Organ and Lesion Segmentation, in Deep Breast Workshop on AI and Imaging for Diagnostic and Treatment Challenges in Breast Care, pages 23–31, Springer, 2024
2024
- [26]
- [27]
-
[28]
C.-W. Wang, T.-S. Su, and K.-W. Liu, Dual channel CW nnU-Net for 3D PET-CT Lesion Segmentation in 2024 autoPET III Challenge, arXiv preprint arXiv:2409.07144 (2024)
-
[29]
Ronneberger, P
O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, inInternational Conference on Medical image computing and computer-assisted intervention, pages 234–241, Springer, 2015
2015
-
[30]
Isensee, P
F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation, Nature methods18, 203–211 (2021)
2021
-
[31]
Glorot and Y
X. Glorot and Y . Bengio, Understanding the difficulty of training deep feedforward neural networks, inPro- ceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256, JMLR Workshop and Conference Proceedings, 2010
2010
-
[32]
N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, On large-batch training for deep learning: Generalization gap and sharp minima, arXiv preprint arXiv:1609.04836 (2016)
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[33]
Schott, V
B. Schott, V . Santoro-Fernandes, ˇZ. Klane ˇcek, S. Perlman, and R. Jeraj, Uncertainty quantification for deep learning-based metastatic lesion segmentation on whole body pet/ct, Physics in Medicine & Biology70, 115009 (2025)
2025
-
[34]
Kendall and Y
A. Kendall and Y . Gal, What uncertainties do we need in bayesian deep learning for computer vision?, Advances in neural information processing systems30(2017)
2017
-
[35]
F. Yu, A. Moehring, O. Banerjee, T. Salz, N. Agarwal, and P. Rajpurkar, Heterogeneity and predictors of the effects of AI assistance on radiologists, Nature Medicine30, 837–849 (2024)
2024
-
[36]
Lakshminarayanan, A
B. Lakshminarayanan, A. Pritzel, and C. Blundell, Simple and scalable predictive uncertainty estimation using deep ensembles, Advances in neural information processing systems30(2017)
2017
-
[37]
Abdar et al., A review of uncertainty quantification in deep learning: Techniques, applications and challenges, Information fusion76, 243–297 (2021)
M. Abdar et al., A review of uncertainty quantification in deep learning: Techniques, applications and challenges, Information fusion76, 243–297 (2021)
2021
-
[38]
Gal and Z
Y . Gal and Z. Ghahramani, Dropout as a bayesian approximation: Representing model uncertainty in deep learning, ininternational conference on machine learning, pages 1050–1059, PMLR, 2016
2016
-
[39]
Y . Zhao, C. Yang, A. Schweidtmann, and Q. Tao, Efficient Bayesian uncertainty estimation for nnU-Net, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 535–544, Springer, 2022
2022
-
[40]
Sensoy, L
M. Sensoy, L. Kaplan, and M. Kandemir, Evidential deep learning to quantify classification uncertainty, Ad- vances in neural information processing systems31(2018). 30 Improving PET/CT-based WBLS Using Uncertainy-Augmented ModelsA PREPRINT
2018
-
[41]
G. Wang, W. Li, M. Aertsen, J. Deprest, S. Ourselin, and T. Vercauteren, Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks, Neurocomputing 338, 34–45 (2019)
2019
-
[42]
G. D. Ruxton, The unequal variance t-test is an underused alternative to Student’s t-test and the Mann–Whitney U test, Behavioral Ecology17, 688–690 (2006)
2006
-
[43]
N. Nachar et al., The Mann-Whitney U: A test for assessing whether two independent samples come from the same distribution, Tutorials in quantitative Methods for Psychology4, 13–20 (2008)
2008
- [44]
-
[45]
J. S. Yoon, K. Oh, Y . Shin, M. A. Mazurowski, and H.-I. Suk, Domain generalization for medical image analysis: A review, Proceedings of the IEEE112, 1583–1609 (2024)
2024
-
[46]
C. Chen, Z. Li, C. Ouyang, M. Sinclair, W. Bai, and D. Rueckert, Maxstyle: Adversarial style composition for robust medical image segmentation, inInternational Conference on Medical Image Computing and Computer- Assisted Intervention, pages 151–161, Springer, 2022
2022
-
[47]
Q. Liu, Q. Dou, and P.-A. Heng, Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains, inInternational conference on medical image computing and computer-assisted intervention, pages 475–485, Springer, 2020
2020
-
[48]
C. Xu, Z. Wen, Z. Liu, and C. Ye, Improved domain generalization for cell detection in histopathology images via test-time stain augmentation, inInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 150–159, Springer, 2022
2022
-
[49]
R. Wen, H. Yuan, D. Ni, W. Xiao, and Y . Wu, From denoising training to test-time adaptation: Enhancing domain generalization for medical image segmentation, inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 464–474, 2024
2024
-
[50]
Der Kiureghian and O
A. Der Kiureghian and O. Ditlevsen, Aleatory or epistemic? Does it matter?, Structural safety31, 105–112 (2009)
2009
-
[51]
Hartmann, A
A. Hartmann, A. Davari, T. Seehaus, M. Braun, A. Maier, and V . Christlein, Bayesian u-net for segmenting glaciers in sar imagery, in2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pages 3479–3482, IEEE, 2021. 31 Improving PET/CT-based WBLS Using Uncertainy-Augmented ModelsA PREPRINT
2021
-
[52]
Meakin, P
J. Meakin, P. K. Gerke, S. Kerkstra, T. Koopman, A. Mickan, C. van Run, H. van Zeeland, F. Ciompi, A. Her- ing, C. Jacobs, N. Khalili, P. Koopmans, J. van der Laak, G. Litjens, S. Quax, C. I. S ´anchez, J. Tannhauser, M. Groeneveld, and H. Huisman, Grand-Challenge.org, 2025
2025
-
[53]
Gatidis, T
S. Gatidis, T. Hepp, M. Fr ¨uh, C. La Foug`ere, K. Nikolaou, C. Pfannenberg, B. Sch¨olkopf, T. K¨ustner, C. Cyran, and D. Rubin, A whole-body FDG-PET/CT dataset with manually annotated tumor lesions, Scientific Data9, 601 (2022)
2022
-
[54]
Jeblick et al., A Whole-Body PSMA-PET/CT Dataset with Manually Annotated Tumor Lesions (PSMA- PET-CT-Lesions), 2024
K. Jeblick et al., A Whole-Body PSMA-PET/CT Dataset with Manually Annotated Tumor Lesions (PSMA- PET-CT-Lesions), 2024
2024
-
[55]
Depeweg, J.-M
S. Depeweg, J.-M. Hernandez-Lobato, F. Doshi-Velez, and S. Udluft, Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning, inInternational conference on machine learning, pages 1184–1193, PMLR, 2018
2018
-
[56]
Gal et al., Uncertainty in deep learning, (2016)
Y . Gal et al., Uncertainty in deep learning, (2016)
2016
-
[57]
Malinin and M
A. Malinin and M. Gales, Predictive uncertainty estimation via prior networks, Advances in neural information processing systems31(2018)
2018
-
[58]
Jungo and M
A. Jungo and M. Reyes, Assessing reliability and challenges of uncertainty estimations for medical image segmentation, inInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 48–56, Springer, 2019
2019
-
[59]
M. J. Roberts et al., A prospective, multi-centre trial of PSMA-PET compared to FDG-PET for staging of newly diagnosed high risk prostate cancer, EJNMMI research15, 92 (2025). 32
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.