REVIEW 3 major objections 7 minor 26 references
Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation
T0 review · 3 major / 7 minor · reviewed 2026-08-27 · deepseek-v4-flash
Pith's one-line read This paper shows that a memory-efficient 3D V-Net trained on 64-cube patches can segment glioma subregions and, more importantly, that voxel-wise uncertainty from test-time dropout and augmentation produces maps flagging the model's…
desk verdict Clean, honest BraTS write-up, but the central uncertainty claim is never actually demonstrated; useful as a cautionary example, not as a building block. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the modified V-Net together with the uncertainty recipe applied on top of it. The V-Net is an encoder-decoder fully convolutional network for volumetric segmentation, using strided-convolution downsampling, instance normalization, ELU activations, and four softmax output channels; it is trained on $64^{3}$ patches, half centered on tumor, with a dice loss. At test time the epistemic uncertainty map for each evaluation region is the voxel-wise variance $\mathrm{var} = \frac{1}{B}\sum_{b=1}^{B}(y_b^i - y_{\mathrm{mean}}^i)^2$ over $B=50$ dropout passes, and aleatoric uncertainty is obtained identically under random flips, intensity shifts, intensity scales, and Gaussian noise. The mechanism that carries the argument is the repeated-sampling variance: it converts a deterministic segmenter into a distribution over segmentations whose spread is read as doubt.
What would settle it
Rank all predicted voxels on a ground-truth validation set by their uncertainty value and compute segmentation accuracy within deciles: if the uncertainty is informative, accuracy should rise steadily from the most-uncertain decile to the most-certain decile; if accuracy is flat across deciles, the low FTP/FTN numbers are just an artifact of the thresholding metric and the maps carry no real information.
Extended reading notes
Core claim
The paper's central discovery is that a V-Net modified with instance normalization, doubled feature channels, and $64^{3}$ patch training can estimate two distinct flavors of voxel-wise uncertainty at test time: epistemic uncertainty from the variance of $B=50$ predictions with dropout active, and aleatoric uncertainty from the variance of predictions under test-time augmentation and Gaussian noise. The authors argue that the resulting maps are informative: the model shows low ratios of filtered true positives and true negatives in the BraTS uncertainty evaluation, and visual inspection shows higher variance near tumor boundaries and in misclassified regions. They also report that averaging the 50 dropout predictions does not beat the single-pass post-processed model on segmentation metrics, so the value of the Monte-Carlo procedure lies in the uncertainty maps rather than in improved masks.
Load-bearing premise
The whole uncertainty story depends on the spread of 50 dropout runs and of augmented test passes really measuring how much the model does not know; if those approximations fail, the maps do not say what the paper claims.
Editorial extensions
If this is right
- If the uncertainty estimates are right, a clinician can be shown a color overlay of where the model is guessing, rather than a single hard mask.
- Because the direct post-processed prediction outperforms the averaged dropout prediction, future work can keep the fast single-pass segmentation and run the 50 passes only when a confidence map is needed.
- The patch-training setup (64^3 patches, batch 8) makes the method run on a 12 GB GPU, so the uncertainty recipe is available outside large compute centres.
- The BraTS filtered-ratios being low implies that removing voxels the model marks uncertain removes few correct predictions, which is the property the challenge rewards.
Reading between the lines
- A natural next step the paper does not take is to aggregate the per-subregion variance maps into one overall doubt volume; the max or entropy across regions would give a single clinical display.
- The choice of $B=50$ is untested; plotting variance versus number of passes on a few subjects would show whether the epistemic map has converged or is dominated by sampling noise.
- The same two-channel uncertainty recipe transfers to any patch-trained volumetric segmentation task, but the paper only demonstrates it on BraTS gliomas.
- Uncertainty maps could drive active learning: send the highest-variance patches to an expert for relabeling, since the paper shows uncertainty concentrates near boundaries and errors.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript describes a 3D V-Net-based encoder-decoder for brain tumor segmentation in multimodal MRI, trained on BraTS 2020 with patch sampling, instance normalization, and data augmentation. The authors estimate voxel-wise epistemic uncertainty via Monte Carlo dropout (B=50) and aleatoric uncertainty via test-time data augmentation, then evaluate segmentation with Dice and Hausdorff distance on the BraTS training and validation sets. They report low FTP and FTN values as evidence that the model is confident in its correct predictions, and they present qualitative uncertainty maps. The paper concludes that uncertainty maps could help clinicians identify unreliable segmentations, while acknowledging difficulties in whole-tumor and especially enhancing-tumor segmentation.
Significance. If the uncertainty maps were properly validated, the practical value for clinicians reviewing automated segmentations would be real. The paper has some strengths: it uses a public benchmark (BraTS 2020), reports external validation results, and describes the architecture and training choices in sufficient detail for reproduction, although no code is provided. However, the uncertainty component, which is the main novelty, is evaluated only through two ratio metrics and a few visual examples, so the significance of the uncertainty contribution is currently unestablished.
major comments (3)
- [Section 4.2, Table 3] The claim that 'our model has low FTP and FTN, meaning that it is certain of those predictions' is not supported by the reported quantities. FTP and FTN measure the fraction of correct true positives and true negatives that are removed when voxels above an uncertainty threshold are filtered; low values only show that the filtering removes few correct voxels, and they carry no information about whether the removed voxels are enriched in errors. To establish the abstract's utility claim, the paper must report the threshold T, the uncertainty distribution, and a direct measure of how well uncertainty discriminates correct from incorrect voxels (for example, an ROC curve or AUC for uncertainty as an error detector, or error rates by uncertainty quantile).
- [Section 4.3, Figures 2-3] The visual assessment is anecdotal. The statement that 'the uncertainty values show that the model is more uncertain in the tumor surroundings and areas where the prediction has been miss-classified' is not quantified; a few selected slices cannot substantiate a general property. The authors should add a quantitative analysis over the full validation set, such as the mean uncertainty on false-positive, false-negative, true-positive, and true-negative voxels, or a reliability/calibration plot.
- [Section 3.4, Tables 1-2] The post-processing connected-component threshold is chosen by analyzing the training set, and the same threshold is then applied to the validation set. While this is not circular because the validation set is external, the paper neither reports the threshold value nor a sensitivity analysis, so the contribution of post-processing to the reported Dice/Hausdorff improvements cannot be assessed. Please report the chosen threshold and show how validation results vary over a range of threshold values.
minor comments (7)
- [Figure 3 caption] The caption lists image order as '(5) WT uncertainty map (5) ET uncertainty map'; the last panel should be labeled (6).
- [Section 3.5] Please clarify whether y_i^b in the variance formula are hard labels or softmax probabilities; the variance formula is defined for scalar values, and the paper says 'for each evaluation region' without specifying how the scalar per-region variance is obtained from the four output channels.
- [Section 4.2] The symbols TP100 and TP_T in the definition of FTP are not defined; define them and the threshold T explicitly.
- [Discussion] There are typographical errors: 'enchancing tumor' should be 'enhancing tumor' and 'Haurdorff' should be 'Hausdorff'.
- [Abstract] The phrase 'Automation of brain tumors' should be 'Automation of brain tumor segmentation'.
- [Section 3.3] 'in the passed 10 epochs' should be 'in the past 10 epochs'.
- [Tables 1-2] The reported differences between variants are small and no confidence intervals or significance tests are given; at least per-patient standard deviations should be reported.
Circularity Check
No circularity: uncertainty maps are computed from the model's own stochastic outputs and evaluated on an external benchmark.
full rationale
No significant circularity found. The paper's derivation chain is empirical: it trains a V-Net on BraTS 2020 training data, applies standard post-processing whose threshold is chosen on the training set, and evaluates Dice/Hausdorff and FTP/FTN on the BraTS validation set. Uncertainty maps are computed from B=50 Monte-Carlo dropout samples and test-time augmentations (Section 3.5) directly from predictive variance and entropy; these quantities do not incorporate the target labels, FTP, or any fitted parameter, and the evaluation against the external BraTS benchmark is independent of the method's inputs. The two self-citations ([11], [13]) are contextual related-work comparisons and are not load-bearing for the architecture or uncertainty claim. The statement 'our model has low FTP and FTN, meaning that it is certain of those predictions' is an over-interpretation: FTP/FTN measure only how many true positives/negatives are removed at a threshold, not whether removed voxels are errors. This is an evidentiary weakness, not circularity, because low FTP/FTN are not used to define or derive uncertainty. Similarly, the clinical utility claim in the abstract is untested, but lack of validation is not circularity.
Assumptions & free parameters
free parameters (7)
- post-processing connected component threshold
- patch size =
64x64x64
- batch size =
8
- test-time dropout probability =
0.5
- number of MC dropout iterations B =
50
- data augmentation ranges =
flip 50%, intensity shift in (-0.1, 0.1) std, scale (0.9, 1.1)
- learning rate and scheduler settings =
lr=1e-2, momentum=0.99, patience=10, factor=0.1
assumptions (4)
- domain assumption BraTS 2020 training ground truth annotations are correct and representative.
- domain assumption Monte Carlo dropout is a valid Bayesian approximation for epistemic uncertainty.
- domain assumption Test-time data augmentation captures aleatoric uncertainty.
- domain assumption The V-Net architecture is a suitable backbone for this segmentation task.
Cite this review
Pith. "Pith review of Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation." pith.science (2026). https://pith.science/paper/CEBE2TOM
@misc{pith2026200912188,
author = {Pith},
title = {Pith review of: Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/CEBE2TOM}},
note = {Machine review of arXiv:2009.12188}
}
read the original abstract
Automation of brain tumors in 3D magnetic resonance images (MRIs) is key to assess the diagnostic and treatment of the disease. In recent years, convolutional neural networks (CNNs) have shown improved results in the task. However, high memory consumption is still a problem in 3D-CNNs. Moreover, most methods do not include uncertainty information, which is specially critical in medical diagnosis. This work proposes a 3D encoder-decoder architecture, based on V-Net \cite{vnet} which is trained with patching techniques to reduce memory consumption and decrease the effect of unbalanced data. We also introduce voxel-wise uncertainty, both epistemic and aleatoric using test-time dropout and data-augmentation respectively. Uncertainty maps can provide extra information to expert neurologists, useful for detecting when the model is not confident on the provided segmentation.
Figures
Reference graph
Works this paper leans on
- [19]
- [1]
-
[2]
S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J.S. Kirby, et al.: ”Ad- vancing The Cancer Genome Atlas glioma MRI collections with expert segmen- tation labels and radiomic features”, Nature Scientific Data, 4:170117 (2017) https://doi.org/10.1038/sdata.2017.117
-
[3]
S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, et al.: ”Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progres- sion Assessment, and Overall Survival Prediction in the BRATS Challenge”, arXiv preprint arXiv:1811.02629 (2018)
work page Pith review arXiv 2018
-
[4]
The Cancer Imaging Archive (2017)
S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., ”Segmentation Labels and Radiomic Features for the Pre-operative Scans of the TCGA-GBM collection”, The Cancer Imaging Archive, 2017. DOI: 10.7937/K9/TCIA.2017.KLXWJJ1Q
-
[5]
The Cancer Imaging Archive (2017)
S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., ”Segmentation Labels and Radiomic Features for the Pre-operative Scans of the TCGA-LGG collection”, The Cancer Imaging Archive, 2017. DOI: 10.7937/K9/TCIA.2017.GJQ7R0EF
-
[6]
V-net: Fully convo- lu- tional neural networks for volumetric medical image segmentation
Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convo- lu- tional neural networks for volumetric medical image segmentation. 3D Vision (3DV), 2016 Fourth International Conference on. IEEE, 2016
work page 2016
-
[7]
Neuro-oncology vol.17 01-2015 https://doi.org/10.1093/neuonc/nou358
Morgan, L Lloyd: The epidemiology of glioma in adults: A ”state of the science” review. Neuro-oncology vol.17 01-2015 https://doi.org/10.1093/neuonc/nou358
Show all 26 references
-
[8]
Armen Der Kiureghian and Ove Ditlevsen: Aleatory or epistemic? does it matter? Structural safety, 31(2):105112, 2009
2009
-
[10]
Newcombe, Joanna P
Konstantinos Kamnitsas, Christian Ledig, Virginia F.J. Newcombe, Joanna P. Simpson, Andrew D. Kane, David K. Menon, Daniel Rueckert, Ben Glocker: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Medical Image Analysis, Volume 36, 2017...
2017 doi
-
[11]
In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries
Casamitjana, A., Puch, S., Aduriz, A., Vilaplana, V., ”3D Convolutional Neural Networks for Brain Tumor Segmentation: a comparison of multi-resolution archi- tectures”. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2016. Lecture Not...
2016
-
[12]
Kamnitsas, K., Bai, W., Ferrante, E., McDonagh, S., Sinclair, M., Pawlowski, N: Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmen- tation in International MICCAI Brainlesion Workshop (Quebec, QC), 450462 arXiv preprint arXiv:1711.01468, 2017
2017 arXiv
-
[13]
In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries
Casamitjana, A., Cat, M., Snchez, I., Combalia, M., Vilaplana, V., ”Cascaded V- Net Using ROI Masks for Brain Tumor Segmentation”. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2017. Lecture Notes in Computer Science, vol 10670. Spr...
2017
-
[14]
arXiv preprint arXiv:1810.11654, 2016
Andriy Myronenko: 3D MRI brain tumor segmentation using autoencoder regu- larization. arXiv preprint arXiv:1810.11654, 2016
2016 arXiv
-
[15]
International MICCAI Brainlesion Workshop, pp
Isensee, F., et al.: No new-net. International MICCAI Brainlesion Workshop, pp. 234244. Springer (2018)
2018
-
[16]
(2020) Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task
Jiang Z., Ding C., Liu M., Tao D. (2020) Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task. In: Crimi A., Bakas S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Brain- Les 2019. Lecture Notes in Compute...
2020
-
[17]
(2020) Bag of Tricks for 3D MRI Brain Tumor Segmentation
Zhao YX., Zhang YM., Liu CL. (2020) Bag of Tricks for 3D MRI Brain Tumor Segmentation. In: Crimi A., Bakas S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2019. Lecture Notes in Computer Science, vol 11992. Springer, Cham
2020
-
[18]
Frontiers in Com- putational Neuroscience vol.14 page 6 https://doi.org/10.3389/fncom.2020.00006, 2020
Natekar Parth, Kori Avinash, Krishnamurthi Ganapathy AUTHOR=Natekar Parth, Kori Avinash, Krishnamurthi Ganapathy: Demystifying Brain Tumor Seg- mentation Networks: Interpretability and Uncertainty Analysis. Frontiers in Com- putational Neuroscience vol.14 page 6 https://doi.or...
-
[20]
(2019) Ensembles of Densely-Connected CNNs with Label-Uncertainty for Brain Tumor Segmentation
McKinley R., Meier R., Wiest R. (2019) Ensembles of Densely-Connected CNNs with Label-Uncertainty for Brain Tumor Segmentation. In: Crimi A., Bakas S., Kuijf H., Keyvan F., Reyes M., van Walsum T. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuri...
2019
-
[21]
Instance Normaliza- tion: The Missing Ingredient for Fast Stylization
Dmitry Ulyanov and Andrea Vedaldi and Victor Lempitsky. Instance Normaliza- tion: The Missing Ingredient for Fast Stylization. arXiv preprint arXiv:1607.08022, 2016
2016 arXiv
-
[22]
Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36 (2017) 6178
2017
-
[23]
arXiv preprint arXiv:1506.02142, 2015
Yarin Gal and Zoubin Ghahraman.Dropout as a Bayesian Approximation: Rep- resenting Model Uncertainty in Deep Learning. arXiv preprint arXiv:1506.02142, 2015
2015 arXiv
-
[24]
Automatic differentiation in PyTorch, NIPS-W 2017 Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation 11
Paszke, Adam and Gross, Sam and Chintala, Soumith and Chanan, Gregory and Yang, Edward and DeVito, Zachary and Lin, Zeming and Desmaison, Alban and Antiga, Luca and Lerer, Adam. Automatic differentiation in PyTorch, NIPS-W 2017 Brain Tumor Segmentation using 3D-CNNs with Uncert...
2017
-
[25]
U-net: Convolu- tional net- works for biomedical image segmentation
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. U-net: Convolu- tional net- works for biomedical image segmentation. MICCAI. Springer, 2015. https://doi.org/10.1007/978-3-319-24574-4 28
2015 doi
-
[26]
Lecture Notes in Computer Science 240-248, Springer International Publishing
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmenta- tions. Lecture Notes in Computer Science 240-248, Springer International Publishing
-
[2017]
https://doi.org/10.1007/978-3-319-67558-9 28
Reviewed August 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.