Pith. sign in

REVIEW 3 major objections 7 minor 26 references

Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation

T0 review · 3 major / 7 minor · reviewed 2026-08-27 · deepseek-v4-flash

Pith's one-line read This paper shows that a memory-efficient 3D V-Net trained on 64-cube patches can segment glioma subregions and, more importantly, that voxel-wise uncertainty from test-time dropout and augmentation produces maps flagging the model's…

desk verdict Clean, honest BraTS write-up, but the central uncertainty claim is never actually demonstrated; useful as a cautionary example, not as a building block. read the letter →

arxiv 2009.12188 v1 pith:CEBE2TOM submitted 2020-09-24 eess.IV cs.CVcs.LG

classification eess.IVcs.CVcs.LG
keywords braintumorsegmentationglioma3DconvolutionalneuralnetworksV-Netuncertaintyestimationtest-timedropoutepistemicaleatoric
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish two things: a patch-trained 3D V-Net can segment brain tumor subregions on BraTS 2020 MRIs within a 12 GB GPU memory budget, and the same network can produce voxel-wise uncertainty maps—epistemic via 50 test-time dropout passes, aleatoric via test-time data augmentation—that highlight where the segmentation is unreliable. This matters because automated tumor segmentation is only clinically useful if a neurologist knows when to trust it; the uncertainty maps are meant to supply that trust signal. On the validation set the post-processed model reaches Dice 0.846 for whole tumor, 0.753 for tumor core, and 0.618 for enhancing tumor, and the authors report low filtered-true-positive and filtered-true-negative ratios, which they read as the model being certain when it is correct. The main novel contribution is therefore not the architecture but the demonstration that cheap Monte-Carlo-style uncertainty estimation attaches useful confidence information to an existing segmentation network.

What carries the argument

The load-bearing object is the modified V-Net together with the uncertainty recipe applied on top of it. The V-Net is an encoder-decoder fully convolutional network for volumetric segmentation, using strided-convolution downsampling, instance normalization, ELU activations, and four softmax output channels; it is trained on $64^{3}$ patches, half centered on tumor, with a dice loss. At test time the epistemic uncertainty map for each evaluation region is the voxel-wise variance $\mathrm{var} = \frac{1}{B}\sum_{b=1}^{B}(y_b^i - y_{\mathrm{mean}}^i)^2$ over $B=50$ dropout passes, and aleatoric uncertainty is obtained identically under random flips, intensity shifts, intensity scales, and Gaussian noise. The mechanism that carries the argument is the repeated-sampling variance: it converts a deterministic segmenter into a distribution over segmentations whose spread is read as doubt.

What would settle it

Rank all predicted voxels on a ground-truth validation set by their uncertainty value and compute segmentation accuracy within deciles: if the uncertainty is informative, accuracy should rise steadily from the most-uncertain decile to the most-certain decile; if accuracy is flat across deciles, the low FTP/FTN numbers are just an artifact of the thresholding metric and the maps carry no real information.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a V-Net modified with instance normalization, doubled feature channels, and $64^{3}$ patch training can estimate two distinct flavors of voxel-wise uncertainty at test time: epistemic uncertainty from the variance of $B=50$ predictions with dropout active, and aleatoric uncertainty from the variance of predictions under test-time augmentation and Gaussian noise. The authors argue that the resulting maps are informative: the model shows low ratios of filtered true positives and true negatives in the BraTS uncertainty evaluation, and visual inspection shows higher variance near tumor boundaries and in misclassified regions. They also report that averaging the 50 dropout predictions does not beat the single-pass post-processed model on segmentation metrics, so the value of the Monte-Carlo procedure lies in the uncertainty maps rather than in improved masks.

Load-bearing premise

The whole uncertainty story depends on the spread of 50 dropout runs and of augmented test passes really measuring how much the model does not know; if those approximations fail, the maps do not say what the paper claims.

Editorial extensions

If this is right

  • If the uncertainty estimates are right, a clinician can be shown a color overlay of where the model is guessing, rather than a single hard mask.
  • Because the direct post-processed prediction outperforms the averaged dropout prediction, future work can keep the fast single-pass segmentation and run the 50 passes only when a confidence map is needed.
  • The patch-training setup (64^3 patches, batch 8) makes the method run on a 12 GB GPU, so the uncertainty recipe is available outside large compute centres.
  • The BraTS filtered-ratios being low implies that removing voxels the model marks uncertain removes few correct predictions, which is the property the challenge rewards.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the paper does not take is to aggregate the per-subregion variance maps into one overall doubt volume; the max or entropy across regions would give a single clinical display.
  • The choice of $B=50$ is untested; plotting variance versus number of passes on a few subjects would show whether the epistemic map has converged or is dominated by sampling noise.
  • The same two-channel uncertainty recipe transfers to any patch-trained volumetric segmentation task, but the paper only demonstrates it on BraTS gliomas.
  • Uncertainty maps could drive active learning: send the highest-variance patches to an expert for relabeling, since the paper shows uncertainty concentrates near boundaries and errors.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript describes a 3D V-Net-based encoder-decoder for brain tumor segmentation in multimodal MRI, trained on BraTS 2020 with patch sampling, instance normalization, and data augmentation. The authors estimate voxel-wise epistemic uncertainty via Monte Carlo dropout (B=50) and aleatoric uncertainty via test-time data augmentation, then evaluate segmentation with Dice and Hausdorff distance on the BraTS training and validation sets. They report low FTP and FTN values as evidence that the model is confident in its correct predictions, and they present qualitative uncertainty maps. The paper concludes that uncertainty maps could help clinicians identify unreliable segmentations, while acknowledging difficulties in whole-tumor and especially enhancing-tumor segmentation.

Significance. If the uncertainty maps were properly validated, the practical value for clinicians reviewing automated segmentations would be real. The paper has some strengths: it uses a public benchmark (BraTS 2020), reports external validation results, and describes the architecture and training choices in sufficient detail for reproduction, although no code is provided. However, the uncertainty component, which is the main novelty, is evaluated only through two ratio metrics and a few visual examples, so the significance of the uncertainty contribution is currently unestablished.

major comments (3)
  1. [Section 4.2, Table 3] The claim that 'our model has low FTP and FTN, meaning that it is certain of those predictions' is not supported by the reported quantities. FTP and FTN measure the fraction of correct true positives and true negatives that are removed when voxels above an uncertainty threshold are filtered; low values only show that the filtering removes few correct voxels, and they carry no information about whether the removed voxels are enriched in errors. To establish the abstract's utility claim, the paper must report the threshold T, the uncertainty distribution, and a direct measure of how well uncertainty discriminates correct from incorrect voxels (for example, an ROC curve or AUC for uncertainty as an error detector, or error rates by uncertainty quantile).
  2. [Section 4.3, Figures 2-3] The visual assessment is anecdotal. The statement that 'the uncertainty values show that the model is more uncertain in the tumor surroundings and areas where the prediction has been miss-classified' is not quantified; a few selected slices cannot substantiate a general property. The authors should add a quantitative analysis over the full validation set, such as the mean uncertainty on false-positive, false-negative, true-positive, and true-negative voxels, or a reliability/calibration plot.
  3. [Section 3.4, Tables 1-2] The post-processing connected-component threshold is chosen by analyzing the training set, and the same threshold is then applied to the validation set. While this is not circular because the validation set is external, the paper neither reports the threshold value nor a sensitivity analysis, so the contribution of post-processing to the reported Dice/Hausdorff improvements cannot be assessed. Please report the chosen threshold and show how validation results vary over a range of threshold values.
minor comments (7)
  1. [Figure 3 caption] The caption lists image order as '(5) WT uncertainty map (5) ET uncertainty map'; the last panel should be labeled (6).
  2. [Section 3.5] Please clarify whether y_i^b in the variance formula are hard labels or softmax probabilities; the variance formula is defined for scalar values, and the paper says 'for each evaluation region' without specifying how the scalar per-region variance is obtained from the four output channels.
  3. [Section 4.2] The symbols TP100 and TP_T in the definition of FTP are not defined; define them and the threshold T explicitly.
  4. [Discussion] There are typographical errors: 'enchancing tumor' should be 'enhancing tumor' and 'Haurdorff' should be 'Hausdorff'.
  5. [Abstract] The phrase 'Automation of brain tumors' should be 'Automation of brain tumor segmentation'.
  6. [Section 3.3] 'in the passed 10 epochs' should be 'in the past 10 epochs'.
  7. [Tables 1-2] The reported differences between variants are small and no confidence intervals or significance tests are given; at least per-patient standard deviations should be reported.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: uncertainty maps are computed from the model's own stochastic outputs and evaluated on an external benchmark.

full rationale

No significant circularity found. The paper's derivation chain is empirical: it trains a V-Net on BraTS 2020 training data, applies standard post-processing whose threshold is chosen on the training set, and evaluates Dice/Hausdorff and FTP/FTN on the BraTS validation set. Uncertainty maps are computed from B=50 Monte-Carlo dropout samples and test-time augmentations (Section 3.5) directly from predictive variance and entropy; these quantities do not incorporate the target labels, FTP, or any fitted parameter, and the evaluation against the external BraTS benchmark is independent of the method's inputs. The two self-citations ([11], [13]) are contextual related-work comparisons and are not load-bearing for the architecture or uncertainty claim. The statement 'our model has low FTP and FTN, meaning that it is certain of those predictions' is an over-interpretation: FTP/FTN measure only how many true positives/negatives are removed at a threshold, not whether removed voxels are errors. This is an evidentiary weakness, not circularity, because low FTP/FTN are not used to define or derive uncertainty. Similarly, the clinical utility claim in the abstract is untested, but lack of validation is not circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. The main assumptions are the validity of dropout-based and augmentation-based uncertainty estimates, both carried over from cited work. Several hyperparameters are chosen by hand or tuned on the training set, and none are given uncertainty intervals.

free parameters (7)
  • post-processing connected component threshold
    Section 3.4: threshold on the proportion of the two largest components is 'obtained by analysing the training set', i.e., tuned to the training data; value not reported.
  • patch size = 64x64x64
    Section 3.3: chosen by experimentation as the best trade-off between memory and overfitting.
  • batch size = 8
    Section 3.3: chosen alongside patch size based on GPU memory and validation performance.
  • test-time dropout probability = 0.5
    Section 3.5: set to 50% to zero out a channel; not justified beyond citing [23].
  • number of MC dropout iterations B = 50
    Section 3.5: [23] suggests a few hundred iterations, but the paper uses 50 without justification.
  • data augmentation ranges = flip 50%, intensity shift in (-0.1, 0.1) std, scale (0.9, 1.1)
    Section 3.1: hand-picked ranges to 'disrupt minimally the data', no sensitivity analysis.
  • learning rate and scheduler settings = lr=1e-2, momentum=0.99, patience=10, factor=0.1
    Section 3.3: standard SGD settings used without ablation.
assumptions (4)
  • domain assumption BraTS 2020 training ground truth annotations are correct and representative.
    The entire training and evaluation relies on these labels as ground truth, Section 1.
  • domain assumption Monte Carlo dropout is a valid Bayesian approximation for epistemic uncertainty.
    Section 3.5 invokes Gal et al. [23] without re-deriving or validating the approximation in this setting.
  • domain assumption Test-time data augmentation captures aleatoric uncertainty.
    Section 3.5 follows Wang et al. [19]; this is a heuristic mapping from input sensitivity to aleatoric uncertainty, not a proven equivalence.
  • domain assumption The V-Net architecture is a suitable backbone for this segmentation task.
    The paper adopts V-Net [6] as a proven baseline, Section 3.2, without comparative justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation." pith.science (2026). https://pith.science/paper/CEBE2TOM

@misc{pith2026200912188,
  author       = {Pith},
  title        = {Pith review of: Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CEBE2TOM}},
  note         = {Machine review of arXiv:2009.12188}
}
read the original abstract

Automation of brain tumors in 3D magnetic resonance images (MRIs) is key to assess the diagnostic and treatment of the disease. In recent years, convolutional neural networks (CNNs) have shown improved results in the task. However, high memory consumption is still a problem in 3D-CNNs. Moreover, most methods do not include uncertainty information, which is specially critical in medical diagnosis. This work proposes a 3D encoder-decoder architecture, based on V-Net \cite{vnet} which is trained with patching techniques to reduce memory consumption and decrease the effect of unbalanced data. We also introduce voxel-wise uncertainty, both epistemic and aleatoric using test-time dropout and data-augmentation respectively. Uncertainty maps can provide extra information to expert neurologists, useful for detecting when the model is not confident on the provided segmentation.

Figures

Figures reproduced from arXiv: 2009.12188 by the authors.

Figure 1
Figure 1. We use the V-Net [6] architecture with instance normalization, ELU non [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Training results on patients: 223 and 325 (top-bottom). Image order: (1) [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Validation results on patients: 007 and 035 (top-bottom). Image order: (1) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [19]

    Morel, M

    Wang G., Li W., Ourselin S. and Vercauteren T. Automatic Brain Tumor Segmentation Based on Cascaded Convolutional Neural Networks With Un- certainty Estimation. Frontiers in Computational Neuroscience vol.13 pages 56 https://doi.org/10.3389/fncom.2019.00056, 2019

  2. [1]

    B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al.: ”The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)”, IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) https://doi.org/ 10.1109/TMI.2014.2377694

  3. [2]

    Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features

    S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J.S. Kirby, et al.: ”Ad- vancing The Cancer Genome Atlas glioma MRI collections with expert segmen- tation labels and radiomic features”, Nature Scientific Data, 4:170117 (2017) https://doi.org/10.1038/sdata.2017.117

  4. [3]

    Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge

    S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, et al.: ”Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progres- sion Assessment, and Overall Survival Prediction in the BRATS Challenge”, arXiv preprint arXiv:1811.02629 (2018)

  5. [4]

    The Cancer Imaging Archive (2017)

    S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., ”Segmentation Labels and Radiomic Features for the Pre-operative Scans of the TCGA-GBM collection”, The Cancer Imaging Archive, 2017. DOI: 10.7937/K9/TCIA.2017.KLXWJJ1Q

  6. [5]

    The Cancer Imaging Archive (2017)

    S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., ”Segmentation Labels and Radiomic Features for the Pre-operative Scans of the TCGA-LGG collection”, The Cancer Imaging Archive, 2017. DOI: 10.7937/K9/TCIA.2017.GJQ7R0EF

  7. [6]

    V-net: Fully convo- lu- tional neural networks for volumetric medical image segmentation

    Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convo- lu- tional neural networks for volumetric medical image segmentation. 3D Vision (3DV), 2016 Fourth International Conference on. IEEE, 2016

  8. [7]

    Neuro-oncology vol.17 01-2015 https://doi.org/10.1093/neuonc/nou358

    Morgan, L Lloyd: The epidemiology of glioma in adults: A ”state of the science” review. Neuro-oncology vol.17 01-2015 https://doi.org/10.1093/neuonc/nou358

Show all 26 references
  1. [8]

    Armen Der Kiureghian and Ove Ditlevsen: Aleatory or epistemic? does it matter? Structural safety, 31(2):105112, 2009

  2. [10]

    Newcombe, Joanna P

    Konstantinos Kamnitsas, Christian Ledig, Virginia F.J. Newcombe, Joanna P. Simpson, Andrew D. Kane, David K. Menon, Daniel Rueckert, Ben Glocker: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Medical Image Analysis, Volume 36, 2017...

  3. [11]

    In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries

    Casamitjana, A., Puch, S., Aduriz, A., Vilaplana, V., ”3D Convolutional Neural Networks for Brain Tumor Segmentation: a comparison of multi-resolution archi- tectures”. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2016. Lecture Not...

  4. [12]

    Kamnitsas, K., Bai, W., Ferrante, E., McDonagh, S., Sinclair, M., Pawlowski, N: Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmen- tation in International MICCAI Brainlesion Workshop (Quebec, QC), 450462 arXiv preprint arXiv:1711.01468, 2017

  5. [13]

    In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries

    Casamitjana, A., Cat, M., Snchez, I., Combalia, M., Vilaplana, V., ”Cascaded V- Net Using ROI Masks for Brain Tumor Segmentation”. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2017. Lecture Notes in Computer Science, vol 10670. Spr...

  6. [14]

    arXiv preprint arXiv:1810.11654, 2016

    Andriy Myronenko: 3D MRI brain tumor segmentation using autoencoder regu- larization. arXiv preprint arXiv:1810.11654, 2016

  7. [15]

    International MICCAI Brainlesion Workshop, pp

    Isensee, F., et al.: No new-net. International MICCAI Brainlesion Workshop, pp. 234244. Springer (2018)

  8. [16]

    (2020) Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task

    Jiang Z., Ding C., Liu M., Tao D. (2020) Two-Stage Cascaded U-Net: 1st Place Solution to BraTS Challenge 2019 Segmentation Task. In: Crimi A., Bakas S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Brain- Les 2019. Lecture Notes in Compute...

  9. [17]

    (2020) Bag of Tricks for 3D MRI Brain Tumor Segmentation

    Zhao YX., Zhang YM., Liu CL. (2020) Bag of Tricks for 3D MRI Brain Tumor Segmentation. In: Crimi A., Bakas S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2019. Lecture Notes in Computer Science, vol 11992. Springer, Cham

  10. [18]

    Frontiers in Com- putational Neuroscience vol.14 page 6 https://doi.org/10.3389/fncom.2020.00006, 2020

    Natekar Parth, Kori Avinash, Krishnamurthi Ganapathy AUTHOR=Natekar Parth, Kori Avinash, Krishnamurthi Ganapathy: Demystifying Brain Tumor Seg- mentation Networks: Interpretability and Uncertainty Analysis. Frontiers in Com- putational Neuroscience vol.14 page 6 https://doi.or...

  11. [20]

    (2019) Ensembles of Densely-Connected CNNs with Label-Uncertainty for Brain Tumor Segmentation

    McKinley R., Meier R., Wiest R. (2019) Ensembles of Densely-Connected CNNs with Label-Uncertainty for Brain Tumor Segmentation. In: Crimi A., Bakas S., Kuijf H., Keyvan F., Reyes M., van Walsum T. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuri...

  12. [21]

    Instance Normaliza- tion: The Missing Ingredient for Fast Stylization

    Dmitry Ulyanov and Andrea Vedaldi and Victor Lempitsky. Instance Normaliza- tion: The Missing Ingredient for Fast Stylization. arXiv preprint arXiv:1607.08022, 2016

  13. [22]

    Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36 (2017) 6178

  14. [23]

    arXiv preprint arXiv:1506.02142, 2015

    Yarin Gal and Zoubin Ghahraman.Dropout as a Bayesian Approximation: Rep- resenting Model Uncertainty in Deep Learning. arXiv preprint arXiv:1506.02142, 2015

  15. [24]

    Automatic differentiation in PyTorch, NIPS-W 2017 Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation 11

    Paszke, Adam and Gross, Sam and Chintala, Soumith and Chanan, Gregory and Yang, Edward and DeVito, Zachary and Lin, Zeming and Desmaison, Alban and Antiga, Luca and Lerer, Adam. Automatic differentiation in PyTorch, NIPS-W 2017 Brain Tumor Segmentation using 3D-CNNs with Uncert...

  16. [25]

    U-net: Convolu- tional net- works for biomedical image segmentation

    Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. U-net: Convolu- tional net- works for biomedical image segmentation. MICCAI. Springer, 2015. https://doi.org/10.1007/978-3-319-24574-4 28

  17. [26]

    Lecture Notes in Computer Science 240-248, Springer International Publishing

    Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmenta- tions. Lecture Notes in Computer Science 240-248, Springer International Publishing

  18. [2017]

    https://doi.org/10.1007/978-3-319-67558-9 28

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.