REVIEW 4 major objections 7 minor 1 cited by
Optimizing Brain Tumor Segmentation with MedNeXt: BraTS 2024 SSA and Pediatrics
T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Using the MedNeXt architecture with ensembling, finetuning, and size-threshold postprocessing, this paper reports average Dice scores of 0.896 on the BraTS 2024 sub-Saharan Africa validation set and 0.830 on the pediatric validation set.
desk verdict Solid BraTS 2024 challenge report, but the headline Dice numbers are inflated by validation-set threshold tuning and should not be read as unbiased generalization. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is MedNeXt itself: a convolutional U-Net whose blocks replace standard convolutions with depthwise 3D convolutions that mimic the large attention windows of transformers, followed by channel expansion and compression, with deep supervision at each decoder stage. The paper uses the Base and Medium sizes and ensembles the five models from 5-fold cross-validation by averaging their output probability maps. A schedule-free AdamW optimizer with learning rates near 0.0027 replaces the usual cosine-annealed schedule. Postprocessing removes predicted tumor components smaller than experimentally chosen voxel thresholds, and for Africa the last ten decoder layers are finetuned on the target population after training on combined adult and African data. These components together carry the performance; the paper's ablations show each addition raises Dice or reduces boundary error.
What would settle it
Fix every training choice and postprocessing threshold using only the five cross-validation folds, then freeze the pipeline and recompute Dice and HD95 on the same 35 African and pediatric validation cases; if the scores fall clearly below 0.896 and 0.830, the gap is the validation-tuning contribution. Alternatively, run the released pipeline unchanged on a newly collected, never-seen cohort of African and pediatric MRIs and compare.
Extended reading notes
Core claim
The paper's central claim is that MedNeXt, a U-Net-like network built from inverted-bottleneck convolutional blocks, can be adapted to segment gliomas under distribution shift, and that the adaptation comes mostly from data and training choices rather than architectural novelty. For the sub-Saharan Africa task the authors train MedNeXt-Base and MedNeXt-Medium on the 60 African training scans together with 1,470 adult glioma scans, finetune the last decoder layers on the African data alone, and ensemble the five cross-validation models by averaging probability maps; the best configuration reports average Dice 0.896 and HD95 14.682 on the 35-case validation set. For the pediatric task they train MedNeXt-Base on 464 pediatric high-grade glioma cases, predict enhancing tumor, tumor core, and whole tumor, and use minimum-size thresholds to remove small false-positive components, reporting average Dice 0.830 and HD95 37.508. The paper attributes the remaining performance gap to data heterogeneity and to children's different anatomy, and it treats postprocessing thresholds as an important lever, noting that thresholds were tuned partly against the validation set.
Load-bearing premise
The validation set is treated as an unbiased estimate of generalization even though the postprocessing thresholds were manually adjusted to fit it; if that adjustment leaks information, the reported Dice and HD95 numbers overstate performance on genuinely new scans.
Editorial extensions
If this is right
- If the Africa ablations generalize, the recipe is transferable: pretrain on a large adult glioma set, finetune only the decoder on a small target-population cohort, and tune per-region binarization thresholds; each step moved average Dice upward, ending at 0.896.
- A schedule-free optimizer near learning rate 0.0027 with no weight decay matched or beat the cosine-annealed AdamW baseline in both tasks, so explicit learning-rate schedules are not required for strong segmentation.
- The best pediatric configuration used learning rate 0.0005 and voxel thresholds of 50, 75, and 250 for enhancing tumor, tumor core, and whole tumor, averaging Dice 0.830 and HD95 37.508; other thresholds degraded both metrics.
- Pediatric HD95 of 37.5 is much larger than Africa HD95 of 14.7, showing boundary accuracy lags overlap accuracy when anatomy and image quality differ, and overlap-focused metrics alone would mask that gap.
Reading between the lines
- A natural test the paper does not run is to freeze the full pipeline, including thresholds, after cross-validation and evaluate on the eventual test set; if validation-tuned thresholds were a major source of the reported scores, the test-set gap would show it.
- The same recipe could transfer to other underrepresented populations: combine a large adult benchmark, finetune decoder layers on a small local cohort, and tune thresholds per region; the paper's SSA results make this concrete enough to try.
- The Base-plus-Medium ensemble scored lower Dice than either model alone in the Africa task, suggesting simple equal-weight probability averaging is not always beneficial; selecting or weighting ensemble members by validation performance might recover the lost margin.
- For clinical deployment, the large pediatric HD95 implies some segmentations have gross boundary outliers; combining the size-threshold step with a shape prior or outlier-aware loss would be the next lever to test.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript describes a MedNeXt-based segmentation pipeline for the BraTS 2024 SSA and Pediatric tumor tasks. The authors train MedNeXt-B and MedNeXt-M models on combined adult glioma and SSA data, use the schedule-free AdamW optimizer, 5-fold cross-validation ensembles, and postprocessing with minimum-size and binarization thresholds. They report average validation Dice scores of 0.896 (SSA) and 0.830 (Pediatric) with HD95 of 14.682 and 37.508, and they ablate patch size, finetuning strategy, learning rate, and postprocessing thresholds.
Significance. The paper provides a practical, reproducible challenge pipeline with a public code repository and a recent optimizer, and its ensembles and finetuning are standard and internally consistent. If the reported validation numbers were unbiased estimates of generalization, the results would be competitive for the two BraTS 2024 tasks. However, the central evidence is weakened by explicit tuning of postprocessing hyperparameters on the validation set and by selecting the best of several validation-set runs, so the quantitative claims need to be reframed or supplemented with cross-validation or held-out test results before they can be taken at face value.
major comments (4)
- [Section 2.3; Tables 1 and 2] The sentence in Section 2.3, 'We tuned postprocessing hyperparameters using a Wandb sweep on the 5-fold CV and then manually adjusted them to fit the unseen validation set,' breaks the independence of the validation set as a generalization estimate. Because the binarization thresholds and minimum tumor sizes were adjusted using validation-set performance, the reported Dice of 0.896 and 0.830 are not unbiased estimates for truly unseen data. The effect is visible in Table 1: the average Dice improves from 0.8930 with CV-default thresholds (row 5) to 0.8963 after manual adjustment (row 6), and HD95 improves from 15.37 to 14.68. The authors should report the 5-fold cross-validation performance of the final configuration, or clearly label the validation numbers as leaderboard results obtained with validation-set-informed postprocessing and report an independent test-set result if available.
- [Section 3.2, Table 2] The abstract's Pediatric Dice of 0.830 is the best among the six learning-rate and minimum-size-threshold combinations shown in Table 2; selecting the best row after observing validation scores inflates the expected performance on future data through selection bias. For example, the 0.0027 learning-rate runs in Table 2 have average Dice below 0.80, while the 0.0005 runs reach 0.853 in some rows. The authors should report all configurations considered, provide the selection rule, and give per-case variability (e.g., standard deviation across validation subjects) or confidence intervals for the chosen configuration so that the reader can separate genuine improvement from selection noise.
- [Section 4] The conclusion calls the Africa results 'state-of-the-art average Dice scores above 0.895' without comparing to other BraTS 2024 submissions or published methods on the same tasks. Since 'state-of-the-art' is a comparative claim, the authors should provide a comparison table with leaderboard positions, other challenge entries, or prior published results on the BraTS SSA and Pediatrics validation sets, or temper the claim to 'competitive with' or 'among the top entries.'
- [Section 3.1, Table 1] The text names configurations such as 'MedNeXt Finetuned False 0.7' (Dice 0.895) and 'MedNeXt Finetuned True 0.7,0.5' (Dice 0.896), but Table 1 does not show finetuning initialization and threshold values as explicit columns, so the mapping between table rows and the described models is ambiguous. This makes it difficult to verify which configuration produced the headline number. Please add explicit columns for finetuning initialization (True/False), binarization thresholds, and minimum-size thresholds, and align the table rows with the bullet descriptions.
minor comments (7)
- [Section 2.3] The sentence 'For the objective function (loss), deep supervision, brain MRI preprocessing, and data augmentation, we followed the approach outlined in .' contains an empty citation; please fill in the intended reference.
- [Section 3.1] The bullet 'MedNeXt Base achieved an average Dice of 0.889, which was the highest among the non-finetuned models. It performed well, especially in segmenting WT, with a Dice score of 0.932.' is duplicated verbatim in the list of MedNeXt models; please remove the duplicate.
- [Section 3.2, Table 2] The column header for Table 2 is garbled ('Dice Scores HD 95ETTCWTETTCWTNETCCCED ET TC WT NETCCC ED'); please reformat the table so that the six tumor-region columns and their metric groupings are clearly labeled.
- [Section 2.4] The text says sliding window inference uses 'aX% (X ∈ {50, 70}) overlap' but does not state which overlap value was used for each task or for the final submission; please specify this in the experimental setup.
- [Section 2.4, Section 3] The final ensemble composition is not specified: the text mentions ensembling 'models derived from different training strategies' but does not state which models (Base, Medium, finetuned variants) were combined, in what number, and with what weighting for the reported validation numbers. Please describe the final ensemble explicitly.
- [Section 2.1] For the SSA dataset, the three sub-regions are introduced as ET, NETC, and SNFH, but the results in Table 1 report ET, TC, and WT. Please clarify the relationship between these label sets (e.g., TC = ET + NETC and WT includes surrounding FLAIR hyperintensity) to avoid confusion.
- [Throughout] There are minor typographical issues, including 'accross' in Section 3.1 and 'LesionWise' in the Table 2 caption; please proofread for spelling and capitalization.
Circularity Check
Validation-set fitting of postprocessing thresholds means the reported leaderboard scores are partly the result of fitting, not independent prediction; the core MedNeXt pipeline itself is otherwise self-contained.
-
fitted input called prediction
[Section 2.3, Model Development; Table 1, rows 5-6]
"We tuned postprocessing hyperparameters using a Wandb sweep on the 5-fold CV and then manually adjusted them to fit the unseen validation set."
The abstract's headline SSA numbers, Dice 0.896 and HD95 14.682, are obtained immediately after changing the binarization thresholds to 0.7 for ET and TC and 0.5 for WT (Table 1, row 6). Row 5, before that manual adjustment, gives Dice 0.8930 and HD95 15.37. The postprocessing thresholds were therefore fitted to the same validation set whose scores are then reported as the paper's main result, so the 'prediction' on the validation set is partly an optimization target rather than an independent evaluation.
-
fitted input called prediction
[Section 3.2, Table 2]
"The best-performing model was obtained using threshold values of 50, 75, and 250 for ET, TC, and WT, respectively. These results highlight the importance of the Data post-processing chosen parameters and the learning rate in our model performance."
The abstract reports Pediatric DSC 0.830 and HD95 37.508, which correspond to the best row selected after trying multiple learning rates and minimum-size thresholds on the validation leaderboard. Because the configuration was chosen after viewing validation Dice and HD95 results, the reported validation performance is the maximum of a small selection distribution over validation-fitted parameters, not an unbiased estimate of generalization on a truly unseen set. The paper explicitly calls these 'chosen parameters' and reports their validation outcome as the headline result.
full rationale
The circularity is real but localized. The paper explicitly admits that postprocessing thresholds were 'manually adjusted to fit the unseen validation set' (Section 2.3), and Table 1 quantifies the resulting gain: moving from 0.8930/15.37 to 0.8963/14.68 after threshold adjustment. Table 2 similarly reports the best of six validation-set configurations for Pediatrics. Hence the abstract's Dice and HD95 values are partly fitted to the same validation labels used to produce them, which is the 'fitted input called prediction' pattern. However, the underlying model pipeline is not circular in a definitional sense: MedNeXt is adopted from external work [18], the schedule-free optimizer is from an external source [6], the loss and augmentation follow nnU-Net [8], and the training/ensemble strategy is standard challenge practice. Prior MBZUAI papers [11,12] are cited as inspiration for adopting MedNeXt and ensemble design, but they are not used as a load-bearing uniqueness argument or as a substitute for the experiments reported here. The main threat is therefore statistical validity of the reported validation estimates, not self-citation or definitional equivalence. A score of 6 reflects that the central empirical claims are partially constructed by validation-set fitting while the method itself retains independent content; no score of 8 or 10 is warranted because the network architecture and training procedure are not derived from the target metrics.
Assumptions & free parameters
free parameters (5)
- Binarization thresholds (SSA) =
0.7 for ET/TC, 0.5 for WT
- Minimum size thresholds (pediatric) =
50 (ET), 75 (TC), 250 (WT) in the best run
- Learning rate =
0.0027 (SSA); 0.0027 and 0.0005 (pediatric)
- Patch size =
128 x 160 x 112
- Sliding window overlap =
50% or 70%
assumptions (4)
- domain assumption Manual tumor annotations by radiologists are ground truth
- domain assumption Combining BraTS Adult Glioma and SSA data improves generalization to SSA
- domain assumption Schedule-free AdamW trains without a scheduler as effectively as scheduled AdamW
- domain assumption Organizer-provided preprocessing (SR124 co-registration, skull-stripping for SSA) preserves segmentation-relevant information
Cite this review
Pith. "Pith review of Optimizing Brain Tumor Segmentation with MedNeXt: BraTS 2024 SSA and Pediatrics." pith.science (2026). https://pith.science/paper/WPOGOE4S
@misc{pith2026241115872,
author = {Pith},
title = {Pith review of: Optimizing Brain Tumor Segmentation with MedNeXt: BraTS 2024 SSA and Pediatrics},
year = {2026},
howpublished = {\url{https://pith.science/paper/WPOGOE4S}},
note = {Machine review of arXiv:2411.15872}
}
read the original abstract
Identifying key pathological features in brain MRIs is crucial for the long-term survival of glioma patients. However, manual segmentation is time-consuming, requiring expert intervention and is susceptible to human error. Therefore, significant research has been devoted to developing machine learning methods that can accurately segment tumors in 3D multimodal brain MRI scans. Despite their progress, state-of-the-art models are often limited by the data they are trained on, raising concerns about their reliability when applied to diverse populations that may introduce distribution shifts. Such shifts can stem from lower quality MRI technology (e.g., in sub-Saharan Africa) or variations in patient demographics (e.g., children). The BraTS-2024 challenge provides a platform to address these issues. This study presents our methodology for segmenting tumors in the BraTS-2024 SSA and Pediatric Tumors tasks using MedNeXt, comprehensive model ensembling, and thorough postprocessing. Our approach demonstrated strong performance on the unseen validation set, achieving an average Dice Similarity Coefficient (DSC) of 0.896 on the BraTS-2024 SSA dataset and an average DSC of 0.830 on the BraTS Pediatric Tumor dataset. Additionally, our method achieved an average Hausdorff Distance (HD95) of 14.682 on the BraTS-2024 SSA dataset and an average HD95 of 37.508 on the BraTS Pediatric dataset. Our GitHub repository can be accessed here: Project Repository : https://github.com/python-arch/BioMbz-Optimizing-Brain-Tumor-Segmentation-with-MedNeXt-BraTS-2024-SSA-and-Pediatrics
Figures
Forward citations
Cited by 1 Pith paper
-
BraTS orchestrator : Democratizing and Disseminating state-of-the-art brain tumor image analysis
BraTS orchestrator is a new open-source package that provides uniform, tutorial-based access to winning BraTS segmentation and synthesis algorithms for brain tumor MRI.
Reference graph
Works this paper leans on
-
[1]
Adewole, M., Rudie, J.D., Gbadamosi, A., Toyobo, O., Raymond, C., Zhang, D., Omidiji, O., Akinola, R., Suwaid, M.A., Emegoakor, A., Ojo, N., Aguh, K., Kalaiwo, C., Babatunde, G., Ogunleye, A., Gbadamosi, Y., Iorpagher, K., Cal- abrese, E., Aboian, M., Linguraru, M., Albrecht, J., Wiestler, B., Kofler, F., Janas, A., LaBella, D., Kzerooni, A.F., Li, H.B., ...
arXiv 2023
-
[2]
Adewole, M., Rudie, J.D., Gbdamosi, A., Toyobo, O., Raymond, C., Zhang, D., Omidiji, O., Akinola, R., Suwaid, M.A., Emegoakor, A., et al.: The brain tumor segmentation (brats) challenge 2023: glioma segmentation in sub-saharan africa patient population (brats-africa). ArXiv (2023)
work page 2023
-
[3]
arXiv preprint arXiv:2107.02314 (2021)
Baid, U., Ghodasara, S., Mohan, S., Bilello, M., Calabrese, E., Colak, E., Farahani, K., Kalpathy-Cramer, J., Kitamura, F.C., Pati, S., et al.: The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classifica- tion. arXiv preprint arXiv:2107.02314 (2021)
arXiv 2021
-
[4]
The cancer imaging archive286 (2017)
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Farahani, K., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the tcga-lgg collection. The cancer imaging archive286 (2017)
2017
-
[5]
Scientific data 4(1), 1–13 (2017)
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data 4(1), 1–13 (2017)
2017
-
[6]
arXiv preprint arXiv:2405.15682 (2024)
Defazio, A., Mehta, H., Mishchenko, K., Khaled, A., Cutkosky, A., et al.: The road less scheduled. arXiv preprint arXiv:2405.15682 (2024)
arXiv 2024
-
[7]
Ferreira, A., Solak, N., Li, J., Dammann, P., Kleesiek, J., Alves, V., Egger, J.: How we won brats 2023 adult glioma challenge? just faking it! enhanced synthetic data augmentation and model ensemble for brain tumour segmentation (2024), https://arxiv.org/abs/2402.17317
arXiv 2024
-
[8]
Isensee, F., Jäger, P.F., Full, P.M., Vollmuth, P., Maier-Hein, K.H.: nnu-net for brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. pp. 118–132. Springer International Publishing, Cham (2021)
work page 2021
Show all 20 references
-
[9]
Kazerooni, A.F., Khalili, N., Liu, X., Gandhi, D., Jiang, Z., Anwar, S.M., Albrecht, J., Adewole, M., Anazodo, U., Anderson, H., Baid, U., Bergquist, T., Borja, A.J., Calabrese, E., Chung, V., Conte, G.M., Dako, F., Eddy, J., Ezhov, I., Familiar, A., Farahani, K., Franson, A.,...
2024 arXiv
-
[10]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11976–11986 (2022) 12 S. Hashmi et al
2022
-
[11]
Maani, F., Hashmi, A.U.R., Aljuboory, M., Saeed, N., Sobirov, I., Yaqub, M.: Advanced tumor segmentation in medical imaging: An ensemble approach for brats 2023 adult glioma and pediatric tumor tasks (2024), https://arxiv.org/abs/ 2403.09262
2024 arXiv
-
[12]
In: 2024 IEEE International Symposium on Biomedical Imaging (ISBI)
Maani, F.A., Ur Rehman Hashmi, A., Saeed, N., Yaqub, M.: On enhancing brain tumor segmentation across diverse populations with convolutional neural networks. In: 2024 IEEE International Symposium on Biomedical Imaging (ISBI). pp. 1–4 (2024). https://doi.org/10.1109/ISBI56570.2...
2024
-
[13]
IEEE transactions on medical imaging 34(10), 1993–2024 (2014)
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34(10), 1993–2024 (2014)
2014
-
[14]
StatPearls [Internet] (2024),https://www.ncbi.nlm
Mesfin, F.B., et al.: Gliomas. StatPearls [Internet] (2024),https://www.ncbi.nlm. nih.gov/books/NBK441874/, updated 2023 May 20
2024
-
[15]
Physics in Medicine & Biology63(5), 05TR01 (2018)
Owrangi, A.M., Greer, P.B., Glide-Hurst, C.K.: Mri-only treatment planning: ben- efits and challenges. Physics in Medicine & Biology63(5), 05TR01 (2018)
2018
-
[16]
Human brain mapping31, 798–819 (05 2009)
Rohlfing, T., Zahr, N., Sullivan, E., Pfefferbaum, A.: The sri24 multichannel atlas of normal adult human brain structure. Human brain mapping31, 798–819 (05 2009). https://doi.org/10.1002/hbm.20906
2009 doi
-
[17]
In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI
-
[18]
arXiv preprint arXiv:2303.09975 (2023)
Roy, S., Koehler, G., Ulrich, C., Baumgartner, M., Petersen, J., Isensee, F., Jaeger, P.F., Maier-Hein, K.: Mednext: Transformer-driven scaling of convnets for medical image segmentation. arXiv preprint arXiv:2303.09975 (2023)
2023 arXiv
-
[19]
In: Bakas, S., Crimi, A., Baid, U., Malec, S., Pytlarz, M., Baheti, B., Zenk, M., Dorent, R
Zeineldin, R.A., Karar, M.E., Burgert, O., Mathis-Ullrich, F.: Multimodal cnn networks for brain tumor segmentation in mri: A brats 2022 challenge solution. In: Bakas, S., Crimi, A., Baid, U., Malec, S., Pytlarz, M., Baheti, B., Zenk, M., Dorent, R. (eds.) Brainlesion: Glioma,...
2023
-
[2015]
pp. 234–241. Springer International Publishing, Cham (2015)
2015
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.