REVIEW 4 major objections 4 minor 39 references
BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read BrainNext is a single self-supervised 3D encoder that transfers across brain-MRI classification, segmentation, and regression, ranking second overall and first on meningioma segmentation in the FOMO 2025 Method Track.
desk verdict A legitimate FOMO challenge entry with a new MAE plus 3D xLSTM-UNet recipe, but the general-purpose foundation-model claim outruns the evidence because the paper lacks within-paper baselines, ablations, and error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the native 3D Bi-Directional xLSTM-UNet, a five-stage encoder-decoder in which each encoder stage combines 3D convolutional layers with bidirectional xLSTM blocks that read the volume forward and backward, fuse their outputs through a learnable projection with a residual connection ($H_l = F_l + \phi([\vec H_l, \overleftarrow H_l])$), and feed the decoder through skip connections. During pretraining, the model is treated as an MAE: input volumes are split into $96 \times 96 \times 96$ voxel patches, a random subset $\mathcal{M}$ is masked, and the loss is the mean squared reconstruction error over masked patches only. This objective forces the encoder to represent global anatomical structure from visible context, while the xLSTM blocks supply long-range volumetric dependencies that plain convolutions would miss. After pretraining, the decoder is discarded and task-specific heads are attached to the encoder.
What would settle it
Run the same three downstream tasks with the identical fine-tuning pipeline but replace the pretrained encoder with a randomly initialized one of the same architecture. If the from-scratch model matches or beats BrainNext's AUROC of 0.684, Dice of 0.261, and brain-age MAE of 12.67 years, then the reported transfer is not caused by MAE pretraining. A second check is external validation: fine-tune the released weights on an independent brain MRI dataset and compare against a from-scratch baseline under matched label counts.
Extended reading notes
Core claim
The paper's central claim is that large-scale masked autoencoder pretraining in native 3D produces a single encoder whose lightweight fine-tuning is competitive across classification, segmentation, and regression. On the official FOMO 2025 Method Track test set, BrainNext reports an AUROC of 0.684 for infarct classification, a Dice of 0.261 and Normalized Surface Dice of 0.232 for meningioma segmentation, and a mean absolute error of 12.67 years with correlation 0.448 for brain-age estimation, ranking second overall and first on meningioma segmentation. The same pretrained encoder supports all three tasks via task-specific heads; the reconstruction decoder is discarded after pretraining. These numbers are the paper's evidence that transferable anatomical representations emerge from unlabeled volumetric data at this scale.
Load-bearing premise
The load-bearing premise is that the official FOMO 2025 leaderboard is a complete, unbiased measure of cross-task generalization for brain MRI foundation models; if that single benchmark is unrepresentative or unfair, the claim that BrainNext transfers across tasks weakens.
Editorial extensions
If this is right
- A single pretrained 3D encoder can be adapted to classification, segmentation, and regression through lightweight task-specific heads, so new brain MRI tasks may not require training a full model from scratch.
- MAE-style masked reconstruction at 60,551 scans is sufficient to make the encoder competitive on a common benchmark, supporting the use of unlabeled hospital-scale MRI collections for pretraining.
- Because BrainNext is natively 3D, the learned representations carry volumetric context that slice-based models discard, which is directly relevant to anatomy-dependent tasks such as meningioma segmentation.
- The planned public release of weights and inference code would make BrainNext a reusable starting point for downstream neuroimaging studies.
Reading between the lines
- Inference, not the paper's claim: the 12.67-year brain-age MAE is likely inflated by the out-of-domain, limited-label protocol; a matched external evaluation would reveal whether the same encoder approaches errors typical of dedicated brain-age models.
- Inference: because no ablation separates MAE pretraining from the xLSTM-UNet backbone, the relative contributions of the objective and the architecture remain entangled; a two-factor ablation would isolate them.
- Inference: a natural extension is label-efficiency testing—freeze the encoder, train heads on small labelled subsets, and measure how much annotation the pretrained representation actually saves.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes BrainNext, a volumetric brain MRI foundation model combining masked autoencoder (MAE) pretraining with a native 3D Bi-Directional xLSTM-UNet architecture. The model is pretrained on the FOMO60K dataset (reported as 60,551 or 60,529 examinations in different sections) and then fine-tuned for three downstream tasks in the FOMO 2025 Method Track: infarct classification, meningioma segmentation, and brain-age estimation. The authors report official leaderboard results of second place overall and first place in meningioma segmentation, with AUROC 0.684, DSC/NSD 0.261/0.232, and MAE/CORR 12.67/0.448. The paper claims that a single pretrained encoder can transfer across classification, segmentation, and regression, establishing BrainNext as a general-purpose foundation model.
Significance. If the claims are well supported, the paper makes a useful contribution by demonstrating a native 3D self-supervised foundation model for brain MRI and by providing results under the standardized FOMO 2025 evaluation protocol. The use of an official blind test set and a common pretraining dataset is a clear strength, and the plan to release model weights is commendable. However, the current evidence is insufficient to attribute the reported performance to MAE pretraining or to the proposed xLSTM-UNet architecture, because the paper presents no within-manuscript controls, no per-task comparison with other leaderboard entries, and no statistical uncertainty. The modest absolute metric values and the challenge's own finding that MAE favors segmentation further temper the 'general-purpose' claim. The significance is therefore conditional on an additional round of analysis.
major comments (4)
- [§5.1, Table 3] The central claim that BrainNext's self-supervised pretraining and architecture yield generalizable representations is not supported by any within-paper control. The paper reports only its own absolute test-set metrics (AUROC 0.684, DSC 0.261, NSD 0.232, MAE 12.67, CORR 0.448) and an aggregate rank. There is no randomly initialized encoder, no standard 3D U-Net or UNETR baseline, no MAE without the xLSTM component, and no per-task scores of other participants. Consequently, the observed leaderboard position cannot be attributed to MAE pretraining or to the Bi-Directional xLSTM-UNet design; the official rank alone is not evidence for the proposed method's components. Please add at least a from-scratch control and direct per-task leaderboard comparisons, including margins to the best and median methods.
- [§2.3 vs §3.2] The size of the pretraining dataset is inconsistent across the manuscript. Section 2.3 states that FOMO60K comprises 60,529 MRI scans, while the Abstract, Section 3.2, Section 4.1, and the description of Figure 1 state 60,551. If the discrepancy reflects the difference between scans and examinations, or between raw and preprocessed volumes, this should be stated explicitly. As written, the two numbers are presented as the same dataset without clarification, which is a reproducibility issue.
- [§2.4 vs §3.4] The handling of multiple MRI sequences is described contradictorily. Section 2.4 says 'each MRI sequence treated independently during pretraining,' while Section 3.4 says 'Multiple MRI sequences are treated as separate input channels, allowing the network to jointly learn complementary information across imaging modalities.' These two statements define different input tensors (single-channel volume processed separately per sequence versus multi-channel volume processed jointly). Please clarify the actual input representation and unify the two descriptions, as this affects the architecture and the pretraining objective.
- [§5, Table 3; §2.3] The paper reports no error bars or confidence intervals for any metric, despite the five-fold cross-validation and ensembling described in Section 3.6. Without uncertainty estimates, the reader cannot assess whether DSC 0.261 or MAE 12.67 is stable. In addition, Section 2.3 itself notes that no single SSL objective wins all FOMO tasks and that MAE-style objectives favor segmentation; this paper reports second overall with a first-place segmentation score but a modest AUROC (0.684) and MAE (12.67). To substantiate the 'general-purpose' claim, please report the official per-task rankings and per-task scores of competing methods, and discuss explicitly whether the overall rank is driven predominantly by the segmentation result.
minor comments (4)
- [§2.4] The reference to 'the authors’ previously published BrainFound [15]' is incorrect: BrainFound is introduced as [14] in Section 2.2, while [15] is cited for BrainIAC. Please fix the citation.
- [§3.6] Several implementation details are missing or imprecise: the masking ratio for MAE, the patch size used during fine-tuning, the fine-tuning learning rate schedule, and the per-task head architecture are not specified. The sentence 'He . initialization' contains a typo and should read 'He initialization.'
- [§5.1, Figure 2] If Figure 2 is a screenshot of the official leaderboard, the resolution or font size in the manuscript may make team names and scores unreadable. Please provide a table with the top entries' per-task and overall scores so that the comparison is legible and machine-readable.
- [References] Several references contain formatting errors, such as a double period at the end of entries [10] and [11], and reference [29] should include the date of access consistently with the URL. Please proofread the reference list.
Circularity Check
BrainNext's claims rest on an external blind-test leaderboard and standard SSL losses; no step reduces to its inputs.
full rationale
No significant circularity was found. BrainNext's pretraining uses a masked autoencoder reconstruction loss on unlabeled FOMO60K volumes, while the downstream evaluations use separate task-specific losses (BCE, Dice + CE, MSE) and metrics (AUROC, DSC/NSD, MAE/CORR). The downstream metrics are not used as pretraining objectives, and no parameter is fitted to the official test leaderboard within the paper; reported numbers are from the blind FOMO 2025 challenge evaluation. The paper's reliance on the FOMO leaderboard is an external benchmark result, not a self-derivation, and the MAE reconstruction objective (Equation for L_MAE) is independent of the downstream task formulations. Self-citations to prior BrainFound work appear only as positioning and related work, not as load-bearing justification for the reported results. The admitted limitation that evaluation was performed exclusively within FOMO concerns external validity and generalizability, not circular reasoning. Inconsistencies in dataset counts or channel descriptions are reproducibility concerns, not circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption Masked autoencoder reconstruction on 96x96x96 patches learns transferable anatomical representations.
- domain assumption The FOMO 2025 benchmark is a valid and representative measure of generalizability for brain MRI foundation models.
- domain assumption Bidirectional xLSTM modules capture long-range dependencies that improve volumetric representation learning.
- domain assumption Z-score normalization and patch-based sampling preserve the anatomical information needed for downstream tasks.
Cite this review
Pith. "Pith review of BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis." pith.science (2026). https://pith.science/paper/R5LPOOAK
@misc{pith2026260717782,
author = {Pith},
title = {Pith review of: BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/R5LPOOAK}},
note = {Machine review of arXiv:2607.17782}
}
read the original abstract
Foundation models pretrained using self-supervised learning have transformed computer vision by learning transferable representations from large-scale unlabeled data. However, existing foundation models for neuroimaging remain limited by task-specific training, slice-based learning strategies, or relatively small pretraining datasets, restricting their generalizability across diverse brain MRI applications. In this work, we present BrainNext, a general-purpose self-supervised foundation model for volumetric brain MRI analysis. BrainNext combines masked autoencoder (MAE) pretraining with a native three-dimensional Bi-Directional xLSTM-UNet architecture to learn rich anatomical representations from 60,551 unlabeled brain MRI examinations spanning multiple MRI modalities. The pretrained model is subsequently adapted to downstream tasks through lightweight task-specific fine-tuning. We evaluate BrainNext on the Foundation Models for Medical Imaging (FOMO) 2025 Method Track, encompassing classification, segmentation, and brain-age estimation, where it achieved second place overall and ranked first in the meningioma segmentation task on the official FOMO 2025 challenge leaderboard, demonstrating strong transferability across heterogeneous neuroimaging tasks. These results highlight the potential of large-scale self-supervised pretraining to learn robust and transferable volumetric representations, establishing BrainNext as a scalable foundation model for diverse brain MRI applications.
Figures
Reference graph
Works this paper leans on
-
[1]
Introduction Magnetic resonance imaging (MRI) has become an indispensable tool [1,2] for the diagnosis, prognosis, and longitudinal monitoring of neurological disorders, including stroke, brain tumours , Alzheimer's disease, multiple sclerosis, and other neurodegenerative conditions. The widespread adoption of MRI in both clinical practice and large -scal...
work page 2025
-
[2]
Related Work 2.1 Self-supervised learning for medical imaging Deep learning methods for brain MRI have traditionally been developed for individual tasks such as classification, segmentation, or regression [3-5]. Although task -specific supervised models can achieve strong performance, their development depends on labelled datasets that are costly to curat...
-
[3]
BrainNext 3.1 Overview BrainNext is a general-purpose self-supervised foundation model designed for volumetric brain MRI analysis. The proposed framework follows a two -stage learning paradigm consisting of large-scale self-supervised pretraining followed by task-specific fine-tuning, as illustrated in Figure 1. During the first stage, the model learns tr...
work page 2025
-
[4]
Experimental Setup 4.1 FOMO 2025 Benchmark BrainNext was evaluated on the Foundation Models for Medical Imaging (FOMO) 2025
work page 2025
-
[5]
Experimental Results 5.1 Overall Performance BrainNext was evaluated on the official Foundation Models for Medical Imaging (FOMO) 2025 Method Track, a standardized benchmark designed to assess the transferability of self -supervised foundation models across multiple neuroimaging applications under limited -label and out -of-domain conditions. The proposed...
work page 2025
-
[6]
Discussion This study presented BrainNext, a general-purpose self-supervised foundation model [14, 15, 2 7] for volumetric brain MRI analysis based on masked autoencoder (MAE) pretraining and a native three -dimensional Bi -Directional xLSTM-UNet architecture. Unlike conventional supervised approaches that require independent models for individual applica...
work page 2025
-
[7]
Conclusion We presented BrainNext , a general -purpose self -supervised foundation model for volumetric MRI analysis. BrainNext combines masked autoencoder pretraining with a native three -dimensional Bi -Directional xLSTM -UNet architecture to learn transferable anatomical representations from 60,551 unlabeled brain MRI examinations without requiring man...
work page 2025
-
[8]
Emerging properties in self -supervised vision transformers
Caron, Mathilde, et al. "Emerging properties in self -supervised vision transformers." Proceedings of the IEEE/CVF international conference on computer vision. 2021
work page 2021
Show all 39 references
-
[9]
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, Alexey, et al. "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020)
2020 arXiv
-
[10]
Dinov2: Learning robust visual features without supervision
Oquab, Maxime, et al. "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023)
2023 arXiv
-
[11]
Looking into the functional architecture of the brain with diffusion MRI
Le Bihan, Denis. "Looking into the functional architecture of the brain with diffusion MRI." Nature reviews neuroscience 4.6 (2003): 469-480
2003
-
[12]
The clinical use of structural MRI in Alzheimer disease
Frisoni, Giovanni B., et al. "The clinical use of structural MRI in Alzheimer disease." Nature reviews neurology 6.2 (2010): 67-77
2010
-
[13]
A survey on deep learning in medical image analysis
Litjens, Geert, et al. "A survey on deep learning in medical image analysis." Medical image analysis 42 (2017): 60-88
2017
-
[14]
Robust and data -efficient generalization of self-supervised machine learning for diagnostic imaging
Azizi, Shekoofeh, et al. "Robust and data -efficient generalization of self-supervised machine learning for diagnostic imaging." Nature Biomedical Engineerin g 7.6 (2023): 756-779
2023
-
[15]
Foundation model for whole-heart segmentation: leveraging student-teacher learning in multi -modal medical imaging
Qayyum, Abdul, et al. "Foundation model for whole-heart segmentation: leveraging student-teacher learning in multi -modal medical imaging." arXiv preprint arXiv:2503.19005 (2025)
2025 arXiv
-
[16]
AMAES [17] and related MAE-based approaches focused more specifically on three -dimensional brain MRI segmentation, while BrainAgeNeXt
pretrained an autoencoder-based model on 131,170 three -dimensional MRI volumes from multiple anatomical regions and evaluated transfer across segmentation, classification, and registration. AMAES [17] and related MAE-based approaches focused more specifically on three -dimens...
-
[17]
Simclr: A simple framework for contrastive learning of visual representations [C]
Chen, T., et al. "Simclr: A simple framework for contrastive learning of visual representations [C]." International Con -ference on Learning Representation s 2 (2020)
2020
-
[18]
Momentum contrast for unsupervised visual representation learning
He, Kaiming, et al. "Momentum contrast for unsupervised visual representation learning." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020
2020
-
[19]
investigated pretrained representations for brain -age modelling. Together, these studies demonstrate increasing interest in volumetric MRI pretraining, but they also show that models are often optimized or evaluated primarily for a restricted family of downstream tasks. 2.3 B...
-
[20]
U -net: Convolutional networks for biomedical image segmentation
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U -net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computer -assisted interventio n. Cham: Springer international publishing, 2015
2015
-
[21]
Masked autoencoders are scalable vision learners
He, Kaiming, et al. "Masked autoencoders are scalable vision learners." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022
2022
-
[22]
Models genesis: Generic autodidactic models for 3d medical image analysis
Zhou, Zongwei, et al. "Models genesis: Generic autodidactic models for 3d medical image analysis." International conference on medical image computing and computer-assisted intervention. Cham: Springer International Publishing, 2019
2019
-
[23]
Self -supervised pre -training of swin transformers for 3d medical image analysis
Tang, Yucheng, et al. "Self -supervised pre -training of swin transformers for 3d medical image analysis." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022
2022
-
[24]
Towards generalisable foundation models for brain MRI
Mazher, Moona, Geoff JM Parker, and Daniel C. Alexander. "Towards generalisable foundation models for brain MRI." npj Imaging (2026)
2026
-
[25]
A generalizable foundation model for analysis of human brain MRI
Tak, Divyanshu, et al. "A generalizable foundation model for analysis of human brain MRI." Nature Neuroscience (2026): 1-12
2026
-
[26]
Triad: Vision foundation model for 3d magnetic resonance imaging
Wang, Shansong, et al. "Triad: Vision foundation model for 3d magnetic resonance imaging." Research Square (2025): rs-3
2025
-
[27]
Amaes: Augmented masked autoencoder pretraining on public brain mri data for 3d -native segmentation
Munk, Asbjørn, et al. "Amaes: Augmented masked autoencoder pretraining on public brain mri data for 3d -native segmentation." arXiv preprint arXiv:2408.00640 (2024)
2024 arXiv
-
[28]
Method Track, a large -scale benchmark designed to assess the generalization capability of self -supervised foundation models under limited -label and out -of-domain conditions. The Method Track provides a standardized evaluation protocol in which all participating methods are...
2025
-
[29]
An OpenMind for 3D medical vision self -supervised learning
Wald, Tassilo, et al. "An OpenMind for 3D medical vision self -supervised learning." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025
2025
-
[30]
BrainAgeNeXt: advancing brain age modeling for individuals with multiple sclerosis
La Rosa, Francesco, et al. "BrainAgeNeXt: advancing brain age modeling for individuals with multiple sclerosis." Imaging Neuroscienc e 3 (2025): imag_a_00487
2025
-
[31]
nnU -Net: a self-configuring method for deep learning -based biomedical image segmentation
Isensee, Fabian, et al. "nnU -Net: a self-configuring method for deep learning -based biomedical image segmentation." Nature methods 18.2 (2021): 203-211
2021
-
[32]
Unetr: Transformers for 3d medical image segmentation
Hatamizadeh, Ali, et al. "Unetr: Transformers for 3d medical image segmentation." Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2022
2022
-
[33]
xlstm: Extended long short -term memory
Beck, Maximilian, et al. "xlstm: Extended long short -term memory." Advances in Neural Information Processing Systems 37 (2024): 107547-107603
2024
-
[34]
Decoupled weight decay regularization
Loshchilov, Ilya, and Frank Hutter. "Decoupled weight decay regularization." arXiv preprint arXiv:1711.05101 (2017)
2017 arXiv
-
[35]
Measures of the amount of ecologic association between species
Dice, Lee R. "Measures of the amount of ecologic association between species." Ecology 26.3 (1945): 297-302
1945
-
[36]
Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy
Nikolov, Stanislav, et al. "Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy." arXiv preprint arXiv:1809.04430 (2018)
2018 arXiv
-
[37]
Foundation Model for Medical Imaging: A Comprehensive Review
Jiao, Licheng, et al. "Foundation Model for Medical Imaging: A Comprehensive Review." IEEE Transactions on Artificial Intelligence (2025)
2025
-
[38]
Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challenge
Munk, Asbjørn, et al. "Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challenge." arXiv preprint arXiv:2604.11679 (2026)
2026 arXiv
-
[39]
Foundation Models for Medical Imaging (FOMO) 2025 Challenge: Official Results
FOMO 2025 Challenge Organizing Committee. Foundation Models for Medical Imaging (FOMO) 2025 Challenge: Official Results . Available: https://fomo25.github.io/results. Accessed: Jul. 17, 2026
2025
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.