REVIEW 4 major objections 5 minor 25 references
Understanding Sources of Demographic Predictability in Brain MRI via Disentangling Anatomy and Contrast
T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read Demographic predictability in brain MRI is primarily anatomical; acquisition contrast contributes only a weak, site-specific signal.
desk verdict A useful first decomposition of demographic signal in brain MRI, but the 'anatomy dominates' conclusion rests on an unvalidated self-cited disentangler and no error bars — worth reviewing, not yet trustworthy at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The decomposition x → (z_anat, z_contrast) produced by two pre-trained contrastive representation models: MR-CLIP, which aligns 3D MRI volumes with DICOM acquisition parameters to learn a 512-dimensional contrast embedding, and DIST-CLIP, which adds an anatomy mapper trained so that corresponding anatomical patches from different protocols remain similar, yielding an acquisition-invariant anatomical representation. The predictive models f_raw, f_anat, and f_contrast are trained under identical splits, learning rates, and regularisation; the performance gap between f_anat and f_contrast quantifies the relative contribution of structure vs. acquisition to demographic predictability.
What would settle it
Scan the same participants on two different scanners with different protocols, train a demographic predictor on z_contrast from the first scan, and test on the second scan. If it performs above chance (same anatomy, new contrast), the contrast channel carries cross-scan demographic signal and the attribution is wrong; if a z_anat predictor changes its output between the two scans, anatomy leakage is present.
Extended reading notes
Core claim
The central discovery is that demographic predictability in brain MRI is primarily anatomical. Models trained on anatomy-focused representations (which suppress intensity and contrast cues) achieve performance close to raw-image models across T1w, T2w, and FLAIR sequences, in within-dataset and joint training settings. Contrast-only embeddings (which minimize anatomical content) retain above-chance predictive power for sex, age, and sometimes race, but this signal is dataset-specific: in cross-dataset evaluations it collapses toward chance, while anatomy-based models transfer more reliably. The authors conclude that the demographic signal is structured biological variation in brain morpholog
Load-bearing premise
The entire attribution depends on the identity x → (z_anat, z_contrast) being a true separation, so z_anat contains no acquisition contrast and z_contrast contains no anatomy; the paper provides qualitative examples but no quantitative leakage tests.
Editorial extensions
If this is right
- Scan harmonization and intensity normalization will leave most of the demographic signal intact, because it is carried by anatomy.
- Contrast-based demographic shortcuts are inherently site-specific; models relying on them will not transfer across scanners or protocols.
- The same framework can audit debiasing methods: if a method removes demographic predictability, one can check whether it acted on the anatomical or contrast channel.
- Anatomical dominance holds across T1w, T2w, and FLAIR, so the result is not an artifact of a single sequence type.
- Effective mitigation must either target the anatomical component or accept that removing all demographic signal may remove clinically meaningful morphological information.
Reading between the lines
- A paired re-scan experiment (same subjects, different scanners) would test the decomposition: if z_contrast predicts demographics on the second scan above chance, the contrast channel carries generalizable signal or the decomposition leaks.
- The site-specific failure of contrast embeddings suggests that multi-site training may encode site as a proxy for demographics; the anatomy/contrast split could serve as a diagnostic for such shortcut learning.
- Since the decomposition comes from pre-trained models, the conclusions inherit any leakage in those models; a self-supervised decomposition within each dataset would be a stronger test of the attribution.
- If anatomical dominance is causal, image-warping debiasing approaches that erase group differences may be misdirected; preserving anatomical fidelity while removing group-correlated contrast may be the more tractable target.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper asks whether demographic predictability in brain MRI is carried primarily by anatomical structure or by acquisition-dependent contrast. Using the authors' MR-CLIP and DIST-CLIP models to decompose each image into an anatomy-focused representation z_anat and a contrast embedding z_contrast, the authors train predictive models for age, sex, and race on raw images, anatomy representations, and contrast embeddings. Across OASIS, ADNI, HCP, and joint training, they report that anatomy-focused representations largely preserve raw-image performance, while contrast embeddings retain weaker, dataset-specific signal that drops toward chance under cross-dataset transfer. They conclude that demographic signal in brain MRI is primarily anatomical, with a secondary acquisition-dependent component.
Significance. If the conclusion is correct, it has direct implications for bias mitigation in MRI-based models: harmonization or intensity normalization alone would be insufficient, and methods that suppress anatomical variation could remove clinically meaningful signal. The paper's strengths are its use of three public datasets, multiple MRI sequences, matched training protocols, balanced accuracy, and explicit cross-dataset evaluation. However, the central claim is conditional on an untested decomposition and on comparability between very different predictor architectures. Because the decomposition is supplied by the authors' own models without independent validation or released code/weights, and because no uncertainty quantification is provided, the evidence as presented is not yet sufficient to establish the claimed attribution. The framework is promising and the empirical pattern is internally consistent, but the load-bearing assumptions require additional validation.
major comments (4)
- [§2.1, Eq. (1)] The entire attribution in Tables 2–4 depends on the validity of the decomposition x → (z_anat, z_contrast). The manuscript offers only qualitative evidence: Fig. 1A shows example anatomy images and Fig. 1B shows t-SNE clustering of contrast embeddings by dataset/sequence. Clustering by dataset establishes that z_contrast carries acquisition-related information, but not that it excludes anatomy; similarly, visually plausible anatomy images do not establish that z_anat excludes acquisition contrast. No quantitative test is reported. I request: (i) train a classifier to predict scanner/site/sequence from z_anat and compare to chance; (ii) train a predictor of age or anatomical volume (e.g., total brain volume, cortical thickness) from z_contrast; (iii) report these alongside raw-image baselines. Since MR-CLIP and DIST-CLIP are self-cited and no code or weights are released, the decompositio
- [§2.2] The three predictors are not comparable. f_full and f_anat use a 3D ResNet-50 on 128³ volumes, while f_contrast is an MLP operating on a 512-dimensional embedding. The contrast pathway therefore has vastly lower capacity and different input statistics. The observed gap between Contrast and Anat/Raw could be due to architecture and input dimensionality rather than the intrinsic amount of demographic signal in z_contrast versus z_anat. The text says 'All models are trained under matched data splits, learning rates, and regularization schemes,' but this does not address architectural mismatch. I request a control where z_anat is passed through the same MLP head used for z_contrast (or a linear probe is used on both embeddings), and ideally where the raw image is also compressed to a 512-d embedding and evaluated with the same MLP. Without this, the quantitative claim that 'contrast embeddin
- [Tables 2–4 and §3] No uncertainty quantification is reported. All performance numbers are single point estimates; there are no repeated runs, cross-validation folds, confidence intervals, or significance tests. This is especially problematic for ADNI race predictions, where Table 1 shows only 84 Black subjects (5.2%). A balanced accuracy of 0.50 for ADNI Race Contrast could be either chance or slightly above chance given the small minority class; the paper does not allow the reader to distinguish. Similarly, differences such as OASIS Sex Raw 0.92 vs Anat 0.93, or ADNI Race Raw 0.97 vs Anat 0.82, are reported without any measure of variance. To support the quantitative claims, the authors should report mean±std over multiple seeds or bootstrap confidence intervals.
- [§3, Table 3] The text states that 'anatomy-focused representations (z_anat) tend to generalize more robustly than raw images.' This is supported for sex and some race rows, but for age the opposite is true: Anat produces larger MAE than Raw in five of the six transfer pairs (e.g., OASIS→HCP: 28.03 vs 26.37; HCP→ADNI: 44.24 vs 42.39). The conclusion as written overgeneralizes. The authors should either restrict the robustness claim to classification tasks or analyze age separately. This does not overturn the within-dataset primary result, but it is a claim in the experimental section that needs correction.
minor comments (5)
- [Table 1 (caption)] Typo: 'T able 1' should be 'Table 1.'
- [Table 4] For T2w and FLAIR, Race Anat substantially exceeds Race Raw (0.88 vs 0.74 and 0.85 vs 0.65). This is surprising and unexplained; it may indicate that the anatomy representation removes nuisance variation that hurts raw-image race prediction, or it may be a chance fluctuation given the lack of uncertainty intervals. Please discuss.
- [Abstract and §3] The abstract claims contrast embeddings retain 'a weaker but systematic signal,' but ADNI Race Contrast is exactly 0.50 (Table 2), which is chance. The 'systematic' claim should be qualified to the datasets/tasks where above-chance performance is actually shown.
- [§2.2] The phrase 'HCP age uses range midpoint' is undefined. Please specify how age was assigned for HCP subjects (e.g., exact ages vs. reported ranges) and how this affects the age regression task.
- [§2.2] Using ImageNet-pretrained weights inflated to 3D is unusual for volumetric MRI. Please clarify the inflation procedure and cite the specific reference; also discuss whether this pretraining is appropriate for MRI intensities.
Circularity Check
The anatomy/contrast labels driving the central attribution are imported from the authors' own MR-CLIP/DIST-CLIP papers without independent validation; the predictor comparisons themselves are not tautological, so circularity is partial.
-
self citation load bearing
[Section 2.1, Eq. (1) and the paragraph beginning 'To operationalize this separation...']
"To operationalize this separation, we leverage pre-trained disentangled representation learning frameworks, MR-CLIP [4] and DIST-CLIP [3]. ... As a result, the model achieves a disentangled decomposition in which z_anat captures acquisition-invariant information, while z_contrast encodes acquisition-dependent characteristics [3]."
The conclusion that demographic predictability is 'primarily rooted in anatomical variation' is obtained by labeling z_anat as anatomy and z_contrast as contrast. Those labels are not independently measured in this paper: there is no quantitative check that z_anat is free of acquisition influence or z_contrast free of anatomy. Instead, the labels rest on the authors' own earlier MR-CLIP/DIST-CLIP papers, cited as [3,4]. Thus the central attribution step reduces to a load-bearing self-citation: if the self-cited decomposition is impure, Tables 2-4 mislabel the source of demographic signal. The predictive comparisons are not forced by construction (an anatomy-only representation could have failed), so this is partial, not total, circularity.
full rationale
The paper's predictive experiments are genuine out-of-sample evaluations: f_anat and f_contrast are trained on fixed representations, and their performance could in principle have been at chance. That part is not circular. However, the scientific claim that these representations isolate 'anatomical variation' and 'acquisition-dependent contrast' is imported from the same group's MR-CLIP and DIST-CLIP models, with no quantitative validation of decomposition purity in this paper. The t-SNE clustering of z_contrast by dataset/sequence is offered as evidence, but MR-CLIP was trained to align embeddings with acquisition metadata, so this is partially expected from the training objective and does not establish the absence of anatomy. Likewise, the visual plausibility of z_anat images does not establish the absence of contrast leakage. Thus the abstract's central assertion is conditional on a load-bearing self-citation. Because the empirical comparisons have independent content and could have refuted the anatomy-dominance claim, the appropriate score is 4 rather than 6-10. The unvalidated assumption is a circularity-adjacent validity threat, and the paper would be strengthened by quantitative leakage checks (e.g., predicting site/scanner from z_anat, predicting anatomy from z_contrast, or releasing weights/code for independent audit).
Assumptions & free parameters
assumptions (3)
- domain assumption DIST-CLIP and MR-CLIP produce a valid disentanglement: z_anat is truly acquisition-invariant and z_contrast truly minimizes anatomical content (Eq. 1).
- ad hoc to paper f_full, f_anat, and f_contrast are comparable predictors; differences in capacity do not determine the observed performance gaps.
- domain assumption Demographic signal in these cohorts is measurable with MAE and balanced accuracy, and the race results are statistically reliable despite small minority subgroups.
Cite this review
Pith. "Pith review of Understanding Sources of Demographic Predictability in Brain MRI via Disentangling Anatomy and Contrast." pith.science (2026). https://pith.science/paper/ZNGPQEGJ
@misc{pith2026260304113,
author = {Pith},
title = {Pith review of: Understanding Sources of Demographic Predictability in Brain MRI via Disentangling Anatomy and Contrast},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZNGPQEGJ}},
note = {Machine review of arXiv:2603.04113}
}
read the original abstract
Demographic attributes can be predicted from medical images, raising concerns about bias in clinical AI systems. In X-ray imaging, acquisition characteristics have been shown to contribute substantially to this predictability. Whether the same holds in brain MRI remains unclear, as anatomical variation and acquisition-dependent contrast are deeply entangled in the image formation process, obscuring the origins of demographic signal. To address this, we propose a controlled framework based on disentangled representation learning, decomposing brain MRI into anatomy-focused representations that suppress acquisition influence and contrast embeddings that capture acquisition-dependent characteristics. Training predictive models for age, sex, and race on full images, anatomical representations, and contrast embeddings allows us to quantify the relative contributions of structure and acquisition to the demographic signal. Across three datasets and multiple MRI sequences, demographic predictability is found to be driven primarily by anatomical variation, with anatomy-focused representations largely preserving the performance of models trained on raw images. Contrast embeddings retain a weaker signal that is dataset-specific and does not generalise across sites. These findings suggest that effective mitigation must explicitly account for the primarily anatomical and secondarily acquisition-dependent origins of demographic signal, ensuring that any bias reduction generalizes robustly across domains.
Figures
Reference graph
Works this paper leans on
-
[1]
Achara, A., Anton, E.P., Hammers, A., King, A.P.: Invisible attributes, visible biases: Exploring demographic shortcuts in mri-based alzheimer’s disease classifi- cation (2025),https://arxiv.org/abs/2509.09558
arXiv 2025
-
[2]
In: Cui, Z., Rekik, A., Suk, H.I., Ouyang, X., Sun, K., Wang, S
Avci, M.Y., Borges, P., Wright, P., Yigitsoy, M., Ourselin, S., Cardoso, J.: MR- CLIP: Efficient metadata-guided learning of MRI contrast representations. In: Cui, Z., Rekik, A., Suk, H.I., Ouyang, X., Sun, K., Wang, S. (eds.) Machine Learning in Medical Imaging. MLMI 2025, Lecture Notes in Computer Science, vol. 16241. Springer, Cham (2026)
2025
-
[3]
Avci, M.Y., Borges, P., Fernandez, V., Wright, P., Yigitsoy, M., Ourselin, S., Car- doso, J.: DIST-CLIP: Arbitrary metadata and image guided MRI harmonization via disentangled anatomy-contrast representations (2025),https://arxiv.org/ abs/2512.07674
arXiv 2025
-
[4]
Avci, M.Y., Borges, P., Fernandez, V., Wright, P., Yigitsoy, M., Ourselin, S., Car- doso, J.: Metadata-aligned 3D MRI representations for contrast understanding and quality control. arXiv:2511.00681 (2025)
arXiv 2025
-
[5]
Archives of Neurology65(8), 1053–1061 (08 2008)
Brickman, A.M., Schupf, N., Manly, J.J., Luchsinger, J.A., Andrews, H., Tang, M.X., Reitz, C., Small, S.A., Mayeux, R., DeCarli, C., Brown, T.R.: Brain mor- phology in older african americans, caribbean hispanics, and whites from northern manhattan. Archives of Neurology65(8), 1053–1061 (08 2008)
2008
-
[6]
Machine Learning for Biomedical Imaging3, 792–808 (2025) 10 M.Y
Danaee, G., Niethammer, M., Rushmore, J., Bouix, S.: Investigating demographic bias in brain mri segmentation: A comparative study of deep-learning and non- deep-learning methods. Machine Learning for Biomedical Imaging3, 792–808 (2025) 10 M.Y. Avci et al
2025
-
[7]
The Lancet Digital Health4(6), e406–e414 (2022)
Gichoya, J.W., Banerjee, I., Bhimireddy, A.R., Burns, J.L., Celi, L.A., Chen, L.C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.C., Kuo, P.C., Lungren, M.P., Palmer, L.J., Price, B.J., Purkayastha, S., Pyrros, A.T., Oakden-Rayner, L., Okechukwu, C., Seyyed-Kalantari, L., Trivedi, H., Wang, R., Zaiman, Z., Zhang, H.: AI recognition of patient race in ...
2022
-
[8]
Neuroim- age14(3), 685–700 (2001)
Good, C.D., Johnsrude, I., Ashburner, J., Henson, R.N., Friston, K.J., Frackowiak, R.S.: Cerebral asymmetry and the effects of sex and handedness on brain structure: a voxel-based morphometric analysis of 465 normal adult human brains. Neuroim- age14(3), 685–700 (2001)
2001
Show all 25 references
-
[9]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015),https://arxiv.org/abs/1512.03385
2015 arXiv
-
[10]
NeuroImage260, 119474 (2022)
Hoopes, A., Mora, J.S., Dalca, A.V., Fischl, B., Hoffmann, M.: SynthStrip: skull- stripping for any brain image. NeuroImage260, 119474 (2022)
2022
-
[11]
Journal of Magnetic Resonance Imaging27(4), 685–691 (2008)
Jack, C.R.J., Bernstein, M.A., Fox, N.C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P.J., Whitwell, J.L., Ward, C., Dale, A.M., Felmlee, J.P., Gunter, J.L., Hill, D.L., Killiany, R., Schuff, N., Fox-Bosetti, S., Lin, C.P., Studholme, C., DeCarli, C.S., Kr...
2008
-
[12]
Scien- tific reports10(1), 20905 (2020)
Kang, D.W., Wang, S.M., Na, H.R., Park, S.Y., Kim, N.Y., Lee, C.U., Kim, D., Son, S.J., Lim, H.K.: Differences in cortical structure between cognitively normal east asian and caucasian older adults: a surface-based morphometry study. Scien- tific reports10(1), 20905 (2020)
2020
-
[13]
medRxiv (2019)
LaMontagne, P.J., Benzinger, T.L., Morris, J.C., Keefe, S., Hornbeck, R., Xiong, C., Grant, E., Hassenstab, J., Moulder, K., Vlassenko, A., Raichle, M.E., Cruchaga, C., Marcus, D.: Oasis-3: Longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheim...
2019
-
[14]
Proceedings of the National Academy of Sciences117(23), 12592–12594 (2020)
Larrazabal, A.J., Nieto, N., Peterson, V., Milone, D.H., Ferrante, E.: Gender im- balance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences117(23), 12592–12594 (2020)
2020
-
[15]
Machine Learning for Biomedical Imaging3(Special on FAIMI), 808–824 (2025)
Lee, T., Puyol-Anton, E., Ruijsink, B., Masci, P.G., Keehn, L., Chowienczyk, P., Haseler, E., Shi, M., King, A.: Understanding-informed bias mitigation for fair CMR segmentation. Machine Learning for Biomedical Imaging3(Special on FAIMI), 808–824 (2025)
2025
-
[16]
Nature Communications 15, 7465 (2024)
Lotter, W.: Acquisition parameters influence AI recognition of race in chest X-rays and mitigating these factors reduces underdiagnosis bias. Nature Communications 15, 7465 (2024)
2024
-
[17]
Progress in Neuro- Psychopharmacology and Biological Psychiatry21(8), 1231–1237 (1997)
Passe, T.J., Rajagopalan, P., Tupler, L.A., Byrum, C.E., Macfall, J.R., Kr- ishnan, K.R.: Age and sex effects on brain morphology. Progress in Neuro- Psychopharmacology and Biological Psychiatry21(8), 1231–1237 (1997)
1997
-
[18]
In: "Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S
Petersen, E., Feragen, A., da Costa Zemsch, M.L., Henriksen, A., Wiese Chris- tensen, O.E., Ganz, M.: Feature robustness and sex differences in medical imaging: A case study in MRI-based Alzheimer’s Disease detection. In: "Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S....
2022
-
[19]
In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C
Puyol-Antón, E., Ruijsink, B., Piechnik, S.K., Neubauer, S., Petersen, S.E., Razavi, R., King, A.P.: Fairness in cardiac mr image analysis: An investigation of bias due to data imbalance in deep learning based segmentation. In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N...
2021
-
[20]
Neuroscience & Biobehavioral Reviews39, 34–50 (2014)
Ruigrok, A.N., Salimi-Khorshidi, G., Lai, M.C., Baron-Cohen, S., Lombardo, M.V., Tait,R.J.,Suckling,J.:Ameta-analysisofsexdifferencesinhumanbrainstructure. Neuroscience & Biobehavioral Reviews39, 34–50 (2014)
2014
-
[21]
Schrouff, J., Harris, N., Koyejo, O., Alabdulmohsin, I., Schnider, E., Opsahl-Ong, K., Brown, A., Roy, S., Mincu, D., Chen, C., Dieng, A., Liu, Y., Natarajan, V., Karthikesalingam, A., Heller, K., Chiappa, S., D’Amour, A.: Diagnosing failures of fairness transfer across distri...
2023 arXiv
-
[22]
Nature Medicine27, 2176–2182 (2021)
Seyyed-Kalantari, L., Zhang, H., McDermott, M., Chen, I., Marzyeh, G.: Under- diagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine27, 2176–2182 (2021)
2021
-
[23]
Computers in Biology and Medicine141, 105089 (2022).https://doi.org/10.1016/j.compbiomed.2021.105089
Solovyev, R., Kalinin, A.A., Gabruseva, T.: 3d convolutional neural networks for stalled brain capillary detection. Computers in Biology and Medicine141, 105089 (2022).https://doi.org/10.1016/j.compbiomed.2021.105089
2022
-
[24]
NeuroImage80, 62–79 (2013)
Van Essen, D.C., Smith, S.M., Barch, D.M., Behrens, T.E., Yacoub, E., Ugurbil, K.: The WU-Minn Human Connectome Project: An overview. NeuroImage80, 62–79 (2013)
2013
-
[25]
Radiology299(1), 27–35 (2021), pMID: 33560191
Waite, S., Scott, J., Colombo, D.: Narrowing the gap: Imaging disparities in radi- ology. Radiology299(1), 27–35 (2021), pMID: 33560191
2021
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.