REVIEW 3 major objections 3 minor 1 cited by
Demographic Predictability in 3D CT Foundation Embeddings
T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read 3D CT foundation embeddings reveal a patient's age and sex
desk verdict A short, honest linear-probing study with strong headline numbers; the main open question is whether the CT Foundation embedding was trained on the same NLST data. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the 1408-dimensional embedding vector produced by the CT Foundation model for each 3D CT image; these embeddings are the fixed input features for a set of shallow predictors (linear regression, softmax regression, linear SVM, random forest, decision tree) trained to estimate age, sex, and race. The machinery works by showing that simple linear probes on the embeddings achieve near-saturating performance on age and sex, which demonstrates that the demographic signal is directly accessible in the representation rather than requiring a bespoke deep model.
What would settle it
Evaluate the same demographic probes on a CT dataset that the CT Foundation model is known to have been excluded from during pretraining (for example, an independent lung-cancer screening cohort with similar acquisition parameters). If age RMSE and sex AUC drop substantially on that external set, the paper's conclusion that demographic information is broadly encoded in the embeddings would need to be weakened to a statement about in-distribution memorization.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that self-supervised 3D CT foundation embeddings encode demographic information with high fidelity: a linear regression model predicts patient age within 3.8 years RMSE, and a softmax regression model distinguishes sex with an AUC of 0.998 and accuracy of 0.993. Race prediction is weaker, with AUC 0.878 and accuracy 0.471, a gap the authors attribute to race's limited direct anatomical representation combined with a heavily imbalanced dataset. The authors conclude that these embeddings contain demographic characteristics, which carries both potential benefits for personalized decision support and risks for fairness and privacy in clinical AI.
Load-bearing premise
The key assumption is that the CT Foundation model's embeddings are not boosted by the foundation model having been pretrained on the very same NLST scans used for evaluation, so the predictive accuracy reflects general anatomical encoding rather than memorization of specific patients.
Editorial extensions
If this is right
- Any downstream model trained on CT Foundation embeddings inherits age and sex information, so a clinical algorithm that means to be blind to demographics is not actually blind.
- The near-perfect sex classification (AUC 0.998) means that sex can be recovered from a chest CT even when it is not explicitly labeled, which has direct privacy implications for medical data sharing.
- Age prediction to within 3.8 years suggests the embedding carries fine-grained chronological information, not just broad age-group structure.
- The weak race result may be an artifact of the 94%-White NLST cohort; the question of whether race is encoded remains unresolved with this dataset.
Reading between the lines
- If the embedding encodes age and sex this strongly, then any performance gain reported for downstream tasks such as lung-cancer risk forecasting could be partly confounded by demographic stratification, which a careful bias audit should measure.
- The same linear-probe methodology could be turned into a standard auditing tool for any medical foundation model: report age RMSE and sex AUC as part of model cards.
- A testable prediction follows: applying this probe to foundation models trained with explicit demographic-debiasing objectives should lower age RMSE and sex AUC; if it does not, those debiasing methods are not altering the embedding geometry.
- Because accuracy for race is only 0.471 despite AUC 0.878, the result hints that race information is present but only in a small subset of patients, so future work with balanced multi-site data is needed before concluding that CT embeddings are race-neutral.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This short letter evaluates whether embeddings from Google's CT Foundation model encode demographic attributes. Using the NLST public dataset with patient-wise training (10,299 patients) and test (2,199 patients) splits, the authors train linear regression, softmax regression, linear SVM, random forest, and decision tree classifiers to predict age, sex, and race from 1408-dimensional CT embeddings. They report RMSE 3.8 years for age with linear regression, AUC 0.998 for sex with softmax regression, and AUC 0.878 for race, and conclude that the embeddings effectively encode age and sex but less so race, partly due to the 94% White class imbalance. The manuscript frames this as a preliminary investigation with implications for fairness and privacy in medical AI.
Significance. If the central claim holds, the result is significant: it would indicate that widely used self-supervised 3D CT foundation embeddings contain easily extractable demographic signals, with direct implications for privacy, fairness, and downstream clinical decision support. The paper has several strengths: it uses a large public dataset with held-out patients, evaluates multiple classifier families, releases code, and explicitly acknowledges its preliminary scope and the race imbalance. However, the absence of trivial baselines, confidence intervals, and external validation means the magnitude and generality of the effect are not yet established; the undisclosed pretraining data of the CT Foundation model is the most consequential uncertainty.
major comments (3)
- [Materials and Methods / Table 1(b)] The reported predictive performances are not compared with any trivial baseline. Reporting RMSE=3.8 years for age without comparing to predicting the mean age in the training set, and AUC=0.998 for sex without a majority-class baseline, makes the claim that the embeddings 'effectively encoded' demographics difficult to interpret. Please add these baselines, preferably along with simple image-derived proxies (e.g., lung volume or body-size measures), so that the effect attributable to the embeddings is established.
- [Materials and Methods] The manuscript states 'Statistical significance was set at p < 0.05', but no p-values, confidence intervals, or standard errors are reported for any RMSE or AUC value. Given the near-perfect sex AUC and the heavily imbalanced race data, the reader cannot assess the stability of the metrics or whether the sex/age results are significantly better than chance. Please report bootstrap confidence intervals or repeated cross-validation intervals for the main metrics.
- [Materials and Methods (CT Foundation embeddings)] The manuscript does not disclose whether the CT Foundation model was pretrained on NLST. If the NLST scans were part of the foundation model's pretraining data, the high demographic predictability may partly reflect patient-specific memorization rather than a general property of 3D CT embeddings. This is load-bearing for the generalizable conclusion in the Discussion. Please either provide evidence of no training/evaluation overlap or validate the findings on an external CT dataset that was not used in the foundation model's pretraining.
minor comments (3)
- [Figure 1 caption] The caption for panel (a) refers to 'Logistic regression models predicted age vs. actual age', while the text and Table 1(b) report linear regression as the age model; please correct the caption for consistency.
- [Materials and Methods] The data splitting procedure ('patient-wise training and test data splits') should specify whether the split is random, stratified by demographic attributes, and whether it is taken from the CT Foundation demo notebook or generated by the authors; this detail is needed for reproducibility.
- [Discussion] The conclusion that race is 'encoded less effectively' is not fully supported given the 94% White class imbalance; the text acknowledges this, but the abstract still presents race AUC 0.878 as a substantive finding, so the abstract should more clearly label the race result as inconclusive.
Circularity Check
No circularity: the demographic predictions are held-out classifier evaluations on externally provided embeddings; possible pretraining overlap is a leakage risk, not a circular derivation.
full rationale
The paper performs an empirical evaluation rather than a derivation: pretrained CT Foundation embeddings are taken as fixed inputs, classifiers are trained on a patient-wise training split, and performance is reported on a patient-wise test split ("patient-wise training (N=10299 patients, 52696 images) and test (N=2199 patients, 11421 images) data splits"). The age and sex results are therefore genuine held-out predictions from the fitted classifiers, not quantities that were fitted and then renamed as predictions. No equation in the paper defines the embedding in terms of the demographic labels, and no fitted parameter is claimed as a first-principles result. The cited works [1-3] describe the CT Foundation embedding tool, but the paper does not derive its central claim from those citations; they are used only to identify the source of the embeddings. Reference [5] includes a current author, but it is cited only as background on adversarial bias and is not load-bearing for the demographic predictability result. The main validity concern is that the CT Foundation model's pretraining data are not disclosed, so it is possible that the embeddings were influenced by NLST images; however, that is a data-leakage and generalizability concern, not a circularity of the paper's own derivation. The paper itself acknowledges its preliminary scope and the race-class imbalance, neither of which constitutes circular reasoning. Thus no circular step can be exhibited from the text, and the appropriate score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption CT Foundation embeddings faithfully represent the 3D CT images for probing demographic information.
- domain assumption The patient-wise data split prevents patient-level leakage between classifier training and test sets.
- domain assumption The NLST demographic labels (age, sex, race) are accurate and representative.
- ad hoc to paper The CT Foundation model was not pretrained on the same NLST scans used in this evaluation, or any overlap is negligible.
Cite this review
Pith. "Pith review of Demographic Predictability in 3D CT Foundation Embeddings." pith.science (2026). https://pith.science/paper/XLS4FZXE
@misc{pith2026241200110,
author = {Pith},
title = {Pith review of: Demographic Predictability in 3D CT Foundation Embeddings},
year = {2026},
howpublished = {\url{https://pith.science/paper/XLS4FZXE}},
note = {Machine review of arXiv:2412.00110}
}
read the original abstract
Self-supervised foundation models have recently been successfully extended to encode three-dimensional (3D) computed tomography (CT) images, with excellent performance across several downstream tasks, such as intracranial hemorrhage detection and lung cancer risk forecasting. However, as self-supervised models learn from complex data distributions, questions arise concerning whether these embeddings capture demographic information, such as age, sex, or race. Using the National Lung Screening Trial (NLST) dataset, which contains 3D CT images and demographic data, we evaluated a range of classifiers: softmax regression, linear regression, linear support vector machine, random forest, and decision tree, to predict sex, race, and age of the patients in the images. Our results indicate that the embeddings effectively encoded age and sex information, with a linear regression model achieving a root mean square error (RMSE) of 3.8 years for age prediction and a softmax regression model attaining an AUC of 0.998 for sex classification. Race prediction was less effective, with an AUC of 0.878. These findings suggest a detailed exploration into the information encoded in self-supervised learning frameworks is needed to help ensure fair, responsible, and patient privacy-protected healthcare AI.
Figures
Forward citations
Cited by 1 Pith paper
-
Towards Fair Medical AI: Adversarial Debiasing of 3D CT Foundation Embeddings
A variational autoencoder with adversarial branches reduces sex and age signal in 3D CT foundation embeddings while preserving lung cancer risk prediction accuracy.
Reference graph
Works this paper leans on
-
[1]
Advancing multimodal medical capabilities of gemini
Lin Yang, Shawn Xu, Andrew Sellergren, Timo Kohlberger, Yuchen Zhou, Ira Ktena, Atilla Kiraly, Faruk Ahmed, Farhad Hormozdiari, Tiam Jaroensri, et al. Advancing multimodal medical capabilities of gemini. arXiv preprint arXiv:2405.03162, 2024
arXiv 2024
-
[2]
Taking medical imaging embeddings 3d, Accessed: Oct 26 2024
Atilla Kiraly and Madeleine Traverse. Taking medical imaging embeddings 3d, Accessed: Oct 26 2024. URL https://research.google/blog/taking-medical-imaging-embeddings-3d/
2024
-
[3]
Nlst embeddings in ct foundation demo.ipynb, Accessed: Oct 26 2024
Google Health. Nlst embeddings in ct foundation demo.ipynb, Accessed: Oct 26 2024. URL https://github.com/Google-Health/imaging-research/tree/master/ct-foundation
work page 2024
-
[4]
Ai recognition of patient race in medical imaging: a modelling study
Judy Wawira Gichoya, Imon Banerjee, Ananth Reddy Bhimireddy, John L Burns, Leo Anthony Celi, Li-Ching Chen, Ramon Correa, Natalie Dullerud, Marzyeh Ghassemi, Shih-Cheng Huang, et al. Ai recognition of patient race in medical imaging: a modelling study. The Lancet Digital Health, 4(6):e406–e414, 2022
2022
-
[5]
Hidden in plain sight: Undetectable adversarial bias attacks on vulnerable patient populations
Pranav Kulkarni, Andrew Chan, Nithya Navarathna, Skylar Chan, Paul H Yi, and Vishwa S Parekh. Hidden in plain sight: Undetectable adversarial bias attacks on vulnerable patient populations. arXiv preprint arXiv:2402.05713, 2024
arXiv 2024
-
[6]
The national lung screening trial: overview and study design
National Lung Screening Trial Research Team. The national lung screening trial: overview and study design. Radiology, 258(1):243–253, 2011. 4
work page 2011
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.