{"id":"fa4e224a-a02b-425f-9194-ca7b6b321a04","arxiv_id":"2502.04386","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A variational autoencoder with adversarial branches reduces sex and age signal in 3D CT foundation embeddings while preserving lung cancer risk prediction accuracy.","lead":"The authors present a VAE-based adversarial debiasing method that maps 3D CT foundation model embeddings into a new latent space with reduced sex and age information, tested on the NLST lung cancer dataset. The work could make clinical AI systems fairer by letting them use foundation model features without carrying demographic signals, while keeping predictive accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Demographic 'elimination' is established only through linear probes, so nonlinear leakage could invalidate the central fairness claim despite the reported linear reductions.","rationale":"The reader's weakest assumption correctly identifies the central vulnerability: demographic 'elimination' is asserted from linear probe results, while the underlying representation could retain nonlinear demographic information. This is load-bearing because the abstract and discussion use 'no longer encoded' and 'effectively eliminate' as the basis for both fairness and security claims. If nonlinear leakage exists, the debiased embeddings are not demographically neutral and the poisoning-robustness experiment becomes a demonstration that linear demographic signal is removed, not that demographic information is absent. The proposed nonlinear probe test directly settles this. Secondary issues noted by the reader and visible in the manuscript—missing numerical cancer AUC values in the main text, latent-dimension selection on an unspecified tune split in Appendix B, and absence of error bars—are real but addressable and do not change the verdict. The manuscript does acknowledge limitations around other attributes and domains, but that does not repair the linear-probe gap for the attributes actually claimed to be debiased. Overall, the work is plausible and the code is promised; a conditional acceptance with a request for nonlinear adversary evaluation is appropriate.","tokens_in":7568,"tokens_out":4190,"duration_ms":45603,"concrete_test":"Train a nonlinear probe on the same train/test split used in Section 3.1: a multi-layer perceptron with at least two hidden layers (e.g., 256-128, ReLU) plus a radial-basis-function SVM, predicting sex (AUC) and age (MAE) from the 500-dimensional debiased embeddings. Compare against the same probes trained on the original 1408-dimensional embeddings. If the nonlinear sex AUC on debiased embeddings exceeds roughly 0.85, or the nonlinear age MAE falls below roughly 3.0, the claim that demographic information is eliminated fails. Also report adversary training loss and accuracy curves to confirm the adversary was not underfit.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that demographic information is 'no longer encoded' rests entirely on linear classifier results in Section 2.2 and Section 3.1: sigmoid classification for sex and linear regression for age. The adversary is described only as 'multiple branches' with no hidden-layer specification, so both the defense and the evaluation may be restricted to linear families. A latent representation can be linearly unreadable while still containing demographic information in nonlinear subspaces, e.g., through high-order interactions or low-dimensional manifolds. The reported sex AUC drop from 0.999 to 0.669 and age MAE increase from 2.734 to 4.169 show attenuation, not elimination; a nonlinear probe could plausibly recover near-original predictability. Furthermore, downstream task performance is also evaluated with linear classifiers, so the full trade-off 'eliminate demographics without compromising accuracy' is demonstrated only for linear decoding. If a nonlinear adversary recovers demographic attributes, the fairness and security arguments in Section 4 lose their foundation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes a variational autoencoder (VAE) with adversarial branches that maps 1408-dimensional 3D CT foundation embeddings into a lower-dimensional latent space intended to remove sex and age information while preserving lung cancer risk prediction. Using the NLST dataset, the authors report that after debiasing, linear sex-prediction AUC drops from 0.999 to 0.669, age-prediction MAE increases from 2.734 to 4.169, 1-year and 2-year cancer-prediction accuracy remain around 0.986 and 0.977, equal opportunity difference (EOD) decreases, and EOD stays low under demographic label-poisoning attacks. The paper claims that demographic information is 'eliminated' while clinical predictive accuracy is maintained.","tokens_in":7726,"tokens_out":5794,"duration_ms":51800,"significance":"If substantiated, the approach would be practically valuable because it provides post-hoc debiasing of closed, black-box foundation-model embeddings, handles multiple sensitive attributes simultaneously, and is downstream-task independent. The manuscript has clear strengths: it uses a public dataset with patient-wise splits, releases code, evaluates on a clinically relevant task, and includes a poisoning-robustness analysis. However, the central claims are currently supported only by linear-probe evaluations, the reported numbers do not match the 'elimination' language, and the downstream evaluation omits key metrics such as AUC and uncertainty estimates. The work is promising but needs substantial additional evidence before the main claims can be accepted.","major_comments":[{"comment":"The abstract's claim that the debiased embeddings 'effectively eliminate multiple encoded demographic information' is not supported by the results in Section 3.1: after debiasing, sex-prediction AUC is 0.669, which remains far above chance (0.5), and age-prediction MAE is 4.169, which is close to the trivial baseline of predicting the mean age given the NLST age standard deviation of 5.0 years. The paper should either weaken the claim to 'substantially reduce linearly decodable demographic information' or provide evidence of elimination, such as AUC at chance level with confidence intervals and comparison to a no-information baseline.","section":"Abstract; Section 3.1"},{"comment":"Both the debiasing adversary and the evaluation probes appear to be restricted to linear families, so the central fairness result may not generalize to nonlinear adversaries. The adversary is described only as 'multiple branches' with no hidden-layer specification, and the evaluation uses a sigmoid classifier for sex and linear regression for age. A latent representation can be linearly unreadable while still encoding demographics nonlinearly. The authors should specify the adversary architecture, test with nonlinear probes such as an MLP or kernel SVM, and report how demographic predictability varies with adversary capacity.","section":"Section 2.3; Section 3.1"},{"comment":"The claim that debiasing 'does not compromise predictive accuracy' rests on accuracy values of 0.986 and 0.977 for an imbalanced cancer-prediction task, where always predicting the majority class can yield high accuracy. No AUC values from Figure 3 are reported in the main text, and Appendix B reports post-debiasing AUCs of 0.897 and 0.856 for latent dimension 500 but does not give the corresponding original-embedding AUCs or any variability estimate. The paper should report AUC with confidence intervals for both original and debiased embeddings, along with class prevalence.","section":"Section 3.1; Figure 3"},{"comment":"The latent dimension is selected using a 'tune split' that is not described in Section 2.1, and no repeated runs or random seeds are reported anywhere in the paper. This makes it impossible to assess whether the choice of 500 and the reported improvements are stable or overfit to the tuning set. The authors should define the tune split, explain how it was derived from the training data, and provide means and standard deviations over multiple runs.","section":"Appendix B"},{"comment":"The poisoning-robustness result is presented as an independent finding, but it follows directly from reduced demographic predictability: if the model cannot distinguish demographic groups, targeted label flipping for one group cannot produce large EOD. This experiment is useful as an application of the debiased embeddings, but it should be framed as a consequence rather than a separate validation of the method.","section":"Section 3.2"}],"minor_comments":[{"comment":"The term 'Variation Autoencoder' should be 'Variational Autoencoder'.","section":"Abstract"},{"comment":"The percentages appear off by a factor of 100; for example, White patients are listed as '0.93%' of the training set, but they actually constitute approximately 93%.","section":"Table A.1"},{"comment":"The phrase 'Similar to experiment 2' should refer to Experiment 1.","section":"Section 3.2"},{"comment":"The keywords include 'Progressive Streaming' and 'Data-Efficiency', which appear unrelated to the manuscript's content and may be a copy-paste artifact.","section":"Keywords"},{"comment":"The caption 'The difference for sex is the larger, the better' should be reworded for clarity.","section":"Figure B.1 caption"},{"comment":"The manuscript states that statistical significance was set at p < 0.05, but no significance tests or p-values are reported anywhere in the results.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of MIDL, and the code release plus use of a public dataset are commendable. My main concern is the mismatch between the 'elimination' language in the title and abstract and the evidence, which shows only partial reduction of linearly decodable demographic information. The requested revisions—nonlinear probes, full AUC reporting with uncertainty, and a defined tuning protocol—are within reach and would materially strengthen the paper. I would not recommend acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something useful and slightly understated: it shows a VAE-based adversarial debiasing layer can be trained on top of black-box 3D CT foundation embeddings, stripping linear demographic predictability for sex and age while keeping lung cancer risk accuracy roughly intact. That is a genuinely convenient property—no need to retrain the foundation model, and no need to know its internals. The NLST experiments are real, the code is up, and the EOD reductions plus the poisoning robustness plots are consistent with the method working as advertised. It is an incremental combination of known components (VAE + adversarial branches), not a new theory, and the paper is honest enough to list race and other tasks as missing.\n\nThe soft spots are mostly about how far the claims outrun the evidence. \"Demographic information is no longer encoded\" is too strong. The evaluation only uses linear probes; sex AUC goes from 0.999 to 0.669 and age MAE from 2.73 to 4.17. That is attenuation, not elimination. A nonlinear probe could plausibly recover more, and the paper gives no adversary-strength analysis or latent-visualization sanity check. The same linear-only evaluation applies to the downstream task, so the \"without compromising accuracy\" claim is really \"without compromising linear classifier accuracy.\" Also, the main text reports only accuracy for cancer prediction (0.986, 0.977) and hides the AUCs; the appendix has AUCs for debiased embeddings only, with no original-embedding AUC for comparison, which makes the trade-off hard to assess. No error bars, and the latent dimension of 500 is selected on a tune split with no discussion of variance across seeds.\n\nNone of this sinks the paper, but it does mean the headline should be \"reduces linear demographic leakage\" rather than \"eliminates encoded demographics.\" The poisoning-robustness result is a downstream consequence of removing group signal, not an independent discovery, but it is still nice to see.\n\nFor a reader working on fairness in medical imaging or on embedding-space interventions, this is a solid, citable data point. It deserves a proper peer review: the method is reproducible, the evaluation is substantial, and the missing analyses (nonlinear probes, error bars, full AUC tables) are exactly what reviewers should ask for. I would send it to review, with the expectation of a major-revision request.","headline":"Useful embedding-level debiasing with clear limits: linear probes show attenuation, not elimination, and the paper overstates the claim.","tokens_in":8223,"tokens_out":2280,"would_cite":true,"duration_ms":21083,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a VAE trained against a demographic adversary can strip sex and age information out of 3D CT foundation embeddings while preserving lung-cancer risk prediction and reducing fairness gaps.","keywords":["fairness in medical AI","adversarial debiasing","variational autoencoder","3D CT foundation embeddings","lung cancer risk prediction","equal opportunity difference","data poisoning defense"],"falsifier":"Train a multilayer perceptron or kernel SVM on the debiased embeddings to predict sex and age on the held-out test split; if sex AUC returns toward the original 0.999 or age MAE falls back toward 2.7, demographic information was not actually eliminated and the fairness claim is false.","tokens_in":7376,"feed_emoji":"🩻","tokens_out":9075,"duration_ms":75477,"temperature":0.7,"pith_summary":"Self-supervised embeddings of 3D CT scans carry demographic signals such as age and sex, and those signals can leak into downstream clinical predictions. This paper proposes a variational autoencoder trained in opposition to a demographic predictor: the predictor tries to recover age and sex from the latent code, and the encoder is pushed to make that difficult. On the NLST lung-cancer screening cohort, the transformed embeddings reduce sex-prediction AUC from 0.999 to 0.669, raise age-prediction error, and cut equal-opportunity differences for 1-year and 2-year cancer risk, while cancer-prediction accuracy stays at 0.986 and 0.977. The paper's central point is that fairness for black-box foundation embeddings can be achieved as a post-processing step, without retraining the foundation model.","feed_headline":"Debiased CT embeddings hide sex and age, keep cancer risk","feed_subtitle":"Post-hoc VAE fairness layer cuts demographic bias in 3D CT features without hurting lung-cancer prediction.","key_machinery":"The central object is a variational autoencoder with a multi-branch demographic adversary attached to its latent code. The encoder maps the original 1408-dimensional CT embedding to mean and log-variance parameters, a reparameterization trick allows gradient flow, and a linear decoder reconstructs the input. The loss sums reconstruction mean squared error, KL divergence toward a unit Gaussian, and the adversary's prediction losses for sex (binary cross-entropy) and age (mean squared error). The encoder is trained to minimize the adversarial loss, pushing the latent space to be demographically neutral, while the reconstruction term keeps clinically useful information available. Because the transformation operates only on existing embeddings, it is downstream-task independent and compatible with black-box foundation models.","core_discovery":"The central discovery, on the paper's own terms, is that demographic encoding in a pretrained 3D CT embedding is separable from the clinical signal it carries. By routing 1408-dimension embeddings through a VAE with a 500-dimensional bottleneck and simultaneously training an adversary with separate branches for sex and age, the authors obtain transformed embeddings that preserve, and in AUC terms slightly improve, 1-year and 2-year lung-cancer risk prediction while making demographic attributes much harder to decode linearly. Reported numbers include a drop in sex-prediction AUC from 0.999 to 0.669, an increase in age-prediction mean absolute error from 2.734 to 4.169, and reductions in equal-opportunity difference for both sensitive attributes on both prediction horizons. The same debiased embeddings keep EOD close to zero when up to 100% of one demographic group's cancer labels are flipped, whereas original embeddings become severely unfair. The authors take this as evidence that the method eliminates multiple encoded demographic attributes simultaneously and independently of the downstream task.","pith_inferences":["A direct extension the paper leaves implicit is adding race as a third adversary branch; the NLST demographic table shows race is available, so the framework could be tested on the attribute most commonly associated with imaging bias.","Because the transformation is downstream-agnostic, one trained VAE could act as a fairness layer in front of any consumer of the same foundation embeddings, making debiasing a deployment-time operation rather than a training-time one.","The linear-probe evaluation sets a low bar for 'elimination'; measuring how demographic AUC grows as adversary capacity increases would turn the central claim into a quantifiable leakage curve.","Since age is itself a risk factor for lung cancer, the reported accuracy preservation may partly reflect the model ignoring age; a clinically deployable version would need to separate demographic leakage from legitimate age-related clinical signal."],"forward_implications":["Debiasing can be applied after the fact to embeddings from a black-box 3D CT foundation model, so no retraining of the foundation model is required.","Equal-opportunity differences for sex and age drop for both 1-year and 2-year lung-cancer risk prediction when the debiased embeddings are used.","The debiased embeddings keep EOD near zero even when 100% of one demographic group's cancer labels are flipped, whereas the original embeddings show EOD approaching 1.","Multiple sensitive attributes are handled in a single transformation because the adversary has one branch per attribute.","A latent dimension of 500 marks a chosen balance: smaller bottlenecks erase more demography but also erase clinical signal, while larger ones preserve more of both."],"supporting_citations":[{"why":"Cited as the source of the 3D CT foundation model whose 1408-dimension embeddings are the inputs to the debiasing framework.","marker":"Yang et al., 2024"},{"why":"Provides the CT Foundation tool implementation and the patient-wise data splits used for training and evaluation.","marker":"Health, 2024"},{"why":"Establishes that these 3D CT embeddings encode demographic attributes, the problem the paper addresses.","marker":"Zheng et al., 2024"},{"why":"Documents AI recognition of patient race in medical imaging, motivating the fairness and security concern.","marker":"Gichoya et al., 2022"},{"why":"Supplies the adversarial-learning method for mitigating unwanted biases, which the VAE framework builds on.","marker":"Zhang et al., 2018"},{"why":"Source of the public lung-cancer screening dataset and demographic labels used for training and evaluation.","marker":"NLST, 2011"},{"why":"Defines the equal-opportunity difference used to quantify bias in downstream predictions.","marker":"Obermeyer et al., 2019"},{"why":"Defines the targeted label-poisoning attack that the debiased embeddings are tested against.","marker":"Kulkarni et al., 2024"}],"fun_headline_variants":["Adversarial VAE hides demographics in 3D CT features","Demographic-free CT embeddings still predict lung cancer","VAE wipes sex and age from CT embeddings, keeps cancer risk","Adversarial debiasing strips demographics from CT embeddings","Debiasing CT embeddings: sex and age erased, cancer risk intact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that demographic information is eliminated is tested only with linear classifiers; if sex or age remains recoverable by a nonlinear adversary, the central fairness guarantee collapses.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial VAE hides demographics in 3D CT features","Demographic-free CT embeddings still predict lung cancer","VAE wipes sex and age from CT embeddings, keeps cancer risk","Adversarial debiasing strips demographics from CT embeddings","Debiasing CT embeddings: sex and age erased, cancer risk intact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000666,"raw_usage":{"total_tokens":3064,"prompt_tokens":996,"completion_tokens":2068,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":1991}},"tokens_in":612,"tokens_out":2068,"duration_ms":14562,"temperature":1.0,"reasoning_tokens":1991,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T04:22:35.414078+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a multilayer perceptron or kernel SVM on the debiased embeddings to predict sex and age on the held-out test split; if sex AUC returns toward the original 0.999 or age MAE falls back toward 2.7, demographic information was not actually eliminated and the fairness claim is false.","supporting_citations":[{"cited_title":"Nlst embeddings in ct\\_foundation\\_demo.ipynb, Accessed: Oct 26 2024","cited_arxiv_id":null,"evidence_quote":"Provides the CT Foundation tool implementation and the patient-wise data splits used for training and evaluation."},{"cited_title":"Demographic Predictability in 3D CT Foundation Embeddings","cited_arxiv_id":"2412.00110","evidence_quote":"Establishes that these 3D CT embeddings encode demographic attributes, the problem the paper addresses."},{"cited_title":"Ai recognition of patient race in medical imaging: a modelling study","cited_arxiv_id":null,"evidence_quote":"Documents AI recognition of patient race in medical imaging, motivating the fairness and security concern."},{"cited_title":"Mitigating unwanted biases with adversarial learning","cited_arxiv_id":null,"evidence_quote":"Supplies the adversarial-learning method for mitigating unwanted biases, which the VAE framework builds on."},{"cited_title":"The national lung screening trial: overview and study design","cited_arxiv_id":null,"evidence_quote":"Source of the public lung-cancer screening dataset and demographic labels used for training and evaluation."},{"cited_title":"Hidden in Plain Sight: Undetectable Adversarial Bias Attacks on Vulnerable Patient Populations","cited_arxiv_id":"2402.05713","evidence_quote":"Defines the targeted label-poisoning attack that the debiased embeddings are tested against."}],"review_version":1}