{"id":"4cbbe206-8f79-4aee-9848-e311136a52e3","arxiv_id":"2501.05970","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A CNN ensemble predicts age from veteran MRIs with R2 0.816, but the evidence that residuals flag chronic disease is weakened by post hoc analysis and age confounding.","lead":"The paper trains four small neural networks on T2-weighted MRI slices from 1,220 U.S. veterans to predict age, then checks whether the prediction error (brain age residual) tracks five chronic conditions. The model predicts age reasonably well, but the claimed link between residuals and disease count rests on a post-hoc age split and likely confounded statistics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The p=0.002 residual-trend claim is confounded by age-composition differences across ICD-count groups; the direct ANOVA on residuals is null, so the biomarker claim lacks support.","rationale":"The paper's age-prediction model is plausibly sound: the ensemble achieves R2=0.816 on a held-out test set, with cross-validation metrics close to test metrics, and the authors are transparent about mild overfitting and data limitations. However, the central biomarker claim rests on the residual analysis in Section 3.2, and the key evidence for that claim does not survive scrutiny. The abstract's headline p=0.002 comes from comparing regression lines of predicted age against actual age across ICD-count groups. Because ICD count is strongly associated with chronological age, and because brain-age models are known to have age-dependent bias (predicted age regressing toward the mean), differences in the age distributions of the groups can produce significant F-tests even when residuals are entirely unrelated to ICD count. The paper itself provides the critical internal check: a direct one-way ANOVA of residuals grouped by ICD count is null (p=0.1880). This is exactly the test the biomarker hypothesis implies, and it fails. The subsequent restriction to subjects older than 49 produces a significant p=0.0069, but this threshold was not pre-specified; the authors describe using decision trees to identify an age split around 48-49 before testing it, so the p-value is inflated by selection and no multiple-testing correction is applied. The small samples in the 3- and 4-ICD groups further weaken this subgroup analysis. The paper also notes that attempts to predict ICD count from age and residuals were unsuccessful, which is consistent with a weak or absent signal. In sum, the age-prediction model is a reasonable contribution, but the manuscript's central claim that the residual is a biomarker for latent health conditions in veterans is not established. The reader's REJECT verdict is appropriate, and I would not change it.","tokens_in":17756,"tokens_out":4247,"duration_ms":40327,"concrete_test":"Run an ANCOVA on the training-set residuals with ICD-count group as the factor and actual age as a covariate (equivalently, test ICD-group differences in residuals after subtracting the global age-bias regression line). If the group effect becomes non-significant, as the reported direct ANOVA p=0.1880 already suggests, the Section 3.2 F-test result is driven by age-composition differences across groups rather than by residual-disease association. As a secondary check, recompute the over-49 p=0.0069 with a pre-specified threshold (e.g., 50) and apply multiple-testing correction across candidate cutoffs.","verdict_should_be":"REJECT","load_bearing_attack":"In Section 3.2 (Figure 7), the paper claims that residuals grouped by number of ICD codes demonstrate statistically significant different trends (F=4.265, p=0.002) by comparing predicted-versus-actual-age regression lines across ICD-count groups. This test does not directly test residual differences: if the brain-age model exhibits the well-known age-bias (predicted age regresses toward the mean, so residuals correlate with chronological age), then any two groups with different age distributions will have different regression lines even when residual distributions are identical. ICD count increases sharply with age; the paper itself notes the decision-tree split at ages 48-49 and the 'sharp decline in the proportion of subjects with zero ICD codes after this age threshold.' Because the direct one-way ANOVA of residuals grouped by ICD count is null (p=0.1880, reported in Section 3.2), the aggregate F-test result is most plausibly an artifact of age composition interacting with model age-bias. The over-49 subgroup result (p=0.0069) is post-hoc: the threshold was selected after exploratory decision trees on the same data, with no multiple-testing correction, and Table 6 shows that groups with 3-4 ICD codes have small samples. These issues undermine the central claim that the residual is a usable biomarker for latent health conditions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a brain-age prediction model for 1,220 U.S. veterans using four CNNs trained on 2D T2-weighted FSE and FLAIR MRI slices at two anatomical locations, combined through a degree-3 polynomial ensemble. The model achieves R²=0.816 on the test set with honest reporting of training-set overfitting (training R²=0.97). The central claim of the paper is that the model's residuals constitute a biomarker for latent health conditions, based on an analysis where residuals grouped by the number of ICD-coded conditions (HTN, DM, mTBI, SAD, AAD) reportedly show statistically significant different trends (p=0.002), particularly for subjects older than 49 years.","tokens_in":17988,"tokens_out":3969,"duration_ms":38872,"significance":"If the biomarker claim held, the paper would add a modest but useful data point to the brain-age residual literature, especially because it uses T2-weighted images rather than the dominant T1 modality and reports overfitting explicitly. The age-prediction model itself is competently built and its performance is reported with commendable transparency, including cross-validation metrics, test-set metrics, and a comparison of training and test R². However, the central biomarker claim is not supported by the evidence presented. The headline p=0.002 comes from a test that is confounded by age composition across ICD-count groups, and the paper's own direct one-way ANOVA on residuals grouped by ICD count is null (p=0.1880). The over-49 subgroup result is post-hoc and uncorrected. The manuscript therefore does not establish the load-bearing claim that the residual is a usable biomarker.","major_comments":[{"comment":"The headline claim that residuals grouped by the number of ICD codes show statistically significant different trends (p=0.002) is based on an F-test comparing predicted-age-versus-actual-age regression lines across ICD-count groups. This test does not directly compare residuals; it can reject the null when groups have different age distributions even if the residual distributions are identical. Because brain-age models typically exhibit regression to the mean, residuals are correlated with chronological age, so any two groups with different age compositions will show different regression lines. The paper itself notes in §3.2 that decision trees consistently split at actual age 48–49 and that there is a sharp decline in the proportion of subjects with zero ICD codes after this threshold, confirming that ICD-count groups differ in age composition. The direct one-way ANOVA on residuals grouped by ICD count, reported later in the same section, is null (p=0.1880). The p=0.002 result is therefore most plausibly an artifact of age confounding rather than evidence of an association between residuals and disease burden.","section":"Abstract; §3.2 (Figure 7)"},{"comment":"The significant result restricted to subjects older than 49 (p=0.0069) is post-hoc and cannot rescue the null aggregate ANOVA. The threshold was selected after exploratory decision-tree analysis on the same data, and no multiple-comparison correction is reported. Moreover, the groups with 3–4 ICD codes in this age range have small sample sizes, as reflected in Table 6 and Figure 8, making the subgroup test unreliable. A post-hoc subgroup analysis without correction does not provide support for the claim that negative residuals correlate with multiple ICD codes in older subjects.","section":"§3.2 (over-49 subgroup analysis)"},{"comment":"Even the descriptive statistics in Table 6 do not support the paper's narrative of a monotonic trend toward older predicted brain age with more ICD codes. The mean residuals for 0, 1, 2, 3, and 4 codes are +0.07, +0.21, +0.12, −0.82, and −0.72, respectively. The 1-code group has a more positive mean residual than the 0-code group, and the 2-code group is nearly identical to the 0-code group. Only the small 3- and 4-code groups show negative means. Combined with the null one-way ANOVA, the numeric pattern in Table 6 does not substantiate the abstract's assertion of a clear relationship between multiple ICD codes and advanced brain aging.","section":"§3.2 (Table 6)"}],"minor_comments":[{"comment":"The text says 'these conditions would benefit from a BADB'; this should read 'BARB'.","section":"§1.3"},{"comment":"The figure contains a typo: 'Batch Normzalizatoin' should be 'Batch Normalization'.","section":"Figure 4"},{"comment":"The phrase 'ICD codes can lack the subtly of distinguishing' should read 'subtlety'.","section":"§1.3"},{"comment":"The reported p-values of 'p = 0.000' for the HTN and SAD ANOVAs should be reported as 'p < 0.001' rather than an exact zero.","section":"§3.2"},{"comment":"The sentence 'The development for a clinically applicable BARB rests on on the need for...' contains a duplicated 'on'.","section":"§4"},{"comment":"The manuscript does not state whether institutional review board approval or patient consent was obtained for the MRI and ICD data; this should be clarified for a human-subjects study.","section":"§2.1"}],"recommendation":"reject","confidential_remarks":"The age-prediction model is a reasonable engineering contribution with transparent reporting, but the manuscript's central claim is statistically invalid. The p=0.002 result is confounded by age composition, the direct ANOVA is null, and the subgroup finding is post-hoc. This is not a case where a local fix would preserve the abstract's conclusion; the biomarker claim would need to be substantially re-analyzed and, on current evidence, likely retracted. I would welcome a future submission that presents a properly age-adjusted residual analysis, whether or not it supports the biomarker hypothesis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a look for the model, not for the biomarker claim. It builds four small CNNs on two T2 FSE/FLAIR slices from 1,220 veterans and ensembles them with a degree-3 polynomial, reaching R2=0.816 on a held-out test set. That is a legitimate result: the design is clean, the training/test split is honest, and they report training R2=0.97 as evidence of mild overfitting rather than hiding it.\n\nWhat's new: the specific combination of T2 FSE/FLAIR at two anatomical levels, the veteran cohort, and the ensemble. The method itself is established brain-age machinery, and the paper's own tables show that. So novelty is incremental.\n\nThe soft spot is the load-bearing claim in the abstract and Section 3.2: residuals grouped by number of ICD codes show statistically significant different trends (p=0.002). The paper compares regression lines of predicted vs actual age across ICD-count groups. Those groups differ sharply in age composition—the paper itself notes the decision-tree split at 48–49 and the \"sharp decline\" in zero-ICD subjects after that threshold. Since predicted age tracks chronological age, comparing regression lines can reject even when residuals are identical across groups. The direct one-way ANOVA on residuals grouped by ICD count is null (p=0.1880) and the paper reports it. The over-49 subgroup result is post-hoc, with the threshold selected after exploratory decision trees and no multiple-testing correction. So the claim that the residual is a usable biomarker for latent health conditions is not supported by the evidence presented.\n\nCredit where due: the paper is transparent. It reports the null ANOVA, discusses the binary-label limitations, acknowledges possible mislabeling and the lack of a healthy baseline, and flags its own overfitting. That is honest reporting. The theoretical framework section is mostly a restatement of bias-variance decomposition and doesn't add much.\n\nWho this is for: researchers working on brain-age residuals in clinical populations will find a useful negative result and a reasonable model on an unusual dataset. The biomarker claim needs an age-adjusted analysis, pre-registered subgroup thresholds, and external validation. As it stands, I'd treat the residual–disease association as unproven.\n\nRecommendation: send to peer review. The model is solid enough and the analysis flaws are fixable; a serious referee could push the authors to redo the residual analysis properly. I would not cite the biomarker claim until it survives that re-analysis.","headline":"Decent brain-age model on an unusual veteran MRI dataset, but the headline residual–disease claim is confounded by age composition; the paper reports its own null ANOVA, so the honest core is the model, not the biomarker.","tokens_in":18586,"tokens_out":1873,"would_cite":false,"duration_ms":16998,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that the gap between MRI-predicted and chronological brain age—the model residual—tracks how many chronic health conditions a veteran carries, making it a candidate biomarker for latent disease.","keywords":["brain age prediction","brain age residual biomarker","convolutional neural network","MRI","T2-weighted imaging","ICD codes","veteran health","latent disease detection"],"falsifier":"Re-run the residual comparison on age-matched subgroups: for each chronological age, compare residuals of veterans with zero, one, and multiple ICD codes. If the residual differences disappear within matched age bands—or if an age-adjusted residual ANOVA is null—the central claim is falsified.","tokens_in":116,"feed_emoji":"🧠","tokens_out":6779,"duration_ms":127994,"temperature":0.7,"pith_summary":"This paper tries to establish that the residual of a brain-age model—the gap between the age a neural network predicts from MRI scans and the patient's chronological age—can serve as a biomarker for latent health conditions. Using four convolutional neural networks trained on T2-weighted FSE and FLAIR slices of 1,220 U.S. veterans, combined by a degree-3 polynomial ensemble, the authors predict age with an $R^2$ of 0.816 and then test whether residuals track five ICD-coded conditions: hypertension, diabetes, mild traumatic brain injury, and substance or alcohol abuse/dependence. They report that residuals grouped by the number of ICD codes show statistically significant different trend lines ($p = 0.002$), and that in veterans over 49, negative residuals—brains predicted to be older than they are—correlate with carrying multiple ICD codes. The point of the work is that a standard, non-invasive MRI sequence could flag otherwise hidden or underreported disease burden in a veteran population.","feed_headline":"MRI brain-age gap flags hidden illnesses in veterans","feed_subtitle":"Model residuals match the number of ICD-coded conditions, strongest in veterans past 49.","key_machinery":"The central object is the brain age residual biomarker (BARB), the difference between a model's predicted brain age and a patient's chronological age; a negative residual means the brain appears older than its owner's years. The machinery is a four-CNN ensemble: each of four identical shallow convolutional networks learns one of four 2D T2-weighted image types (FSE and FLAIR, at the anterior commissure and at the frontal horns of the lateral ventricles), and a degree-3 polynomial regression without interaction terms combines their outputs into the final age prediction. The residual is then grouped by number of ICD codes and analyzed through trend-line comparison and ANOVA, with the over-49 subgroup carrying the significant signal.","core_discovery":"The central discovery, on the paper's own terms, is that the error of a brain-age predictor is not pure noise: it carries information about disease burden. The ensemble model, built from four lightweight CNNs each trained on one of four T2-weighted image types, produces residuals whose group-specific regression lines differ significantly by the number of ICD-coded conditions ($F = 4.265$, $p = 0.002$ in the training set; $F = 16.4$, $p = 5.77\\times10^{-13}$ in the test set). The authors find the clearest signal in subjects older than 49, where a one-way ANOVA on residuals grouped by ICD count is significant ($p = 0.0069$) and negative residuals (older-appearing brain) accompany multiple diagnoses. They are careful to note that individual conditions, apart from hypertension and substance abuse/dependence in ANOVA, show no consistent association, and they attribute the aggregate signal to the collective burden of chronic conditions rather than to any single code.","pith_inferences":["Editorial extension: the headline association is most fragile where the paper's own evidence is weakest—the age-unadjusted ANOVA on residuals grouped by number of ICD codes was null ($p = 0.1880$), so the positive result depends on the over-49 subgroup and on regression-line comparisons that may absorb age-composition effects.","If the BARB is real, it would most naturally be tested against disease severity (A1C, blood pressure readings, toxicology-confirmed substance use) rather than binary ICD codes, which the paper itself identifies as the main labeling limitation.","The use of routine T2-weighted clinical slices, if reproducible, would make the biomarker cheap enough to screen retrospectively on existing scans, without dedicated research acquisitions."],"forward_implications":["If the association holds, a gap between predicted and chronological brain age becomes a candidate screening signal for otherwise hidden disease burden in veterans, especially after age 49.","The success of the model on two T2-weighted sequences rather than T1 means routine clinical MRIs already collected for other reasons could be reused for brain-age screening.","Refining ICD labels into severity scores should strengthen the residual-disease association, since the paper's binary labels mix well-managed and poorly managed cases.","Any future BARB study should stratify by age before testing residual-disease relationships, since age dominates both ICD count and residual behavior.","The model's test-set $R^2$ of 0.816 with MAE of 5.45 years provides a realistic performance benchmark for small, single-site brain-age models."],"supporting_citations":[{"why":"Establishes the brain-predicted age difference concept and links it to mortality, the core rationale for testing residuals as biomarkers.","marker":"[7]"},{"why":"Introduces the BrainAGE residual biomarker and the regression approach that the paper extends.","marker":"[12]"},{"why":"Shows brain-age residuals associate with genetic variants, supporting the claim that residuals carry biological signal.","marker":"[15]"},{"why":"Supplies the closest methodological precedent: brain-age prediction from routine T2-weighted spin-echo MRI with a deep CNN.","marker":"[14]"},{"why":"Provides the bias-variance-irreducible-error decomposition used to argue residual signal can come from external health factors.","marker":"[51]"},{"why":"Gives the analogous decomposition for mean absolute error, the metric used for the residual analysis.","marker":"[52]"}],"fun_headline_variants":["Brain age gap flags hidden illnesses in veterans","MRI brain age errors reveal disease load in veterans","Veterans' brain age residual signals chronic conditions","Brain age residual biomarker spots latent health issues"],"cache_read_input_tokens":20608,"weakest_assumption_plain":"The load-bearing premise is that the different regression lines and negative residuals across ICD-count groups reflect disease effects on brain aging, not the fact that veterans with more diagnoses tend to be older and that prediction error grows with age.","fun_headline_variants_meta":{"raw":{"variants":["Brain age gap flags hidden illnesses in veterans","MRI brain age errors reveal disease load in veterans","Veterans' brain age residual signals chronic conditions","Brain age residual biomarker spots latent health issues"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000124,"raw_usage":{"total_tokens":1135,"prompt_tokens":1011,"completion_tokens":124,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":66}},"tokens_in":627,"tokens_out":124,"duration_ms":2358,"temperature":1.0,"reasoning_tokens":66,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:06:29.618747+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the residual comparison on age-matched subgroups: for each chronological age, compare residuals of veterans with zero, one, and multiple ICD codes. If the residual differences disappear within matched age bands—or if an age-adjusted residual ANOVA is null—the central claim is falsified.","supporting_citations":[{"cited_title":"H., Ritchie, S","cited_arxiv_id":null,"evidence_quote":"Establishes the brain-predicted age difference concept and links it to mortality, the core rationale for testing residuals as biomarkers."},{"cited_title":"K., Lee, J","cited_arxiv_id":null,"evidence_quote":"Supplies the closest methodological precedent: brain-age prediction from routine T2-weighted spin-echo MRI with a deep CNN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the bias-variance-irreducible-error decomposition used to argue residual signal can come from external health factors."}],"review_version":1}