{"id":"df8594d8-86a9-4a2f-a853-a334f7fe85f1","arxiv_id":"2412.05632","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A multimodal brain-age estimator with sex input is presented, but its reported advantage over prior methods rests on an invalid comparison across different test datasets.","lead":"This paper proposes a sex-aware adversarial variational autoencoder that fuses structural and functional MRI features to estimate biological brain age, reporting a mean absolute error of 2.72 years. However, the model is tested on a small external multimodal dataset while comparison methods are evaluated on the larger OpenBHB benchmark, so the claimed superiority is not established.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline MAE 2.722 is computed on 320 external multimodal scans, not on the OpenBHB test splits used for Table III baselines; the SOTA comparison is therefore invalid.","rationale":"The reader's verdict identifies the most load-bearing flaw: the evaluation protocol for the headline result is not commensurable with the comparison methods. I agree. The strongest claim—that SA-AVAE outperforms SOTA—requires the test sets to be comparable, and this assumption is contradicted by the paper's own description. Section IV-A says multimodal data come from two datasets cited as [67] and [68], which are independent fMRI studies (Sunavsky & Poppenk; Nastase et al.), not OpenBHB sub-datasets. The reported counts are inconsistent (381 combined vs. 320 used in Table V), and OpenBHB is a structural MRI dataset. Thus the 2.722 MAE is an out-of-distribution evaluation on a small sample, while the baselines in Table III (Aqil 4.554, Ahmed 3.250, Cheshmi 3.860, Träuble 3.724) are presumably on OpenBHB's standardized splits. Moreover, the paper's own unimodal SA-AVAE on 3,200 OpenBHB sMRI scans achieves 2.906, which would beat all Table III baselines if evaluated on the same split; the paper's failure to present that comparison is revealing. The concern is not about theoretical soundness but about the central empirical claim. A concrete check—running the model on the exact baseline test split—would settle it. Since this check has not been reported, the rejection stands.","tokens_in":18924,"tokens_out":4802,"duration_ms":39353,"concrete_test":"Verify the composition of OpenBHB (e.g., from the official repository) to confirm whether it contains any fMRI scans. If it does not, then evaluate the SA-AVAE on the exact OpenBHB test split used by Ahmed et al. (2023) using sMRI-only input, and compare the resulting MAE to 3.250. If OpenBHB lacks fMRI, the multimodal model cannot be run on that split, and the Table III comparison is unsupportable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SA-AVAE outperforms state-of-the-art brain-age estimators rests on Table III, where the proposed method reports MAE 2.722 against baselines evaluated on OpenBHB. However, the multimodal experiments described in Section IV-A are not performed on OpenBHB: the paper combines two independent datasets, refs [67] (66 scans) and [68] (315 scans), totaling 381 scans, and Section V-D/Table V later states that only 320 sMRI+fMRI scans were used. OpenBHB itself contains structural T1-weighted MRI only; it has no fMRI data. Consequently, the 2.722 MAE is measured on a small external multimodal test set that is not comparable to the OpenBHB internal/external test splits used for the comparison methods in Table III. Additional evidence of the mismatch is internal: the unimodal SA-AVAE trained on 3,200 OpenBHB sMRI scans achieves MAE 2.906 (Table V), which would already beat every baseline in Table III, yet the paper highlights the multimodal 2.722 from 320 scans instead. The claimed improvement from 3.250 (Ahmed et al.) to 2.722 cannot be attributed to the model when the test sets differ in size, domain, and modality composition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SA-AVAE, a sex-aware adversarial variational autoencoder for biological brain age estimation from sMRI and fMRI. The architecture disentangles latent features into shared and modality-specific codes using adversarial, variational, cross-reconstruction, and shared-distinct distance ratio losses, and feeds the concatenated codes plus sex information into a regressor. Experiments are reported on OpenBHB for the unimodal setting and on a smaller multimodal sample assembled from two additional sources. The paper claims state-of-the-art performance with a multimodal MAE of 2.722 years, robustness across age and sex groups, and a consistent advantage of multimodal over unimodal input.","tokens_in":19177,"tokens_out":4457,"duration_ms":44226,"significance":"If the evaluation were valid, the contribution would be meaningful: the disentanglement objective is clearly specified, sex conditioning is a sensible design choice, and the ablation study shows monotonic improvement when adversarial, variational, and sex-aware components are added. The paper also reports useful implementation details such as architecture sizes, optimizer settings, and training time. However, the central comparison is undermined by the fact that the headline multimodal MAE is measured on a small external sample while the state-of-the-art baselines are evaluated on OpenBHB test splits. As a result, the claimed superiority and the practical significance of the framework are not currently established.","major_comments":[{"comment":"The state-of-the-art comparison is not valid because the proposed method and the baselines are evaluated on different test sets. Section IV-A states that multimodal experiments use 381 scans from references [67] and [68], and Section V-D later says 320 sMRI+fMRI scans were used. These sources are not the OpenBHB internal/external test splits used by the baseline methods in Table III, and OpenBHB is a T1-weighted structural MRI dataset without fMRI. Therefore the MAE of 2.722 cannot be directly compared with the OpenBHB-based MAE values of Aqil et al., Ahmed et al., Cheshmi et al., and Träuble et al. The paper should either re-evaluate all methods on a common held-out set under the same protocol or drop the claim of outperforming state-of-the-art methods.","section":"Section V-B, Table III"},{"comment":"The unimodal-versus-multimodal comparison is confounded by dataset differences. The unimodal SA-AVAE is trained on 3,200 OpenBHB sMRI scans, while the multimodal SA-AVAE uses 320 scans from a different multimodal source. The observed improvement from 2.906 to 2.722 years cannot be attributed to multimodality because training-set size, data domain, and modality composition differ simultaneously. Additionally, Section IV-A reports a total of 381 multimodal scans, while Table V and the surrounding text use 320; this inconsistency must be resolved and the experimental protocol specified precisely.","section":"Section V-D, Table V"},{"comment":"The paper's evidence for disentanglement is largely circular. The adversarial, variational, cross-reconstruction, and distance-ratio losses are designed to enforce separation of shared and distinct codes, and the ablation study in Table II only reports downstream age-prediction metrics. No quantitative evaluation of the latent space is provided, such as similarity of shared codes across modalities, separation of distinct codes, or reconstruction diagnostics. Since disentanglement is a core claimed contribution, the manuscript should report a direct measure of the learned representations, not only the final MAE.","section":"Section III-B, Section V-A"},{"comment":"The paper lists a 'comprehensive strategy for fine-tuning loss weight parameters' as a contribution, but no loss-weight sensitivity analysis or final values of mu_1...mu_5 and eta_1...eta_4 are reported. These weights are empirically determined free parameters, and the robustness of the method to their choice is never examined. At minimum, the final weights and a small sensitivity study should be included, otherwise this claimed contribution is unsupported.","section":"Section I, Section III-B, Eq. (17)"}],"minor_comments":[{"comment":"The text says the MAE for sMRI-only input ranged from 3.52 to 2.72 years and that multimodal fusion yielded values between 3.59 and 2.72 years; this is inconsistent with Table V, where the unimodal SA-AVAE reports MAE 2.906 and the multimodal reports 2.722. The ranges should be reconciled or the description clarified.","section":"Section V-D"},{"comment":"The adversarial loss appears to be written with the roles of real and generated samples reversed. As written, the discriminator receives prior samples as real data and generated shared codes as fake data, which conflicts with the standard adversarial alignment described in the text and with the objective of matching the aggregated posterior to the prior.","section":"Section III-B, Eq. (7)"},{"comment":"The terms 'sex' and 'gender' are used interchangeably in several places (for example, Figure 5 and Table V), although the paper's stated variable is biological sex. The terminology should be made consistent.","section":"Throughout"},{"comment":"It is not clearly stated whether the multimodal subsets from references [67] and [68] are part of the OpenBHB dataset or independent external datasets. The distinction matters because the abstract and conclusion claim evaluation on OpenBHB, while the multimodal experiments appear to use separate sources.","section":"Section IV-A"}],"recommendation":"reject","confidential_remarks":"The central problem is not stylistic but experimental: the multimodal model is evaluated on a small external sample, while the comparison methods are evaluated on OpenBHB test splits. Because OpenBHB does not contain fMRI, the multimodal claim cannot be repaired simply by reporting additional splits of the same data; it would require either a new multimodal evaluation dataset with a common protocol or a substantial reframing of the paper's claims. In addition, the unimodal-versus-multimodal comparison is confounded by dataset and sample-size differences. These issues are load-bearing for the paper's main conclusions, so I recommend rejection, although the architectural ideas could be revisited in a future study with a properly matched evaluation design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on arXiv:2412.05632. The architecture is a sensible combination of established ideas: adversarial and variational losses for disentangling shared vs modality-specific latent codes, cross-reconstruction, a distance-ratio loss, and direct sex input to the regressor. It's clearly described, and the single-modality adaptation is pragmatic. Credit where due: the framework is not a random bag of tricks; the ablations show each added component helps, and sex information improves performance modestly.\n\nThe problem is the evaluation. The headline MAE of 2.722 in Table III is computed on a small external multimodal test set (320-381 scans from refs [67] and [68]), not on the OpenBHB internal/external splits used for the comparison methods. OpenBHB is T1-weighted sMRI only, so there's no fMRI to do the multimodal experiment on that benchmark. That makes the SOTA comparison invalid. This isn't a nitpick; it's the paper's central claim. The internal evidence confirms it: the unimodal sMRI model trained on 3,200 OpenBHB scans reaches MAE 2.906, which already beats every baseline in Table III, yet the paper highlights the multimodal 2.722 from 320 scans instead. Similarly, the unimodal-vs-multimodal comparison in Table V is confounded by training set size: 3,200 vs 320 scans.\n\nOther soft spots: the loss weights (mu's, eta's) are empirically chosen but the paper doesn't give the actual values, which hurts reproducibility. The limitations section is honest about missing-modality sensitivity and healthy-only testing, which is good, but it largely undercuts the clinical applicability claims.\n\nBottom line: the architecture is worth having a look at, but the evaluation doesn't support the claims. This needs major revision: either redo the multimodal experiments on a benchmark that actually has paired sMRI/fMRI and comparable test splits, or frame the contribution as an architectural study without the SOTA claim. I would not cite it as a validated method, but I'd send it to review with the expectation of heavy revision; a good referee could help the authors fix the evaluation.","headline":"A competent architecture but the headline MAE is measured on a different test set than the baselines, so the SOTA claim is unsupported.","tokens_in":19750,"tokens_out":2737,"would_cite":false,"duration_ms":23954,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sex-aware adversarial variational autoencoder that disentangles shared and modality-specific MRI features estimates brain age with 2.72-year mean absolute error, beating prior OpenBHB methods.","keywords":["brain age estimation","multimodal MRI","variational autoencoder","adversarial learning","disentangled representation learning","sex-aware model","OpenBHB dataset","neurodegenerative disease biomarker"],"falsifier":"Re-run SA-AVAE and all Table III baselines on one common held-out test set from the official OpenBHB internal and external splits, and compare MAEs; if SA-AVAE no longer beats the 3.250-year prior best, or if its multimodal MAE is not better than its sMRI-only MAE on matched data, the paper's central claim is refuted.","tokens_in":18698,"feed_emoji":"🧠","tokens_out":8637,"duration_ms":74182,"temperature":0.7,"pith_summary":"Brain age—how old the brain looks on imaging rather than chronological age—is a candidate biomarker for neurodegenerative disease, but fusing structural and functional MRI often adds noise and hurts accuracy. This paper tries to establish that a sex-aware adversarial variational autoencoder (SA-AVAE) can fuse sMRI and fMRI by splitting the latent code into shared and modality-specific parts, so the functional modality contributes signal without contaminating the structural one. The paper reports a mean absolute error of 2.722 years on its multimodal evaluation, lower than the four prior OpenBHB-based methods it compares against, and shows the multimodal model beats its own sMRI-only counterpart even when that counterpart trains on ten times more data. If the claim holds, multimodal brain-age estimation becomes accurate enough to be worth deploying in clinical screening for early neurodegeneration.","feed_headline":"Sex-aware MRI fusion estimates brain age to 2.72 years","feed_subtitle":"Combining structural and functional scans with sex info beats sMRI-only and prior multimodal models on OpenBHB.","key_machinery":"The central object is the Sex-Aware Adversarial Variational Autoencoder (SA-AVAE), a paired-encoder architecture that decomposes each modality's latent vector $z_i$ into $\\mathrm{Shared}(z_i)$ and $\\mathrm{Dist}(z_i)$, concatenates them with sex for the regressor, and is trained with five loss families: adversarial loss aligning shared codes to a prior, variational KL loss on distinct codes, reconstruction loss, cross-modality reconstruction, and the shared-distinct distance ratio $L_D = L_{\\mathrm{Shared}}^D / L_{\\mathrm{Dist}}^D$. This ratio loss is what enforces the contract that shared codes converge while distinct codes diverge, and the sex input is what lets the regressor model male- and female-specific aging trajectories.","core_discovery":"On its own terms, the paper claims that the SA-AVAE is the most accurate brain-age estimator among compared methods on the OpenBHB dataset. The architecture takes sMRI and fMRI feature vectors, encodes each into a latent space split into a shared code (modality-invariant) and a distinct code (modality-specific), and feeds the concatenated codes plus sex into a regressor that outputs biological age. Adversarial alignment regularizes the shared codes, variational KL losses regularize the distinct codes, and two extra terms—cross-modality reconstruction and a shared-distinct distance ratio loss—push the split to be meaningful. The reported outcome is an overall MAE of $2.722 \\pm 1.351$ years, RMSE of $3.039$, and $R^2$ of $0.936$, with the sex-aware version outperforming the same model without sex, the multitask sex variant, and all ablated autoencoder variants.","pith_inferences":["Because the paper evaluates only healthy controls (a limitation it states), the 2.72-year error should not yet be read as a clinical diagnostic accuracy for Alzheimer's or Parkinson's; a testable next step is running SA-AVAE on patient cohorts and checking whether brain-age gap separates patients from controls.","The architecture's shared/distinct split is not specific to sMRI versus fMRI; the same loss design could be applied to other paired brain measurements, such as T1 with diffusion MRI or with PET, and the shared-distinct distance ratio would be the component to isolate in an ablation.","The reported comparison in Table III mixes test sets: the multimodal result comes from roughly 320-381 scans drawn from two fMRI datasets, while the listed prior methods were evaluated on OpenBHB splits. Re-running all methods on one matched test set would settle whether the 0.5-year advantage is real or a test-set artifact."],"forward_implications":["The multimodal SA-AVAE reaches 2.722 years MAE, below the best listed OpenBHB comparator at 3.250 years, which would make it the most accurate published method on this benchmark.","Injecting sex information directly into the regressor improves both accuracy and balance across male and female subgroups while using fewer parameters than multitask sex prediction.","Fusion of sMRI and fMRI helps rather than hurts: the multimodal model beats the unimodal sMRI model (2.722 vs 2.906 MAE) even though the unimodal model was trained on roughly ten times more scans.","The disentanglement losses—especially cross-reconstruction and the shared-distinct distance ratio—are what the paper credits for letting fMRI add useful signal without the noise penalty that defeats simple multimodal fusion.","Age-group breakdowns show MAE below about 3.1 years in every group from under 25 to 45-55, which the paper reads as robustness across the adult lifespan."],"supporting_citations":[{"why":"open multi-site brain MRI benchmark that supplies the training and validation data and defines the dataset context for the state-of-the-art comparison.","marker":"[66]"},{"why":"prior disentangled multimodal adversarial autoencoder whose architecture and effectiveness the proposed framework extends to sex-aware brain age estimation.","marker":"[23]"},{"why":"earlier multi-task adversarial variational autoencoder for brain age that supplies the M-AVAE baseline in Table VI.","marker":"[65]"},{"why":"companion multi-task AVAE work cited alongside [65] as the sex-via-multitask baseline that SA-AVAE is compared against.","marker":"[76]"},{"why":"one of the two fMRI datasets (66 scans) combined to form the multimodal evaluation set.","marker":"[67]"},{"why":"the other fMRI dataset (315 scans) combined with [67] to form the multimodal evaluation set.","marker":"[68]"},{"why":"the strongest prior OpenBHB comparator in Table III (MAE 3.250), the result the central claim must beat.","marker":"[71]"},{"why":"prior OpenBHB-based autoencoder method included in Table III as a direct comparison point.","marker":"[72]"}],"fun_headline_variants":["Sex-aware AI fuses brain scans to estimate age within 2.7 years","Multimodal brain age model beats single-scan, uses sex info","Sex-aware adversarial autoencoder improves brain age from MRI+fMRI","Sex-aware fusion of sMRI and fMRI yields brain age error of 2.72 years"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline 2.722-year MAE is computed on a small multimodal set assembled from two fMRI datasets (roughly 320-381 scans), while the four methods it is compared against in Table III were evaluated on OpenBHB test splits; the central claim of superior accuracy collapses if those test sets are not comparable enough for a direct MAE comparison.","fun_headline_variants_meta":{"raw":{"variants":["Sex-aware AI fuses brain scans to estimate age within 2.7 years","Multimodal brain age model beats single-scan, uses sex info","Sex-aware adversarial autoencoder improves brain age from MRI+fMRI","Sex-aware fusion of sMRI and fMRI yields brain age error of 2.72 years"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000764,"raw_usage":{"total_tokens":3417,"prompt_tokens":1003,"completion_tokens":2414,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":2331}},"tokens_in":619,"tokens_out":2414,"duration_ms":15622,"temperature":1.0,"reasoning_tokens":2331,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:30:57.963079+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run SA-AVAE and all Table III baselines on one common held-out test set from the official OpenBHB internal and external splits, and compare MAEs; if SA-AVAE no longer beats the 3.250-year prior best, or if its multimodal MAE is not better than its sMRI-only MAE on matched data, the paper's central claim is refuted.","supporting_citations":[{"cited_title":"Openbhb: a large-scale multi-site brain mri data-set for age prediction and debiasing,","cited_arxiv_id":null,"evidence_quote":"open multi-site brain MRI benchmark that supplies the training and validation data and defines the dataset context for the state-of-the-art comparison."},{"cited_title":"Disentangled-multimodal adversarial autoencoder: Application to infant age prediction with incomplete multimodal neu- roimages,","cited_arxiv_id":null,"evidence_quote":"prior disentangled multimodal adversarial autoencoder whose architecture and effectiveness the proposed framework extends to sex-aware brain age estimation."},{"cited_title":"Advancing metaverse-based healthcare with multimodal neuroimaging fusion via multi-task adversarial variational autoencoder for brain age estimation,","cited_arxiv_id":null,"evidence_quote":"earlier multi-task adversarial variational autoencoder for brain age that supplies the M-AVAE baseline in Table VI."},{"cited_title":"Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging","cited_arxiv_id":"2411.10100","evidence_quote":"companion multi-task AVAE work cited alongside [65] as the sex-via-multitask baseline that SA-AVAE is compared against."},{"cited_title":"Neuroimaging predictors of creativity in healthy adults,","cited_arxiv_id":null,"evidence_quote":"one of the two fMRI datasets (66 scans) combined to form the multimodal evaluation set."},{"cited_title":"The “narratives","cited_arxiv_id":null,"evidence_quote":"the other fMRI dataset (315 scans) combined with [67] to form the multimodal evaluation set."},{"cited_title":"Robust brain age estimation via regression models and mri-derived features,","cited_arxiv_id":null,"evidence_quote":"the strongest prior OpenBHB comparator in Table III (MAE 3.250), the result the central claim must beat."},{"cited_title":"Confounding factors mitigation in brain age prediction using mri with deformation fields,","cited_arxiv_id":null,"evidence_quote":"prior OpenBHB-based autoencoder method included in Table III as a direct comparison point."}],"review_version":1}