{"id":"33cf1924-984c-467c-bea0-dde861b4b341","arxiv_id":"2411.10100","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"M-AVAE, a multitask adversarial variational autoencoder, predicts brain age from multimodal MRI with a mean absolute error of 2.77 years on a 381-subject subset of OpenBHB.","lead":"This paper presents a deep learning model that predicts a person's brain age from both structural and functional MRI scans, achieving an average error of 2.77 years on a small dataset. The model separates shared and modality-specific features and uses sex classification as a secondary task to improve accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Full-data Random Forest feature selection before 10-fold CV leaks target information, inflating the reported 2.77-year MAE and the claimed advantage over M-AAE.","rationale":"The reader's weakest assumption identifies full-data feature selection before cross-validation as the key flaw, and the text supports that reading: feature selection is described as a preprocessing step producing m1 and m2 features, while cross-validation is described only later as an evaluation procedure. This is the most load-bearing concern because it directly affects the quantitative headline (2.77 years) and the relative ranking in Table II, which are the core of the paper's central claim. A secondary but compounding issue is the absence of any paired significance test; the gap between M-AVAE and M-AAE is small relative to the reported standard deviations. I do not take the CAE comparison in Table III as the main issue, since that is a cross-dataset comparison and the strongest claim is centered on Table II. Credit is due for releasing source code, which makes the proposed check feasible, but the existence of code does not by itself establish that the evaluation protocol is leak-free. I would keep the reader's REJECT because the manuscript as written does not support its central claim without re-analysis; if the code inspection and nested-CV rerun show no leakage and a significant improvement, the verdict would need to be revisited.","tokens_in":949,"tokens_out":1484,"duration_ms":78523,"concrete_test":"Run the released code (github.com/engrussman/MAVAE) with a corrected evaluation protocol: refit the RandomForest feature selector inside each training fold, apply the fitted selector only to that fold's validation split, and repeat the same 10-fold CV for M-AVAE and M-AAE. Compare MAE, RMSE, PCC, and a paired test (e.g., Wilcoxon signed-rank or paired t-test on per-subject absolute errors) between M-AVAE and M-AAE. If the nested-CV MAE rises materially above 2.77 years or the M-AVAE advantage over M-AAE becomes non-significant, the leakage concern lands and the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that M-AVAE achieves 2.77 years MAE and outperforms all Table II baselines rests on an evaluation protocol that appears to select features once on the full 381-subject dataset before cross-validation. The Feature Extraction Process section states 'we use the filter method ... After selecting the features, we obtain the m1 and m2 features', with no indication that the Random Forest selector is refit inside each training fold. Section H then describes a 10-fold cross-validation on the resulting features. Because the Random Forest feature importance is computed with access to the age labels of all 381 subjects, the selected feature set can encode target-dependent information from validation folds; the subsequent CV performance is therefore optimistic. This bias is not necessarily uniform: flexible models such as M-AVAE can exploit leakage features more aggressively than linear baselines such as PLSR and SVR, so the claimed superiority over M-AAE and other methods may be inflated. The paper also provides no paired statistical test; the reported 2.773±1.567 vs 3.125±1.976 MAE for M-AVAE versus M-AAE has overlapping standard deviations, so even on the current feature set the 0.35-year improvement is not established. The load-bearing assumption is that feature selection is nested inside the CV loop, or that any leakage is negligible; the manuscript provides no evidence for either.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes M-AVAE, a multitask adversarial variational autoencoder that integrates sMRI and fMRI for brain-age estimation and sex classification. The architecture separates latent variables into generic (shared) and unique (modality-specific) codes, uses adversarial and variational losses, and adds sex classification as an auxiliary task. The authors evaluate M-AVAE on a 381-subject subset of the OpenBHB dataset and report a mean absolute error of 2.77 years, comparing against model-agnostic, model-based, and AAE-based baselines. The paper claims that M-AVAE outperforms all compared methods and positions it as a tool for metaverse-based healthcare.","tokens_in":13553,"tokens_out":3552,"duration_ms":37902,"significance":"If the reported performance is validated under a leakage-free evaluation protocol, the disentangled multimodal architecture with a sex-classification auxiliary task would be a useful contribution to brain-age estimation. The manuscript has strengths: the source code is made public, the evaluation uses a public dataset, and the model combines several mechanisms (adversarial learning, variational regularization, cross-reconstruction, and multitask learning) in a principled way. However, the central empirical claim is currently undermined by a likely information-leakage problem in feature selection and by the absence of statistical significance testing. The claimed advantage over M-AAE, the closest baseline, is within one standard deviation, so the headline result is not yet established.","major_comments":[{"comment":"The manuscript describes Random Forest feature selection before cross-validation, but does not state that the selector is refit inside each training fold. In the Feature Extraction Process, the authors write that after Random Forest feature selection they 'obtain the m1 and m2 features'; Section H then describes a 10-fold cross-validation on the resulting features. Because Random Forest importance is computed using the target variables y (age and sex) in Eq. (1), a selector fit once on all 381 subjects can leak validation-fold label information into the training folds. This can inflate the reported MAE and can also exaggerate the difference between flexible deep models such as M-AVAE and simpler baselines. The authors must either confirm that feature selection was nested inside each CV fold or rerun all experiments with feature selection performed only on training folds, and then report the resulting metrics.","section":"Feature Extraction Process; Section H (Robustness Analysis)"},{"comment":"No statistical significance tests are reported for the comparison between M-AVAE and the baselines. The headline improvement over M-AAE is 2.773 +/- 1.567 vs. 3.125 +/- 1.976 years MAE; these standard deviations overlap substantially, and the manuscript does not report paired tests. Since the same subjects are evaluated under all models, a paired test (e.g., a paired bootstrap or Wilcoxon signed-rank test on per-subject absolute errors) is appropriate and should be added. Without such a test, the claim that M-AVAE 'outperformed all the methodologies compared' is not supported.","section":"Section IV-G, Table II"},{"comment":"The comparison with prior published methods in Table III mixes datasets, preprocessing pipelines, and sample sizes, so it cannot support the claim that M-AVAE outperforms those methods. For example, CAE on UKB achieves MAE 2.71 vs. 2.773 for M-AVAE, with lower RMSE (3.68 vs. 3.185) and higher PCC (0.868 vs. 0.824). The authors acknowledge the larger dataset for CAE, but the conclusion 'our model surpassed in terms of MAE' is based on a 0.06-year difference that is not shown to be statistically meaningful. This table should be reframed as an indicative comparison, or the authors should perform a same-data comparison.","section":"Section IV-G, Table III"}],"minor_comments":[{"comment":"The sentence 'we applied the preprocessing pipeline outlined in Section to both datasets' has a missing section number; please provide the correct reference.","section":"Section E (Datasets)"},{"comment":"The reconstruction loss in Eq. (14) has typographical problems: the norm expression 'xi−Deci' should likely be '||xi - Deci(...)||', and the notation Exi~Pd(xi) is unclear about the summation indices. Please rewrite this equation for readability.","section":"Equation (14)"},{"comment":"The paper calls Random Forest feature selection a 'filter method,' but Random Forest importance is typically classified as an embedded method. This terminology should be corrected or clarified.","section":"Feature Extraction Process"},{"comment":"The selected feature counts m1 and m2 are never reported, even though they are central to the input dimensionality. Please state these values explicitly.","section":"Feature Extraction Process"},{"comment":"The claim that M-AVAE is 'a powerful tool for metaverse-based healthcare applications' is speculative, since no metaverse or deployment experiments are performed. Please soften this claim to match the scope of the study.","section":"Abstract and Conclusions"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the feature-selection leakage, which affects all numbers in Table II. If the authors cannot rerun the pipeline with nested feature selection, the paper should be rejected; however, since the fix is methodologically straightforward, a major revision is appropriate at this stage. I would also ask the editor to ensure that the claimed comparison with M-AAE is supported by paired significance tests before any acceptance decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on 2411.10100. The architecture is a reasonable incremental combination of published ideas: it merges adversarial autoencoders, VAEs, multitask learning, and the generic/unique latent split from Hu et al., and adds a distance ratio loss plus a sex classifier. The writing is clear, and the authors ship code. If the reported 2.77-year MAE on the 381-subject sMRI+fMRI OpenBHB subset is genuine, it's a competitive result.\n\nThe trouble is the evaluation. The Random Forest feature selection is described once, before the data is split for 10-fold CV. There is no sign that the selector is refit inside each training fold. That means the selected features are influenced by the labels of all subjects, including the validation folds, so the test numbers have target leakage. The text gives no basis for assuming the leakage is negligible, and flexible models like M-AVAE are exactly the kind that can exploit such leakage more than the linear baselines. That undermines the claimed win over M-AAE.\n\nEven if the protocol were clean, the 0.35-year MAE gap between M-AVAE and M-AAE is inside the reported standard deviations, and there is no paired significance test. So the headline difference isn't statistically established. There's also a factual slip in the comparison with CAE: the paper says M-AVAE surpassed CAE in MAE, but Table III shows CAE at 2.71 and M-AVAE at 2.773. The datasets differ, so it's not a controlled comparison, but the sentence is simply wrong.\n\nThe missing lambda values and the lack of ablations on the loss weights also make it hard to judge how much each component buys. The 'metaverse' framing can be dropped without loss.\n\nAll that said, this is a real attempt at a real problem. The architecture is clearly specified, the code is available, and multimodal brain age on both sMRI and fMRI is genuinely underexplored. With a corrected CV protocol, statistical testing, and some honest re-analysis, it could be a useful paper.\n\nMy recommendation: send it to review, but the referee should treat it as a major-revision case. The central numeric claim needs to be re-run with nested feature selection, and the authors need to show the leakage is not responsible for the result. I would not cite the numbers as they stand.","headline":"Solid incremental architecture, but the evaluation leaks target information and the reported advantage lacks statistical support.","tokens_in":14105,"tokens_out":3349,"would_cite":false,"duration_ms":31519,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multimodal adversarial variational autoencoder estimates biological brain age with a mean absolute error of 2.77 years by disentangling shared and modality-specific features.","keywords":["brain age estimation","multimodal neuroimaging","adversarial variational autoencoder","disentangled representation learning","multitask learning","sex classification","structural MRI","functional MRI"],"falsifier":"Run the same 10-fold cross-validation but perform Random Forest feature selection independently inside each training fold, then record the held-out MAE for M-AVAE and the M-AAE baseline; if the M-AVAE advantage disappears or its MAE rises substantially above 2.77 years, the central performance claim is not supported.","tokens_in":13112,"feed_emoji":"🧠","tokens_out":4669,"duration_ms":42710,"temperature":0.7,"pith_summary":"The paper introduces M-AVAE, a multitask adversarial variational autoencoder that estimates biological brain age from paired structural and functional MRI. The central claim is that splitting each modality's latent representation into shared (generic) and modality-specific (unique) codes, then regularizing those codes adversarially and variationally, lets the model fuse multimodal data better than prior autoencoder and regression baselines. On a 381-subject subset of the OpenBHB dataset, the model reports a mean absolute error of 2.77 years, outperforming the compared methods. The authors argue this improves the accuracy and robustness of brain age estimation, with potential for personalized healthcare in virtual environments.","feed_headline":"Brain age predicted to 2.77 years with multimodal MRI","feed_subtitle":"A model that splits shared and unique brain features and adds sex classification beats prior methods.","key_machinery":"The central object is the M-AVAE architecture, which pairs two encoder-decoder branches (one for sMRI, one for fMRI) and splits each latent vector into a generic code $\\operatorname{Gen}(z_i)$ and a unique code $\\operatorname{Unq}(z_i)$. Cross-reconstruction forces the generic code of one modality to reconstruct the other modality, the distance ratio loss $L_{\\mathrm{Dist}} = L^{\\mathrm{Gen}}_{\\mathrm{Dist}} / L^{\\mathrm{Unq}}_{\\mathrm{Dist}}$ pushes shared codes together and unique codes apart, an adversarial discriminator regularizes the generic latent space toward a Gaussian prior, and a KL divergence regularizes the unique codes. A regressor and a classifier predict age and sex from the combined representation.","core_discovery":"The paper claims that M-AVAE outperforms existing brain age estimation methods on multimodal sMRI+fMRI data by disentangling latent variables into generic and unique parts, using cross-reconstruction between modalities, and adding sex classification as an auxiliary task. With this architecture the model achieves a mean absolute error of 2.77 years and a root mean square error of 3.185 years on a merged two-dataset subset of OpenBHB, compared with 3.125 years for a multitask adversarial autoencoder without variational regularization. The authors attribute the gain to the hybrid adversarial-variational regularization, which imposes distinct priors on generic and unique codes and promotes a disentangled latent space.","pith_inferences":["The reported performance may be optimistic because Random Forest feature selection is applied to the full 381-subject dataset before the 10-fold split, allowing test-fold information to influence training features; a nested selection procedure is likely to raise the error and narrow the gap to the baselines.","If the disentanglement is as effective as claimed, a testable prediction is that the generic codes alone should carry most of the age signal, so an age regressor trained on $\\operatorname{Gen}(z_1), \\operatorname{Gen}(z_2)$ should nearly match the full-model performance.","The comparison with CAE [22] suggests the performance gap may shrink or reverse on datasets with thousands of subjects, since CAE reports lower RMSE on the much larger UK Biobank.","The metaverse and digital-twin framing is speculative; the concrete contribution is the architecture and the evaluation protocol, not the virtual healthcare deployment."],"forward_implications":["If the 2.77-year MAE holds in independent validation, the model offers a practical way to combine structural and functional MRI for brain age estimation without the noise that often degrades multimodal fusion.","The disentangled generic/unique coding scheme should make the model robust to missing or incomplete modalities, since shared information can be reconstructed from the other modality.","Adding sex classification as an auxiliary task improves age prediction, supporting the view that sex-specific aging patterns carry useful signal for brain age models.","The framework could be adapted to other multimodal biomedical inputs where separating shared and modality-specific variation matters."],"supporting_citations":[{"why":"Supplies the adversarial autoencoder mechanism that matches the latent aggregate posterior to a prior distribution.","marker":"[55]"},{"why":"Supplies the variational KL loss that regularizes the unique latent codes toward a standard Gaussian.","marker":"[56]"},{"why":"Provides the base architecture of modality-specific encoders and decoders with generic and unique codes and cross-reconstruction that M-AVAE extends.","marker":"[23]"},{"why":"Is the multisite dataset from which the 381 sMRI/fMRI subjects are drawn for evaluation.","marker":"[57]"},{"why":"Supports the Random Forest filter method used to select the m1 and m2 features from the two modalities.","marker":"[54]"},{"why":"Serves as a state-of-the-art comparison method whose MAE, RMSE, and PCC are contrasted with M-AVAE in Table III.","marker":"[22]"},{"why":"Provides a gender-informed brain age baseline compared against M-AVAE in Table III.","marker":"[16]"}],"fun_headline_variants":["M-AVAE hits 2.77-year brain age error on multimodal MRI","Multimodal brain age: MAE 2.77 years with adversarial VAE","Disentangled latent codes cut brain age error to 2.77 years","Sex-aware multi-task model estimates brain age within 2.77 years","Adversarial VAE fuses sMRI and fMRI for brain age with 2.77-yr MAE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that feature selection performed once on all 381 subjects does not leak test information into the 10-fold cross-validation; if that leakage occurs, the reported 2.77-year error is inflated.","fun_headline_variants_meta":{"raw":{"variants":["M-AVAE hits 2.77-year brain age error on multimodal MRI","Multimodal brain age: MAE 2.77 years with adversarial VAE","Disentangled latent codes cut brain age error to 2.77 years","Sex-aware multi-task model estimates brain age within 2.77 years","Adversarial VAE fuses sMRI and fMRI for brain age with 2.77-yr MAE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2585,"prompt_tokens":819,"completion_tokens":1766,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":435,"completion_tokens_details":{"reasoning_tokens":1655}},"tokens_in":435,"tokens_out":1766,"duration_ms":10661,"temperature":1.0,"reasoning_tokens":1655,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:57:49.115289+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 10-fold cross-validation but perform Random Forest feature selection independently inside each training fold, then record the held-out MAE for M-AVAE and the M-AAE baseline; if the M-AVAE advantage disappears or its MAE rises substantially above 2.77 years, the central performance claim is not supported.","supporting_citations":[],"review_version":1}