{"id":"b1a74dcd-7e96-4123-a219-5a7f82dbe1b6","arxiv_id":"2509.10527","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper claims a new ensemble of GAT, VAE, elastic-net, and FDR achieves top biomarker-discovery performance, but provides no reproducible evidence.","lead":"This paper proposes combining four existing machine learning and statistics tools to find dementia biomarkers in high-dimensional, low-sample omics data, and reports high accuracy on simulated data and ADNI. It provides no code, no detailed experimental setup, and the reported performance tables cannot be verified.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Storey FDR step has no specified p-value source from elastic net; without it, the q-value filtered gene lists and 'statistical reliability' claims are unsupported.","rationale":"The paper's central claim is superiority in predictive accuracy, feature selection precision, and biological relevance. That claim rests on the pipeline: GAT captures interactions, MOVE compresses, elastic net selects, Storey FDR validates. The weakest link is the missing p-value derivation: Storey FDR requires valid p-values, but the text only gives equations for the elastic-net objective and Storey's q-value formula, with no connection between them. This is a correctness risk, not merely a consensus disagreement. Sparse-regression p-values are subtle; post-selection inference is needed for valid FDR control, and the paper is silent on which procedure is used. If the p-values are invalid, the q≤0.05 gene lists in Tables 2 and 5 lose their statistical meaning, and the framework degenerates to an unvalidated feature-ranking scheme. The reader's weakest assumption was broader—the implementability of the entire GAT/MOVE/elastic-net/Storey pipeline—so I partially agree, focusing specifically on the p-value/Storey connection as the load-bearing point. The benchmarking tables add a second reproducibility concern, but that is downstream of the statistical gate. The verdict remains REJECT because the current manuscript does not supply the critical algorithmic detail or any evidence that the claimed FDR control actually holds. If the authors later provide the p-value procedure and pass a null-calibration test, a CONDITIONAL acceptance could be considered; as written, the central claim is unverified and arguably not verifiable from the text alone.","tokens_in":5965,"tokens_out":3888,"duration_ms":36607,"concrete_test":"Ask the authors to release code and specify the exact procedure that turns elastic-net coefficient estimates into the p-values used in Storey's q-values (e.g., residual bootstrap per selected variable, permutation of labels, or selective inference). Then run the stated pipeline on a simulated null dataset with all features independent of the outcome (same dimension and sample size as Section 4.1). Under the complete null, the number of genes with q<0.05 should be at most 5% of the tests on average. If the authors cannot provide the p-value derivation, or if the empirical FDR exceeds the nominal 5%, the Storey-FDR filtering step—and therefore the claimed statistical robustness—fails. As a secondary check, recompute Tables 1 and 4 from the released code over 10 random train/test splits and report mean ± SD; without this, the 'superior accuracy' comparison is not statistically meaningful.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The framework's central statistical claim—Storey-FDR-controlled biomarker selection—depends on a set of valid p-values for the elastic-net coefficients, but the manuscript never defines how those p-values are obtained. Section 2.3 specifies only the elastic-net objective (β minimizing penalized squared error); Section 2.4 assumes p-values {p1,...,pm} as inputs; Section 3 states that features are ranked with Storey's FDR using q-value thresholds. The connective step is absent. There is no mention of whether p-values come from a post-selection inference procedure, bootstrap/permutation, a Gaussian approximation on the selected coefficients, or a refit on an independent validation set. Each choice has materially different null behavior. A naive procedure—e.g., running an OLS t-test on the features selected by elastic net—does not produce valid p-values, and Storey's q-values would not control FDR even at the nominal level. Consequently, Tables 2 and 5 (FDR < 0.05 gene lists) and the 'statistical reliability' part of the superiority claim are unsupported. The same gap affects the simulation benchmark: no hyperparameters, architecture dimensions, training/validation split, number of repeats, or code repository are given, so the reported AUC/F1/feature precision numbers cannot be reproduced. This is not merely a presentation problem; if the p-values are invalid, the biological-relevance claims built on q≤0.05 lists are not grounded. I do not accuse the authors of fabrication; the issue is that the argument's logic is incomplete at exactly the step that separates ad hoc feature ranking from FDR-controlled discovery.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an ensemble framework for multi-omics biomarker discovery in dementia, combining Graph Attention Networks (GAT), Multi-Omics Variational AutoEncoder (MOVE), elastic-net regression, and Storey's False Discovery Rate (FDR). The authors benchmark against DIABLO, MOCAT, AMOGEL, and MOMLIN on both a synthetic Alzheimer's disease dataset and the ADNI cohort, reporting higher AUC, F1, and feature-selection precision (Tables 1 and 4). They also present lists of significant genes and gene-gene interactions, claiming superior predictive accuracy, feature selection precision, and biological relevance. The manuscript is a draft-level document: the method is described only through textbook equations, the integration between components is unspecified, and the experimental reporting omits reproducibility-critical details.","tokens_in":6368,"tokens_out":4320,"duration_ms":37021,"significance":"The problem addressed—interpretable biomarker discovery from high-dimensional, low-sample-size multi-omics data—is important, and the broad idea of combining sparse regression with an FDR filter is sensible. If the framework were fully specified and validated with rigorous experiments, it could be of interest to the multi-omics integration community. The inclusion of four benchmark methods and an attempt at realistic synthetic data generation are commendable. However, in its current form the manuscript does not provide an implementable algorithm, does not supply valid statistical evidence for its central FDR claim, and does not disclose enough experimental detail to support the numerical results. The significance of the claimed contributions therefore cannot be assessed.","major_comments":[{"comment":"The central statistical claim—Storey-FDR-controlled biomarker selection—depends on p-values for elastic-net coefficients, but the manuscript never defines how these p-values are obtained. Section 2.3 gives only the elastic-net objective (Eq. 2); Section 2.4 assumes an input list {p1,...,pm} without specifying its origin. Section 3 states that features are ranked using Storey's FDR with q-value thresholds. No post-selection inference, bootstrap/permutation procedure, independent refit, or other p-value source is described. Without valid p-values, Storey's q-values do not control FDR, and Tables 2 and 5 are unsupported.","section":"§2.3–2.4, §3"},{"comment":"The numerical superiority claim rests entirely on Tables 1 and 4, but the tables report point estimates only. No standard deviations, number of repeated runs, train/test splits, cross-validation scheme, hyperparameter values (GAT heads, MOVE weights β/λ, elastic-net λ/α), or sample sizes are given. No code or data availability is stated. The AUC values (0.93 and 0.91), F1 scores, and feature precision (0.88 and 0.87) therefore cannot be independently verified, and the claim of outperforming DIABLO, MOCAT, AMOGEL, and MOMLIN is not substantiated.","section":"§4.2, §5.2 (Tables 1 and 4)"},{"comment":"The simulation in Section 4.1 embeds 'established AD pathways' into synthetic data. The same well-known genes (TREM2, APOE, BIN1, SORL1, MAPT, etc.) then appear as 'discoveries' in both the simulated and ADNI analyses. As reported, this is a known-answer test rather than evidence of novel biological discovery. The manuscript does not quantify how accurately the framework recovers the embedded ground truth, nor does it compare the ADNI findings against a chance baseline or account for the high prior probability of these genes. The 'biological relevance' highlight is therefore partly circular and not supported by the reported analysis.","section":"§4.1, §5.3 (Tables 2, 3, 5, 6)"},{"comment":"The proposed ensemble is not concretely defined as an integrated pipeline. Section 2.1 gives the GAT update; Section 2.2 gives the MOVE objective; Section 2.3 gives elastic net; Section 2.4 gives Storey's FDR. There are no equations or descriptions that connect the GAT output to MOVE's latent space, nor the latent variables to the elastic-net design matrix. Architecture dimensions, training procedure, loss weighting, and the target variable y for classification/regression are all unspecified. As written, the framework is a list of standard components, not an implementable algorithm.","section":"§2.1–2.2, §3"},{"comment":"Several results are presented only as figure captions or placeholders, e.g., 'Top 10 Significant Genes and Interactions' (Section 4.3) and 'Biomarker Gene Map Visualization' (Section 5.3). No actual figure content is included. The interaction strengths in Tables 3 and 6 are stated without any description of how they were computed from the framework. The claim of 'visualized and interpreted' biomarker gene maps is therefore unverifiable.","section":"§4.3, §5.3 (figures and Tables 3, 6)"}],"minor_comments":[{"comment":"A citation is missing: '[?]' appears in Section 3 after 'q-value thresholds of 0.01, 0.05, and 0.1'.","section":"§3"},{"comment":"MOCAT is inconsistently cited: Section 4.2 cites [Chen et al., 2021] while the reference list also contains Yao et al. (2024) as another MOCAT entry. The bibliography should be reconciled.","section":"References"},{"comment":"Typo: 'Biormarker' should be 'Biomarker'.","section":"§7.2"},{"comment":"Tables in the text show 'T able' (e.g., 'Table 1: Performance Comparison') and column headers like 'F eature Precision' with extra spaces; these should be fixed.","section":"Table captions"},{"comment":"The acronym inconsistency: Section 2.2 calls MOVE a 'Multi-Omics Variational AutoEncoder,' while Section 4.3 refers to a 'Conditional Variational Autoencoder (CV AE) for multi-omics data (MOVE).' Clarify whether the VAE is conditional or not.","section":"§2.2, §4.3"},{"comment":"Several references have incomplete author lists (e.g., 'H. et al. Fang', 'Y. et al. Hasin') or are formatted informally. Full author lists and journal details should be provided.","section":"References"}],"recommendation":"reject","confidential_remarks":"This manuscript appears to be an incomplete draft: the method is not specified at a level that permits implementation, the statistical validation chain is broken (no p-value source for elastic net), and the experimental reporting lacks reproducibility-critical details. The known-answer simulation design further limits the strength of the biological claims. Even substantial revision would require new experiments, a full algorithmic specification, and re-analysis; these go beyond standard minor/major revision and are closer to a rewrite. I do not see a path to acceptance within the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this manuscript describes a multi-omics pipeline composed of four established methods—GAT, MOVE (a VAE variant), elastic net, and Storey's FDR—but it never actually specifies how the pieces fit together. The reported performance numbers (AUC 0.93 on simulation, 0.91 on ADNI) are the only evidence for the central claim, and they come without standard deviations, train/test splits, hyperparameters, or any code. I agree with the stress-test note: there is no defined source for the p-values fed into Storey's FDR, so the q-value-filtered gene lists in Tables 2 and 5 are unsupported. This is not a cosmetic flaw; it's the load-bearing step that is supposed to convert ad hoc feature ranking into FDR-controlled discovery.\n\nThat said, the paper isn't worthless. It addresses a real problem—HDLSS multi-omics integration—and the selected components are reasonable for that problem. The benchmark against DIABLO, MOCAT, AMOGEL, and MOMLIN is an appropriate set of comparators. The simulation is a known-answer test, which the authors are transparent about; they embed established AD pathways and then recover well-known genes. That validates the mechanics, but it does not constitute a novel discovery, and presenting those genes as \"biologically relevant\" findings is partly circular.\n\nThe soft spots are substantial. Beyond the missing p-value link, there are no architectural dimensions, no training procedure, no number of repeats, and no details on how GAT outputs enter MOVE's latent space. Some sections and figures are empty placeholders, and the bibliography is inconsistent (AMOGEL and MOCAT are each cited with conflicting authors and years). These problems are severe enough that the paper cannot be reproduced or even fully understood as a method.\n\nWho is this for? A reader who wants a quick survey of what an ensemble of these methods might look like could skim the intro and conclusion, but anyone trying to implement it would have to go back to the original papers. As a research contribution, it is not ready. I would not send it to referees in its current form; a desk rejection is appropriate, with an invitation to resubmit once the method is concretely defined and the results are reproducible.\n\nRecommendation: reject/desk-reject, but tell the authors exactly what is missing.","headline":"A plausible-sounding pipeline with no implementable method and no verifiable results; the missing p-value link leaves the FDR filter as decoration.","tokens_in":6813,"tokens_out":4373,"would_cite":false,"duration_ms":37103,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An ensemble of four methods claims to beat single-model multi-omics integrators at dementia biomarker discovery.","keywords":["multi-omics integration","dementia biomarkers","Alzheimer's disease","graph attention networks","variational autoencoder","elastic-net regression","false discovery rate","ADNI"],"falsifier":"Attempt the reproduction: to obtain the reported AUC of 0.93 and 0.91, F1 of 0.91 and 0.89, and feature precision of 0.88 and 0.87, one needs the exact interface between the GAT representation and the MOVE encoder, the encoder's latent dimension, the elastic-net hyperparameters, and the way latent variables are mapped to p-values for Storey's procedure, none of which the manuscript gives. A reader following the text cannot reconstruct a working pipeline; specifying those details and rerunning the two benchmarks with cross-validation would settle whether the claimed margins over DIABLO, MOCAT,","tokens_in":5889,"feed_emoji":"🧬","tokens_out":9843,"duration_ms":81614,"temperature":0.7,"pith_summary":"The paper sets out to show that dementia biomarker discovery from high-dimensional, low-sample-size multi-omics data works best when four complementary ideas are chained together: graph attention to model gene-gene interactions, a variational autoencoder to compress modalities into one latent space, elastic-net regression to pick a sparse feature set, and Storey's FDR to filter those features statistically. Against four published multi-omics integrators, the proposed ensemble reports the best AUC, F1, and feature-selection precision on both a simulated Alzheimer's dataset (0.93/0.91/0.88) and the ADNI cohort (0.91/0.89/0.87), with interpretability as the stated advantage. The resulting biomarker gene map, built from top genes such as TREM2, APOE, BIN1, and MAPT connected by interaction strengths, is offered as a bridge between statistical genomics and systems biology. A reader should care because the paper claims a principled answer to a real clinical constraint: few patients, many molecular measurements, and the need to know not just which genes matter but how they interact.","feed_headline":"Four-method ensemble claims sharper multi-omics dementia biomarkers","feed_subtitle":"Reports AUC 0.93/0.91 on simulated and ADNI data, topping four published integrators with interpretable output.","key_machinery":"The carrying object is the four-stage pipeline itself. GAT assigns learned attention weights to gene-gene edges, replacing the uniform neighbor weighting of graph convolutions and avoiding over-smoothing. MOVE, a multi-omics variational autoencoder with an objective of reconstruction accuracy minus a beta-weighted KL divergence plus a cross-modal alignment term, compresses the omics layers into a shared latent space that preserves GAT-derived structure. Elastic-net solves a mixed L1/L2 penalized regression to select a sparse, grouped feature set. Storey's q-values, computed from an estimated null proportion, threshold features at 0.01, 0.05, and 0.1. Each stage counters one failure mode of h","core_discovery":"On the paper's own terms, the central claim is that an ensemble of four established techniques, GAT, MOVE, elastic-net, and Storey's FDR, outperforms each family of single-paradigm integrators across every reported metric while remaining interpretable enough that the selected genes and their interactions can be read as a biological map. The author would state it as: by letting a graph attention network first learn which gene-gene associations matter, a multi-omics variational autoencoder then harmonize modalities into a shared latent space, elastic-net then select a sparse and grouped feature set, and Storey's procedure then certify those features against false-discovery error, a practitione","pith_inferences":["My reading: the reported margins over the strongest baseline are small, an AUC of 0.93 versus 0.89, and the paper reports no variance, cross-validation, or significance testing for its comparisons; a fair method ranking would need confidence intervals or repeated resampling.","My reading: the gene-gene interaction strengths, such as TREM2-TYROBP at 0.93, are presented as outputs but appear to be pre-assigned rather than learned; a genuine test of the GAT's contribution would compare its learned attention weights against these listed strengths to see whether the network rediscovers them.","My reading: the paper's central novelty claim, that graph structure plus latent compression plus sparse regression plus FDR beats each alone, could be tested directly by ablation on the same simulated data, dropping one stage at a time; the paper does not run this experiment.","Testable extension: applying the identical pipeline to another well-phenotyped high-dimensional low-sample-size cohort, such as cancer omics with fewer than one hundred samples, and checking whether the top hits reproduce known driver genes would show whether the ADNI result generalizes or is dataset-specific."],"forward_implications":["If the reported numbers hold, the ensemble beats four published multi-omics integrators on both simulated and ADNI data without sacrificing interpretability, giving high-dimensional low-sample-size studies a viable alternative to single-model pipelines.","The top-ranked genes, TREM2, APOE, BIN1, SORL1, MAPT, CD33, BACE1, INPP5D, CR1, and APP, and their mapped interactions align with established Alzheimer's risk and pathology, so the framework's outputs are directly usable as a hypothesis-generating biomarker map.","Because the design is modular, the same GAT-MOVE-elastic-net-FDR chain is claimed to transfer to cancer, type 2 diabetes, autoimmune disorders, and Parkinson's disease with only modality swaps.","Every selected feature carries a statistical warranty, Storey's FDR, layered on a biological prior, the graph structure, which the paper argues is what makes its results interpretable in a way the benchmark methods are not.","Future extensions claimed include longitudinal disease-progression modeling, integration with neuroimaging, spatial transcriptomics, and prospective clinical validation."],"supporting_citations":[{"why":"Supplies the graph attention network (GAT) attention mechanism used to model gene-gene interactions.","marker":"[Veličković et al., 2018]"},{"why":"Defines MOVE, the variational embedding used to compress multi-omics modalities into a shared latent space.","marker":"[Wang et al., 2021b]"},{"why":"Provides elastic-net regression, the sparse and grouped feature-selection step.","marker":"[Zou and Hastie, 2005]"},{"why":"Defines the q-value and false discovery rate procedure that filters selected features for statistical reliability.","marker":"[Storey, 2002]"},{"why":"DIABLO, the first multi-omics integration baseline the proposed framework must beat.","marker":"[Singh et al., 2019]"},{"why":"MOCAT, the multi-omics clustering baseline against which performance is benchmarked.","marker":"[Yao et al., 2024]"},{"why":"AMOGEL, the graph-embedding baseline compared in the benchmarking tables.","marker":"[Tan et al., 2025]"},{"why":"MOMLIN, the machine-learning integration baseline compared in the benchmarking tables.","marker":"[Rashid and Selvarajoo, 2024]"},{"why":"Supplies established Alzheimer's risk loci used to embed latent biological modules in the synthetic data.","marker":"[Kunkle et al., 2019]"}],"fun_headline_variants":["Four-method ensemble outperforms integrators on dementia omics","Interpretable ensemble beats single integrators on dementia multi-omics","AUC 0.93/0.91: four-method ensemble tops integrators","Four-way ensemble with interpretable maps wins on dementia omics","Ensemble of four methods tops published integrators on dementia"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The pipeline is assumed to be concretely implementable: GAT's graph outputs must connect to MOVE's latent space, and that latent space must yield p-values on which Storey's FDR can validly run, but the manuscript specifies none of these connections, dimensions, training procedures, or p-value derivations; if any link is missing, the framework is a list of component names and the reported scores are unsupported.","fun_headline_variants_meta":{"raw":{"variants":["Four-method ensemble outperforms integrators on dementia omics","Interpretable ensemble beats single integrators on dementia multi-omics","AUC 0.93/0.91: four-method ensemble tops integrators","Four-way ensemble with interpretable maps wins on dementia omics","Ensemble of four methods tops published integrators on dementia"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001865,"raw_usage":{"total_tokens":7118,"prompt_tokens":662,"completion_tokens":6456,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":406,"completion_tokens_details":{"reasoning_tokens":6375}},"tokens_in":406,"tokens_out":6456,"duration_ms":38725,"temperature":1.0,"reasoning_tokens":6375,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:12:43.011187+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Attempt the reproduction: to obtain the reported AUC of 0.93 and 0.91, F1 of 0.91 and 0.89, and feature precision of 0.88 and 0.87, one needs the exact interface between the GAT representation and the MOVE encoder, the encoder's latent dimension, the elastic-net hyperparameters, and the way latent variables are mapped to p-values for Storey's procedure, none of which the manuscript gives. A reader following the text cannot reconstruct a working pipeline; specifying those details and rerunning the two benchmarks with cross-validation would settle whether the claimed margins over DIABLO, MOCAT,","supporting_citations":[{"cited_title":"Regularization and variable selection via the elastic net","cited_arxiv_id":null,"evidence_quote":"Provides elastic-net regression, the sparse and grouped feature-selection step."},{"cited_title":"Mocat: Multi-omics clustering and annotation toolkit for disease subtyping","cited_arxiv_id":null,"evidence_quote":"MOCAT, the multi-omics clustering baseline against which performance is benchmarked."},{"cited_title":"Amogel: Associative multi-omics graph embedding learning","cited_arxiv_id":null,"evidence_quote":"AMOGEL, the graph-embedding baseline compared in the benchmarking tables."},{"cited_title":"Momlin: Multi-omics machine learning integration for neurodegeneration","cited_arxiv_id":null,"evidence_quote":"MOMLIN, the machine-learning integration baseline compared in the benchmarking tables."},{"cited_title":"Genetic meta-analysis of diagnosed alzheimer’s disease identifies new risk loci and implicates a , tau, immunity and lipid processing","cited_arxiv_id":null,"evidence_quote":"Supplies established Alzheimer's risk loci used to embed latent biological modules in the synthetic data."}],"review_version":1}