REVIEW 3 major objections 4 minor 21 references
Decoding Phenotypes: A Framework for Fusing Genomic Language Models and Neuroimaging
T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A fusion framework that conditions multi-scale brain-MRI features on sequence embeddings of a 1024-base-pair DNA window around APOE improves early cognitive decline and dementia screening over imaging alone, reporting AUROCs of 0.77 and…
desk verdict Genuinely new fusion architecture that deserves refereeing, but the claim that GLM sequence context adds signal is currently confounded by a multi-SNP baseline gap. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by two fusion modules plus a frozen genomic encoder. GCFM (Genotype-Conditioned Feature Modulation) adapts feature-wise linear modulation (FiLM) by predicting channel-wise scale $\gamma^s$ and shift $\beta^s$ from the genomic embedding and applying $F' = \gamma \odot F + \beta$ to image feature maps at the 1/2, 1/4, and 1/8 encoder scales, so DNA context recalibrates MRI features before global pooling. U-GRF (Uncertainty-aware Genomic Residual Fusion) computes predictive entropy of the image-only logits, maps it together with the image latent to a gate $\alpha \in [0,1]$, and forms $z_{\mathrm{final}} = (1-\alpha) z_{\mathrm{img}} + \alpha z'_{\mathrm{gene}}$, suppressing genomic input when the image is confident. The genomic embedding itself comes from a frozen pre-trained Nucleotide Transformer v2 (100M) applied to a patient-specific 1024-bp window centered on the risk variant, with mean pooling over loci in multi-locus settings.
What would settle it
Retrain GeneFuse after randomly shuffling the nucleotide order within each subject's 1024-base-pair window while keeping imaging, labels, and all other settings fixed; if AUROC remains at 0.83 for NC vs AD, the reported gain is not attributable to DNA sequence context. A simpler check is comparing against a model that uses only APOE ε2/ε3/ε4 allele counts; matching 0.83 would mean the extra sequence context is unnecessary.
Extended reading notes
Core claim
The paper's central claim is that genomic language model (GLM) embeddings provide additional information to imaging for early cognitive decline and dementia screening. In the APOE-centered setting, GeneFuse improves AUROC by 0.08 (NC vs MCI) and 0.12 (NC vs AD) over the image-only backbone, with the gains statistically significant at $p<0.05$ by paired DeLong tests on subject-level out-of-fold predictions; scalar SNP encoding gives only a small gain, and late concatenation, FiLM-style conditioning, cross-attention, and fixed residual fusion all underperform. Extending to multiple AD-risk loci raises AUROC to 0.79 and 0.84, which the authors read as evidence that loci beyond APOE contribute complementary diagnostic information. They conclude that preserving nucleotide context around disease-associated variants, rather than reducing variants to allele counts, is what makes the fusion work.
Load-bearing premise
The result depends on the assumption that the order of DNA bases in a 1024-base-pair window around APOE, as read by a pre-trained DNA language model, carries Alzheimer's-relevant information beyond the known risk variants themselves, and that this information transfers to a small single-cohort sample without fine-tuning.
Editorial extensions
If this is right
- If GeneFuse's result is right, pre-trained genomic language models can be used as parameter-free encoders that add signal to MRI-based diagnosis without needing to fine-tune the DNA model on medical data.
- The comparison with scalar SNP encoding implies that nucleotide context around risk variants contains phenotype-relevant information beyond allele dosage, so future imaging-genetics pipelines may replace hard-coded genotype labels with sequence windows.
- The ablation results indicate that conditioning multiple encoder scales beats single-scale or late conditioning, so fusion should happen at the feature-map level rather than only at the final latent.
- The uncertainty-gated residual design implies that genomic features should be weighted by the confidence of the stronger modality, not by a fixed rule, when the genotype-phenotype link is loose.
- Multi-locus integration improving over APOE alone suggests the same framework generalizes beyond a single well-established risk gene to collections of risk loci.
Reading between the lines
- The uncertainty-gating principle is generic: any weak or loosely coupled auxiliary modality could be added this way, letting the confident primary modality decide how much weight the auxiliary signal receives; testing GeneFuse with fluid biomarkers or clinical scores would show whether the mechanism transfers.
- The multi-scale conditioning design suggests imaging-genetics fusion is best applied at feature-map level rather than at the final prediction, a principle that could transfer to other pairings of genomic embeddings with volumetric imaging, such as cardiac or oncological MRI.
- Because the cohort is small and single-source, external replication on another MRI and whole-genome-sequencing cohort using the same frozen encoder would show whether the AUROC gains transfer before any clinical use.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GeneFuse, a multimodal classifier that combines 3D T1w MRI features from a 3D TransUNet with genomic embeddings from the frozen Nucleotide Transformer v2 (NT-v2). For each subject, a 1024 bp patient-specific window centered on an AD risk variant (APOE in the main analysis) is encoded by the GLM; a Genotype-Conditioned Feature Modulation (GCFM) block applies channel-wise FiLM conditioning to multi-scale imaging features, and an Uncertainty-aware Genomic Residual Fusion (U-GRF) module gates the genomic contribution using imaging-derived predictive entropy. The authors evaluate on ADNI (N=182; 52 NC, 53 MCI, 77 AD) for NC vs MCI and NC vs AD, reporting AUROCs of 0.77 and 0.83 for the full APOE-based GeneFuse, compared with 0.69 and 0.71 for image-only, and claim that GLM-derived genomic embeddings provide additional information to imaging. The paper also ablates GCFM and U-GRF, compares several GLM encoders, and reports multi-locus results.
Significance. Strengths: the architecture is clearly specified; subject-level out-of-fold predictions with paired DeLong tests are the right protocol for correlated AUROC comparison; the use of a frozen pre-trained GLM with no fine-tuning makes the comparison to scalar SNP encoding interpretable in principle; ablations of GCFM insertion scales and GLM backbones are informative. I agree with the reader that the evaluation is not circular: the frozen GLM and out-of-fold predictions make the comparison legitimate in principle. If the reported effect is real, the paper would be among the first to show that pre-trained genomic language model embeddings of non-coding sequence context can add diagnostic signal beyond SNP allele counts when fused with volumetric imaging. The uncertainty gating idea is novel and clinically sensible. However, because the central comparison is potentially confounded by multi-variant coverage in the sequence window, and because only one of the two headline significance claims is supported by Table 1, the contribution is not yet established with the confidence required for publication.
major comments (3)
- [Section 3, Table 1] The text in Section 3 states that GeneFuse's AUROC improvements over the image-only backbone 'are statistically significant under paired DeLong tests on subject-level out-of-fold predictions (p < 0.05).' However, Table 1 places the dagger marker only on the NC vs AD row of GeneFuse (Full); the NC vs MCI row (0.77 vs 0.69) carries no dagger or double-dagger marker. As written, the central abstract claim covers both tasks, so the NC vs MCI significance assertion is contradicted by the manuscript's own table. Please report exact DeLong p-values for all comparisons to image-only and correct the text, or explicitly qualify the NC vs MCI result as a non-significant trend.
- [Section 2.1 and Table 1] The headline comparison between GeneFuse and 'Image + SNP scalar' is confounded by the amount of genetic information each input carries. A 1024 bp window centered on the APOE locus can contain both APOE-defining variants (rs429358 and rs7412) as well as nearby polymorphic sites, so the frozen NT-v2 embedding can separate patients by multi-SNP haplotype; the scalar baseline by definition encodes a single variant call. The observed AUROC gain (0.77 vs 0.70 for NC vs MCI; 0.83 vs 0.73 for NC vs AD) may therefore reflect the addition of more allele information rather than nucleotide context learned by the GLM. To make the central claim ('GLM-derived genomic embeddings provide additional information to imaging') load-bearing, add a control that encodes all variants in the same 1024 bp window in an allele-only form (e.g., one-hot or allele counts per polymorphic site) under the same fusion framework, and a control that masks or shuffles non-variant positions in the NT-v2 input. If the allele-only control matches or outperforms the GLM, the conclusion should be revised.
- [Abstract and Section 3, Table 1] The abstract says GeneFuse 'outperforms existing imaging–genetics fusion methods,' but no significance test is reported for comparisons against MADDi, MMDL, stage-wise DNN, Late Concat, Cross-Attention, or Residual in Table 1. Several baselines have overlapping standard deviations with GeneFuse (e.g., NC vs MCI Cross-Attention 0.74±0.07 vs 0.77±0.05; NC vs AD MADDi 0.77±0.12 vs 0.83±0.05), so the mean AUROC differences are not evidently significant. Report DeLong p-values for all pairwise baseline comparisons, or temper the claim to mean-AUROC advantage.
minor comments (4)
- [Section 2.1] The genomic tokenization step is underspecified: the paper does not state whether NT-v2 is applied with 6-mer tokenization, whether all 1024 positions are used as a single sequence, or how the forward/reverse strand is chosen; please add these details for reproducibility.
- [Section 3 and Table 3] The multi-locus comparison reports mean AUROC gains over APOE alone (0.79 vs 0.77; 0.84 vs 0.83) without significance tests; given the small sample, please report DeLong p-values or label the result as exploratory.
- [Section 3, Figure 2] Figure 2 is described only qualitatively; please state how many subjects were used, whether the maps are averaged across the cohort and across five folds, and whether the reported periventricular/medial temporal pattern was consistent across subjects.
- [Section 4] There is a typo ('adddtional') in the conclusion, and the paper lacks a data/code availability statement; even a brief statement would improve reproducibility.
Circularity Check
No significant circularity: the reported AUROCs are out-of-fold empirical results, and no load-bearing derivation reduces to its own inputs.
full rationale
The paper's central empirical claim is that frozen pre-trained GLM embeddings of a 1024 bp patient-specific APOE window add diagnostic signal to MRI features. This is evaluated with subject-level stratified 5-fold cross-validation, with AUROC computed on out-of-fold predictions and significance assessed by paired DeLong tests. The genomic backbone (NT-v2-100M) is frozen and not fitted to ADNI labels, and the fusion modules (GCFM and U-GRF) are ordinary trained networks whose reported metrics are out-of-fold performance, not quantities defined by the fitted parameters. No equation in the paper defines a predicted quantity in terms of the same quantity being explained, and no fitted parameter is renamed as a prediction. The paper contains no load-bearing self-citation: the cited GLM backbones, imaging-genetics baselines, and ADNI are independent external resources, and the authors make no uniqueness or forced-choice argument resting on their own prior work. The skeptic concern that the 1024 bp window contains multiple linked variants, so the GLM embedding may encode haplotype information beyond the scalar SNP baseline, is a possible confound or benchmark-fairness issue about what signal the GLM contributes, but it is not circularity under the required definition: it does not show that the result is equivalent to its input by construction. Accordingly, the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (6)
- Trainable network weights (image encoder, decoder, GCFM MLPs, U-GRF gate, linear transform, classifier)
- lambda_reg (L1 coefficient on fusion gate alpha)
- DNA sequence window length L =
1024 bp
- Latent dimensions (dimg=256, dg=512, GCFM hidden=256) =
dimg=256, dg=512, hidden=256
- GCFM insertion scales =
1/2, 1/4, 1/8
- Residual baseline alpha =
0.05
assumptions (6)
- domain assumption APOE is a well-established genetic risk factor for late-onset Alzheimer's disease.
- ad hoc to paper Frozen NT-v2 embeddings of 1024 bp human DNA windows carry phenotype-relevant context without fine-tuning.
- ad hoc to paper Imaging-derived predictive entropy is a reliable signal for gating genomic contribution.
- ad hoc to paper A 1024 bp window centered on the target variant covers principal proximal regulatory elements.
- domain assumption The ADNI 182-subject subset and baseline clinical labels are representative and accurate.
- domain assumption The genomic preprocessing pipeline (VCFtools, Beagle, vcf2diploid) produces correct personal genome sequences.
Cite this review
Pith. "Pith review of Decoding Phenotypes: A Framework for Fusing Genomic Language Models and Neuroimaging." pith.science (2026). https://pith.science/paper/R7CEJIJE
@misc{pith2026260808926,
author = {Pith},
title = {Pith review of: Decoding Phenotypes: A Framework for Fusing Genomic Language Models and Neuroimaging},
year = {2026},
howpublished = {\url{https://pith.science/paper/R7CEJIJE}},
note = {Machine review of arXiv:2608.08926}
}
read the original abstract
Neuroimaging and genetic testing are two important clinical references for nervous system diseases, offering complementary diagnostic information. However, integrating genomic and neuroimaging data for precise disease diagnosis is challenging due to cross-modality heterogeneity. Existing imaging-genetics approaches mainly encode genetic information as hard-coded labels, which lose the local sequence context around disease-associated variants. To address this limitation, we propose GeneFuse, a multimodal learning framework that aligns genetic representations from pre-trained Genomic Language Models (GLMs) with features extracted from images. GeneFuse integrates two components: (1) Genotype-Conditioned Feature Modulation (GCFM), a FiLM-inspired module that uses genomic embeddings to modulate image feature maps; and (2) Uncertainty-aware Genomic Residual Fusion (U-GRF), a fusion strategy that uses imaging-derived predictive uncertainty to gate the contribution of genotypic features. We evaluate GeneFuse on early cognitive decline identification (NC vs. MCI) and dementia screening (NC vs. AD). In the APOE-centered setting, GeneFuse achieves AUROCs of 0.77 and 0.83, outperforming existing imaging-genetics fusion methods. These results indicate that GLM-derived genomic embeddings provide additional information to imaging.
Figures
Reference graph
Works this paper leans on
-
[1]
The American Journal of Human Genetics108(10), 1880–1890 (2021)
Browning, B.L., Tian, X., Zhou, Y., Browning, S.R.: Fast two-stage phasing of large-scale sequence data. The American Journal of Human Genetics108(10), 1880–1890 (2021)
work page 2021
-
[2]
arXiv preprint arXiv:2102.04306 (2021)
Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.:Transunet:Transformersmakestrongencodersformedicalimagesegmentation. arXiv preprint arXiv:2102.04306 (2021)
arXiv 2021
-
[3]
Nature Methods22(2), 287–297 (2025)
Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Lopez Carranza, N., Grzywaczewski, A.H., Oteri, F., Dallago, C., Trop, E., de Almeida, B.P., Sirelkha- tim, H., et al.: Nucleotide transformer: building and evaluating robust foundation models for human genomics. Nature Methods22(2), 287–297 (2025)
work page 2025
-
[4]
Bioinformatics27(15), 2156–2158 (2011)
Danecek, P., Auton, A., Abecasis, G., Albers, C.A., Banks, E., DePristo, M.A., Handsaker, R.E., Lunter, G., Marth, G.T., Sherry, S.T., et al.: The variant call format and vcftools. Bioinformatics27(15), 2156–2158 (2011)
work page 2011
-
[5]
Nucleic Acids Research53(2), gkae1310 (2025)
Fishman, V., Kuratov, Y., Shmelev, A., Petrov, M., Penzar, D., Shepelin, D., Chekanov, N., Kardymon, O., Burtsev, M.: Gena-lm: a family of open-source foun- dational dna language models for long sequences. Nucleic Acids Research53(2), gkae1310 (2025)
work page 2025
-
[6]
Journal of the American Medical Informatics Association29(12), 2014–2022 (2022)
Golovanevsky, M., Eickhoff, C., Singh, R.: Multimodal attention-based deep learn- ing for alzheimer’s disease diagnosis. Journal of the American Medical Informatics Association29(12), 2014–2022 (2022)
work page 2022
-
[7]
Whitwell, J., Ward, C., et al.: The alzheimer’s disease neuroimaging initiative (adni): Mri methods
Jack Jr, C.R., Bernstein, M.A., Fox, N.C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P.J., L. Whitwell, J., Ward, C., et al.: The alzheimer’s disease neuroimaging initiative (adni): Mri methods. Journal of Magnetic Reso- nance Imaging: An Official Journal of the International Society for Magnetic Res- onance in Medicine27(4), 685–691 (2008)
2008
-
[8]
Bioinformat- ics37(15), 2112–2120 (2021)
Ji, Y., Zhou, Z., Liu, H., Davuluri, R.V.: Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformat- ics37(15), 2112–2120 (2021)
work page 2021
Show all 21 references
-
[9]
Frontiers in Aging Neuroscience 17, 1532470 (2025)
Li, Y., Niu, D., Qi, K., Liang, D., Long, X.: An imaging and genetic-based deep learning network for alzheimer’s disease diagnosis. Frontiers in Aging Neuroscience 17, 1532470 (2025)
2025
-
[10]
Nature Reviews Neurology9(2), 106–118 (2013)
Liu, C.C., Kanekiyo, T., Xu, H., Bu, G.: Apolipoprotein e and alzheimer disease: risk, mechanisms and therapy. Nature Reviews Neurology9(2), 106–118 (2013)
2013
-
[11]
medRxiv pp
Liu, Q., Zeng, W., Zhu, H., Li, L., Wong, W.H., Initiative, A.D.N.: Leveraging genomic large language models to enhance causal genotype-brain-clinical pathways in alzheimer’s disease. medRxiv pp. 2024–10 (2024)
2024
-
[12]
Journal of Molecular Neuroscience75(2), 66 (2025)
Mashhour, M.A., Youssef, I., Wahed, M.A., Mabrouk, M.S.: The intersection of genetics and neuroimaging: A systematic review of imaging genetics in neurological disease for personalized treatment. Journal of Molecular Neuroscience75(2), 66 (2025)
2025
-
[13]
arXiv preprint arXiv:2306.15794 (2023)
Nguyen, E., Poli, M., Faizi, M., Thomas, A., Birch-Sykes, C., Wornow, M., Patel, A., Rabideau, C., Massaroli, S., Bengio, Y., et al.: Hyenadna: long-range genomic sequence modeling at single nucleotide resolution. arXiv preprint arXiv:2306.15794 (2023)
2023 arXiv
-
[14]
Nanomaterials15(12), 895 (2025) 10 T
Parvin, N., Joo, S.W., Jung, J.H., Mandal, T.K.: Multimodal ai in biomedicine: Pioneering the future of biomaterials, diagnostics, and personalized healthcare. Nanomaterials15(12), 895 (2025) 10 T. Tao et al
2025
-
[15]
In: Proceedings of the AAAI conference on artificial intelligence
Perez, E., Strub, F., De Vries, H., Dumoulin, V., Courville, A.: Film: Visual rea- soning with a general conditioning layer. In: Proceedings of the AAAI conference on artificial intelligence. vol. 32 (2018)
2018
-
[16]
In: International conference on machine learning
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)
2021
-
[17]
Molecular systems biology7(1), 522 (2011)
Rozowsky, J., Abyzov, A., Wang, J., Alves, P., Raha, D., Harmanci, A., Leng, J., Bjornson, R., Kong, Y., Kitabayashi, N., et al.: Alleleseq: analysis of allele-specific expression and binding in a network framework. Molecular systems biology7(1), 522 (2011)
2011
-
[18]
Nature Machine Intelligence6(8), 911–923 (2024)
Sanabria, M., Hirsch, J., Joubert, P.M., Poetsch, A.R.: Dna language model grover learns sequence context in the human genome. Nature Machine Intelligence6(8), 911–923 (2024)
2024
-
[19]
Scientific Reports 11(1), 3254 (2021)
Venugopalan, J., Tong, L., Hassanzadeh, H.R., Wang, M.D.: Multimodal deep learning models for early detection of alzheimer’s disease stage. Scientific Reports 11(1), 3254 (2021)
2021
-
[20]
arXiv preprint arXiv:2307.05628 (2023)
Zhang, D., Zhang, W., Zhao, Y., Zhang, J., He, B., Qin, C., Yao, J.: Dnagpt: A generalized pre-trained tool for versatile dna sequence analysis tasks. arXiv preprint arXiv:2307.05628 (2023)
2023 arXiv
-
[21]
Human Brain Mapping40(3), 1001–1016 (2019)
Zhou, T., Thung, K.H., Zhu, X., Shen, D.: Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Human Brain Mapping40(3), 1001–1016 (2019)
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.