{"id":"6b52ad2e-851b-4eb3-9101-e7af8cb6c403","arxiv_id":"2502.03938","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A new attribution method backtracks through a fine-tuned Geneformer to rank genes by how strongly they activate disease-predicting neurons, but the causal claim is not supported.","lead":"This paper fine-tunes a genomic foundation model on Alzheimer's single-cell data and traces which genes most activate the neurons that drive the model's AD prediction. The authors call these genes novel causal biomarkers, but the method only measures attention within the model, not biological causation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'must-have' postulate in §2.2.1 equates attention-based backtracking with gene-to-neuron causality; a reversal test on the top MCGs would settle whether the causal claim holds.","rationale":"The paper proposes a clear pipeline: fine-tune Geneformer, use causal tracing on masked known AD genes to select MCNs, then backtrack attention to rank gene tokens. The central claim is the causal status of the resulting top-10 genes. For that claim to hold, the backtracked attention weights must correspond to a real gene-to-neuron causal pathway. The paper does not provide evidence for this equivalence; the 'must have' sentence in §2.2.1 is asserted, not derived. The proposed reversal test checks the equivalence directly: if a top MCG truly activates an MCN, then masking that gene should reduce the MCN's indirect effect or the model's AD prediction, similar to the masking of known AD genes. If top MCGs behave no differently from random genes, the attention backtracking is merely correlational attribution, not causal discovery. The reader's weakest assumption identifies the same unproven equivalence, so I agree. This is not a charge of fraud; the paper is an interesting interpretability exercise, but the causal biomarker conclusion is not supported. The verdict should remain REJECT.","tokens_in":13167,"tokens_out":3421,"duration_ms":33003,"concrete_test":"Perform a reversal test: for each of the top-10 MCGs (PLXDC2, MT-CO3, DOCK4, ARHGAP24, MEF2A, RUNX1, ITPR2, FOXN3, MT-CO1, SORL1), mask that gene's token as done for known AD genes in §3.2.1 and recompute the indirect effects of the MCN set identified in §3.2. Compare the average drop in MCN indirect effect when masking top MCGs versus masking the same number of frequency-matched random genes (e.g., selected from the same expression-rank bins). If top MCGs do not produce significantly larger drops than random genes, the backtracking weights do not reflect causal gene-to-neuron influence, and the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the top-10 MCGs are 'novel causal genetic biomarkers' of AD (§4.1). This rests on the postulate in §2.2.1 that MCGs must have the highest probability of activating MCNs, and on Algorithm 1, which backtracks from MCNs to input tokens using self-attention weights. But self-attention weights measure the model's learned association between tokens and hidden states, not a causal effect of a gene on a neuron. The causal tracing step in §3.2 intervenes only on known AD genes to locate MCNs; it never intervenes on the candidate MCGs to verify that perturbing them changes MCN activation or AD prediction. Thus the backtracking may simply propagate model-internal correlations learned from one microglial cluster (cluster 10 of the ROSMAP microglia, §3.1.1). If the identified MCGs do not actually drive MCN activity, the gene list is an artifact of the model and the selected data subset, not a causal biomarker discovery. A reversal test would make the causal assumption empirically checkable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Reverse-Gene-Finder, an interpretability pipeline applied to Geneformer for Alzheimer's disease (AD) gene discovery. The authors fine-tune a 12-layer Geneformer on single-cell microglial RNA-seq data from ROSMAP for early-AD versus non-AD classification, use causal tracing (masking ten known AD genes) to identify 'most causal neurons' (MCNs), and then backtrack through self-attention weights to score input gene tokens. The top-10 scored genes (PLXDC2, MT-CO3, DOCK4, ARHGAP24, MEF2A, RUNX1, ITPR2, FOXN3, MT-CO1, SORL1) are presented as novel causal genetic biomarkers of AD, with validation claimed through literature review and KEGG pathway enrichment. The paper positions the 'must' postulate connecting MCGs to MCNs as the central methodological novelty.","tokens_in":13303,"tokens_out":4790,"duration_ms":45405,"significance":"If the causal claim were supported, the approach would be a useful hypothesis-generation tool for AD and other diseases. The paper has concrete strengths: it ships a public code repository, builds on a publicly available foundation model (Geneformer), uses a public single-cell dataset, and reports cross-validated classification performance. However, the evidence presented does not establish the causal interpretation. The core postulate equating biological causation with neuron activation is unproven; the candidate genes are never intervened on; the ranking is derived from model-internal attention; and the reported gene scores come without uncertainty quantification or negative controls. The computational pipeline is coherent, but the central claim of 'novel causal genetic biomarkers' is not supported by the current experiments.","major_comments":[{"comment":"The load-bearing postulate—that MCGs 'must have the highest probability of activating' MCNs—is asserted as an observation but is neither derived nor empirically tested. The manuscript provides no link between biological causation of AD and activation of a hidden state in a fine-tuned transformer. Since Algorithm 1's scores and the resulting top-10 list are constructed entirely on this equivalence, the central claim collapses if the postulate fails. A reversal test (e.g., masking the top-ranked candidate and measuring whether MCN activation or AD prediction changes) is needed to make the claim empirically checkable.","section":"§2.2.1"},{"comment":"The causal tracing step intervenes only on ten known AD genes to locate MCNs; the candidate MCGs identified in §4.1 are never perturbed. The backtracking recurrence uses self-attention weights as 'model weights,' but in a transformer, attention weights are per-layer, per-head, normalized coefficients over tokens, not causal connections from a neuron at one layer to the input. The recurrence therefore propagates a model-internal association measure, not a counterfactual effect of a gene on an MCN. Without an intervention on the candidates, labeling them 'causal' is not justified.","section":"§3.2 and Algorithm 1"},{"comment":"The pipeline contains several under-analyzed free choices: the single microglial cluster (cluster 10 from the prefrontal cortex), the early-AD-only split, the 95th percentile MCN cutoff, the Gaussian noise level (set to 1), the 256-token truncation, and the top-10 cutoff. No sensitivity or robustness analysis is reported for any of these choices, and the gene scores in §4.1 are presented without standard deviations or confidence intervals across the five cross-validation folds. The top-10 list may therefore be an artifact of these choices rather than a stable discovery.","section":"§3.1.1, §3.2.1, §4.1"},{"comment":"The validation consists of citing prior literature for each gene and reporting enriched KEGG pathways. A literature review cannot distinguish a true causal gene from a well-known correlated one, and the pathway analysis on ten genes is post-hoc and vulnerable to selection bias. There are no negative controls (e.g., genes not believed to be AD-related), no permutation test on the ranking, no independent cohort replication, and no functional or interventional follow-up. Consequently, the abstract and §4.1 claim of 'novel causal genetic biomarkers' is unsupported by the evidence presented.","section":"§4.2–§4.3"},{"comment":"The paper introduces its definition of causality only in the discussion: 'This study defines \"causal\" as \"the most probable\" genetic biomarkers from a statistical perspective.' This is not the sense of causality used in the title and abstract, and it is considerably weaker than the counterfactual interpretation that the causal-tracing and 'intervention' language suggests. The manuscript should either operationalize causal claims with actual interventions or explicitly limit the contribution to hypothesis ranking, which would avoid overclaiming.","section":"§4.4"}],"minor_comments":[{"comment":"The abstract contains a typo: 'ave-nue' should be 'avenue'.","section":"Abstract"},{"comment":"The text reads 'fine-turned model performance'; this should be 'fine-tuned model performance'.","section":"§4.1"},{"comment":"Section number 3.2.1 is used twice: once for 'Known Genes Related to AD' and once for 'Use of Causal Tracing Analysis'. The second subsection should be renumbered (e.g., 3.2.2).","section":"§3.2.1"},{"comment":"The phrase 'interconnected MCNs in the next layer' is ambiguous because self-attention operates within a layer, not between layers. Please clarify whether MCNs are selected across all layers and how the recurrence maps from layer l+1 to layer l.","section":"Algorithm 1"},{"comment":"Figure 2 shows the locations of MCNs, but the axes and the marker encoding are not defined. Please specify what the x- and y-axes represent and how the 95th percentile cutoff is reflected in the plot.","section":"Figure 2"}],"recommendation":"reject","confidential_remarks":"The manuscript overclaims causal discovery from model attention; the implementation is coherent but the central premise is untested and the candidate genes are never intervened on. The paper would need a substantial redesign (reversal tests, negative controls, uncertainty quantification, and a reduction of the causal claim) before it could be considered for publication here. The authors disclose a pending patent and state acceptance at AAAI-25; neither affects the technical assessment. The public code and use of standard public data are positive aspects that should be preserved in any revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this before reading it: the paper is a well-written, transparent application of causal tracing (Meng et al.) to a genomic foundation model for Alzheimer's disease, but the headline claim—that it discovers novel causal genetic biomarkers—is not supported by the evidence. The method is a plausible hypothesis generator, not a validated discovery.\n\nWhat's actually new: the gene-token input representation and the backtracking from MCNs to input tokens via self-attention weights. Adapting causal tracing from LLMs to single-cell gene expression is a legitimate extension, and the authors are honest about their data, code, and hyperparameters. The fine-tuned Geneformer reaches 74.67% AUC, which is modest but reasonable for genetic-only data.\n\nThe soft spots are serious. The load-bearing postulate in §2.2.1—that MCGs must have the highest probability of activating MCNs—is asserted, not demonstrated. Self-attention weights measure learned associations, not causal effects. The causal tracing step intervenes only on known AD genes to locate MCNs; it never intervenes on the candidate MCGs to test whether perturbing them changes MCN activation or AD prediction. So the final gene ranking is essentially a correlation-based artifact of the model and the chosen data subset (one microglial cluster). The validation is a post-hoc literature review, and several top-10 genes (SORL1, RUNX1, MT-CO1) already appear in the AD literature cited elsewhere in the paper. There are no negative controls, no error bars on gene scores, and the cross-validation may leak subject-level information because single cells from the same subject can appear in both train and test folds.\n\nThe paper itself, in §4.4, redefines 'causal' as 'most probable'—an honest but significant downgrade. The reversal test suggested in the stress-test would make the causal claim empirically checkable, and its absence is the core gap.\n\nWho this is for: readers working on interpretability of genomic foundation models. It's a useful demonstration that causal tracing can be adapted to this setting, and the code and experimental detail support reproduction. But it should not be read as a credible causal gene discovery without functional validation or at least a reversal test.\n\nMy recommendation: I would send this to peer review despite my skepticism. The methodological transparency and the potential value as a hypothesis-generation tool justify referee time. I expect reviewers to push for interventional validation or a reframing away from 'causal' language.","headline":"A clear, transparent interpretability pipeline for Geneformer, but the 'causal biomarker' claim rests on an unproven equivalence between attention backtracking and gene-to-neuron causality; worth serious review as hypothesis generation, not as validated discovery.","tokens_in":13935,"tokens_out":2072,"would_cite":true,"duration_ms":20742,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Backtracking from causal neurons to input tokens yields ten novel candidate causal genes for Alzheimer's.","keywords":["Alzheimer's disease","genetic biomarkers","causal tracing","backtracking","genomic foundation model","single-cell transcriptomics","interpretability","gene tokens"],"falsifier":"Run the same Reverse-Gene-Finder procedure on an independent AD single-cell dataset or on a different microglial cluster from the same cohort and check whether the top-10 most causal genes reproduce; if the list changes entirely, the top-10 result is an artifact of the specific model and data subset rather than a robust causal signal. A complementary check is to mask the identified top-10 genes and test whether the fine-tuned model's AD prediction shifts significantly beyond what random gene masking produces.","tokens_in":12808,"feed_emoji":"🧬","tokens_out":7962,"duration_ms":62571,"temperature":0.7,"pith_summary":"This paper aims to extract candidate causal genetic biomarkers for Alzheimer's disease (AD) from a fine-tuned genomic foundation model by working backwards from the neurons most responsible for AD prediction to the input gene tokens that activate them. The authors claim that genes most likely to cause AD are the genes most likely to activate the neurons most likely to cause AD, and they operationalize that idea by masking known AD genes, locating the most causal neurons via causal tracing, and backtracking through self-attention weights to score every input gene token. The top ten genes produced by this procedure are PLXDC2, MT-CO3, DOCK4, ARHGAP24, MEF2A, RUNX1, ITPR2, FOXN3, MT-CO1, and SORL1, none of which are in the known AD gene list used for masking. If these candidates hold up, the method would provide an interpretable route from a black-box model to prioritized, testable genetic hypotheses about disease onset.","feed_headline":"Backtracking neurons finds 10 candidate causal genes for Alzheimer's","feed_subtitle":"A new backtracking method flags the strongest gene-token drivers of AD prediction for lab validation.","key_machinery":"The central machinery is the Reverse-Gene-Finder backtracking algorithm, which computes a score for the $i$-th unit at layer $l$ as $s_i^l = \\sum_j W_{i,j}^l \\left( IE(h_j^{l+1}) + s_j^{l+1} \\right)$, where $IE$ is the indirect causal effect of a most causal neuron measured by causal tracing and $W$ is the self-attention weight between units in adjacent layers. The recurrence is evaluated from the last layer to the input layer, so each input gene token accumulates the weighted causal influence of all MCNs it can reach. The load-bearing identity is the assumed equivalence behind the method: the most causal genes are exactly those whose tokens most strongly activate the most causal neurons. That equivalence turns a neural-network interpretability question into a biological causation claim.","core_discovery":"The paper's central discovery is a procedure that turns a fine-tuned genomic foundation model for AD classification into a reverse gene finder. The authors posit that the genes with the highest probability of causing AD, which they call the most causal genes (MCGs), must have the highest probability of activating the neurons with the highest probability of causing AD, the most causal neurons (MCNs). Using causal tracing, they perturb the embeddings of ten known AD-associated genes, measure the indirect effect of each hidden state on the AD prediction, and select the top hidden states as MCNs. They then compute, for each input position, a cumulative influence score by summing MCN effects weighted by self-attention weights layer by layer back to the input, and the input gene tokens with the highest averaged scores are designated the most causal tokens and their genes the MCGs. The resulting top-10 list is presented as novel causal genetic biomarkers for AD, supported by literature references and by pathway enrichment that flags Alzheimer's, oxidative phosphorylation, and related neurodegenerative and metabolic pathways.","pith_inferences":["If the equivalence between model activation and biological causation is only approximate, the top-10 list is better read as a ranked set of hypotheses than as established causal genes; the paper's own definition of 'causal' as 'most probable from a statistical perspective' already leans toward this weaker reading.","The procedure uses a single microglial cluster from one public cohort, so a natural stress test is to run the same backtracking on other clusters and other AD cohorts; a stable top-10 list across those settings would substantially strengthen the case that the genes are disease-relevant rather than cluster-specific.","The backtracking score depends on a 95th-percentile cutoff for selecting most causal neurons and on the noise level used for perturbations; probing how sensitive the top-10 list is to those hyperparameters would tell future users how much weight to place on the exact ranking."],"forward_implications":["The top-10 MCGs (PLXDC2, MT-CO3, DOCK4, ARHGAP24, MEF2A, RUNX1, ITPR2, FOXN3, MT-CO1, SORL1) become prioritized candidates for experimental validation in AD models.","The same neuron-to-token backtracking can be applied to other diseases by fine-tuning a genomic foundation model on a different phenotype and backtracking from that phenotype's most causal neurons.","Because most causal neurons concentrate in the early and middle layers of the model, the method suggests that causal signal for AD is distributed across low- and mid-level genetic interaction features rather than only in the final prediction layer.","The enriched pathways (Alzheimer's, Parkinson's, Huntington's, oxidative phosphorylation, thermogenesis) indicate that the candidate genes point to shared bioenergetic and neuroinflammatory mechanisms across neurodegenerative diseases."],"supporting_citations":[{"why":"Supplies the causal tracing technique used to identify most causal neurons by measuring indirect effects of hidden states.","marker":"Meng et al. 2022"},{"why":"Provides the pre-trained genomic foundation model and the rank-value encoding that represents genes as tokens.","marker":"Theodoris et al. 2023"},{"why":"Supplies the single-cell microglial transcriptomic dataset from an AD case/control study used for fine-tuning and analysis.","marker":"Sun et al. 2023"},{"why":"Defines the curated list of known AD-associated genes that are masked to locate most causal neurons.","marker":"Bertram et al. 2007"},{"why":"Describes the ROSMAP study cohort from which the single-cell expression data originate.","marker":"Bennett et al. 2018"},{"why":"Provides the STRING pathway enrichment analysis used to validate the biological relevance of the identified most causal genes.","marker":"Szklarczyk et al. 2019"}],"fun_headline_variants":["Reverse gene finder pinpoints top 10 Alzheimer's causing genes","Neuron backtracking exposes 10 causal Alzheimer's genes","Backtracking from neurons to genes ranks 10 AD candidates","Neuron-to-gene backtracking crowns 10 top Alzheimer's genes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the strength with which an input gene token activates the model's most influential neurons is a faithful ranking of that gene's real-world causal role in Alzheimer's disease.","fun_headline_variants_meta":{"raw":{"variants":["Reverse gene finder pinpoints top 10 Alzheimer's causing genes","Neuron backtracking exposes 10 causal Alzheimer's genes","Backtracking from neurons to genes ranks 10 AD candidates","Neuron-to-gene backtracking crowns 10 top Alzheimer's genes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000565,"raw_usage":{"total_tokens":2710,"prompt_tokens":1009,"completion_tokens":1701,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1628}},"tokens_in":625,"tokens_out":1701,"duration_ms":12074,"temperature":1.0,"reasoning_tokens":1628,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T00:10:39.799396+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same Reverse-Gene-Finder procedure on an independent AD single-cell dataset or on a different microglial cluster from the same cohort and check whether the top-10 most causal genes reproduce; if the list changes entirely, the top-10 result is an artifact of the specific model and data subset rather than a robust causal signal. A complementary check is to mask the identified top-10 genes and test whether the fine-tuned model's AD prediction shifts significantly beyond what random gene masking produces.","supporting_citations":[{"cited_title":"2023) and a large genomic foundation model (namely, Geneformer (Theodoris et al","cited_arxiv_id":null,"evidence_quote":"Supplies the single-cell microglial transcriptomic dataset from an AD case/control study used for fine-tuning and analysis."}],"review_version":1}