{"id":"a02af1ed-d1c0-434d-bd57-0f105c13c635","arxiv_id":"2505.04631","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"An ICA-based model on EHR data identified latent factors associated with 10-year cryptogenic stroke risk in migraine patients, with allergic rhinitis emerging as a candidate cause and medications as protective.","lead":"This paper uses machine learning on electronic health records to separate migraine patients' medical histories into hidden factors and then predict which patients will have a cryptogenic stroke within 10 years. The model was moderately accurate and pointed to medications as protective and allergic rhinitis as a possible risk factor, but the causal claims need stronger validation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal claim requires the evaluation-set sources to be the exact exogenous errors, but with k=2000 and m=9000 the mixing matrix is not invertible; the non-square projection makes SHAP values non-causal even if LiNGAM assumptions held.","rationale":"Reader's weakest assumption is essentially right: the causal claim turns on identifying ICA components with structural error terms. I agree, and I would keep the verdict conditional. My stress-test sharpens the same concern in a way that is checkable from the code and data: with m=9000 and k=2000, A is rectangular, so 'A^{-1}E' cannot be an exact inverse, and the evaluation-set projections are not the true independent errors unless an additional low-rank generative model holds exactly. The paper itself flags information loss from k=2000 in the Discussion, which I take as an in-scope limitation that reinforces this. Even if the projection were corrected, the LiNGAM assumptions remain untested; but that is the reader's point, so I only partially claim it as a new concern. The SHAP-to-causal step adds a second non sequitur (SHAP attributes the learned RF function, not the true interventional effect), but the projection issue is the more primary load-bearing failure. A reconstruction-error and independence check would settle whether the concern lands. The predictive claims and the hypothesis-generating allergic-rhinitis finding can still stand with appropriate tempering, so conditional acceptance remains appropriate; no verdict change.","tokens_in":11232,"tokens_out":11084,"duration_ms":118104,"concrete_test":"Inspect the saved FastICA model: report the shape of A and the exact transform applied to E. If A is not square, verify whether S_eval = pinv(A) E or S_eval = W E, and compute the relative reconstruction error ||E - A S_eval||_F / ||E||_F on the evaluation set. If this error is above a small threshold (e.g. 5%) or if the resulting S_eval rows fail a mutual-independence test (e.g. HSIC or dCov permutation test), the projected sources are not exact exogenous errors; the allergic-rhinitis SHAP finding is then an associational result, not a causal one. Additionally, re-running the pipeline with k set by a data-driven rank estimate (e.g., PCA elbow) and checking whether the top SHAP sources change would show how load-bearing the k=2000 constraint is.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Central claim requires that the S fed to the random forest (and later to SHAP) are the true exogenous error terms of an SCM. This is where the paper is least secure, for three concrete reasons. (1) LiNGAM identification (Shimizu et al. 2006) presupposes a square, invertible mixing of observed variables by as many independent non-Gaussian errors as there are variables. Here m=9000 and k=2000, so A is 9000x2000. The Methods state 'S = A^{-1}E'; no inverse exists. If a pseudo-inverse or the FastICA unmixing matrix is used instead, S_eval is a least-squares projection, not an exact recovery of independent errors, so the source expressions are not the structural error terms. (2) The choice k=2000 is a memory constraint, not a model-derived dimension; the Discussion admits this loses information and may merge multiple disease sources into one signature. Even under LiNGAM, dimension reduction invalidates the standard identifiability result: the recovered components are independent by construction, but need not be the root causes of the label. (3) The LiNGAM assumptions (linearity, acyclicity, no unobserved common causes) are asserted, not tested; with time-sampled repeated EHR rows and a future label, the i.i.d. and causal-sufficiency conditions are implausible. The SHAP values then measure the fitted random forest's association between the projected coordinates and the label, not an interventional causal effect. The predictive AUROC can survive all of these issues; the causal interpretation does not.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an unsupervised ICA-based decomposition of longitudinal EHR curves from roughly 310,000 neurology patients into 2000 latent 'disease sources', then projects a migraine cohort (N=72,876; 1,670 cryptogenic stroke cases) onto these sources and trains a random forest to predict 10-year cryptogenic stroke risk. The model achieves a test AUROC of 0.782. The authors interpret the ICA sources as the exogenous error terms of a LiNGAM structural causal model and interpret the SHAP values on the source expressions as quantitative causal effects, concluding that medication-related sources are protective and that an allergic rhinitis-related source is a potential cause of cryptogenic stroke in migraine patients. A secondary model for general ischemic stroke is used for comparison.","tokens_in":11580,"tokens_out":6587,"duration_ms":64629,"significance":"If the causal interpretation were valid, the paper would offer a scalable method for generating hypotheses about root causes of heterogeneous diseases from EHR data, and the specific findings would merit clinical follow-up. The study has genuine strengths: a large real-world cohort, a prospective sampling window relative to the stroke event, a described end-to-end pipeline, and a held-out test set. However, the central claim is not supported. The mixing matrix is non-square and non-invertible, the LiNGAM identifiability conditions are neither met nor tested, SHAP values are not causal effect estimators without additional assumptions, and the predictive evaluation lacks confidence intervals, calibration, and baseline comparators. As presented, the paper is best read as an exploratory predictive and associational study rather than a causal analysis, and the causal language would need to be substantially revised or removed.","major_comments":[{"comment":"The projection step is mathematically invalid as stated. The mixing matrix A is m×k with m=9000 and k=2000, so A^{-1} does not exist. If the authors instead used the FastICA unmixing matrix W or a pseudo-inverse, the resulting source expressions S_eval are a projection onto the learned subspace, not the exact exogenous error terms of a structural causal model. Since the label may depend on the omitted m-k components, the SHAP values computed on S_eval cannot be interpreted as causal effects of latent sources on cryptogenic stroke. This issue is load-bearing for the 'Latent causes' and 'Feature importance' sections and must be resolved before the causal claims can be considered.","section":"Model creation, S = A^{-1}E"},{"comment":"The paper invokes LiNGAM to equate ICA sources with exogenous error terms, but LiNGAM requires a square, invertible mixing of as many non-Gaussian error terms as observed variables, together with linearity, acyclicity, and no unobserved common causes. With k=2000 and m=9000, the identifiability theorem does not apply, and the Discussion itself acknowledges that the dimension reduction may merge multiple disease sources into one signature. The assumptions of linearity, acyclicity, and causal sufficiency are asserted rather than tested, and the longitudinal, repeatedly sampled EHR structure with a future label makes these assumptions implausible. Consequently, the claim that the recovered sources 'do in fact exert a causal relationship onto the final label' is unsupported.","section":"Latent causes (LiNGAM assumptions)"},{"comment":"SHAP values are additive feature attributions that decompose a model's prediction, and they do not generally estimate interventional causal effects. Even if the source expressions were true exogenous errors, a random forest trained on those expressions does not automatically provide causal effect estimates unless the causal structure, the functional form, and the absence of confounding are all correctly specified. The statement that 'the SHAP value of each input feature is a quantitative estimate of the causal effect of that latent source on the record's final label' is not justified by the cited literature and should be reframed as an associational measure.","section":"Feature importance (SHAP values)"},{"comment":"The predictive evaluation is incomplete: the test AUROC of 0.782 is reported without confidence intervals or calibration measures, and no comparator is trained on raw EHR features or a standard clinical risk score such as ASCVD. The 'external validation' in the Discussion is a comparison with published literature, not validation on independent data. Additionally, the authors candidly report that their sampling window made prior stroke appear protective; this demonstrates that study-design artifacts can dominate the SHAP interpretations, which weakens confidence in the remaining causal-sounding conclusions such as the allergic rhinitis source.","section":"Model performance and Discussion"}],"minor_comments":[{"comment":"The word 'hyperparemeter' should be 'hyperparameter'.","section":"Model creation"},{"comment":"The phrase 'quantify the the causal effect' contains a duplicated article and should read 'quantify the causal effect'.","section":"Discussion and Conclusions"},{"comment":"The Abstract reports ROC 0.771 while the Results report a test AUROC of 0.782 (Figure 4); the manuscript should clearly distinguish the cross-validated AUROC from the held-out test AUROC.","section":"Abstract and Results"},{"comment":"The header 'N◦' appears to be a typographical artifact; it should read 'N'.","section":"Table 1"},{"comment":"Reference [34], the custom FastICA implementation, should include a version number or an access date.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript's central methodological claim is invalid as written: the non-square mixing matrix makes the stated inverse projection impossible, and the LiNGAM identifiability conditions are not satisfied with k=2000 sources for m=9000 observed variables. The causal conclusions, including the allergic rhinitis finding, are therefore unsupported. A purely predictive and associational version of the study could be publishable in a suitable venue, but that would be a substantially different contribution requiring reanalysis and reframing, not minor edits."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—\n\nThe thing worth knowing about this paper is that it finds something plausible and new—a latent EHR signature dominated by allergic rhinitis that associates with cryptogenic stroke in migraine patients, and a phenytoin-dominated source that looks protective—but the causal machinery it uses to get there has a load-bearing mathematical error. The matrix A in the ICA decomposition is 9000×2000, and the paper says S = A^{-1}E. That inverse does not exist. Even if you read it as a pseudo-inverse, the projected sources are not the true exogenous error terms of a structural model; they're a least-squares fit into a 2000-dimensional subspace. So the SHAP values on those projections are not causal effects, no matter how cleanly the LiNGAM assumptions are stated.\n\nThat said, the paper does a lot right. The preprocessing of messy, asynchronous EHR data into longitudinal curves is careful and follows a published pipeline. The labeling algorithm for cryptogenic stroke is transparent and produces a prevalence that matches literature estimates. The model achieves AUROC 0.782 on a held-out test set, which is competitive with clinical risk scores. And the authors are unusually honest about their own limitations: they flag the memory constraint that forced k=2000, the information loss from dimensional reduction, and they devote a whole paragraph to the recurrent-stroke sampling artifact that reverses the direction of a known risk factor. That kind of openness is worth crediting.\n\nThe soft spots, in proportion: the causal claim is the main one, and it's not minor. The LiNGAM assumptions (linearity, acyclicity, causal sufficiency) are asserted, not tested, and with repeated time-sampled rows per patient and a future label, the i.i.d. assumption is implausible. The \"external validation\" is really a comparison to the literature, not an independent dataset. And there are no confidence intervals or a direct baseline against raw EHR features, so we don't know if the source projection is actually buying predictive power.\n\nWho's this for? Someone building hypothesis generators from EHRs will find the pipeline instructive and the allergic rhinitis lead worth chasing. But the paper shouldn't be taken as causal evidence as it stands. It deserves a serious referee—the empirical result is genuinely interesting and the core method, properly fixed (use the square unmixing matrix or explicitly treat the projection as approximate, and temper the causal language), could be publishable. Right now it overclaims.","headline":"A promising EHR hypothesis-generation result undermined by a non-invertible mixing matrix that breaks the causal interpretation.","tokens_in":12119,"tokens_out":2289,"would_cite":false,"duration_ms":21537,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62P10","62H25"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that latent causes of cryptogenic stroke in migraine patients can be recovered from electronic health record data as probabilistically independent sources, so that SHAP scores on those sources estimate causal effects…","keywords":["cryptogenic stroke","migraine","electronic health records","independent component analysis","latent causal sources","SHAP values","LiNGAM","risk prediction"],"falsifier":"Feed the same pipeline synthetic EHR data generated from a known causal graph with known cryptogenic-stroke incidence: if the top-SHAP sources do not match the planted causes, the causal interpretation fails. Clinically, a prospective study or natural experiment showing that treating allergic rhinitis or removing decongestants does not change cryptogenic stroke incidence would refute the allergic-rhinitis-as-cause claim.","tokens_in":10999,"feed_emoji":"🧠","tokens_out":4755,"duration_ms":43081,"temperature":0.7,"pith_summary":"The paper tries to show that latent causes of cryptogenic stroke in migraine patients can be recovered from messy electronic health record data without human-curated features. It uses independent component analysis on 9000 clinical variables to extract 2000 probabilistically independent sources, then treats those sources as the independent error terms of a causal model, so that a random forest's SHAP scores on those sources estimate causal effects. On a held-out test set the model achieves an AUROC of 0.782 for 10-year cryptogenic stroke risk, with cross-validated AUROC 0.771. The top predictive sources are mostly protective medication signatures, and one source tied to allergic rhinitis is consistently associated with increased risk, which the authors interpret as a candidate root cause. If this interpretation is right, the method turns routine EHR data into a hypothesis generator for disease causes.","feed_headline":"Allergic rhinitis surfaces as a candidate cryptogenic stroke cause","feed_subtitle":"Machine learning on EHR data finds a consistent allergic-rhinitis signal for cryptogenic stroke in migraine patients.","key_machinery":"The central object is the ICA decomposition $X = AS$, where $X$ is the stacked EHR data matrix, $S$ holds the 2000 probabilistically independent source expressions, and $A$ is the mixing matrix whose columns are clinical signatures. The machinery links these sources to the independent error terms of a LiNGAM structural equation model, then uses SHAP values computed on a random forest trained on $S$ to quantify each source's causal contribution to the cryptogenic stroke label.","core_discovery":"Under the LiNGAM assumptions, the paper claims that the 2000 probabilistically independent sources recovered by ICA from EHR data correspond to the exogenous error terms of the structural causal model that generated the observations. Because causal relations are transitive, these error terms exert a causal relationship onto the final cryptogenic stroke label, and the SHAP value of each source is therefore a quantitative estimate of its causal effect on a patient's predicted risk. Applying this to 72,876 migraine patients, the model found that the largest protective effects come from common preventive medications, while a source characterized by allergic rhinitis consistently increases cryptogenic stroke risk, albeit by a small amount.","pith_inferences":["A natural next experiment the paper does not run: compare cryptogenic stroke incidence in migraine patients with and without allergy immunotherapy or decongestant use; a difference would strengthen the allergic-rhinitis causal claim, while no difference would weaken it.","Because the 2000-source ICA was truncated by memory, some discovered signatures may merge several distinct causes; re-running with more sources could split the allergic-rhinitis signature into separate inflammation and decongestant components.","The causal interpretation depends on EHR documentation capturing the real causes; patients who use over-the-counter decongestants without recording them would bias the allergic-rhinitis source, a limitation the paper itself notes.","If the LiNGAM assumptions are violated in real EHR data, the SHAP ranks still describe predictive associations and could remain useful for risk stratification, but not for causal intervention."],"forward_implications":["If the causal reading holds, the highest-ranked protective sources imply that common preventive medications—antiplatelet, lipid-lowering, and antihypertensive regimens—are the largest modifiable factors reducing cryptogenic stroke risk in migraine patients.","The allergic-rhinitis source, though small in effect, is consistently positive, which the paper identifies as a candidate root cause that should be investigated prospectively.","Patient-level SHAP waterfall explanations become actionable: a clinician could see which latent sources push a given patient's 10-year stroke risk up or down.","The same ICA-plus-SHAP pipeline can be applied to other disease labels, turning the EHR into a root-cause hypothesis generator.","The apparent protective effect of prior stroke is a sampling artifact of excluding post-stroke windows, so recurrent-stroke risk is not captured by this model."],"supporting_citations":[{"why":"Supplies the LiNGAM structural-equation assumptions that make the ICA sources exogenous error terms with causal meaning.","marker":"[36]"},{"why":"Provides the data-driven method for discovering clinical signatures and training causal models from EHR data.","marker":"[22]"},{"why":"Establishes the probabilistic-independence signature discovery and preprocessing approach used here.","marker":"[26]"},{"why":"Supports the identification of independent error terms as patient-specific root causes and justifies using SHAP on those terms as causal-effect estimates.","marker":"[27]"},{"why":"Defines independent component analysis and the independence criterion used to disentangle the sources.","marker":"[33]"},{"why":"Defines SHAP values, the tool used to quantify each source's effect on the model's prediction.","marker":"[37]"},{"why":"Provides the de-identified EHR data source that all analyses are built on.","marker":"[29]"}],"fun_headline_variants":["Allergic rhinitis tied to cryptogenic stroke risk","Migraine-stroke link: allergic rhinitis emerges as factor","EHR analysis flags allergic rhinitis in cryptogenic stroke","Machine learning points to allergic rhinitis in stroke","Latent sources reveal allergic rhinitis in cryptogenic stroke"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the EHR observation network is linear, non-Gaussian, acyclic, and free of unmeasured common causes outside the 2000 extracted sources, so that each ICA source really is an independent exogenous cause rather than an algebraic artifact.","fun_headline_variants_meta":{"raw":{"variants":["Allergic rhinitis tied to cryptogenic stroke risk","Migraine-stroke link: allergic rhinitis emerges as factor","EHR analysis flags allergic rhinitis in cryptogenic stroke","Machine learning points to allergic rhinitis in stroke","Latent sources reveal allergic rhinitis in cryptogenic stroke"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000159,"raw_usage":{"total_tokens":1171,"prompt_tokens":831,"completion_tokens":340,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":261}},"tokens_in":447,"tokens_out":340,"duration_ms":3490,"temperature":1.0,"reasoning_tokens":261,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:12:41.272959+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed the same pipeline synthetic EHR data generated from a known causal graph with known cryptogenic-stroke incidence: if the top-SHAP sources do not match the planted causes, the causal interpretation fails. Clinically, a prospective study or natural experiment showing that treating allergic rhinitis or removing decongestants does not change cryptogenic stroke incidence would refute the allergic-rhinitis-as-cause claim.","supporting_citations":[{"cited_title":"Fast and robust fixed-point algorithms for independent component analysis","cited_arxiv_id":null,"evidence_quote":"Supplies the LiNGAM structural-equation assumptions that make the ICA sources exogenous error terms with causal meaning."},{"cited_title":"Distilling knowledge from deep networks with applications to healthcare domain; 2015","cited_arxiv_id":null,"evidence_quote":"Provides the data-driven method for discovering clinical signatures and training causal models from EHR data."},{"cited_title":"Why do probabilistic clinical models fail to transport between sites","cited_arxiv_id":null,"evidence_quote":"Establishes the probabilistic-independence signature discovery and preprocessing approach used here."},{"cited_title":"Unsupervised Discovery of Clinical Disease Signatures Using Probabilistic Independence","cited_arxiv_id":"2402.05802","evidence_quote":"Supports the identification of independent error terms as patient-specific root causes and justifies using SHAP on those terms as causal-effect estimates."},{"cited_title":"Scaling regression inputs by dividing by two standard deviations","cited_arxiv_id":null,"evidence_quote":"Defines independent component analysis and the independence criterion used to disentangle the sources."},{"cited_title":"A linear non-gaussian acyclic model for causal discovery","cited_arxiv_id":null,"evidence_quote":"Defines SHAP values, the tool used to quantify each source's effect on the model's prediction."}],"review_version":1}