{"id":"149338be-9989-43c5-849a-5d7546d73569","arxiv_id":"2607.28212","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Inverted sparse self-attention plus global aggregation and permutation verification recovers multivariate Granger causal graphs better than six baselines on four benchmarks.","lead":"The paper introduces an inverted self-attention mechanism that treats each variable's full time series as a token, then uses sparse attention scores plus permutation checks to recover Granger-style causal graphs. It reports higher F1 than standard baselines on four synthetic and simulated multivariate time-series benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Sparse inverted-attention weights after free rank-k thresholding are not shown to equal Granger edges","rationale":"The reader correctly isolates the same load-bearing step: the map from inverted Sparsemax weights through global sum and rank-k threshold onto Granger edges (Assumptions 1–4 + Eqs. 5–10). Ablations in Fig. 4 show that removing inversion or Sparsemax hurts, yet they do not test whether the retained weights are causal rather than merely predictive, nor do they stabilize k. Sensitivity Table 2 varies heads/blocks but leaves the discrete threshold unexamined. No multi-seed statistics, code, or formal link to the Granger definition are supplied. Consequently the empirical superiority on the two non-linear benchmarks remains suggestive but conditional on the unproven identification and on favorable k. No stronger internal contradiction appears; the concern is precisely the one the reader flagged, so the CONDITIONAL verdict is unchanged.","tokens_in":10124,"tokens_out":623,"duration_ms":26504,"concrete_test":"On Lorenz-96 (and fMRI), sweep the rank parameter k across the full feasible range (top-5 % to top-50 % of possible directed pairs), re-train with 5 random seeds, and recompute Precision/Recall/F1 exactly as in Table 1. If the headline F1 = 0.769 (0.837) is attained only inside a narrow k-band that must be chosen with knowledge of the ground-truth edge count, or if mean F1 across the sweep falls to or below PCMCI/TCDF, the attention-to-edge identification is not robust and the outperformance claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim (highest F1 on Lorenz-96/fMRI via CSAM+global+PI) rests on equating non-zero Sparsemax scores on inverted tokens to Granger-causal edges. After token inversion (§4.1, Fig. 1b), attention is computed as e_i = Q̂_i K̂^T / √d_k, sparsified by Sparsemax (Eq. 5), aggregated by simple summation H_ij = Σ_t C^t_ij (Eq. 8), and hard-thresholded at the free rank-k value τ = sorted(H_ij)[k] (Eqs. 9–10, Alg. 1). Nothing in the construction or Assumptions 1–4 proves that the surviving (i→j) pairs satisfy the predictive-improvement definition of Granger causality rather than residual associations, common-driver artifacts, or optimization biases of the transformer. PI verification (§4.3) is applied only after this selection and cannot rescue edges that were never candidates. The reported gains therefore hinge on an unproven attention-to-causality leap plus an untuned discrete hyper-parameter k.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes a transformer-based framework for Granger-style causal discovery in multivariate time series. It introduces an inverted causal self-attention mechanism (CSAM) that treats each full series as a token, computes attention scores, applies Sparsemax for sparsity, aggregates scores via a global causal algorithm (Algorithm 1) that thresholds at the k-th largest strength, and verifies candidates with permutation importance. Experiments on Hénon, FINANCE, Lorenz-96 and fMRI report higher Precision/Recall/F1 than BGGranger, KGC, tsFCI, TCDF, PCMCI and DYNOTEARS, with ablations on n2n prediction and Softmax vs Sparsemax plus a small sensitivity table on heads/blocks.","tokens_in":10414,"tokens_out":1330,"duration_ms":45063,"significance":"If the empirical gains hold under tighter controls, the work is a useful incremental contribution to attention-based causal discovery for nonlinear multivariate time series, extending the line of TCDF-style methods with token inversion, Sparsemax, global aggregation and PI verification. Strengths include a fully specified pipeline (CSAM + Algorithm 1 + PI), consistent outperformance on the two nonlinear benchmarks (F1 0.769 / 0.837), and ablations (Fig. 4) plus sensitivity (Table 2) that support the two main design choices. The result is of practical interest for domains that already rely on Granger graphs, but the theoretical link from sparse inverted attention to true Granger edges remains empirical rather than proven.","major_comments":[{"comment":"§4.1 (Eqs. 2–3 and surrounding text) contains a load-bearing dimensional inconsistency: after inversion the authors state ˆK, ˆQ, ˆV ∈ R^{T×d}, yet immediately extract the i-th row of ˆQ for each time-series variable i and refer to N as the number of variables. Inverted tokens should yield matrices whose first dimension equals the number of series M, not T. This makes the construction of ei, αi and the causal tensor Cijt difficult to reproduce or verify.","section":"§4.1"},{"comment":"Table 1 reports single-point Precision/Recall/F1 with no error bars, multiple random seeds, or statistical tests. Several margins are small (e.g., Finance F1 0.586 vs PCMCI 0.556; Lorenz recall lower than PCMCI). Without variability estimates it is impossible to judge whether the claimed superiority is stable or an artifact of a single training run.","section":"Table 1"},{"comment":"The global causal algorithm (Algorithm 1, Eqs. 8–10) selects edges by a free rank parameter k that sets τ = sorted(Hij)[k]. The paper states that k is chosen “following [10]” but never ablates or reports sensitivity to k, unlike the heads/blocks analysis in Table 2. Because the entire candidate set P (and therefore the final graph after PI) is determined by this discrete threshold, the central empirical claim rests on an untuned hyper-parameter whose effect size is unknown.","section":"§4.2, Algorithm 1"},{"comment":"Assumptions 1–4 and the construction in §4.1–4.2 equate non-zero Sparsemax weights on inverted tokens (after summation and rank-k thresholding) with Granger-causal edges. Nothing in the derivation shows that surviving (i→j) pairs satisfy the predictive-improvement definition of Granger causality rather than residual associations or optimization artifacts. PI verification (§4.3) is applied only after selection and cannot recover edges that were never candidates; the attention-to-causality step therefore remains an unproven modeling assumption on which the reported gains depend.","section":"§4.1–4.3"},{"comment":"Assumption 3 (“if Xi causes Xj then i < j”) imposes a strict total order on variable indices. Variable labels are arbitrary; enforcing i < j either silently renumbers the series or incorrectly forbids legitimate edges. The algorithm itself does not appear to enforce this constraint, creating an internal inconsistency between the stated assumptions and the implemented procedure.","section":"§3, Assumption 3"}],"minor_comments":[{"comment":"Fig. 3 caption and color scheme (dark blue = true, red = error) are helpful, but the matrices themselves are small and hard to compare quantitatively; a numeric Hamming or SHD overlay would improve readability.","section":"Fig. 3"},{"comment":"Table 2 sensitivity is reported only for one dataset (apparently fMRI) and only for heads/blocks; window size, learning-rate schedule and dropout are fixed without comment.","section":"Table 2"},{"comment":"Several typos and spacing artifacts appear throughout (e.g., “acausalverificationmoduletoensurerobustness”, “intheidentifiedrelationships”, missing spaces after periods in the contributions list).","section":"§1"},{"comment":"Related-work discussion of neural Granger methods is thin; recent continuous-optimization and attention-based baselines beyond TCDF/DYNOTEARS could be acknowledged for completeness.","section":"§2"},{"comment":"Input/output window sizes are fixed at 10/1 with no justification relative to the intrinsic time scales of Hénon, Lorenz-96 or fMRI.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"Primary category cs.CL is a poor fit; the paper is squarely cs.LG / stat.ML / time-series causal discovery. The 2026 arXiv stamp is anomalous but irrelevant to content. Novelty relative to TCDF is real but modest; the main risks are reproducibility (dimensions, free k, single-run tables) rather than outright error. Major revision is appropriate; reject would be disproportionate given the empirical package and ablations."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that this is a solid, fully-specified neural Granger pipeline: invert the tokens so each series is a token, run Sparsemax on the cross-series scores, sum them into a global strength matrix, hard-threshold at rank-k, then filter survivors with permutation importance. That combination is not in TCDF or the other cited baselines, and Table 1 plus the fMRI matrix show clear F1 gains on the two nonlinear sets (0.769 Lorenz, 0.837 fMRI). Ablations in Fig. 4 and the small head/block table back the two design choices they claim matter.\n\nWhat is actually new is the packaging, not a new causal principle. Inverted tokens are already in the forecasting literature, Sparsemax is off-the-shelf, global rank thresholding follows TCDF’s style, and PI is standard post-hoc. They put the pieces together cleanly, write the algorithm out, and beat BGGranger/KGC/tsFCI/TCDF/PCMCI/DYNOTEARS on the four public benchmarks they chose. That is real incremental value for anyone already running attention-based Granger tools.\n\nSoft spots are the usual ones for this genre and should be weighed as such. Nothing in Assumptions 1–4 or Eqs. 5–10 proves that a non-zero Sparsemax weight after summation equals a Granger edge rather than residual predictive association or optimizer artifact; PI only checks the candidates that already survived the free rank-k cut. k itself, heads, and blocks are free parameters with only light sensitivity. No error bars, multi-seed numbers, or code. Circularity is mild—they train for next-step prediction and post-process—so the recovered graph is Granger by construction. None of this sinks the empirical claim; it just keeps the paper inside the normal neural-discovery envelope.\n\nThis is for people who need a stronger exploratory tool on high-dimensional nonlinear series (climate, finance, fMRI-style). It will not change theory or give interventional guarantees. I would send it to referees; the method is reproducible on paper, the gains are consistent, and the gaps are fixable with code and stats. Worth a look if you already cite TCDF or DYNOTEARS; skip if you only care about identification guarantees.","headline":"Clean engineering combo of inverted tokens + Sparsemax + rank aggregation + PI that beats standard baselines on the usual nonlinear suites, still resting on the usual unproven attention-to-Granger leap.","tokens_in":11039,"tokens_out":578,"would_cite":false,"duration_ms":22013,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Inverting the tokens in a transformer and sparsifying its attention scores recovers Granger-causal graphs in multivariate time series better than existing methods.","keywords":["causal discovery","multivariate time series","self-attention","transformer","Granger causality","Sparsemax","permutation importance"],"falsifier":"On a controlled synthetic system whose true Granger graph is known, replace Sparsemax with ordinary Softmax or disable the permutation check; if precision and F1 collapse to the level of the weaker baselines, the claimed advantage of the inverted sparse pipeline is falsified.","tokens_in":10999,"feed_emoji":"⏱️","tokens_out":842,"duration_ms":20109,"temperature":0.7,"pith_summary":"Causal discovery among many interacting time series is hard because ordinary correlations mix with true directed influences, especially when the links are nonlinear or high-dimensional. This paper claims that a simple change to the transformer’s self-attention block solves much of the problem: treat each variable’s entire history as one token, compute attention across variables rather than across time, and force the resulting scores to be sparse. The sparse scores are then summed globally and checked by a permutation test, yielding a directed graph of Granger causes. On both linear chaotic maps and nonlinear systems such as Lorenz-96 and simulated fMRI, the resulting graphs match ground truth more accurately than classic statistical tests and recent neural baselines. If the method generalizes, analysts in climate, finance and neuroscience could extract usable causal structure directly from the same architectures already used for forecasting.","feed_headline":"Inverted sparse attention recovers causal graphs in time series","feed_subtitle":"Token flip plus Sparsemax beats six standard methods on linear and nonlinear benchmarks","key_machinery":"The inverted causal self-attention mechanism (CSAM): each token is a complete univariate series; attention is computed across series, sparsified by Sparsemax, then globally thresholded and permutation-verified to produce the edge set.","core_discovery":"By inverting the usual tokenization of a multivariate series so that each token is one variable’s full trajectory, applying Sparsemax attention, aggregating the sparse scores into a global causal matrix, and verifying each retained edge with permutation importance, one obtains Granger-causal graphs that outperform BGGranger, KGC, tsFCI, TCDF, PCMCI and DYNOTEARS on precision, recall and F1 across four standard benchmarks, with the largest gains on the nonlinear sets.","pith_inferences":["Because the method never assumes linearity, it may serve as a drop-in replacement for kernel Granger causality in domains where the appropriate kernel is unknown.","The rank-k threshold choice is still a free hyper-parameter; an information-theoretic or stability-based rule for choosing k would make the pipeline fully automatic.","If contemporaneous causation is later allowed, the same inverted-token construction could be paired with a structural-equation layer to recover instantaneous edges."],"forward_implications":["Forecasting models that already use transformers can extract an explicit causal graph as a free by-product of training.","High-dimensional nonlinear systems (climate, BOLD fMRI, multi-asset finance) become amenable to automated causal screening without hand-crafted kernels or conditional-independence tests.","Sparsity induced by Sparsemax systematically reduces false-positive edges relative to dense attention baselines.","The same global aggregation-plus-verification step can be attached to other attention architectures to convert predictive attention into claimed causality."],"fun_headline_variants":["Inverted self-attention recovers causal graphs in multivariate time series","Token inversion plus Sparsemax yields Granger-causal graphs beating six baselines","CSAM sparse scores plus verification outperform BGGranger to DYNOTEARS","Variable-trajectory tokens and global causal matrix lift F1 on nonlinear sets","Inverted attention with permutation checks finds latent causal links in time series"],"cache_read_input_tokens":128,"weakest_assumption_plain":"Non-zero sparse attention weights obtained from the inverted tokens, once summed and rank-thresholded, correspond to true Granger-causal edges rather than leftover predictive associations or training artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Inverted self-attention recovers causal graphs in multivariate time series","Token inversion plus Sparsemax yields Granger-causal graphs beating six baselines","CSAM sparse scores plus verification outperform BGGranger to DYNOTEARS","Variable-trajectory tokens and global causal matrix lift F1 on nonlinear sets","Inverted attention with permutation checks finds latent causal links in time series"]},"model":"grok-4.5","effort":"low","cost_usd":0.004163,"raw_usage":{"total_tokens":1247,"prompt_tokens":721,"num_sources_used":0,"completion_tokens":74,"cost_in_usd_ticks":41628000,"prompt_tokens_details":{"text_tokens":721,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":452,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":721,"tokens_out":74,"duration_ms":8221,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T14:29:49.199270+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a controlled synthetic system whose true Granger graph is known, replace Sparsemax with ordinary Softmax or disable the permutation check; if precision and F1 collapse to the level of the weaker baselines, the claimed advantage of the inverted sparse pipeline is falsified.","supporting_citations":[],"review_version":1}