{"id":"83c2726e-9e91-426d-8b33-10d2b6694485","arxiv_id":"2505.19355","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper proposes causal-Mamba, a joint treatment-outcome deep sequential model, to estimate average treatment effects of Google Trends-like external signals on social media engagement and to rank influential users.","lead":"This paper builds a joint model of external attention signals and engagement to predict social media engagement under hypothetical policy changes, and claims this yields a causal measure of online influence. The authors report that their model beats prior baselines by 15-22% and that their causal effect scores align better with an expert-based influence gold standard than follower counts do.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Assumption 2 (no hidden pathways) is the load-bearing link: it is defended by predictive fit alone, contradicted in §6, and never backed by the sequential exchangeability G-computation requires; without a placebo-signal test, the Table 3 ATEs may be artifacts of shared confounders.","rationale":"The paper makes two linked empirical claims: (1) the counterfactual prediction differences are valid ATEs, and (2) the resulting influence scores rank users better than follower counts when checked against expert-based empirical influence. Claim (1) is foundational; if it fails, claim (2) merely shows that a predictive score correlates with human judgments, which is exactly the correlation-versus-causation conflation the introduction warns against. I therefore concentrated on the identification argument. The chain is weakest at Assumption 2. Three specific problems: its defense cites the predictive performance of OMM and HIP, but predictive fit cannot establish the absence of unobserved confounders; Section 6 concedes unobserved confounding from algorithmic amplification and treatment interference through network effects, which are the hidden pathways Assumption 2 rules out, making the manuscript internally contradictory; and the 'G-computation' label in Table 3 is not supported by the Section 3.3 procedure, which is written as a conditional outcome difference under a transformed signal, with no statement of the sequential ignorability that g-methods require. The placebo test targets exactly this: a non-zero ATE for a future-shifted or unrelated signal demonstrates that the estimator attributes causal effect to signals that cannot have one. I considered other problems but judged them secondary: the 492-user validation is underdescribed and appears to mix X/Twitter users with a model trained on Facebook posts, and the abstract's 15–22% improvement is not cleanly derivable from Table 1 (averaged over the nine counterfactual columns, Mamba+Adapter RMSE is about 0.192 versus 0.187 for Transformer+Adapter). These are reporting problems that could be fixed; a failed placebo test would invalidate the method itself. Credit where due: adapting the joint treatment-outcome idea of Hizli et al. (2023) to discrete-time deep sequence models is a sensible contribution, the four integration mechanisms are compared systematically, and the per-dataset appendix tables are internally consistent with the averaged results. None of that supports the causal label, however. Because my read agrees with the reader's weakest-assumption analysis, the REJECT verdict stands unchanged.","tokens_in":20902,"tokens_out":13457,"duration_ms":124343,"concrete_test":"Re-run the CF1/CF2/CF3 pipeline with two falsification treatments: (a) the same Google Trends series time-shifted to lie entirely before the post's creation time, and (b) a matched placebo keyword series (unrelated topic) aligned to the same timestamps. Under Assumptions 1–3 and correct specification, both estimated ATEs must be statistically indistinguishable from zero, since neither can causally precede or drive engagement. If the model yields substantial non-zero ATEs with bootstrap CIs excluding zero for either placebo, the Table 3 effects reflect shared temporal processes or learned autocorrelation rather than causal effects of the signal, and the Section 5 'causal effect influence' ranking loses its causal interpretation. This negative-control test requires no new data, only rerunning the existing pipeline with permuted inputs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim — that the counterfactual prediction differences in Table 3 are valid ATEs — requires sequential exchangeability (no unobserved confounding given the modeled history), the condition that actually licenses G-computation. That condition is never stated or tested. Assumption 2 (Fully-Mediated Policy Effect, §3.2) is its stand-in, and it fails on three counts. (i) Its defense is a category error: 'alternative pathways might exist — such as algorithmic amplification or coordinated network activity — these appear to be minimal, as evidenced by both OMM's and HIP's superior predictive performance.' Predictive fit cannot rule out unobserved confounders; a well-fitting model can still produce badly biased causal estimates. (ii) The manuscript contradicts its own assumption: §6 concedes 'unobserved confounding from algorithmic amplification' and 'treatment interference through network effects' — precisely the hidden pathways Assumption 2 denies. (iii) The estimand is mislabeled: §3.3 defines the effect as E[Y|G_C] − E[Y|G], a conditional outcome regression under a transformed signal, with no demonstrated forward simulation or history marginalization, so the 'G-computation' label in Table 3 is not supported by the described procedure. If Google Trends and engagement share any common driver not in the conditioning set — a news event, an algorithm change, coordinated activity — the Table 3 ATEs and the §5 'causal effect influence' scores inherit that bias, and the influence ranking is not causal. This concern is internal to the manuscript's own stated limitations, not a disagreement with outside consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a joint treatment-outcome framework that couples deep sequential models (Transformers and Mamba) with Google Trends signals to predict social media engagement under counterfactual manipulations of those signals. The authors define a counterfactual scenario as a temporal transformation of the Google Trends input, compute the difference between model predictions under transformed and observed inputs, and label this difference an Average Treatment Effect (ATE) obtained by G-computation. They report predictive improvements over baselines across several counterfactual scenarios, and a case study on 492 X/Twitter users in which their 'causal effect influence' scores correlate more strongly with an expert-based empirical influence measure than do follower counts. The paper concludes that the approach offers a reliable, scalable causal measure of online influence.","tokens_in":21189,"tokens_out":6109,"duration_ms":39173,"significance":"The intended contribution is substantial: a scalable causal influence measure and a counterfactual prediction tool for misinformation intervention would be valuable, and the paper addresses a real gap in social media influence research. Strengths include the use of multiple real-world misinformation/disinformation datasets (SocialSense, DiN), the systematic comparison of eight architectural variants, bootstrap confidence intervals in Table 3, and a direct comparison against a human-judgment-based influence benchmark. However, the causal identification is not established, and the key empirical validation is incompletely documented. The significance of the paper therefore depends on a load-bearing causal claim that the current evidence does not support.","major_comments":[{"comment":"The paper's central claim that the counterfactual prediction gaps in Table 3 are valid Average Treatment Effects requires sequential exchangeability, i.e., no unobserved confounding given the modeled history, the condition that licenses G-computation. Assumption 2 (Fully-Mediated Policy Effect) is the only place this condition is addressed, and it is defended by predictive performance ('superior predictive performance' of OMM and HIP), which cannot rule out hidden pathways. Section 6 then concedes 'unobserved confounding from algorithmic amplification' and 'treatment interference through network effects,' directly contradicting Assumption 2. No placebo-signal test or sensitivity analysis is provided, so the ATEs in Table 3 and the 'causal effect influence' scores in Section 5 are not identified as causal effects.","section":"Section 3.2 and Table 3"},{"comment":"The estimand is stated as ΔC = E[Y|G_C] − E[Y|G], which is a conditional outcome regression under a transformed input signal. The 'G-computation' label in Table 3 is not supported by the described procedure: there is no explicit marginalization over the engagement history H, no forward simulation of the treatment process under the counterfactual policy, and no demonstration that the outcome model is used as an estimator of the g-formula. Either a proper G-computation estimator should be implemented and described, or the quantity should be relabeled as a model-based counterfactual prediction and the causal language should be adjusted accordingly.","section":"Section 3.3 and Table 3"},{"comment":"Tables 1 and 2 report single-point RMSE and BCE values without error bars, confidence intervals, or significance tests, even though Table 3 reports bootstrap intervals. The abstract's claim of '15–22%' improvement over existing benchmarks is not directly recoverable from the reported numbers (for example, Mamba+Adapter has base RMSE 0.113 versus MBPP and Transformer at 0.193, which is a much larger margin), and no statistical support is given for the ranking of architectures. This weakens the conclusions drawn for RQ1 and RQ2.","section":"Section 4.3, Tables 1 and 2"},{"comment":"The case study on 492 X/Twitter users does not document how the model's inputs are constructed: which Google Trends keywords are used for these users, how each user's posting behavior is encoded as a treatment signal, what observation and prediction windows are used, and how the per-user causal effect score is aggregated across posts. The empirical influence scores come from a separate human-judgment study on anti-climate-change topics, and the matching procedure between that study's users and the model's input data is not described. Without this documentation, the reported correlations (ρ=0.57 vs. 0.49, W=0.70 vs. 0.67, CCC=0.21 vs. 0.00) cannot be reproduced or assessed.","section":"Section 5"}],"minor_comments":[{"comment":"The sentence 'These signals, collected over l time points, serve as an exogenous signal, capturing how real-world interest.' is incomplete and should be finished, for example with 'evolves over time.'","section":"Section 2"},{"comment":"The text refers to 'Fig. 4' when describing the decile heatmaps of Spearman correlation, but the heatmaps appear as Figure 3; later, 'Figure 4' is used for the relative percentage changes in engagement. The figure numbering should be made consistent.","section":"Section 5"},{"comment":"The phrase 'Mamba+Adaptors, dubbed causal-Mamba' should read 'Mamba+Adapter' to match the naming used elsewhere.","section":"Section 4.3"},{"comment":"The column header 'Scenario 3: Treatment' in Table 2 should be 'Scenario 3: Duration' to match Table 1.","section":"Tables 1 and 2"},{"comment":"The hyperparameters α=0.5 and β=0.1 appear only in the appendix; β is not defined in the main text's loss function or in Section 3.4, leaving the temporal-coherence and attention-consistency losses underspecified.","section":"Appendix A.2"},{"comment":"The caption contains the typo 'Effect Effect Scores'; this should be 'Causal Effect Scores'.","section":"Figure 3"}],"recommendation":"reject","confidential_remarks":"The manuscript relies heavily on the authors' own prior models (HIP, OMM) to justify the causal assumptions, which makes the lack of external validation more consequential; the causal identification issue is central and cannot be resolved by minor edits. I also note that the paper's topic and methods are closer to computational social science and causal inference than to core computational linguistics, so the fit with a cs.CL venue should be argued explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a serious predictive modeling paper that overreaches on causality. The genuinely new and competent part is the discrete-time joint treatment-outcome architecture for social media engagement, with four Transformer and four Mamba integration mechanisms systematically compared. That is a legitimate extension of Hızlı et al. (2023), and the predictive experiments are the strongest part of the paper. The case study comparing causal effect scores against empirical influence (ρ = 0.57 vs 0.49 for followers) is interesting if it holds up.\n\nThe soft spots, in order of severity. The causal interpretation rests entirely on Assumption 2, which denies hidden pathways. Its defense is a category error: the authors argue that OMM's and HIP's predictive performance shows hidden pathways are minimal. Predictive fit cannot rule out unobserved confounding. Worse, Section 6 explicitly concedes \"unobserved confounding from algorithmic amplification\" and \"treatment interference through network effects\" — precisely what Assumption 2 denies. The paper contradicts its own load-bearing assumption. Second, the \"G-computation\" label in Table 3 is not earned: the estimand is defined as E[Y|G_C] − E[Y|G], a conditional outcome regression difference, with no demonstrated forward simulation or history marginalization. As defined, the ATEs are at best sensitivity analyses of the fitted model, not causal effects. Third, the case study is underdescribed: it is not clear how the 492 X/Twitter users with empirical influence scores map onto the Facebook posts in SocialSense and the 41 DiN accounts. Without that mapping, the Spearman correlations do not establish what they claim. Tables 1 and 2 also lack error bars, and the abstract's 15–22% improvement claim is not cleanly derivable from the tables. No code or data are provided, which makes the predictive results harder to verify.\n\nWho this is for: readers working on social media engagement prediction will find the architecture comparison genuinely useful. Readers wanting causal influence estimates should not take Table 3 at face value. The citation pattern is reasonable, and the paper engages seriously with the prior literature.\n\nRecommendation: send to peer review. A serious referee can separate the solid predictive core from the unsupported causal shell, and the authors could be pushed to reframe the contributions as counterfactual prediction rather than causal effect estimation. As it stands, the causal claims need major revision, but the paper deserves referee time.","headline":"Solid engagement-prediction study wearing a causal-inference costume that doesn't fit; worth peer review, but the ATE claims need serious reframing.","tokens_in":21757,"tokens_out":2131,"would_cite":false,"duration_ms":20215,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that online influence can be measured causally by modeling external temporal signals as treatments, and that the resulting causal effect scores rank influential users better than follower counts.","keywords":["causal inference","social media influence","counterfactual analysis","average treatment effect","misinformation","engagement prediction","Mamba","G-computation"],"falsifier":"Carry out a randomized field experiment in which identical posts receive randomly assigned exposure intensities or timings (for example, a platform boosts some posts with promotion while others are held out), and compare the experimentally measured effect of the exposure change with the ATE estimated from passive Google Trends data; systematic divergence would refute the causal claim. A cheaper check is to rerun the model across a known feed-ranking algorithm change and look for a sudden jump in prediction error, which would indicate an unmodeled hidden pathway.","tokens_in":20653,"feed_emoji":"📈","tokens_out":7122,"duration_ms":51394,"temperature":0.7,"pith_summary":"The paper sets out to establish that true online influence can be measured causally: external temporal signals such as Google Trends spikes are treated as time-varying treatments, and a user's influence is the Average Treatment Effect of their posting activity on later engagement. To do this it adapts a joint treatment-outcome framework from healthcare causal inference to discrete-time social media data, fitting deep sequential models that jointly predict treatment intensity and engagement. Reported results on misinformation and disinformation datasets show 15–22% better engagement prediction than existing benchmarks under counterfactual scenarios that vary exposure, timing, and duration. A case study on 492 users reports that the resulting causal-effect scores agree with the expert-based empirical influence better than follower counts do, and that low-baseline misinformation narratives respond super-linearly to external promotion.","feed_headline":"Causal model outranks follower counts for online influence","feed_subtitle":"Counterfactual engagement predictions beat baselines by 15–22% and align with expert influence rankings.","key_machinery":"The load-bearing mechanism is the joint treatment-outcome model: a deep sequential architecture with two heads, one predicting the binary event intensity of external signals (a square-transformed function of baseline, past treatments, past outcomes, and Google Trends windows) and one predicting engagement, trained with a combined loss of mean squared error and binary cross-entropy plus a temporal-coherence or attention-consistency regularizer. Causal quantities are obtained by G-computation: after applying the counterfactual manipulation function $\\Psi_\\theta$ that shifts each signal's timing by $\\delta_\\theta$ and scales its intensity by $\\gamma_\\theta$, the Average Treatment Effect is $\\Delta_C = \\mathbb{E}[Y \\mid G_C] - \\mathbb{E}[Y \\mid G]$. The instantiation that carries the results is causal-Mamba—Mamba, a selective state-space sequence model, augmented with low-rank adapters that condition state transitions on external signal intensities.","core_discovery":"The paper's central discovery claim is that counterfactual prediction differences, computed by transforming observed Google Trends signals through a temporal manipulation function and taking the difference in expected engagement, are valid Average Treatment Effects of external signals on engagement. Under three stated assumptions—consistency, fully-mediated policy effect, and temporal precedence—the joint treatment-outcome model (dubbed causal-Mamba in its best-performing Mamba-plus-adapter instantiation) is presented as identifying these effects from observational data. The evidence offered is threefold: the model outperforms baselines by 15–22% in engagement prediction; ATEs grow super-linearly with exposure, rise with earlier timing, and increase with longer duration; and causal-effect influence scores align with the expert-based gold standard on 492 users (Spearman $\\rho = 0.57$, Kendall's $W = 0.70$, CCC $= 0.21$), whereas follower counts align more weakly ($\\rho = 0.49$, $W = 0.67$, CCC $= 0.00$). The paper's own conclusion is that causal effect is a tighter approximation of influence than account popularity.","pith_inferences":["Editorial inference: The alignment with expert-influence rankings was tested on 492 users discussing anti-climate topics; the paper does not test other topics, so a natural extension is to collect expert rankings for vaccination or election narratives and check whether the ATE ranking still dominates follower counts there.","Editorial inference: The super-linear ATE pattern suggests an epidemic-threshold structure in attention-driven engagement; one could extend the framework to predict ex ante which low-baseline narratives will cross the amplification threshold under a forecast Google Trends shock, which the paper does not formulate.","Editorial inference: The strongest untested validation would be a randomized exposure experiment—staggered or boosted promotion of identical posts—comparing experimentally measured effects with the model's passive-data ATEs; the paper reports no such comparison.","Editorial inference: Because Assumption 2 excludes algorithmic amplification as a hidden pathway, the framework could be stress-tested by locating known platform ranking-algorithm changes and checking whether causal-Mamba's prediction error jumps at those change points; an error jump would signal an unmodeled confounder."],"forward_implications":["If the causal interpretation holds, counterfactual engagement forecasts become usable for policy design: platforms could estimate the engagement reduction from lowering exposure to a misinformation signal before intervening.","The ATE-based influence score can be computed at scale from passive observations, removing the bottleneck of the expert-based empirical influence, which currently exists for only 492 users.","Moderation and platform governance could prioritize users, pages, and groups by measured causal effect rather than follower counts, which the paper finds capture a fundamentally different and weaker signal of influence.","The reported super-linear response of low-baseline narratives to external signals implies that small Google Trends spikes can trigger disproportionately large engagement bursts, so early intervention timing matters more for low-prominence misinformation than for already-popular content."],"supporting_citations":[{"why":"Supplies the joint treatment-outcome formulation from healthcare that the paper adapts to discrete-time social media data.","marker":"Hızlı et al. (2023)"},{"why":"Provides the expert-based empirical influence scores on 492 users used as the gold standard for the influence-ranking comparison.","marker":"Ram & Rizoiu (2024)"},{"why":"Evidence that Google Trends signals drive engagement through an attention-mediated pathway, supporting the treatment interpretation.","marker":"Calderon et al. (2024)"},{"why":"HIP model establishing that external stimuli produce consistent engagement responses, supporting the consistency assumption.","marker":"Rizoiu et al. (2017)"},{"why":"G-computation, the estimator used to compute the Average Treatment Effects from the fitted joint model.","marker":"Robins (1986)"},{"why":"Mamba, the selective state-space architecture whose adapter variant becomes causal-Mamba.","marker":"Gu & Dao (2024)"},{"why":"Transformer, the baseline architecture family the paper extends and compares against.","marker":"Vaswani et al. (2017)"},{"why":"MBPP, the marked-point-process baseline that the joint model must outperform.","marker":"Rizoiu et al. (2022)"},{"why":"SocialSense, the misinformation dataset of Facebook posts used for training and the influence case study.","marker":"Kong et al. (2022)"},{"why":"DiN, the disinformation dataset from coordinated accounts used in experiments.","marker":"Tian et al. (2025)"}],"fun_headline_variants":["Causal model beats follower counts for influence","Counterfactuals improve engagement prediction by 15-22%","ATEs from time series align with expert influence","Influence is causal, not just popularity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that external signals such as Google Trends changes affect engagement only through the observable pathways the model captures, with no hidden routes such as algorithmic amplification or coordinated network activity; if hidden confounders exist, the estimated Average Treatment Effects are biased and the resulting influence ranking is not truly causal.","fun_headline_variants_meta":{"raw":{"variants":["Causal model beats follower counts for influence","Counterfactuals improve engagement prediction by 15-22%","ATEs from time series align with expert influence","Influence is causal, not just popularity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1497,"prompt_tokens":929,"completion_tokens":568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":508}},"tokens_in":545,"tokens_out":568,"duration_ms":5937,"temperature":1.0,"reasoning_tokens":508,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:15:53.127252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Carry out a randomized field experiment in which identical posts receive randomly assigned exposure intensities or timings (for example, a platform boosts some posts with promotion while others are held out), and compare the experimentally measured effect of the exposure change with the ATE estimated from passive Google Trends data; systematic divergence would refute the causal claim. A cheaper check is to rerun the model across a known feed-ranking algorithm change and look for a sudden jump in prediction error, which would indicate an unmodeled hidden pathway.","supporting_citations":[{"cited_title":"Opinion market model: stemming far-right opinion spread using positive interventions","cited_arxiv_id":null,"evidence_quote":"Evidence that Google Trends signals drive engagement through an attention-mediated pathway, supporting the treatment interpretation."},{"cited_title":"Slipping to the E xtreme: A M ixed M ethod to E xplain H ow E xtreme O pinions I nfiltrate O nline D iscussions","cited_arxiv_id":null,"evidence_quote":"SocialSense, the misinformation dataset of Facebook posts used for training and the influence case study."},{"cited_title":"Before It's Too Late: A State Space Model for the Early Prediction of Misinformation and Disinformation Engagement","cited_arxiv_id":"2502.04655","evidence_quote":"DiN, the disinformation dataset from coordinated accounts used in experiments."}],"review_version":1}