{"id":"61d57cb7-9af9-45f3-a429-6e8e15065e50","arxiv_id":"2502.04655","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"IC-Mamba, a Mamba-based model with interval-censored temporal inputs, predicts post and narrative engagement on Facebook and claims early identification of misinformation and disinformation.","lead":"This paper presents IC-Mamba, a state space model that predicts Facebook engagement (likes, shares, comments, and emoji reactions) from irregularly timed observation data. It aims to identify viral misinformation within the first minutes after posting and to forecast how a narrative spreads over weeks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pretraining on all 1.78M posts (Sec. 3.5) may include the test-period 15% of the chronological split described in Sec. 4.2; if so, the reported 4.72% improvement is a leakage artifact and the central claim fails.","rationale":"I read the paper in good faith. The model is well-motivated: interval-censored engagement data, temporal embeddings, and a two-tier post/opinion architecture are sensible, and code plus a dashboard are provided, which are genuine assets. The reader's verdict identifies the evaluation protocol as the weakest link. I agree that the pretraining stage is the critical hinge. Section 3.5's D is 'collected from the two datasets' without any stated temporal restriction, and Section 4.2's split is introduced only in the experimental setup for the fine-tuning stage; the paper never explicitly says the same split is imposed during pretraining. Because pretraining is self-supervised on engagement timelines, including test-period timelines would let IC-Mamba adapt to the actual posts being evaluated, so the small but consistent RMSE advantage over Mamba in Table 3 could be an artifact. This is not an accusation; it is a decidable protocol question, and the released code makes the test concrete. I also note secondary concerns (no variance despite ten seeds; unstated engagement normalization; abstract's RMSE values vs Fig. 5a), but none is as decisive as the pretraining split. If the code check shows the split is enforced, the central claim should be reassessed rather than rejected; until then, REJECT is appropriate. Hence verdict unchanged.","tokens_in":17964,"tokens_out":8347,"duration_ms":71615,"concrete_test":"Inspect the released code at https://github.com/ltian678/ic-mamba, specifically the pretraining DataLoader used by Algorithm 1 and the temporal-split logic of Sec. 4.2. Confirm whether the pretraining batches are restricted to the earliest 70% of posts (chronologically) and exclude the validation/test 30%. If not, re-run the whole pipeline with pretraining restricted to the training split only, and recompute Table 3. If the IC-Mamba vs Mamba RMSE gap disappears or reverses (e.g., Bushfire no longer 0.118 vs 0.124), the reported 4.72% improvement was due to test-set leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The evaluation's validity rests on the temporal holdout described in Sec. 4.2: posts are chronologically ordered and split 70/15/15 into train/validation/test. However, Sec. 3.5 defines the pretraining corpus D as 1.78M posts 'collected from the two datasets' - which, per Table 2, is all posts in SocialSense and DiN combined. Nothing in Algorithm 1 or Sec. 3.5 restricts pretraining to the earliest 70% of posts. If pretraining sees the engagement timelines of test-period posts, the model is adapted to the very trajectories it is later asked to forecast. The gap in Table 3 (e.g., Bushfire RMSE 0.118 vs Mamba 0.124) is small enough (~4.7%) that such leakage could plausibly account for it. The paper's own assurance in Sec. 4.2 that 'no future information leaks into training' is asserted only for the fine-tuning protocol, not for the pretraining stage. This is the single most load-bearing ambiguity because, if the protocol is not split-aware, every reported improvement in Table 3 and the 4.72% headline is suspect. If it is split-aware, the concern is resolved and the credibility of the comparison is restored.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces IC-Mamba, a state space model that extends Mamba with time-aware positional embeddings, interval-aware state representations, and a temporal-coherence regularizer, and applies it to forecasting Facebook engagement (likes, shares, comments, emojis) for misinformation and disinformation content. The authors report post-level engagement prediction improvements over eight baselines (RMSE 0.118–0.143 across datasets in Table 3), early prediction experiments from 15 minutes to 6 hours, opinion-level classification (F1 0.508–0.751), and illustrative 28-day dynamic forecasts for an opinion-level trajectory. The central claim is a 4.72% improvement over state-of-the-art engagement forecasting.","tokens_in":18250,"tokens_out":4945,"duration_ms":42316,"significance":"If the evaluation is valid, the paper makes a practical contribution to early detection of misinformation and disinformation by integrating interval-censored engagement data with a modern state space architecture. Strengths include the use of two real-world Facebook datasets, a temporal holdout protocol, ablations, and a released code repository and dashboard. However, the current manuscript leaves a critical ambiguity about whether pretraining respects the temporal split, and it overstates the early-window results; these issues must be resolved before the claimed improvements can be trusted.","major_comments":[{"comment":"The pretraining corpus D is defined in Section 3.5 as 1.78 million posts 'collected from the two datasets' with their complete engagement timelines, which per Table 2 is all posts in SocialSense and DiN. The temporal holdout in Section 4.2 (70/15/15 chronological split) is described only for the training/validation/test phases after pretraining. If Algorithm 1 is run on all posts including the test-period 15%, the model has already seen the engagement trajectories it is later asked to forecast, and the reported gains in Table 3 (e.g., Bushfire RMSE 0.118 vs. Mamba 0.124) could be leakage artifacts. The manuscript must state explicitly whether pretraining is restricted to the earliest 70% of posts; if it is not, the experiments must be rerun with split-aware pretraining before the 4.72% improvement claim can be accepted.","section":"Section 3.5 / Section 4.2"},{"comment":"The repeated claim that IC-Mamba excels at predicting engagement 'within the crucial first 15-30 minutes of posting (RMSE 0.118-0.143)' is not supported by the reported numbers. Table 3 and Section 4.3 use a 6-hour observation window to forecast cumulative engagement; the 0.118–0.143 range is exactly the Table 3 RMSE for that setting. Figure 5a shows that RMSE at 15 minutes is considerably higher, and Table 8 reports early-stage (next interval ≤1 hour) RMSE values of 0.169–0.235. The abstract and RQ2 should be corrected to state the actual observation window and horizon associated with the 0.118–0.143 RMSE, or the experiments should be rerun to support the early-window claim.","section":"Abstract / Section 4.3 / Section 4.4"},{"comment":"The 28-day opinion-level forecasting results are presented only as illustrative plots in Fig. 5(b)(c), without quantitative error metrics, comparison baselines, or ablations. The abstract claims the model 'successfully forecasting opinion-level engagement up to 28 days ahead using observation windows of 3-10 days,' but no evaluation table or metric is provided for this scenario. The manuscript should either add quantitative results (e.g., RMSE/MAPE over the forecast horizon, with the same baselines as Table 3) or explicitly soften the claim to reflect that the 28-day results are demonstrative only.","section":"Section 4.5 / RQ3"}],"minor_comments":[{"comment":"Appendix B.1 states that all reported figures are averaged over ten runs with different random seeds, but Table 3 reports no variances or significance tests; adding standard deviations or confidence intervals would strengthen the claim that IC-Mamba outperforms every baseline on every metric.","section":"Appendix B.1 / Table 3"},{"comment":"Table 5 reports hyperparameter ranges only; the final selected values and the selection procedure (grid search, Bayesian optimization, etc.) should be reported to make the experiments reproducible.","section":"Table 5"},{"comment":"The observation windows are described inconsistently: Section 4.5 and Figure 5 use 7-day and 10-day windows, while the abstract says '3-10 days.' Clarify which windows are actually used in the 28-day forecasts.","section":"Section 4.5 / Figure 5"},{"comment":"The sentence 'Fig. 4(c) offers examines comment distributions' contains a typo and should be revised to 'examines' or 'offers an examination of.'","section":"Section 4.1"},{"comment":"The 95% confidence intervals are described as 'obtained from all previous prediction for this time'; the computation should be clarified, including whether they come from the ten random seeds, Monte Carlo sampling, or empirical quantiles.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the pretraining leakage ambiguity: if pretraining is not restricted to the training partition, the central comparison is invalid. The good news is that this is resolvable either by clarifying that pretraining is split-aware or by rerunning the experiments, so I would not reject outright. The abstract's early-window claim is also materially overstated relative to the reported evaluation setup and should be corrected regardless of the leakage outcome."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a genuine extension of Mamba to interval-censored engagement data, with time-aware embeddings and a two-tier post/opinion architecture, and it ships code and a dashboard. That's real, and the early-prediction focus (15–30 minutes) is a useful selling point. But the evaluation has a potentially critical ambiguity: Section 3.5 defines the pretraining corpus as all 1.78M posts 'collected from the two datasets,' while the chronological 70/15/15 split is described only for fine-tuning. If pretraining includes test-period posts, the model has seen the trajectories it is later asked to forecast, and the consistent 4–5% RMSE gains in Table 3 become a leakage artifact. The paper's statement that 'no future information leaks into training' does not explicitly cover the pretraining stage. This is the load-bearing question, and it needs an explicit answer from the authors.\n\nWhat the paper does well: the architecture is sound and the ablations (removing text, user, time) show consistent contributions across both SocialSense and DiN. The interval-aware state representation and the temporal coherence regularizer are sensible adaptations of selective SSMs to irregularly sampled data. The opinion-level forecasting at 28 days with 3–10 day windows is a useful demonstration.\n\nSoft spots beyond the split question: no standard deviations are reported anywhere despite Appendix B claiming ten runs; the small gaps (0.118 vs 0.124) may be within noise. The engagement normalization is never stated, so RMSE units are ambiguous. The exclusions (posts with fewer than four intervals, opinions with fewer than 5,000 posts) are post hoc and should be justified.\n\nIf the pretraining split is confirmed, this is a solid applied contribution. As written, the headline claim is not supported by the evidence in the text. That is a fixable problem, not a fatal one. The paper deserves a serious referee; the code is available, the datasets are real, and the questions are answerable. My recommendation: send it to review with a request that the authors clarify or redo the pretraining split, report variance, and define normalization.","headline":"A real Mamba adaptation for interval-censored engagement, but the pretraining protocol is underspecified enough that the reported 4.72% gain is not trustworthy until the split is confirmed.","tokens_in":18793,"tokens_out":2837,"would_cite":false,"duration_ms":24978,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"IC-Mamba forecasts engagement with misinformation within the first 15–30 minutes after posting, and carries the forecast to opinion-level curves 28 days ahead.","keywords":["state space model","interval-censored data","early prediction","engagement forecasting","misinformation","disinformation","opinion classification","Mamba"],"falsifier":"Reproduce IC-Mamba with pre-training restricted to the earliest 70% of chronologically ordered posts, the same training period used for fine-tuning, and check that no validation- or test-period post appears in the pre-training corpus; if the 4.72% advantage over the best baseline shrinks or disappears, the central claim fails. A simpler audit is to compare the timestamps of the pretraining corpus against the test period.","tokens_in":17727,"feed_emoji":"📈","tokens_out":12484,"duration_ms":108478,"temperature":0.7,"pith_summary":"The paper introduces IC-Mamba, a state space model—that is, a sequence model that advances a hidden state over time—that forecasts social-media engagement by treating engagement counts as interval-censored: each observation is a cumulative count at a discrete time, hiding the exact instants of likes, shares, comments, and emoji reactions. Its stated goal is early warning: within the first 15–30 minutes after posting it reaches RMSE between 0.118 and 0.143, and with 3–10 day observation windows it predicts opinion-level engagement up to 28 days ahead. Across four misinformation themes and a disinformation network, the authors report a 4.72% improvement over the best baselines on engagement metrics and F1 scores of 0.508 to 0.751 for classifying posts into opinion or narrative classes. If these results hold, platforms and researchers would have extra lead time to intervene before harmful content goes viral.","feed_headline":"Model predicts viral misinformation within 30 minutes","feed_subtitle":"IC-Mamba reads early likes, shares, comments, and emojis to flag harmful content before it spreads.","key_machinery":"The load-bearing component is the interval-aware state representation $v_j = [\\Delta^-_j; \\log(1+e_j); \\Delta^+_j; \\log(1+\\hat{e}_{j+1})]$, which encodes how long since the last observation, the observed engagement in the current interval, the forward interval length, and the predicted next engagement. This vector feeds a selective state space model with time-dependent transitions $A_t(\\Delta t) = \\exp(\\Delta t \\tilde{A}_t)$, so the hidden state advances by the actual elapsed time rather than by a fixed tick. Time-aware positional embeddings combine relative and absolute encodings, modulated by engagement level, and a two-tier architecture carries predictions from individual posts to aggregate opinion curves. Pretraining on 1.78 million posts with an engagement-prediction loss plus a temporal-coherence loss supplies the initialization that makes fine-tuning on smaller labeled sets feasible.","core_discovery":"The central claim is that interval-censored engagement histories can be embedded directly into a selective state space model so that the model learns fine-grained temporal dynamics of engagement growth, and that this design forecasts engagement better than transformer, Hawkes, and plain Mamba baselines. IC-Mamba predicts the future trajectory of likes, shares, comments, and emojis for each post, sums those trajectories to opinion-level curves, and classifies posts into predefined opinions. On the paper's temporal holdout, it reports the best RMSE, MAPE, and $R^2$ on every dataset, with the largest qualitative advantage in the first hour after posting.","pith_inferences":["Editorial extension: the interval-aware representation is not tied to Facebook; the same construction should transfer to any platform that exposes only binned counts, such as view clocks or reshare totals.","Editorial extension: if the temporal-coherence loss genuinely enforces consistent hidden-state transitions across irregular intervals, IC-Mamba could also serve as a generative simulator of engagement trajectories for counterfactual intervention experiments.","Editorial extension: the paper's power-law analysis hints at a deployable discriminator: naturally diffusing content plateaus near human-scale engagement limits while coordinated content reaches far higher counts, so the plateau itself could flag suspected artificial amplification.","Editorial extension: because IC-Mamba's reported advantage over baselines grows with the length of the observation window, a head-to-head test on longer horizons, such as 90 days, would show whether the gap keeps widening."],"forward_implications":["Early reach assessment: from 15 minutes of observed engagement, IC-Mamba predicts the trajectory with RMSE 0.118–0.143, so a post's likely reach can be judged before it peaks.","Opinion-level early warning: with 3-, 7-, or 10-day observation windows, the two-tier model forecasts an opinion's total engagement up to 28 days ahead, and the reported confidence intervals tighten as more data streams in.","Better narrative classification than text-only models: on the disinformation network dataset, IC-Mamba's F1 of 0.508 substantially exceeds the reported text-only baseline of 0.107, indicating temporal and user features carry much of the signal in coordinated campaigns.","Ablations locate the signal: removing text hurts most on organic SocialSense content, removing temporal features hurts most on the coordinated DiN dataset, and removing user features hurts uniformly.","A reusable pretraining recipe: pretraining on 1.78 million posts lets the model fine-tune on smaller labeled misinformation sets, which matters when expert-labeled campaigns are scarce."],"supporting_citations":[{"why":"Selective state space baseline that IC-Mamba extends and outperforms; supplies the SSM mechanism.","marker":"[11]"},{"why":"Introduces the Mamba selective state space architecture that IC-Mamba modifies with interval-aware transitions.","marker":"[15]"},{"why":"Interval-censored Transformer Hawkes baseline; the interval-censored approach that IC-Mamba brings into the state space setting.","marker":"[21]"},{"why":"Interval-censored Hawkes process baseline and the formal treatment of interval-censored engagement counts.","marker":"[34]"},{"why":"Constructs the SocialSense Facebook dataset with expert-assigned opinion labels used for evaluation.","marker":"[20]"},{"why":"CrowdTangle API is the source of the observed engagement histories.","marker":"[1]"},{"why":"Transformer-Hawkes baseline; the temporal event model against which interval-censored variants are compared.","marker":"[49]"}],"fun_headline_variants":["Early warning: AI spots viral misinformation in 30 minutes","State space model flags misinformation within 30 minutes","IC-Mamba predicts engagement to stop misinformation fast","New model forecasts misinformation spread in first 30 minutes","30-minute head start: model forecasts misinformation impact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage depends on the large set of posts used to pre-train the model containing none of the posts used later to validate or test it; if test-period posts were included in pre-training, the gains could come from data leakage rather than from the model's design.","fun_headline_variants_meta":{"raw":{"variants":["Early warning: AI spots viral misinformation in 30 minutes","State space model flags misinformation within 30 minutes","IC-Mamba predicts engagement to stop misinformation fast","New model forecasts misinformation spread in first 30 minutes","30-minute head start: model forecasts misinformation impact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000314,"raw_usage":{"total_tokens":1783,"prompt_tokens":948,"completion_tokens":835,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":761}},"tokens_in":564,"tokens_out":835,"duration_ms":8297,"temperature":1.0,"reasoning_tokens":761,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T21:58:11.632227+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce IC-Mamba with pre-training restricted to the earliest 70% of chronologically ordered posts, the same training period used for fine-tuning, and check that no validation- or test-period post appears in the pre-training corpus; if the 4.72% advantage over the best baseline shrinks or disappears, the central claim fails. A simpler audit is to compare the timestamps of the pretraining corpus against the test period.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Selective state space baseline that IC-Mamba extends and outperforms; supplies the SSM mechanism."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Mamba selective state space architecture that IC-Mamba modifies with interval-aware transitions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Interval-censored Transformer Hawkes baseline; the interval-censored approach that IC-Mamba brings into the state space setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Interval-censored Hawkes process baseline and the formal treatment of interval-censored engagement counts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Constructs the SocialSense Facebook dataset with expert-assigned opinion labels used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CrowdTangle API is the source of the observed engagement histories."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Transformer-Hawkes baseline; the temporal event model against which interval-censored variants are compared."}],"review_version":1}