{"id":"1c0aa946-47f3-4b81-b0b5-eff5f09ee1ce","arxiv_id":"2501.09304","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A new video benchmark and graph-based method for identifying the root-cause trigger event behind a target event, using counterfactually generated labels.","lead":"This paper introduces CARVE, a video task where a model must find which earlier event caused a given target event, and builds two datasets with counterfactually generated trigger-target labels. It also proposes CERN, a graph neural network that models temporal and semantic relations between events and outperforms several sequence and video-language baselines on the new benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CARVE's trigger labels are produced by an unvalidated heuristic in Algorithm 1, not by the counterfactual deletion itself, so CERN's reported causal gains may be measured against an arbitrary annotation rule.","rationale":"The reader's verdict is CONDITIONAL and identifies Algorithm 1's trigger selection as the weakest assumption; my reading agrees. The load-bearing concern is not that CERN's implementation is wrong (the message-passing architecture is plausible and the ablations are internally consistent), but that the benchmark labels defining the target concept are not derived from the counterfactual intervention. The task definition requires identifying root-cause trigger events, but the labels only satisfy a weaker object-level necessity. This makes the main empirical claim—CERN identifies triggers better than baselines—conditional on an unvalidated label-generation rule. A focused simulator study comparing Algorithm 1 labels to per-event counterfactual necessity would resolve this. Because the paper is otherwise valuable as a first benchmark and the method is described in enough detail to reproduce, I would not reject outright; the verdict should remain CONDITIONAL until the label rule is validated or revised.","tokens_in":14872,"tokens_out":8145,"duration_ms":81661,"concrete_test":"On a 200-video subset of CARVE, perform per-event interventions in the simulator: for each affecting object c, suppress only the Algorithm-1 trigger h (e.g., by adjusting c's initial conditions so that h's collision does not occur, while keeping c present) and rerun to test whether etarget still occurs. Build an alternative label set from the events whose individual suppression removes the target. Then re-evaluate CERN and baselines on this corrected label set. If Algorithm-1's h is frequently not individually necessary, or if CERN's margin over baselines collapses, the current labels are an annotation heuristic rather than causal ground truth.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CARVE supplies counterfactual trigger-target labels and that CERN outperforms baselines at causal trigger identification. The object-level counterfactual is sound: removing a dynamic object c and rerunning the simulator establishes that c affects the target. But Algorithm 1 (Section 3.2.1) does not derive event-level triggers from this counterfactual. It selects, for each affecting object c, 'the first event of interactions of c and a dynamic object partner' (line 9), falling back to the first event in c's chain if no such interaction exists (line 11). This rule is not implied by the deletion experiment: deleting c only shows that some event in c's history is necessary for the target, not which event. The rule also does not check that the chosen event is itself counterfactually necessary for the target, and it a priori excludes static-scene interactions (e.g., a wall collision that redirects c) even though those can be the true initiators. Since the same h is added for every DFS path, the path loop does not disambiguate among candidate events. Consequently, the benchmark's ground truth is a design heuristic presented as causal ground truth, and CERN's 43.86% test accuracy on CARVE (Table 1) may reward matching this heuristic rather than reasoning about causes. The EpicKitchen-AR labels are similarly oracle-dependent, a limitation the paper acknowledges in Section 4.4.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CARVE, a new video causal abductive reasoning task in which a model must identify, among a set of premise events, the trigger events that caused a given target event. The authors build two datasets: a synthetic physics-simulator dataset (CARVE) with trigger-target labels obtained by counterfactually removing a dynamic object and re-running the simulation, and a realistic dataset (EpicKitchen-AR) constructed by masking past events and using an AFFT action-anticipation oracle to generate pseudo-counterfactual labels. They also propose CERN, a temporal-semantic event graph network with message passing and skip connections, and report experiments showing that CERN outperforms LSTM, Transformer, and Video-LLaVA baselines on both datasets. The central claims are that CARVE is a new, learnable task and that CERN is an effective model for it, supported by ablations that identify the roles of temporal relations, semantic relations, message passing depth, and skip connections.","tokens_in":15146,"tokens_out":4582,"duration_ms":44319,"significance":"If the trigger-label construction is valid, the paper would make a useful contribution by introducing a benchmark that goes beyond linguistic or human-annotated abductive reasoning and toward intervention-based causal labels. The synthetic CARVE dataset has the strength that object-level counterfactual labels come from actual simulator interventions, not from human judgment, and the CERN architecture is thoughtfully designed with message- and layer-level skip connections and a distance-based semantic edge weighting. The paper also includes extensive ablations on feature representations and model components. However, the significance is conditional on resolving a load-bearing issue: the event-level trigger labels in CARVE are not derived from the counterfactual deletion itself but from a manually specified heuristic in Algorithm 1, and the EpicKitchen-AR labels depend entirely on an oracle model whose causal judgments are not validated. The empirical claims also rest on single-run accuracies without confidence intervals, which weakens the quantitative comparisons.","major_comments":[{"comment":"The trigger labels in CARVE are not directly implied by the counterfactual deletion experiment. Deleting a dynamic object c at the beginning of the video establishes only that c's participation is necessary for the target event; it does not identify which event on c's causal path is the trigger. Algorithm 1 instead selects 'the first event of interactions of c and a dynamic object partner' (line 9) or 'the first in the event chain' (line 11). This heuristic is not validated against any event-level counterfactual test, and it a priori excludes interactions with static scene elements (e.g., a wall collision) that could be the true initiators. Moreover, because the same h is added for every DFS path, the path enumeration in lines 8-16 does not disambiguate among candidate events. Consequently, the benchmark's ground truth is a design heuristic presented as causal ground truth, and CERN's accuracy on CARVE (Table 1) may reflect how well the model matches this heuristic rather than how well it performs causal reasoning. Please either validate the heuristic (e.g., by checking whether removing or modifying the chosen event alone changes the target, or by human annotation on a sample) or explicitly reframe the dataset as 'counterfactual object-deletion labels with a deterministic trigger rule' and adjust the causal claims accordingly.","section":"Algorithm 1, Section 3.2.1"},{"comment":"The EpicKitchen-AR dataset relies on pseudo-counterfactual labels generated by an AFFT oracle: a past event is a trigger if masking it flips the oracle's top-5 prediction for the target event. The authors acknowledge in Section 4.4 that this creates a 'relative upper bound accuracy of the labels depending on this oracle,' but the paper still presents Table 3 as evidence that CERN identifies triggers better than baselines in realistic videos. Without any validation of the oracle's causal judgments—for example, human agreement on a subset, or an analysis of the oracle's error rate on the selected videos—the labels are uninterpretable as ground-truth triggers. Please add such validation or substantially temper the claims about the realistic benchmark.","section":"Section 3.2.2 and Section 4.4"},{"comment":"All reported accuracies are single-run results with no confidence intervals or significance tests. Some comparisons are small (e.g., Table 1, 1K test: BiLSTM 27.42 vs LSTM 27.17; Table 3, label-only: CERN 46.90 vs LSTM 45.17), and the text frequently uses phrases like 'clearly outperforming' (Section 5.3) that are not supported without variance information. Please report means and standard deviations over multiple seeds, or at least provide confidence intervals, so that the reader can assess whether the observed gaps are reliable.","section":"Section 5, Tables 1-3"}],"minor_comments":[{"comment":"There are several typos, including 'grouth-truth' (Section 3.1), 'grouthtruth' (Supplementary), 'affectting' (Section 3.2.1), and 'Thy are possible' (Section 5.2).","section":"Throughout"},{"comment":"Equation (3) defines a set of four temporal distances, and Table 4 in the Supplementary uses their signs, but the Euclidean distance in Supplementary Eq. (12) uses only the first two of these components. Please clarify how the full set of four distances is used and whether the Euclidean distance is intended to be a separate scalar.","section":"Section 4.2 and Supplementary A.2"},{"comment":"In Equation (3) and the surrounding text, the variable 'ep' is used inconsistently for the second event (it should be 'ej' to match the rest of the section), which is confusing.","section":"Section 4.2"},{"comment":"The paper states that only object features extractable from visual observation are used, but the CARVE event representation in Eq. (2) appears to use simulator ground-truth attributes (color, shape, position, velocity). Please clarify how these features would be obtained from raw pixels, or acknowledge that the CARVE experiments use oracle object state.","section":"Section 5.1"},{"comment":"The random guess baseline is reported as 0.60% accuracy, but the expected accuracy of random guessing depends on the label prevalence in the dataset. Please report the positive-label rate or explain how the 0.60% value is derived.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the validity of the trigger labels in the CARVE dataset: as written, Algorithm 1 imposes a specific annotation rule that is not justified by the counterfactual deletion procedure. This is a load-bearing issue for both the benchmark contribution and the empirical evaluation. If the authors can validate the rule (or reframe the claims), the paper could be suitable for publication. I would also encourage the editor to request the code and data-generation scripts as part of the revision, since reproducibility of the synthetic dataset is central to the contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the new thing is a task and a large synthetic benchmark for event-level causal trigger identification, and a graph network that beats obvious baselines. The catch is that the trigger labels in the benchmark are not actually pinned down by the counterfactual deletions. Algorithm 1 chooses the first event where the counterfactually-removed object interacts with a dynamic partner, and that's a heuristic, not something the deletion experiment tells you. So the benchmark's ground truth is a design choice, and the method's scores are measured against that choice.\n\nCredit where due: the event-level trigger-target formulation is genuinely new relative to CLEVRER and CRAFT, which are QA or force-interaction sets. The object-level counterfactual synthesis is sound for establishing which objects affect the target. The CARVE dataset is large and the EpicKitchen-AR effort is a reasonable first step toward realistic pseudo-counterfactual labels, with the oracle dependence openly acknowledged. The CERN model is a solid message-passing design, and the ablations make a decent case that relational structure and skip connections help. Showing Video-LLaVA fails is useful.\n\nNow the soft spots, in order of importance. The trigger-rule issue is the load-bearing one. Removing an object shows that the object matters, but it doesn't show which event in that object's history is the root cause. The rule also excludes static-scene collisions as triggers, which can be the actual initiators. This should be fixed or at least defended before the benchmark is adopted as a standard. The other issues are less severe: no released code/data, single-run numbers with no confidence intervals, and event-detection thresholds as free parameters. These are standard for a benchmark paper but should be addressed.\n\nWho benefits: people working on video causal reasoning and event-graph methods. The task is worth taking seriously even if the ground truth needs another iteration. I'd send this to peer review, with a request to rework or justify the trigger selection rule, add error bars, and ship the datasets.","headline":"A promising new benchmark for event-level causal trigger identification, but its trigger labels are set by an unvalidated heuristic in Algorithm 1, so the reported causal gains may reflect the heuristic rather than true causation.","tokens_in":15686,"tokens_out":2896,"would_cite":true,"duration_ms":28718,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that root-cause trigger events in video can be learned as a binary classification over premise events, and that its Causal Event Relation Network beats sequence and video-language baselines by modeling temporal and…","keywords":["causal abductive reasoning","video event understanding","counterfactual synthesis","event graph networks","root-cause analysis","temporal reasoning","benchmark dataset","action anticipation"],"falsifier":"Ask human raters to watch a sample of original CARVE videos alongside their counterfactual variants and mark the event they believe is the root cause of the target; if their choices match Algorithm 1's labels no better than chance on videos where the removed object has multiple interactions, the benchmark's ground truth rests on an unverified assumption.","tokens_in":14672,"feed_emoji":"🎯","tokens_out":6200,"duration_ms":61720,"temperature":0.7,"pith_summary":"The paper introduces Causal Abductive Reasoning on Video Events (CARVE), a task in which a model must pick, from the events preceding a target event in a video, the event that triggered it. To make the task learnable, the authors generate ground-truth trigger-target pairs by counterfactual synthesis: remove a dynamic object from a simulated video, rerun the simulation, and mark which earlier events, when ablated, change whether the target occurs. They build two benchmarks, a clean physics-simulator dataset and a realistic EpicKitchen-based set, and propose Causal Event Relation Networks (CERN), a graph network that reasons over temporal and semantic relations between events. A sympathetic reader should care because the task operationalizes abductive reasoning in video, finding why something happened, as a measurable prediction problem rather than free-form explanation. The central result is that CERN outperforms LSTMs, Transformers, and Video-LLaVA on both benchmarks, with the largest gains coming from explicit temporal-order modeling.","feed_headline":"Graph network finds video event causes better than large models","feed_subtitle":"Counterfactual video benchmarks and an event-graph model locate root-cause triggers in simulated and real kitchen videos.","key_machinery":"Causal Event Relation Network (CERN): a message-passing graph neural network that operates on a directed event graph. Nodes are localized video events; edges are the product of a temporal relation (ordering and distance from Allen's interval algebra, with a learnable distance-decay penalty) and a semantic relation (bilinear edge features from event embeddings). The message-passing layers aggregate neighbor information with skip connections along both message and layer axes, then a logistic classifier combines each refined candidate event with the target event to predict whether the candidate is a trigger.","core_discovery":"On its own terms, the paper's discovery is that root-cause triggers in video events are identifiable by learning from counterfactually labeled event graphs. CERN encodes each video as a directed graph whose nodes are events and whose edges carry both temporal interval relations and learned semantic relations, then refines candidate trigger embeddings by message passing with skip connections and scores them against the target embedding. Tested on the CARVE benchmark, CERN reaches 43.86% test accuracy (44.38% validation), ahead of Transformer (41.75%), LSTM (38.42%), and a first-collision heuristic (9.30%); on EpicKitchen-AR it reaches 47.29% validation accuracy, ahead of LSTM (45.74%) and far ahead of zero-shot Video-LLaVA (4.26%). The authors read these results as evidence that event relational representation and interaction modeling are the critical ingredients, and that neither large-scale video recognition models nor video-language models currently solve the task.","pith_inferences":["Implicit in the paper: the same counterfactual-removal recipe could generate root-cause labels for any simulator that supports object deletion, so the benchmark's contribution is a template, not just two datasets.","A testable extension the authors do not run: compare Algorithm 1's earliest-interaction trigger rule against a latest-interaction rule; if model rankings are unchanged, the precise heuristic is not the source of CERN's advantage.","Because EpicKitchen-AR labels inherit the oracle's accuracy, the reported 47.29% is bounded by the oracle's quality; a human-annotated subset would separate model error from label error.","Since CERN with visual-only features on EpicKitchen-AR reaches only 37.2%, the practical bottleneck for real videos is likely object-centric representation, not the graph architecture."],"forward_implications":["If the CARVE benchmark holds up, video causal reasoning becomes a measurable prediction problem with quantitative accuracy, not just a language-generation task.","CERN's margin over Transformer and LSTM implies explicit temporal ordering and distance-weighted semantic edges are worth more than generic pairwise attention for this task.","The strong gap between CERN and zero-shot Video-LLaVA implies language priors and large-scale pretraining do not yet supply event-level causal understanding.","The moderate absolute accuracies (43.86% on synthetic, 47.29% on real) imply the benchmark is not saturated and that interaction modeling beyond pairwise events is the next lever."],"supporting_citations":[{"why":"Supplies the 2D physics simulator used to generate the CARVE videos and their controlled physical interactions.","marker":"[16]"},{"why":"Justifies the counterfactual/interventional approach for separating causal from associative relations.","marker":"[34]"},{"why":"Provides the CLEVRER video-reasoning benchmark that the CARVE task extends.","marker":"[8]"},{"why":"Supplies the EpicKitchens-100 videos on which EpicKitchen-AR is built.","marker":"[35]"},{"why":"Acts as the oracle action-anticipation model that produces pseudo-counterfactual trigger labels for EpicKitchen-AR.","marker":"[36]"},{"why":"Serves as the large video-language baseline that CERN must beat on realistic data.","marker":"[5]"},{"why":"Supplies Allen's interval algebra used to define temporal relations and edge directions.","marker":"[39]"},{"why":"Is the large video recognition model whose objectless features are ablated to show the need for object-centric event features.","marker":"[4]"},{"why":"Is the Transformer baseline that CERN is compared against and outperforms.","marker":"[46]"}],"fun_headline_variants":["Graph net beats transformers, LSTMs on video cause finding","Counterfactual video benchmarks train graph net to find cause events","CERN event-graph model tops video causal reasoning benchmarks","Video cause hunting: graph net outperforms large vision-language models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The labels assume that when deleting an object changes the target event, the root cause is that object's earliest interaction with a dynamic partner; counterfactual deletion alone only shows the object mattered, not which interaction was the trigger.","fun_headline_variants_meta":{"raw":{"variants":["Graph net beats transformers, LSTMs on video cause finding","Counterfactual video benchmarks train graph net to find cause events","CERN event-graph model tops video causal reasoning benchmarks","Video cause hunting: graph net outperforms large vision-language models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000535,"raw_usage":{"total_tokens":2551,"prompt_tokens":902,"completion_tokens":1649,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1580}},"tokens_in":518,"tokens_out":1649,"duration_ms":10601,"temperature":1.0,"reasoning_tokens":1580,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:05:58.936193+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ask human raters to watch a sample of original CARVE videos alongside their counterfactual variants and mark the event they believe is the root cause of the target; if their choices match Algorithm 1's labels no better than chance on videos where the removed object has multiple interactions, the benchmark's ground truth rests on an unverified assumption.","supporting_citations":[{"cited_title":"Craft: A benchmark for causal reasoning about forces and interactions","cited_arxiv_id":null,"evidence_quote":"Supplies the 2D physics simulator used to generate the CARVE videos and their controlled physical interactions."},{"cited_title":"Causality: Models, reasoning and inference","cited_arxiv_id":null,"evidence_quote":"Justifies the counterfactual/interventional approach for separating causal from associative relations."},{"cited_title":"Tenenbaum","cited_arxiv_id":null,"evidence_quote":"Provides the CLEVRER video-reasoning benchmark that the CARVE task extends."},{"cited_title":"Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100","cited_arxiv_id":null,"evidence_quote":"Supplies the EpicKitchens-100 videos on which EpicKitchen-AR is built."},{"cited_title":"Anticipative feature fusion transformer for multi-modal action anticipation","cited_arxiv_id":null,"evidence_quote":"Acts as the oracle action-anticipation model that produces pseudo-counterfactual trigger labels for EpicKitchen-AR."},{"cited_title":"Maintaining knowledge about temporal intervals","cited_arxiv_id":null,"evidence_quote":"Supplies Allen's interval algebra used to define temporal relations and edge directions."},{"cited_title":"Mvitv2: Improved multiscale vision transformers for classification and detection","cited_arxiv_id":null,"evidence_quote":"Is the large video recognition model whose objectless features are ablated to show the need for object-centric event features."},{"cited_title":"Attention is all you need","cited_arxiv_id":null,"evidence_quote":"Is the Transformer baseline that CERN is compared against and outperforms."}],"review_version":1}