Pith. sign in

REVIEW 3 major objections 5 minor 48 references

Finding the Trigger: Causal Abductive Reasoning on Video Events

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that root-cause trigger events in video can be learned as a binary classification over premise events, and that its Causal Event Relation Network beats sequence and video-language baselines by modeling temporal and…

desk verdict A promising new benchmark for event-level causal trigger identification, but its trigger labels are set by an unvalidated heuristic in Algorithm 1, so the reported causal gains may reflect the heuristic rather than true causation. read the letter →

arxiv 2501.09304 v1 pith:V66ZIT5X submitted 2025-01-16 cs.CV cs.LG

Thao Minh Le , Vuong Le , Kien Do , Sunil Gupta , Svetha Venkatesh , Truyen Tran This is my paper · ORCID
classification cs.CVcs.LG
keywords causalabductivereasoningvideoeventunderstandingcounterfactualsynthesisgraphnetworksroot-causeanalysistemporalbenchmarkdatasetactionanticipation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Causal Abductive Reasoning on Video Events (CARVE), a task in which a model must pick, from the events preceding a target event in a video, the event that triggered it. To make the task learnable, the authors generate ground-truth trigger-target pairs by counterfactual synthesis: remove a dynamic object from a simulated video, rerun the simulation, and mark which earlier events, when ablated, change whether the target occurs. They build two benchmarks, a clean physics-simulator dataset and a realistic EpicKitchen-based set, and propose Causal Event Relation Networks (CERN), a graph network that reasons over temporal and semantic relations between events. A sympathetic reader should care because the task operationalizes abductive reasoning in video, finding why something happened, as a measurable prediction problem rather than free-form explanation. The central result is that CERN outperforms LSTMs, Transformers, and Video-LLaVA on both benchmarks, with the largest gains coming from explicit temporal-order modeling.

What carries the argument

Causal Event Relation Network (CERN): a message-passing graph neural network that operates on a directed event graph. Nodes are localized video events; edges are the product of a temporal relation (ordering and distance from Allen's interval algebra, with a learnable distance-decay penalty) and a semantic relation (bilinear edge features from event embeddings). The message-passing layers aggregate neighbor information with skip connections along both message and layer axes, then a logistic classifier combines each refined candidate event with the target event to predict whether the candidate is a trigger.

What would settle it

Ask human raters to watch a sample of original CARVE videos alongside their counterfactual variants and mark the event they believe is the root cause of the target; if their choices match Algorithm 1's labels no better than chance on videos where the removed object has multiple interactions, the benchmark's ground truth rests on an unverified assumption.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that root-cause triggers in video events are identifiable by learning from counterfactually labeled event graphs. CERN encodes each video as a directed graph whose nodes are events and whose edges carry both temporal interval relations and learned semantic relations, then refines candidate trigger embeddings by message passing with skip connections and scores them against the target embedding. Tested on the CARVE benchmark, CERN reaches 43.86% test accuracy (44.38% validation), ahead of Transformer (41.75%), LSTM (38.42%), and a first-collision heuristic (9.30%); on EpicKitchen-AR it reaches 47.29% validation accuracy, ahead of LSTM (45.74%) and far ahead of zero-shot Video-LLaVA (4.26%). The authors read these results as evidence that event relational representation and interaction modeling are the critical ingredients, and that neither large-scale video recognition models nor video-language models currently solve the task.

Load-bearing premise

The labels assume that when deleting an object changes the target event, the root cause is that object's earliest interaction with a dynamic partner; counterfactual deletion alone only shows the object mattered, not which interaction was the trigger.

Editorial extensions

If this is right

  • If the CARVE benchmark holds up, video causal reasoning becomes a measurable prediction problem with quantitative accuracy, not just a language-generation task.
  • CERN's margin over Transformer and LSTM implies explicit temporal ordering and distance-weighted semantic edges are worth more than generic pairwise attention for this task.
  • The strong gap between CERN and zero-shot Video-LLaVA implies language priors and large-scale pretraining do not yet supply event-level causal understanding.
  • The moderate absolute accuracies (43.86% on synthetic, 47.29% on real) imply the benchmark is not saturated and that interaction modeling beyond pairwise events is the next lever.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Implicit in the paper: the same counterfactual-removal recipe could generate root-cause labels for any simulator that supports object deletion, so the benchmark's contribution is a template, not just two datasets.
  • A testable extension the authors do not run: compare Algorithm 1's earliest-interaction trigger rule against a latest-interaction rule; if model rankings are unchanged, the precise heuristic is not the source of CERN's advantage.
  • Because EpicKitchen-AR labels inherit the oracle's accuracy, the reported 47.29% is bounded by the oracle's quality; a human-annotated subset would separate model error from label error.
  • Since CERN with visual-only features on EpicKitchen-AR reaches only 37.2%, the practical bottleneck for real videos is likely object-centric representation, not the graph architecture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces CARVE, a new video causal abductive reasoning task in which a model must identify, among a set of premise events, the trigger events that caused a given target event. The authors build two datasets: a synthetic physics-simulator dataset (CARVE) with trigger-target labels obtained by counterfactually removing a dynamic object and re-running the simulation, and a realistic dataset (EpicKitchen-AR) constructed by masking past events and using an AFFT action-anticipation oracle to generate pseudo-counterfactual labels. They also propose CERN, a temporal-semantic event graph network with message passing and skip connections, and report experiments showing that CERN outperforms LSTM, Transformer, and Video-LLaVA baselines on both datasets. The central claims are that CARVE is a new, learnable task and that CERN is an effective model for it, supported by ablations that identify the roles of temporal relations, semantic relations, message passing depth, and skip connections.

Significance. If the trigger-label construction is valid, the paper would make a useful contribution by introducing a benchmark that goes beyond linguistic or human-annotated abductive reasoning and toward intervention-based causal labels. The synthetic CARVE dataset has the strength that object-level counterfactual labels come from actual simulator interventions, not from human judgment, and the CERN architecture is thoughtfully designed with message- and layer-level skip connections and a distance-based semantic edge weighting. The paper also includes extensive ablations on feature representations and model components. However, the significance is conditional on resolving a load-bearing issue: the event-level trigger labels in CARVE are not derived from the counterfactual deletion itself but from a manually specified heuristic in Algorithm 1, and the EpicKitchen-AR labels depend entirely on an oracle model whose causal judgments are not validated. The empirical claims also rest on single-run accuracies without confidence intervals, which weakens the quantitative comparisons.

major comments (3)
  1. [Algorithm 1, Section 3.2.1] The trigger labels in CARVE are not directly implied by the counterfactual deletion experiment. Deleting a dynamic object c at the beginning of the video establishes only that c's participation is necessary for the target event; it does not identify which event on c's causal path is the trigger. Algorithm 1 instead selects 'the first event of interactions of c and a dynamic object partner' (line 9) or 'the first in the event chain' (line 11). This heuristic is not validated against any event-level counterfactual test, and it a priori excludes interactions with static scene elements (e.g., a wall collision) that could be the true initiators. Moreover, because the same h is added for every DFS path, the path enumeration in lines 8-16 does not disambiguate among candidate events. Consequently, the benchmark's ground truth is a design heuristic presented as causal ground truth, and CERN's accuracy on CARVE (Table 1) may reflect how well the model matches this heuristic rather than how well it performs causal reasoning. Please either validate the heuristic (e.g., by checking whether removing or modifying the chosen event alone changes the target, or by human annotation on a sample) or explicitly reframe the dataset as 'counterfactual object-deletion labels with a deterministic trigger rule' and adjust the causal claims accordingly.
  2. [Section 3.2.2 and Section 4.4] The EpicKitchen-AR dataset relies on pseudo-counterfactual labels generated by an AFFT oracle: a past event is a trigger if masking it flips the oracle's top-5 prediction for the target event. The authors acknowledge in Section 4.4 that this creates a 'relative upper bound accuracy of the labels depending on this oracle,' but the paper still presents Table 3 as evidence that CERN identifies triggers better than baselines in realistic videos. Without any validation of the oracle's causal judgments—for example, human agreement on a subset, or an analysis of the oracle's error rate on the selected videos—the labels are uninterpretable as ground-truth triggers. Please add such validation or substantially temper the claims about the realistic benchmark.
  3. [Section 5, Tables 1-3] All reported accuracies are single-run results with no confidence intervals or significance tests. Some comparisons are small (e.g., Table 1, 1K test: BiLSTM 27.42 vs LSTM 27.17; Table 3, label-only: CERN 46.90 vs LSTM 45.17), and the text frequently uses phrases like 'clearly outperforming' (Section 5.3) that are not supported without variance information. Please report means and standard deviations over multiple seeds, or at least provide confidence intervals, so that the reader can assess whether the observed gaps are reliable.
minor comments (5)
  1. [Throughout] There are several typos, including 'grouth-truth' (Section 3.1), 'grouthtruth' (Supplementary), 'affectting' (Section 3.2.1), and 'Thy are possible' (Section 5.2).
  2. [Section 4.2 and Supplementary A.2] Equation (3) defines a set of four temporal distances, and Table 4 in the Supplementary uses their signs, but the Euclidean distance in Supplementary Eq. (12) uses only the first two of these components. Please clarify how the full set of four distances is used and whether the Euclidean distance is intended to be a separate scalar.
  3. [Section 4.2] In Equation (3) and the surrounding text, the variable 'ep' is used inconsistently for the second event (it should be 'ej' to match the rest of the section), which is confusing.
  4. [Section 5.1] The paper states that only object features extractable from visual observation are used, but the CARVE event representation in Eq. (2) appears to use simulator ground-truth attributes (color, shape, position, velocity). Please clarify how these features would be obtained from raw pixels, or acknowledge that the CARVE experiments use oracle object state.
  5. [Table 1] The random guess baseline is reported as 0.60% accuracy, but the expected accuracy of random guessing depends on the label prevalence in the dataset. Please report the positive-label rate or explain how the 0.60% value is derived.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: CERN is trained and evaluated on held-out labels from independent simulation (CARVE) or an acknowledged oracle-dependent pipeline (EpicKitchen-AR), and no equation reduces its predictions to a fitted value or to the label-generation heuristic.

full rationale

The claimed derivation chain is: (1) simulator interventions delete each dynamic object and rerun physics to determine object-level affect on the target; (2) for each affecting object, Algorithm 1 (Section 3.2.1) chooses the first interaction with a dynamic object partner as the trigger, falling back to the first event in the chain; (3) CERN is trained with a binary cross-entropy loss to predict these labels from event features and is evaluated on held-out videos. No step of this chain is equivalent to CERN's output by construction. Algorithm 1 is a fixed annotation rule, not a learned parameter, and CERN must generalize to unseen videos under the same label rule. The same labels are supplied to all baselines, so the reported margins in Tables 1 and 3 reflect differences in inductive bias and are externally falsifiable. The EpicKitchen-AR labels are generated by the AFFT oracle (Section 3.2.2), and CERN uses TSN features from the same pipeline; however, the paper explicitly acknowledges this limitation in Section 4.4: 'we use an oracle event prediction model to create counterfactual causal pair. This creates a relative upper bound accuracy of the labels depending on this oracle.' This is a benchmark-validity limitation rather than circularity, since no oracle parameter is fed into CERN and the oracle produces a fixed label set rather than a differentiable target that CERN could reproduce from a fitted value. The only self-citation, reference [7] in Related Work, is not load-bearing for the task definition, dataset construction, or CERN architecture. The main scientific risk is the unvalidated heuristic in Algorithm 1: the 'first event of interactions of c and a dynamic object partner' rule may not equal the true root cause, so causal conclusions inherit that annotation assumption. That is a correctness risk about ground truth, not a derivation chain that reduces prediction to input.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on label quality from counterfactual simulation and on model architecture choices. The synthetic labels are grounded in simulator interventions, but the trigger extraction rule is heuristic, and the realistic labels are oracle-dependent. No new physical entities, forces, or conserved quantities are postulated.

free parameters (5)
  • Event detection threshold for velocity or direction changes = not reported
    Used to segment simulator videos into events; the threshold determines the event set and hence all labels and model inputs.
  • First-event trigger selection rule = not applicable
    Algorithm 1 chooses the first interaction with a dynamic object as the trigger; this rule is hand-designed, not derived from counterfactual evidence.
  • Distance decay factor beta = learned
    In Eq. (12), beta controls how quickly edge weights decay with temporal distance; it is learned from data and affects CERN's accuracy.
  • AFFT top-5 correctness filter = top-5 correct
    EpicKitchen-AR includes only videos where the oracle AFFT makes a correct top-5 prediction, a hand-chosen inclusion filter.
  • Message passing depth = 4 layers for CARVE, 2 for EpicKitchen-AR
    Selected by validation; the ablation shows depth changes performance by about 3 percent.
assumptions (5)
  • domain assumption Removing an object at video start and observing disappearance of the target event establishes that the object caused the target.
    Section 3.2.1 counterfactual synthesis rests on this intervention-to-causation inference in the simulator.
  • domain assumption Only endogenous events inside the video can be triggers.
    Section 4.4 explicitly limits the task to endogenous causal relations, excluding external causes.
  • domain assumption Event sequences and their features are already localized correctly.
    CARVE and EpicKitchen-AR supply pre-extracted event features; the benchmark does not evaluate event detection errors.
  • standard math Allen's interval algebra gives a correct ordering of event intervals.
    Section 4.2 uses this algebra to impose temporal direction on graph edges.
  • domain assumption AFFT can serve as a reliable oracle for generating pseudo-counterfactual causal labels.
    Section 3.2.2 builds EpicKitchen-AR labels from AFFT prediction flips rather than true interventions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Finding the Trigger: Causal Abductive Reasoning on Video Events." pith.science (2026). https://pith.science/paper/V66ZIT5X

@misc{pith2026250109304,
  author       = {Pith},
  title        = {Pith review of: Finding the Trigger: Causal Abductive Reasoning on Video Events},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V66ZIT5X}},
  note         = {Machine review of arXiv:2501.09304}
}
read the original abstract

This paper introduces a new problem, Causal Abductive Reasoning on Video Events (CARVE), which involves identifying causal relationships between events in a video and generating hypotheses about causal chains that account for the occurrence of a target event. To facilitate research in this direction, we create two new benchmark datasets with both synthetic and realistic videos, accompanied by trigger-target labels generated through a novel counterfactual synthesis approach. To explore the challenge of solving CARVE, we present a Causal Event Relation Network (CERN) that examines the relationships between video events in temporal and semantic spaces to efficiently determine the root-cause trigger events. Through extensive experiments, we demonstrate the critical roles of event relational representation learning and interaction modeling in solving video causal reasoning challenges. The introduction of the CARVE task, along with the accompanying datasets and the CERN framework, will advance future research on video causal reasoning and significantly facilitate various applications, including video surveillance, root-cause analysis and movie content management.

Figures

Figures reproduced from arXiv: 2501.09304 by the authors.

Figure 1
Figure 1. Illustration of event chains and trigger-target event pairs in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Distribution of target and trigger event locations in time in CARVE. Only 100 first events are visible. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overview of CERN. Given a target event e target and its premise events, we build a directed graph of events G based on their temporal distance. Edge feature vectors rij (arrows) represent multi-aspect relations between events. We use a novel message passing scheme (Msg) to refine events in consideration of their surrounding events. Gray box illustrates how event e4 receives information from preceding events e1, e3, … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Message passing with skip connections along “message” and “layer” axis. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Qualitative examples demonstrating that sequential models struggle to identify correct trigger events while [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: A showcase for the challenges of our CARVE dataset that all the studied methods struggle. This is often the case when there are multiple concurrent events occurring within the times of the trigger event. Best viewed in color. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 45 canonical work pages

  1. [1]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV, pages 1026–1034, 2015. 1

    work page 2015

  2. [2]

    Lipnet: End-to-end sentence- level lipreading

    Yannis M Assael, Brendan Shillingford, Shimon Whiteson, and Nando De Freitas. Lipnet: End-to-end sentence- level lipreading. ICLR, 2017. 1

    work page 2017

  3. [3]

    Collected papers of Charles Sanders Peirce, volume 5

    Charles Sanders Peirce. Collected papers of Charles Sanders Peirce, volume 5. Harvard University Press, 1974. 1

    work page 1974

  4. [4]

    Mvitv2: Improved multiscale vision transformers for classification and detection

    Yanghao Li, Chao-Yuan Wu, Haoqi Fan, Karttikeya Mangalam, Bo Xiong, Jitendra Malik, and Christoph Feichtenhofer. Mvitv2: Improved multiscale vision transformers for classification and detection. In CVPR, pages 4804–4814, 2022. 1, 2, 5.3.1

    work page 2022

  5. [5]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 1, 5.2, 5.4, B.2

    arXiv 2023

  6. [6]

    (2.5+ 1) d spatio-temporal scene graphs for video question answering

    Anoop Cherian, Chiori Hori, Tim K Marks, and Jonathan Le Roux. (2.5+ 1) d spatio-temporal scene graphs for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 444–453, 2022. 2

    work page 2022

  7. [7]

    Hierarchical object-oriented spatio-temporal reasoning for video question answering

    Long Hoang Dang, Thao Minh Le, Vuong Le, and Truyen Tran. Hierarchical object-oriented spatio-temporal reasoning for video question answering. IJCAI, 2021. 2

    work page 2021

  8. [8]

    Tenenbaum

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. CLEVRER: CoLlision Events for Video REpresentation and Reasoning. In ICLR, 2020. 2, 3.1, 3.2

    work page 2020

Show all 48 references
  1. [9]

    Mist: Multi-modal iterative spatial-temporal transformer for long-form video question answering

    Difei Gao, Luowei Zhou, Lei Ji, Linchao Zhu, Yi Yang, and Mike Zheng Shou. Mist: Multi-modal iterative spatial-temporal transformer for long-form video question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14773–14783, 2023. 2

    2023

  2. [10]

    Visual abductive reasoning

    Chen Liang, Wenguan Wang, Tianfei Zhou, and Yi Yang. Visual abductive reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15565–15575, 2022. 2, 3.1, 3.2

    2022

  3. [11]

    VideoABC: A Real-World Video Dataset for Abductive Visual Reasoning

    Wenliang Zhao, Yongming Rao, Yansong Tang, Jie Zhou, and Jiwen Lu. VideoABC: A Real-World Video Dataset for Abductive Visual Reasoning. IEEE Transactions on Image Processing, 31:6048–6061, 2022. 2

    2022

  4. [12]

    The abduction of sherlock holmes: A dataset for visual abductive reasoning

    Jack Hessel, Jena D Hwang, Jae Sung Park, Rowan Zellers, Chandra Bhagavatula, Anna Rohrbach, Kate Saenko, and Yejin Choi. The abduction of sherlock holmes: A dataset for visual abductive reasoning. In ECCV, pages 558–575. Springer, 2022. 2

    2022

  5. [13]

    Visual explanation by high-level abduction: On answer-set programming driven reasoning about moving objects

    Jakob Suchan, Mehul Bhatt, Przemysław Wałega, and Carl Schultz. Visual explanation by high-level abduction: On answer-set programming driven reasoning about moving objects. In AAAI, volume 32, 2018. 2 11 Finding the Trigger: Causal Abductive Reasoning on Video Events A PREPRINT

    2018

  6. [14]

    Causal discovery in physical systems from videos

    Yunzhu Li, Antonio Torralba, Animashree Anandkumar, Dieter Fox, and Animesh Garg. Causal discovery in physical systems from videos. Neural Information Processing Systems (NeurIPS), 2020. 2

    2020

  7. [15]

    Dynamic visual reasoning by learning differentiable physics models from video and language

    Mingyu Ding, Zhenfang Chen, Tao Du, Ping Luo, Josh Tenenbaum, and Chuang Gan. Dynamic visual reasoning by learning differentiable physics models from video and language. NeurIPS, 34:887–899, 2021. 2

    2021

  8. [16]

    Craft: A benchmark for causal reasoning about forces and interactions

    Tayfun Ates, M Samil Atesoglu, Cagatay Yigit, Ilker Kesen, Mert Kobas, Erkut Erdem, Aykut Erdem, Tilbe Goksun, and Deniz Yuret. Craft: A benchmark for causal reasoning about forces and interactions. ACL, pages 2602–2627, 2022. 2, 3.2.1

    2022

  9. [17]

    Object-centric learning with slot attention

    Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. NeurIPS, 33: 11525–11538, 2020. 2

    2020

  10. [18]

    Cater: A diagnostic dataset for compositional actions and temporal reasoning

    Rohit Girdhar and Deva Ramanan. Cater: A diagnostic dataset for compositional actions and temporal reasoning. arXiv preprint arXiv:1910.04744, 2019. 2

    1910 arXiv

  11. [19]

    CLEVRER-Humans: Describing Physical and Causal Events the Human Way

    Jiayuan Mao, Xuelin Yang, Xikun Zhang, Noah Goodman, and Jiajun Wu. CLEVRER-Humans: Describing Physical and Causal Events the Human Way. In NeurIPS Datasets and Benchmarks Track, 2022. 2, 3.1, 3.2

    2022

  12. [20]

    Multi-shot temporal event localization: a benchmark

    Xiaolong Liu, Yao Hu, Song Bai, Fei Ding, Xiang Bai, and Philip HS Torr. Multi-shot temporal event localization: a benchmark. In CVPR, pages 12596–12606, 2021. 2

    2021

  13. [21]

    Temporal context network for activity localization in videos

    Xiyang Dai, Bharat Singh, Guyue Zhang, Larry S Davis, and Yan Qiu Chen. Temporal context network for activity localization in videos. In ICCV, pages 5793–5802, 2017. 2

    2017

  14. [22]

    Learning to refactor action and co-occurrence features for temporal action localization

    Kun Xia, Le Wang, Sanping Zhou, Nanning Zheng, and Wei Tang. Learning to refactor action and co-occurrence features for temporal action localization. In CVPR, pages 13884–13893, 2022. 2

    2022

  15. [23]

    What is more likely to happen next? video-and-language future event prediction

    Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. What is more likely to happen next? video-and-language future event prediction. EMNLP, pages 8769–8784, 2020. 2

    2020

  16. [24]

    Visualcomet: Reasoning about the dynamic context of a still image

    Jae Sung Park, Chandra Bhagavatula, Roozbeh Mottaghi, Ali Farhadi, and Yejin Choi. Visualcomet: Reasoning about the dynamic context of a still image. In ECCV, pages 508–524. Springer, 2020. 2

    2020

  17. [25]

    Action genome: Actions as compositions of spatio-temporal scene graphs

    Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles. Action genome: Actions as compositions of spatio-temporal scene graphs. In CVPR, pages 10236–10247, 2020. 2

    2020

  18. [26]

    Visual semantic role labeling for video understanding

    Arka Sadhu, Tanmay Gupta, Mark Yatskar, Ram Nevatia, and Aniruddha Kembhavi. Visual semantic role labeling for video understanding. In CVPR, pages 5589–5600, 2021. 2

    2021

  19. [27]

    Multimodal event graphs: Towards event centric understanding of multimodal world

    Hammad A Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Yulei Niu, Xudong Lin, Long Chen, Jaywon Koo, Sounak Ray, and Shih-Fu Chang. Multimodal event graphs: Towards event centric understanding of multimodal world. arXiv preprint arXiv:2206.07207, 2022. 2

    2022 arXiv

  20. [28]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Luˇci´c, and Cordelia Schmid. Vivit: A video vision transformer. In ICCV, pages 6836–6846, 2021. 2

    2021

  21. [29]

    Video swin transformer

    Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In CVPR, pages 3202–3211, 2022. 2

    2022

  22. [30]

    Temporal reasoning with abductive event calculus

    Marc Denecker, Lode Missiaen, and Maurice Bruynooghe. Temporal reasoning with abductive event calculus. In ECAI, pages 384–388. John Wiley and Sons; Chichester, 1992. 2

    1992

  23. [31]

    Abductive commonsense reasoning.ICLR, 2019

    Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Scott Wen-tau Yih, and Yejin Choi. Abductive commonsense reasoning.ICLR, 2019. 2

    2019

  24. [32]

    A case for abductive reasoning over ontologies

    Corinna Elsenbroich, Oliver Kutz, and Ulrike Sattler. A case for abductive reasoning over ontologies. In OWLED, volume 216, 2006. 2

    2006

  25. [33]

    Root cause analysis in it infrastructures using ontologies and abduction in markov logic networks

    Joerg Schoenfisch, Christian Meilicke, Janno von Stülpnagel, Jens Ortmann, and Heiner Stuckenschmidt. Root cause analysis in it infrastructures using ontologies and abduction in markov logic networks. Information Systems, 74:103–116, 2018. 2 12 Finding the Trigger: Causal Abdu...

    2018

  26. [34]

    Causality: Models, reasoning and inference

    Judea Pearl et al. Causality: Models, reasoning and inference. Cambridge, UK: CambridgeUniversityPress, 19,

  27. [35]

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Evangelos Kazakos, Jian Ma, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100. International Journal ...

    2022

  28. [36]

    Anticipative feature fusion transformer for multi-modal action anticipation

    Zeyun Zhong, David Schneider, Michael V oit, Rainer Stiefelhagen, and Jürgen Beyerer. Anticipative feature fusion transformer for multi-modal action anticipation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6068–6077, 2023. 3.2.2,...

    2023

  29. [37]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. NeurIPS, 2019. 4.1, 5.1

    2019

  30. [38]

    Modeling temporal interactions with interval temporal bayesian networks for complex activity recognition.TPAMI, 35(10):2468–2483,

    Yongmian Zhang, Yifan Zhang, Eran Swears, Natalia Larios, Ziheng Wang, and Qiang Ji. Modeling temporal interactions with interval temporal bayesian networks for complex activity recognition.TPAMI, 35(10):2468–2483,

  31. [39]

    Maintaining knowledge about temporal intervals

    James F Allen. Maintaining knowledge about temporal intervals. Communications of the ACM, 26(11):832–843,

  32. [40]

    Unified graph structured models for video understanding

    Anurag Arnab, Chen Sun, and Cordelia Schmid. Unified graph structured models for video understanding. In ICCV, pages 8117–8126, 2021. 4.2

    2021

  33. [41]

    Hierarchical video frame sequence representation with deep convolutional graph network

    Feng Mao, Xiang Wu, Hui Xue, and Rong Zhang. Hierarchical video frame sequence representation with deep convolutional graph network. In ECCV workshops, pages 0–0, 2018. 4.2

    2018

  34. [42]

    Zero-shot video object segmentation via attentive graph neural networks

    Wenguan Wang, Xiankai Lu, Jianbing Shen, David J Crandall, and Ling Shao. Zero-shot video object segmentation via attentive graph neural networks. In ICCV, pages 9236–9245, 2019. 4.2

    2019

  35. [43]

    How powerful are k-hop message passing graph neural networks

    Jiarui Feng, Yixin Chen, Fuhai Li, Anindya Sarkar, and Muhan Zhang. How powerful are k-hop message passing graph neural networks. NeurIPS, 2022. 4.3

    2022

  36. [44]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. 5.2, B.2

    1997

  37. [45]

    Framewise phoneme classification with bidirectional lstm and other neural network architectures

    Alex Graves and Jürgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5-6):602–610, 2005. 5.2, B.2

    2005

  38. [46]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NIPS, 30, 2017. 5.2, B.2

    2017

  39. [47]

    The kinetics human action video dataset

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017. 5.3.1

    2017 arXiv

  40. [48]

    Adam: A method for stochastic optimization

    Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2014. B.1 13 Finding the Trigger: Causal Abductive Reasoning on Video Events A PREPRINT Supplementary material Introduction In this supplementary material, we provide further details on the impleme...

    2014

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.