Pith. sign in

REVIEW 4 major objections 6 minor 49 references

MCAM: Multimodal Causal Analysis Model for Ego-Vehicle-Level Driving Video Understanding

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MCAM claims to be the first causal-analysis model for ego-vehicle driving video understanding, using a driving-state directed acyclic graph to separate true causes from spurious correlations.

desk verdict The paper's causal analysis claim is unsupported by the implementation; the CAM is feature fusion, not causal inference. read the letter →

arxiv 2507.06072 v1 pith:AIY7VM6W submitted 2025-07-08 cs.CV

Tongtong Cheng , Rongzhen Li , Yixin Xiong , Tao Zhang , Jing Wang , Kai Liu This is my paper
classification cs.CV
keywords multimodalcausalanalysisdrivingvideounderstandingego-vehiclebehaviordirectedacyclicgraphvision-languagetransformerreasoningspuriouscorrelationautonomous
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MCAM, a model for ego-vehicle driving video understanding that turns a video clip into a description of what the car does and why. Its central claim is that these tasks need causal structure, not just correlation: the model defines a Driving State Directed Acyclic Graph whose nodes are the initial safe state, the action, the changed environment, a potential dangerous state, and the final safe state, and it claims to be the first to bring such causal analysis to ego-vehicle-level driving video understanding. On BDD-X and CoVLA, the model reports state-of-the-art captioning and reasoning scores, and the authors argue that the causal module suppresses spurious correlations such as attributing a stop to brake lights instead of the car ahead. If true, this would give autonomous driving systems an explainable, causal way to justify their decisions in language. The paper itself notes that label noise in existing datasets limits recognition quality.

What carries the argument

The load-bearing object is the Driving State Directed Acyclic Graph (DSDAG), grounded in structural causal models. It defines the ego-vehicle's initial safe state $X_s$, the action $Y$, the changed environment $Z$, a dangerous state $W$ reached if no action is taken, and the final safe state $X_e$, and it postulates that $Y$ is caused by $X_s$ and $Z$ while $X_e$ is caused by $Y$ and $Z$. The Causal Analysis Module implements this by fusing initial-frame, end-frame, potential-danger, and action features, applying softmax attention to their concatenation, and using the result to weight a whole-video feature as a residual. That weighted feature, together with a sparse-attention vision-language transformer, carries the causal information into the generated caption.

What would settle it

A concrete test: take a validation set where the visual cue that correlates with an action is inverted relative to the true cause (for example, vehicles stopping behind a lead car whose brake lights are off, with a red traffic light visible). If MCAM keeps its reasoning accuracy, the module is doing causal work; if accuracy collapses, it has learned the correlation, not the cause. A second check is to replace the CAM's attention-weighted sum with a plain linear fusion of the same features and compare BDD-X reasoning scores; the causal claim predicts a clear drop, while a correlation-only account predicts no drop.

Watch

Extended reading notes

Core claim

The discovery the paper wants to establish is that ego-vehicle driving behavior can be decomposed into a small causal graph of discrete states, and that a model built around that graph outperforms prior captioning and reasoning systems while being less fooled by spurious correlations. MCAM treats the start safe state $X_s$, the action $Y$, the changed environment $Z$, the hidden danger state $W$, and the end safe state $X_e$ as nodes of a Driving State Directed Acyclic Graph, with the action and the new state determined by the previous state and the environment. The Causal Analysis Module then identifies which environmental factors $V$ had the strongest influence on the chosen action, and the Vision-Language Transformer converts those causal features into narration and reasoning text. The reported BDD-X and CoVLA results, together with the ablation study, are offered as evidence that the DSDAG, not the larger backbone, drives the improvement.

Load-bearing premise

The paper's claim stands or falls on whether the DSDAG's causal equations are actually what the Causal Analysis Module computes, since the write-up gives no explicit mapping from the theoretical variables ($X_s$, $Y$, $Z$, $W$, $do(Y_c = c)$) to the module's concatenation, attention, and residual operations.

Editorial extensions

If this is right

  • If the DSDAG is doing the work, causal structure can be injected into driving video models without large language models, since MCAM uses a compact transformer.
  • Reasoning outputs on BDD-X improve to a 9.1% BLEU-4 and narration to 35.7% against re-implemented and published baselines.
  • The model's explicit dangerous-state node gives a natural handle for risk-related captioning, not just description.
  • Sparse attention and the total loss are designed to suppress hallucination in vision-language alignment, which would make generated explanations more trustworthy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the causal graph is sound, the DSDAG could be reused as a prior for trajectory prediction, giving autonomous vehicles a way to state why a maneuver was chosen before it happens.
  • The key factors that MCAM is meant to identify are not evaluated against human-annotated salient objects; an annotation study could confirm whether the model's attention picks out the same cues a human driver cites.
  • The same state-transition graph could transfer to other egocentric activity understanding tasks, such as robot navigation or assisted driving, where a narrator must explain state changes in language.
  • The paper's own caveat about label noise implies that current benchmarks may understate the model's ceiling; a cleaner dataset could shift the apparent gap between MCAM and baselines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes MCAM, a three-component model for ego-vehicle driving video captioning and reasoning: a Multi-level Feature Extractor combining 3DResNet and VidSwin, a Causal Analysis Module built around a Driving State Directed Acyclic Graph, and a Vision-Language Transformer. The paper claims that the DSDAG provides the first causal analysis structure for ego-vehicle-level driving video understanding and reports state-of-the-art performance on BDD-X and CoVLA, with ablations and qualitative examples.

Significance. If the causal analysis claim were operational, this work would be a meaningful step toward explainable autonomous driving. The paper includes a code release, experiments on two public benchmarks, and an ablation study. However, as written the central contribution is not a causal model: the CAM is a generic feature-fusion attention block, and the formal causal equations are not connected to the implementation. The performance comparisons also do not support the SOTA claim across all metrics. These issues undermine the main contribution, although the empirical comparison and the code availability are useful artifacts.

major comments (4)
  1. [Section 3.2, Eqs. (6)-(18) vs (20)-(23)] The DSDAG formalization defines interventional quantities such as do(Yc=c) and FV(V) in Eq. (17), but the CAM implementation in Eqs. (20)-(23) contains only linear projections, concatenation, softmax attention, and residual scaling. There is no graph, no do-operator, no structural equation, and no variable corresponding to Xs, Xe, W, Yc, or Zξ. Consequently, the causal mechanism claimed in the title and abstract is not present in the architecture. This is the load-bearing gap: the paper asserts a causal analysis capability that the described implementation does not instantiate.
  2. [Section 3.3, Eqs. (24)-(25)] The total loss L_total defined in Eq. (25) omits the sparse attention regularizer L_sparse defined in Eq. (24). The text states that the sparsity constraint is 'crucial' and 'mitigat[es] the risk of model hallucination,' but because L_total = L_signal + L_caption and contains no λ term, the regularizer is never optimized. This makes the described anti-hallucination mechanism ineffective as specified. The value of λ is also not reported anywhere.
  3. [Table 2, reasoning columns] The claim of 'SOTA performance in visual-language causal relationship learning' in the abstract is contradicted by the reported numbers. On the BDD-X reasoning task, RAG-Driver achieves B4=11.1, CIDEr=109.1, and METEOR=14.8, whereas MCAM achieves B4=9.1, CIDEr=94.1, and METEOR=14.0. MCAM is worse on three of the four listed reasoning metrics. The SOTA claim is therefore not supported by the table for the reasoning task, which is the core of the causal-relationship objective.
  4. [Tables 2-5, overall statistical reporting] No error bars, confidence intervals, or multiple-seed results are reported for any of the experimental tables. Given that several improvements are small in absolute terms (e.g., narration B4 35.7 vs 34.3), the empirical superiority of MCAM over baselines is not established with statistical certainty. This is a major concern because the paper's quantitative claims rest on these comparisons.
minor comments (6)
  1. [Section 3.2, text following Eq. (6)] The sentence 'ξ is random noise ... that is dependent of the environmental factors' appears to mean 'independent of' rather than 'dependent of'; the correct independence condition is used in Eq. (11) and should be stated consistently.
  2. [Section 3.2, Eq. (24)] The sparse attention regularizer uses a double summation over M with indices i and j but the matrix V is defined only as a 'relationship matrix' without explaining its dimensions or whether the sum denotes an L1 norm; the notation is unclear.
  3. [Table 2] ROUGE values are missing for several methods (S2VT, SAA, WAA, DriveGPT4, RAG-Driver), and the table would benefit from a consistent format; the 'R' column is empty for many entries.
  4. [Section 3.1] The feature size description 'B × F C/2 × H×W/32×32' is a likely typo; the intended dimensionality is ambiguous and should be written with explicit multiplication signs.
  5. [Appendix, DRAMA experiments] The DRAMA dataset is introduced in Table 5 of the appendix but is not mentioned in the datasets subsection (Section 4.1) or in the main experimental setup; this makes the DRAMA results disconnected from the rest of the paper.
  6. [Abstract and affiliation] The affiliation contains a typo: 'Deffense' should be 'Defense'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the DSDAG formalism and the CAM implementation are disconnected, but no result reduces to its own inputs by construction.

full rationale

The paper does not derive its empirical results from the causal equations; it trains and evaluates on standard datasets (BDD-X, CoVLA) and compares against baselines. The DSDAG equations (6)-(18) are presented as motivation, while the CAM implementation (20)-(23) is a separate architectural description. There is no fitted parameter that is then renamed as a prediction, no self-citation chain that carries a load-bearing premise, and no uniqueness theorem imported from prior work. The gap between the causal formalism and the implemented attention-based fusion is a real validity and interpretability concern, but it is not a circularity: the output F = alpha * Fori is not equivalent by construction to the argmax over intervention probabilities in Eq. (18), nor is any quantity defined in terms of the result it is claimed to predict. Under the stated rules, unsupported labeling and missing implementation details are correctness risks, not circular steps. Therefore the honest finding is no significant circularity, with score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the DSDAG causal theory, but the theory is disconnected from the implemented feature fusion. The free parameters (lambda, beta) are never specified, and the environmental coefficients gamma_i are never estimated. The axioms are domain assumptions about driving dynamics that are not tested or incorporated into the learning objective.

free parameters (3)
  • lambda (sparse attention regularization weight) = not specified
    Defined in eq (24) as the weight of Lsparse, but Lsparse is omitted from Ltotal in eq (25); the value is never given.
  • beta (KL divergence weight in caption loss) = not specified
    Appears in eq (25) weighting the KL term in Lcaption; no value is reported.
  • gamma_i (environmental coefficients) = not estimated
    Introduced in eq (6) as coefficients of environmental factors in Zxi; never estimated or used in the implementation.
assumptions (4)
  • domain assumption Hypothesis 1: The environment Z remains invariant at the instants corresponding to the initial state Xs and the final state Xe.
    Section 3.2; this stability assumption underlies the DSDAG but is not tested on the datasets.
  • domain assumption Hypothesis 2: The vehicle executes a sequence of actions Y during the transition, and the state change is jointly driven by environment Z and actions Y.
    Section 3.2; this defines the causal flow but is not incorporated into the model's training objective.
  • domain assumption Hypothesis 3: The action Y is determined by the current state Xs and environment Z, and the new state Xe is influenced by Y and Z.
    Section 3.2; this encodes the intended causal graph, but the implemented CAM does not estimate these dependencies.
  • ad hoc to paper Environmental elements of the same kind are linearly related, with Zxi = {Uv, Gamma_i, xi} and xi independent of Gamma_i.
    Equation (6) posits a linear combination and a noise term without empirical justification; this assumption is not used in the implementation.
invented entities (1)
  • Driving State Directed Acyclic Graph (DSDAG)
    purpose: Theoretical model of driving state transitions and causal relationships (Section 3.2)
    The graph is introduced as a theoretical foundation but is not validated against any external data; the implementation does not construct or use the graph, so there is no falsifiable handle from the DSDAG itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MCAM: Multimodal Causal Analysis Model for Ego-Vehicle-Level Driving Video Understanding." pith.science (2026). https://pith.science/paper/AIY7VM6W

@misc{pith2026250706072,
  author       = {Pith},
  title        = {Pith review of: MCAM: Multimodal Causal Analysis Model for Ego-Vehicle-Level Driving Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AIY7VM6W}},
  note         = {Machine review of arXiv:2507.06072}
}
read the original abstract

Accurate driving behavior recognition and reasoning are critical for autonomous driving video understanding. However, existing methods often tend to dig out the shallow causal, fail to address spurious correlations across modalities, and ignore the ego-vehicle level causality modeling. To overcome these limitations, we propose a novel Multimodal Causal Analysis Model (MCAM) that constructs latent causal structures between visual and language modalities. Firstly, we design a multi-level feature extractor to capture long-range dependencies. Secondly, we design a causal analysis module that dynamically models driving scenarios using a directed acyclic graph (DAG) of driving states. Thirdly, we utilize a vision-language transformer to align critical visual features with their corresponding linguistic expressions. Extensive experiments on the BDD-X, and CoVLA datasets demonstrate that MCAM achieves SOTA performance in visual-language causal relationship learning. Furthermore, the model exhibits superior capability in capturing causal characteristics within video sequences, showcasing its effectiveness for autonomous driving applications. The code is available at https://github.com/SixCorePeach/MCAM.

Figures

Figures reproduced from arXiv: 2507.06072 by the authors.

Figure 1
Figure 1. The relationship of justification and action exists in ego-vehicle level driving video understanding. In the initial state, the vehicle [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the MCAM. The MCAM framework consists of three key components: the MFE module, which extracts multi-level [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. the (a) is Driving Status Directed Acylics Graph (DSDAG), and (b) is the Refined and involved causal graph from DSDAG. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Overview of Causal Analysis Module, (a) is the first and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Visualization for comparing the result with ADAPT in BDD-X dataset, and the red boxes are added by manually to support the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The visualization of MCAM on the CoVLA dataset, includes the ground truth (GT) caption, and the result from MCAM [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Four representative counterexamples demonstrate the performance disparity between DriveGPT4, ADAPT, and MCAM against [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 40 canonical work pages

  1. [1]

    Spatio-temporal dynamics and se- mantic attribute enriched visual encoding for video caption- ing

    Nayyer Aafaq, Naveed Akhtar, Wei Liu, Syed Zulqarnain Gilani, and Ajmal Mian. Spatio-temporal dynamics and se- mantic attribute enriched visual encoding for video caption- ing. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 12487–12496,

    work page

  2. [2]

    Spice: Semantic propositional image cap- tion evaluation

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image cap- tion evaluation. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Octo- ber 11-14, 2016, Proceedings, Part V 14 , pages 382–398. Springer, 2016. 7

    work page 2016

  3. [3]

    Covla: Comprehensive vision-language-action dataset for autonomous driving

    Hidehisa Arai, Keita Miwa, Kento Sasaki, Yu Yamaguchi, Kohei Watanabe, Shunsuke Aoki, and Issei Yamamoto. Covla: Comprehensive vision-language-action dataset for autonomous driving. arXiv preprint arXiv:2408.10845 ,

    arXiv

  4. [4]

    Minigpt4-video: Advancing multimodal llms for video understanding with interleaved visual-textual tokens

    Kirolos Ataallah, Xiaoqian Shen, Eslam Abdelrahman, Es- sam Sleiman, Deyao Zhu, Jian Ding, and Mohamed El- hoseiny. Minigpt4-video: Advancing multimodal llms for video understanding with interleaved visual-textual tokens. arXiv preprint arXiv:2404.03413, 2024. 3

    arXiv 2024

  5. [5]

    Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments. In Proceedings of the acl workshop on in- trinsic and extrinsic evaluation measures for machine trans- lation and/or summarization, pages 65–72, 2005. 6

    2005

  6. [6]

    Egocentric vehicle dense video captioning

    Feiyu Chen, Cong Xu, Qi Jia, Yihua Wang, Yuhan Liu, Hao- tian Zhang, and Endong Wang. Egocentric vehicle dense video captioning. In Proceedings of the 32nd ACM Inter- national Conference on Multimedia , pages 137–146, 2024. 1

    work page 2024

  7. [7]

    Tem- adapter: Adapting image-text pretraining for video question answer

    Guangyi Chen, Xiao Liu, Guangrun Wang, Kun Zhang, Philip HS Torr, Xiao-Ping Zhang, and Yansong Tang. Tem- adapter: Adapting image-text pretraining for video question answer. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 13945–13955, 2023. 2

    work page 2023

  8. [8]

    Llcp: Learning latent causal processes for reasoning-based video question answer

    Guangyi Chen, Yuke Li, Xiao Liu, Zijian Li, Eman Al Suradi, Donglai Wei, and Kun Zhang. Llcp: Learning latent causal processes for reasoning-based video question answer. In ICLR, 2024. 2

    work page 2024

Show all 49 references
  1. [9]

    H-mba: Hierarchical mamba adaptation for multi-modal video understanding in autonomous driving

    Siran Chen, Yuxiao Luo, Yue Ma, Yu Qiao, and Yali Wang. H-mba: Hierarchical mamba adaptation for multi-modal video understanding in autonomous driving. arXiv preprint arXiv:2501.04302, 2025. 1

    2025 arXiv

  2. [10]

    Spatial-temporal trans- former for dynamic scene graph generation

    Yuren Cong, Wentong Liao, Hanno Ackermann, Bodo Rosenhahn, and Michael Ying Yang. Spatial-temporal trans- former for dynamic scene graph generation. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 16372–16382, 2021. 2

    2021

  3. [11]

    Trafficvlm: A controllable visual lan- guage model for traffic video captioning

    Quang Minh Dinh, Minh Khoi Ho, Anh Quan Dang, and Hung Phong Tran. Trafficvlm: A controllable visual lan- guage model for traffic video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7134–7143, 2024. 3

    2024

  4. [12]

    Reversed in time: A novel temporal-emphasized benchmark for cross-modal video-text retrieval

    Yang Du, Yuqi Liu, and Qin Jin. Reversed in time: A novel temporal-emphasized benchmark for cross-modal video-text retrieval. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 5260–5269, 2024. 1

    2024

  5. [13]

    Mllm-sul: Mul- timodal large language model for semantic scene under- standing and localization in traffic scenarios

    Jiaqi Fan, Jianhua Wu, Jincheng Gao, Jianhao Yu, Yafei Wang, Hongqing Chu, and Bingzhao Gao. Mllm-sul: Mul- timodal large language model for semantic scene under- standing and localization in traffic scenarios. arXiv preprint arXiv:2412.19406, 2024. 1

    2024 arXiv

  6. [14]

    Hierarchical representation net- work with auxiliary tasks for video captioning and video question answering

    Lianli Gao, Yu Lei, Pengpeng Zeng, Jingkuan Song, Meng Wang, and Heng Tao Shen. Hierarchical representation net- work with auxiliary tasks for video captioning and video question answering. IEEE Transactions on Image Process- ing, 31:202–215, 2021. 3

    2021

  7. [15]

    Text with knowledge graph aug- mented transformer for video captioning

    Xin Gu, Guang Chen, Yufei Wang, Libo Zhang, Tiejian Luo, and Longyin Wen. Text with knowledge graph aug- mented transformer for video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18941–18951, 2023. 1

    2023

  8. [16]

    Video re- cap: Recursive captioning of hour-long videos

    Md Mohaiminul Islam, Ngan Ho, Xitong Yang, Tushar Na- garajan, Lorenzo Torresani, and Gedas Bertasius. Video re- cap: Recursive captioning of hour-long videos. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18198–18208, 2024. 1

    2024

  9. [17]

    Adapt: Action-aware driving caption transformer

    Bu Jin, Xinyu Liu, Yupeng Zheng, Pengfei Li, Hao Zhao, Tong Zhang, Yuhang Zheng, Guyue Zhou, and Jingjing Liu. Adapt: Action-aware driving caption transformer. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7554–7561, 2023. 1, 6

    2023

  10. [18]

    Cladder: Assessing causal reasoning in language models

    Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, LYU Zhiheng, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, et al. Cladder: Assessing causal reasoning in language models. In Thirty-seventh conference on neural information processing ...

    2023

  11. [19]

    Road-waymo: Action awareness at scale for autonomous driving

    Salman Khan, Izzeddin Teeti, Reza Javanmard Alitappeh, Mihaela C Stoian, Eleonora Giunchiglia, Gurkirt Singh, An- drew Bradley, and Fabio Cuzzolin. Road-waymo: Action awareness at scale for autonomous driving. arXiv preprint arXiv:2411.01683, 2024. 1

    2024 arXiv

  12. [20]

    Textual explanations for self-driving ve- hicles

    Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. Textual explanations for self-driving ve- hicles. In Proceedings of the European conference on com- puter vision (ECCV), pages 563–578, 2018. 6

    2018

  13. [21]

    Learning hierarchical modular networks for video captioning

    Guorong Li, Hanhua Ye, Yuankai Qi, Shuhui Wang, Laiyun Qing, Qingming Huang, and Ming-Hsuan Yang. Learning hierarchical modular networks for video captioning. IEEE transactions on pattern analysis and machine intelligence ,

  14. [22]

    Automatic evaluation of machine translation quality using longest common sub- sequence and skip-bigram statistics

    Chin-Yew Lin and Franz Josef Och. Automatic evaluation of machine translation quality using longest common sub- sequence and skip-bigram statistics. In Proceedings of the 42nd annual meeting of the association for computational linguistics (ACL-04), pages 605–612, 2004. 6

    2004

  15. [23]

    Swin- bert: End-to-end transformers with sparse attention for video 9 captioning

    Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, and Lijuan Wang. Swin- bert: End-to-end transformers with sparse attention for video 9 captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages ...

    2022

  16. [24]

    Cross-modal causal relational reasoning for event-level visual question answer- ing

    Yang Liu, Guanbin Li, and Liang Lin. Cross-modal causal relational reasoning for event-level visual question answer- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):11624–11641, 2023. 1, 2

    2023

  17. [25]

    Spatio-temporal pixel- level contrastive learning-based source-free domain adapta- tion for video semantic segmentation

    Shao-Yuan Lo, Poojan Oza, Sumanth Chennupati, Alejan- dro Galindo, and Vishal M Patel. Spatio-temporal pixel- level contrastive learning-based source-free domain adapta- tion for video semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2023

  18. [26]

    Llavilo: Boosting video moment retrieval via adapter-based multimodal modeling

    Kaijing Ma, Xianghao Zang, Zerun Feng, Han Fang, Chao Ban, Yuhan Wei, Zhongjiang He, Yongxiang Li, and Hao Sun. Llavilo: Boosting video moment retrieval via adapter-based multimodal modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 27...

    2023

  19. [27]

    Icsvr: Investigating compositional and syntactic understanding in video retrieval models

    Avinash Madasu and Vasudev Lal. Icsvr: Investigating compositional and syntactic understanding in video retrieval models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1733– 1743, 2024. 1

    2024

  20. [28]

    Drama: Joint risk localization and captioning in driving

    Srikanth Malla, Chiho Choi, Isht Dwivedi, Joon Hee Choi, and Jiachen Li. Drama: Joint risk localization and captioning in driving. In Proceedings of the IEEE/CVF winter confer- ence on applications of computer vision , pages 1043–1052,

  21. [29]

    Lingoqa: Visual question answering for autonomous driv- ing

    Ana-Maria Marcu, Long Chen, Jan H ¨unermann, Alice Karn- sund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driv- ing. In European Conference on Computer Vision , p...

    2024

  22. [30]

    Query-dependent video representa- tion for moment retrieval and highlight detection

    WonJun Moon, Sangeek Hyun, SangUk Park, Dongchan Park, and Jae-Pil Heo. Query-dependent video representa- tion for moment retrieval and highlight detection. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23023–23033, 2023. 1

    2023

  23. [31]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318,

  24. [32]

    Causality

    Judea Pearl. Causality. Cambridge university press, 2009. 2, 3

    2009

  25. [33]

    Modular learn- ing of deep causal generative models for high-dimensional causal inference

    Md Musfiqur Rahman and Murat Kocaoglu. Modular learn- ing of deep causal generative models for high-dimensional causal inference. arXiv preprint arXiv:2401.01426, 2024. 2

    2024 arXiv

  26. [34]

    Clip4caption: Clip for video caption

    Mingkang Tang, Zhanyu Wang, Zhenhua Liu, Fengyun Rao, Dian Li, and Xiu Li. Clip4caption: Clip for video caption. In Proceedings of the 29th ACM International Conference on Multimedia, pages 4858–4862, 2021. 1

    2021

  27. [35]

    Cider: Consensus-based image description evalua- tion

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evalua- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015. 6

    2015

  28. [36]

    Sequence to sequence-video to text

    Subhashini Venugopalan, Marcus Rohrbach, Jeffrey Don- ahue, Raymond Mooney, Trevor Darrell, and Kate Saenko. Sequence to sequence-video to text. In Proceedings of the IEEE international conference on computer vision , pages 4534–4542, 2015. 6

    2015

  29. [37]

    Deconfounding causal inference for zero-shot action recognition

    Junyan Wang, Yiqi Jiang, Yang Long, Xiuyu Sun, Maurice Pagnucco, and Yang Song. Deconfounding causal inference for zero-shot action recognition. IEEE Transactions on Mul- timedia, 26:3976–3986, 2023. 1

    2023

  30. [38]

    Weakly- supervised video object grounding via causal intervention

    Wei Wang, Junyu Gao, and Changsheng Xu. Weakly- supervised video object grounding via causal intervention. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 45(3):3933–3948, 2022. 1

    2022

  31. [39]

    Rac3: Retrieval-augmented corner case comprehen- sion for autonomous driving with vision-language models

    Yujin Wang, Quanfeng Liu, Jiaqi Fan, Jinlong Hong, Hongqing Chu, Mengjian Tian, Bingzhao Gao, and Hong Chen. Rac3: Retrieval-augmented corner case comprehen- sion for autonomous driving with vision-language models. arXiv preprint arXiv:2412.11050, 2024. 1

    2024

  32. [40]

    Visual causal scene refinement for video question an- swering

    Yushen Wei, Yang Liu, Hong Yan, Guanbin Li, and Liang Lin. Visual causal scene refinement for video question an- swering. In Proceedings of the 31st ACM International Con- ference on Multimedia, pages 377–386, 2023. 2

    2023

  33. [41]

    Bridging the gap: A unified video comprehension framework for mo- ment retrieval and highlight detection

    Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Heng- wei Bian, Yatai Ji, Yujiu Yang, and Xiu Li. Bridging the gap: A unified video comprehension framework for mo- ment retrieval and highlight detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  34. [42]

    Retrieval-augmented egocentric video captioning

    Jilan Xu, Yifei Huang, Junlin Hou, Guo Chen, Yuejie Zhang, Rui Feng, and Weidi Xie. Retrieval-augmented egocentric video captioning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition , pages 13525–13536, 2024. 1

    2024

  35. [43]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Let- ters, 2024. 1, 3, 6, 7

    2024

  36. [44]

    Prompt learns prompt: Exploring knowledge-aware generative prompt collaboration for video captioning

    Liqi Yan, Cheng Han, Zenglin Xu, Dongfang Liu, and Qifan Wang. Prompt learns prompt: Exploring knowledge-aware generative prompt collaboration for video captioning. In IJ- CAI, pages 1622–1630, 2023. 3

    2023

  37. [45]

    Causal attention for vision-language tasks

    Xu Yang, Hanwang Zhang, Guojun Qi, and Jianfei Cai. Causal attention for vision-language tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9847–9857, 2021. 2

    2021

  38. [46]

    Rag-driver: Gen- eralisable driving explanations with retrieval-augmented in- context learning in multi-modal large language model

    Jianhao Yuan, Shuyang Sun, Daniel Omeiza, Bo Zhao, Paul Newman, Lars Kunze, and Matthew Gadd. Rag-driver: Gen- eralisable driving explanations with retrieval-augmented in- context learning in multi-modal large language model. arXiv preprint arXiv:2402.10828, 2024. 1, 3, 6, 7

    2024

  39. [47]

    Vision-language models for vision tasks: A survey

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence ,

  40. [48]

    Causal inference with latent variables: 10 Figure 7

    Yaochen Zhu, Yinhan He, Jing Ma, Mengxuan Hu, Sheng Li, and Jundong Li. Causal inference with latent variables: 10 Figure 7. Four representative counterexamples demonstrate the performance disparity between DriveGPT4, ADAPT, and MCAM against Ground Truth (GT) trajectories. Ann...

    2024

  41. [49]

    Appendix Comparing in BDD-X for failure examples : In Fig- ure 7.(a), the vehicle transitions from a stopped state to for- ward motion, as indicated by the speed and acceleration curves. MCAM specifically investigates the acceleration dynamics during vehicle launch phases and ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.