Pith. sign in

REVIEW 5 major objections 6 minor 97 references

Large Language Models for Crash Detection in Video: A Survey of Methods, Datasets, and Challenges

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM-based video crash detection has shifted from pixel anomaly detection to context-aware event interpretation, but it still faces a deployment gap.

desk verdict Useful taxonomy, but the transcribed numbers don't hang together; needs a fixing pass before it can be trusted as a reference. read the letter →

arxiv 2507.02074 v2 pith:YABIDMOS submitted 2025-07-02 cs.CV cs.AI

Sanjeda Akter , Ibne Farabi Shihab , Anuj Sharma This is my paper
classification cs.CVcs.AI
keywords crashdetectionlargelanguagemodelsvision-languagevideounderstandingmultimodallearningautonomousdrivinganomalysurvey
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey argues that crash detection from video has entered a new phase: instead of just flagging pixels that look anomalous, systems built on large language models and vision-language models interpret crashes as events, generate explanations, and reason about causes. The paper's central claim is that this shift brings real capabilities but also a hard deployment gap, because the models with the richest reasoning are too slow, too memory-hungry, and too prone to hallucination for real-time safety-critical use. It organizes the field into a taxonomy of fusion strategies, LLM roles, and architectures, maps the available datasets, and compares methods. If the survey's reading is right, the next research step is not higher detection accuracy but standardized latency reporting, causal-annotation datasets, and efficiency-first architectures.

What carries the argument

The carrying machinery is a three-axis taxonomy: fusion level (early token concatenation, late fusion with separate heads, or cross-attention), prompt strategy (static prompts, dynamic event templates, learned visual questions), and LLM role (passive captioning, active QA/inference, generative reasoning). Around this taxonomy the survey builds an architecture classification (visual encoder plus LLM decoder, frozen LLM plus learned adapter, joint vision-language pretraining) and an accuracy-versus-latency trade-off diagram. The taxonomy does the argument's load-bearing work: it converts scattered recent systems into a design space, then reads the deployment gap off that space.

What would settle it

A reader could go to the original DAD and CrashLLM papers and check the transcribed numbers: if DAD's true size is about 620 videos rather than 1,500+, or if CrashLLM's reported performance is macro-F1 53.8% rather than 'over 92% accuracy', then the comparison tables and the deployment-readiness labels derived from them would need revision.

Watch

Extended reading notes

Core claim

The survey's central finding is that current LLM/VLM crash detection systems cluster around a few design choices, and that both their capabilities and their deployment barriers follow from those choices. Systems like VERA and Holmes-VAD can verbalize why a crash happened, not merely that one occurred, but they pay for this with latencies in the hundreds of milliseconds and multi-gigabyte memory footprints. By contrast, the only systems the survey labels deployment-ready in real time, such as LA V AD and CRASH, are training-free or lightweight and give up most of the contextual reasoning that makes LLMs valuable. The paper also claims that the biggest data bottleneck is the lack of large-scale datasets with explicit causal chains linking fine-grained events to crash outcomes, which blocks progress from correlation to true causal reasoning.

Load-bearing premise

The survey's comparisons rest on the accuracy, latency, and dataset-size numbers it transcribes from the papers it cites, and those transcriptions are not internally consistent: DAD appears as 1,500+ videos in Table II but about 620 in the conclusion, and CrashLLM appears as over 92% accuracy in Section II-D but macro-F1 53.8% in Table IV.

Editorial extensions

If this is right

  • If the survey's map is correct, no current 7-billion-parameter crash detection LLM can run within the sub-100ms latency budgets of safety-critical vehicle systems; only lightweight training-free models are currently labeled deployment-ready.
  • The reported absence of large-scale datasets linking fine-grained temporal events to crash outcomes means current models cannot move from correlation to causal reasoning, and this is the field's main data bottleneck.
  • The taxonomy implies that choosing a fusion strategy is choosing a point on the accuracy-latency-explainability trade-off, so future systems should be compared along all three axes rather than by accuracy alone.
  • The survey's call to report latency alongside accuracy, and to evaluate across datasets, would become standard practice if its assessment is accepted.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the survey's accuracy-versus-latency picture holds, a hierarchical pipeline—a cheap frame-level detector that flags candidate clips, with LLM reasoning applied only over those clips—is the most direct way to close the deployment gap; the paper lists this as a solution direction but does not test it.
  • The survey's critique of heterogeneous metrics implies a concrete next step: a standard crash-detection leaderboard reporting latency, accuracy, and explanation quality on fixed hardware, which would let the field quantify its trade-offs instead of asserting them.
  • Because the survey relies on transcribed numbers, re-verifying each method's reported metric against its original paper is a cheap first test of the survey's map; the visible internal conflicts suggest this re-check is needed before the tables are used to guide decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The manuscript surveys recent (2023–2025) LLM/VLM approaches to video-based crash detection. It proposes a taxonomy along fusion level, prompt strategy, and LLM role; reviews key datasets; describes three architectural families; discusses evaluation metrics; compares recent methods in Table IV; analyzes deployment readiness in Table V; and outlines challenges and future directions. The paper's central claims are that LLM-based methods add contextual understanding and explanation generation, that the field is shifting from reactive pixel-based detection to context-aware event interpretation, and that a substantial deployment gap remains due to latency, compute, and robustness issues.

Significance. If its factual basis were reliable, this survey would be a useful entry point for researchers, with a clear taxonomy and broad coverage of a fast-moving area. The manuscript also deserves credit for explicitly acknowledging in §VI-D that computational classifications and latency figures are approximate. However, the survey's value as a synthesis depends on accurate transcription of primary sources, and the visible internal contradictions in dataset sizes, model metrics, architecture descriptions, and unsourced failure statistics undermine confidence in the comparison tables and in the deployment-gap argument. The paper is repairable, but in its current form it is not yet a reliable map of the field.

major comments (5)
  1. [Table II, §VII-A, §X-A] The DAD dataset size is reported inconsistently: Table II and §VII-A say 1,500+ videos, while §X-A states that DAD contains '~620 dashcam videos (with positive/negative splits)'. Since DAD is used as a benchmark for VERA, Video-LLaMA, and CRASH, this discrepancy affects the dataset map and the data-bottleneck argument. Please verify the count against the primary source [61] and state one consistent number, or clearly explain the version/split difference.
  2. [§II-D2, §II-E1, §IV-B, Table IV] CrashLLM's reported performance is contradictory: §II-D2 and §II-E1 credit it with 'over 92% accuracy', while §IV-B and Table IV report macro-F1 improving from 34.9% to 53.8% on CrashEvent. The paper never reconciles these as different metrics or tasks. In addition, §IV-B describes CrashLLM's visual backbone as a Swin Transformer, while §VI-B describes it as ResNet-50. Since CrashLLM is a flagship example in the taxonomy and comparison, these inconsistencies must be resolved with precise metric definitions and a single architecture description.
  3. [§VII-F, §VII-G] The robustness numbers in §VII-F — 12–16% out-of-distribution drops, 40% occlusion misses, 25% weather degradation, 30% lighting reduction, and 60% adversarial false negatives — are presented as empirical findings without any citation. Section VII-G repeats the '12–16% on average' figure without a source. These figures are load-bearing for the deployment-gap claim, so they must either be traced to specific sources with the evaluation protocol described, or be removed and rephrased as unsupported estimates.
  4. [§IV-A, §II-C, §III-A, Table IV] Video-LLaMA's role in crash detection is described inconsistently. Table IV states 'N/A (no crash benchmarks)' and §VI-B says the original paper does not report crash-specific benchmarks, but §IV-A says Video-LLaMA detects 'crash anomalies on datasets like UCF-Crime', §II-C presents it as a step toward temporal reasoning in crash detection, and §III-A says DAD is used to train VLMs like Video-LLaMA. Please distinguish the original model's reported benchmarks from downstream applications that use it as a backbone.
  5. [§VI-D, Table V] The deployment-readiness conclusions rest partly on unmeasured values. The 'Note on Computational Estimates' in §VI-D concedes that computational classifications and latency estimates are approximate, yet Table V labels methods 'Deployment Ready' or 'No' based on those columns and on reported metrics. Please either obtain latency and memory measurements from the primary sources, or clearly mark all estimate-derived cells in Table V and soften the readiness labels accordingly.
minor comments (6)
  1. [Reference [59], §II-E2] Reference [59] (TrafficLens) lists 'Unknown' as the author and provides no URL or venue, making the citation unverifiable; since this example anchors the 'Dynamic Event Templates' taxonomy category, it should be replaced with a traceable source or removed.
  2. [Reference [64], §III-A] Reference [64] attributes UCF-Crime to 'W. Soomro, A. R. Zamir, and M. Shah'; the actual authors are Sultani, Chen, and Shah (as correctly listed in reference [10]). Please correct the citation.
  3. [§IV-B] The word 'demonstaring' in §IV-B should be 'demonstrating'.
  4. [§V-B, Table IV] Section V-B refers to 'anticipation-focused datasets such as CRASH', but CRASH is presented in Table IV and §VI-B as a method/model. Please clarify whether CRASH is also a dataset and avoid mixing these roles.
  5. [References [44] and [67]] References [44] and [67] refer to the same CRASH system with different author lists and one incomplete entry ('Y. Liao et al.'). Please consolidate them into a single complete citation.
  6. [Figure 3] The timeline in Figure 3 lists 'LLaV A-1.5' with an extra space; this should read 'LLaVA-1.5' for consistency with the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the survey's central claim is an organizational synthesis of external works, not a result derived from its own inputs.

full rationale

This is a survey paper, so there is no derivation chain whose conclusion could reduce to its premises. The central claim—that LLM/VLM methods add contextual understanding and explanation while facing deployment barriers—is a structured summary of the cited literature, not a theorem or fitted prediction. The paper's taxonomy of fusion strategies, prompt strategies, LLM roles, and architectures is an organizational contribution, and its comparison tables transcribe externally reported metrics. The two self-citations, [57] and [86], are not load-bearing: [57] is cited as an example of LLM advantages in contextual understanding, and [86] appears as one surveyed method in Table IV and Table V; neither is used to justify the survey's overall claims, and both are accompanied by many independent external citations. The limitations and internal inconsistencies the reader identifies are correctness and reliability issues, not circularity. For example, the 'Note on Computational Estimates' in Section VI-D explicitly concedes that latency and deployment classifications are approximate, which is an honest caveat; the DAD size mismatch (1,500+ videos in Table II vs. ~620 in Section X-A3), the CrashLLM accuracy discrepancy (over 92% in Section II-D vs. macro-F1 34.9% to 53.8% in Table IV), and the unsourced failure-rate percentages in Section VII-F all concern data quality and verifiability, not an argument that reduces to its own input. Likewise, the unverifiable TrafficLens reference [59] (author listed as 'Unknown', no URL) is a citation-completeness problem, not a circularity. Because the paper's comparative and deployment conclusions rest on external, independently published results rather than on self-referential definitions or fitted parameters, there is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

A survey introduces no fitted parameters or new entities. The main unstated premise is that the reported literature facts are accurate and representative; this premise is partially violated by the internal contradictions listed in the red flags.

assumptions (2)
  • domain assumption The papers selected for review are representative of the field and their reported metrics are accurately transcribed.
    The survey's taxonomy and comparison tables depend on this; internal contradictions (DAD size, CrashLLM metrics, Video-LLaMA benchmark use) show the assumption is violated in places.
  • ad hoc to paper The taxonomy dimensions (fusion level, prompt strategy, LLM role) are jointly exhaustive and meaningful for describing the field.
    No formal basis or inter-annotator agreement is given; the categories are presented as a structuring device for the survey.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Models for Crash Detection in Video: A Survey of Methods, Datasets, and Challenges." pith.science (2026). https://pith.science/paper/YABIDMOS

@misc{pith2026250702074,
  author       = {Pith},
  title        = {Pith review of: Large Language Models for Crash Detection in Video: A Survey of Methods, Datasets, and Challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YABIDMOS}},
  note         = {Machine review of arXiv:2507.02074}
}
read the original abstract

Crash detection from video feeds is a critical problem in intelligent transportation systems. Recent developments in large language models (LLMs) and vision-language models (VLMs) have transformed how we process, reason about, and summarize multimodal information. This paper surveys recent methods leveraging LLMs for crash detection from video data. We present a structured taxonomy of fusion strategies, summarize key datasets, analyze model architectures, compare performance benchmarks, and discuss ongoing challenges and opportunities. Our review provides a foundation for future research in this fast-growing intersection of video understanding and foundation models.

Figures

Figures reproduced from arXiv: 2507.02074 by the authors.

Figure 1
Figure 1. Taxonomy of LLM-based crash detection in videos categorized by fusion strategy, LLM role, and input [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Accuracy vs. Latency Trade-off Analysis for [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Timeline for Video-capable LLMs for Crash Detection [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Diagram illustrating different fusion strategies. (a) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Radar chart comparing key attributes of crash [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: A generic architecture for an LLM-based crash [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: An overview of the key challenges hindering the deployment of LLM-based crash detection systems and [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Integration of an LLM-based crash detection module within a standard Autonomous Vehicle (AV) perception [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

97 extracted references · 51 canonical work pages

  1. [59]

    Trafficlens: A novel system for video-to-text conver- sion in multi-camera traffic environments,

    Unknown, “Trafficlens: A novel system for video-to-text conver- sion in multi-camera traffic environments,”NEC Labs Technical Report, 2024

    work page 2024

  2. [57]

    Leveraging video-llms for crash detection and narrative generation: Performance analysis and challenges,

    I. F. Shihaba, B. I. Alvee, and A. Sharma, “Leveraging video-llms for crash detection and narrative generation: Performance analysis and challenges,” inProceedings of the 30th TRC Conference (TRC-30). https://trc-30. epfl. ch/wpcontent/uploads/2024/09/TRC- 30 paper 238. pdf, 2024

    work page 2024

  3. [86]

    Crash time matters: Hybridmamba for fine-grained temporal localization in traffic surveillance footage,

    I. F. Shihab and A. Sharma, “Crash time matters: Hybridmamba for fine-grained temporal localization in traffic surveillance footage,”arXiv preprint arXiv:2504.03235, 2025

    work page arXiv 2025

  4. [61]

    Dad: A dashcam accident dataset,

    F.-H. Chan, Y .-T. Chen, Y . Xiang, and M. Sun, “Dad: A dashcam accident dataset,” inInternational Conference on Pattern Recognition (ICPR). IEEE, 2016, pp. 3431–3436

    work page 2016

  5. [1]

    Traffic monitoring and accident detection at intersections,

    S. Kamijo, Y . Matsushita, K. Ikeuchi, and M. Sakauchi, “Traffic monitoring and accident detection at intersections,” inIEEE Transactions on Intelligent Transportation Systems, vol. 1, no. 2. IEEE, 2000, pp. 108–118

    2000

  6. [2]

    A survey of vision-based trajec- tory learning and analysis for surveillance,

    B. T. Morris and M. M. Trivedi, “A survey of vision-based trajec- tory learning and analysis for surveillance,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 8, pp. 1114–1127, 2008

    2008

  7. [3]

    Trajectory-based anomalous event detection,

    C. Piciarelli, C. Micheloni, and G. L. Foresti, “Trajectory-based anomalous event detection,” inIEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 11. IEEE, 2008, pp. 1544–1554

    2008

  8. [4]

    Development of artificial neural network models to predict driver injury severity in traffic accidents at signalized intersections,

    H. T. Abdelwahab and M. A. Abdel-Aty, “Development of artificial neural network models to predict driver injury severity in traffic accidents at signalized intersections,”Transportation Research Record, vol. 1746, no. 1, pp. 6–13, 2000

    2000

Show all 97 references
  1. [5]

    Severity of driver injury and vehicle damage in traffic crashes at intersections: a bayesian hierarchical analysis,

    H. Huang, H. C. Chin, and M. M. Haque, “Severity of driver injury and vehicle damage in traffic crashes at intersections: a bayesian hierarchical analysis,”Accident Analysis & Prevention, vol. 40, no. 1, pp. 45–54, 2008

    2008

  2. [6]

    Two-stream convolutional networks for action recognition in videos,

    K. Simonyan and A. Zisserman, “Two-stream convolutional networks for action recognition in videos,” inAdvances in Neural Information Processing Systems, 2014, pp. 568–576

    2014

  3. [7]

    Learning spatiotemporal features with 3d convolutional networks,

    D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” inProceedings of the IEEE International Conference on Computer Vision, 2015, pp. 4489–4497

    2015

  4. [8]

    Quo vadis, action recognition? a new model and the kinetics dataset,

    J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299–6308

    2017

  5. [9]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 6202– 6211. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. XX, NO. X, MONTH YEAR 22

    2019

  6. [10]

    Real-world anomaly detection in surveillance videos,

    W. Sultani, C. Chen, and M. Shah, “Real-world anomaly detection in surveillance videos,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6479–6488

    2018

  7. [11]

    Future frame prediction for anomaly detection–a new baseline,

    W. Liu, W. Luo, D. Lian, and S. Gao, “Future frame prediction for anomaly detection–a new baseline,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6536–6545

    2018

  8. [12]

    Learning memory-guided nor- mality for anomaly detection,

    H. Park, J. Noh, and B. Ham, “Learning memory-guided nor- mality for anomaly detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 14 372–14 381

    2020

  9. [13]

    Uniter: Universal image-text representation learning,

    Y .-C. Chen, L. Li, L. Yu, A. E. Kholy, F. Ahmed, Z. Gan, Y . Cheng, and J. Liu, “Uniter: Universal image-text representation learning,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 104–120

    2020

  10. [14]

    Align before fuse: Vision and language representation learning with momentum distillation,

    J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi, “Align before fuse: Vision and language representation learning with momentum distillation,” inAdvances in Neural Information Processing Systems, vol. 34, 2021, pp. 9694–9705

    2021

  11. [15]

    Less is more: Clipbert for video-and-language learning via sparse sampling,

    J. Lei, L. Li, L. Zhou, Z. Gan, T. L. Berg, M. Bansal, and J. Liu, “Less is more: Clipbert for video-and-language learning via sparse sampling,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 7331– 7341

    2021

  12. [16]

    Videoclip: Contrastive pre-training for zero-shot video-text understanding,

    H. Xu, G. Ghosh, P.-Y . Huang, D. Okhonko, A. Aghajanyan, F. Metze, L. Zettlemoyer, and C. Feichtenhofer, “Videoclip: Contrastive pre-training for zero-shot video-text understanding,” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 202...

    2021

  13. [17]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  14. [18]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: a visual language model for few-shot learning,”Advances in Neural Information Processing Systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  15. [19]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 19 730–19 745

    2023

  16. [20]

    Anomalous video event detection using spatiotemporal context,

    F. Jiang, J. Yuan, S. A. Tsaftaris, and A. K. Katsaggelos, “Anomalous video event detection using spatiotemporal context,” Computer Vision and Image Understanding, vol. 115, no. 3, pp. 323–333, 2011

    2011

  17. [21]

    Detecting anomalies in people’s trajectories using spectral graph analysis,

    S. Calderara, U. Heinemann, A. Prati, R. Cucchiara, and N. Tishby, “Detecting anomalies in people’s trajectories using spectral graph analysis,”Computer Vision and Image Understanding, vol. 115, no. 7, pp. 915–924, 2011

    2011

  18. [22]

    Using support vector machine models for crash injury severity analysis,

    Z. Li, P. Liu, W. Wang, and C. Xu, “Using support vector machine models for crash injury severity analysis,”Accident Analysis & Prevention, vol. 45, pp. 478–486, 2012

    2012

  19. [23]

    Accident detection system using image processing and mdr,

    Y .-K. Ki and D.-Y . Lee, “Accident detection system using image processing and mdr,” inWorld Congress on Engineering and Computer Science, 2008

    2008

  20. [24]

    Temporal segment networks: Towards good practices for deep action recognition,

    L. Wang, Y . Xiong, Z. Wang, Y . Qiao, D. Lin, X. Tang, and L. Van Gool, “Temporal segment networks: Towards good practices for deep action recognition,” inEuropean Conference on Computer Vision. Springer, 2016, pp. 20–36

    2016

  21. [25]

    Multimodal machine learning: A survey and taxonomy,

    T. Baltruˇsaitis, C. Ahuja, and L.-P. Morency, “Multimodal machine learning: A survey and taxonomy,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 2, pp. 423–443, 2018

    2018

  22. [26]

    Multimodal deep learning,

    J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y . Ng, “Multimodal deep learning,” inProceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 689–696

    2011

  23. [27]

    Deep multimodal learning: A survey on recent advances and trends,

    D. Ramachandram and G. W. Taylor, “Deep multimodal learning: A survey on recent advances and trends,”IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 96–108, 2017

    2017

  24. [28]

    Simvlm: Simple visual language model pretraining with weak supervision,

    Z. Wang, J. Yu, A. W. Yu, Z. Dai, Y . Tsvetkov, and Y . Cao, “Simvlm: Simple visual language model pretraining with weak supervision,”arXiv preprint arXiv:2108.10904, 2021

    2021 arXiv

  25. [29]

    All in one: Exploring unified video-language pre-training,

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bow- man, “All in one: Exploring unified video-language pre-training,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 652–661

    2022

  26. [30]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 3213–3223

    2016

  27. [31]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 621–11 631

    2020

  28. [32]

    Are we ready for autonomous driving? the kitti vision benchmark suite,

    A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in2012 IEEE Confer- ence on Computer Vision and Pattern Recognition. IEEE, 2012, pp. 3354–3361

    2012

  29. [33]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,”IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016

    2016

  30. [34]

    Edge intelligence: Paving the last mile of artificial intelligence with edge computing,

    Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,”Proceedings of the IEEE, vol. 107, no. 8, pp. 1738–1762, 2019

    2019

  31. [35]

    Videobert: A joint model for video and language representa- tion learning,

    C. Sun, A. Myers, C. V ondrick, K. Murphy, and C. Schmid, “Videobert: A joint model for video and language representa- tion learning,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 7464–7473

    2019

  32. [36]

    Vivit: A video vision transformer,

    A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: A video vision transformer,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 6836–6846

    2021

  33. [37]

    Videogpt: Video generation using vq-vae and transformers,

    W. Yan, Y . Zhang, P. Abbeel, and A. Srinivas, “Videogpt: Video generation using vq-vae and transformers,” inInternational Conference on Learning Representations (ICLR), 2021

    2021

  34. [38]

    Video-llama: An instruction-tuned audio-visual language model for video understanding,

    H. Zhang, X. Li, and L. Bing, “Video-llama: An instruction-tuned audio-visual language model for video understanding,”arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

  35. [39]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Improved baselines with visual instruction tuning,”arXiv preprint arXiv:2310.03744, 2023

    2023 arXiv

  36. [40]

    Palm-e: An embodied multimodal language model,

    D. Driess, H. Black, J. Kew, X. Tian, S. Hong, B. Mahler, C. Lee, U. Kirmani, E. Hessey, K. Corradoet al., “Palm-e: An embodied multimodal language model,”arXiv preprint arXiv:2303.03378, 2023

    2023 arXiv

  37. [41]

    Gpt-4v(ision) system card,

    OpenAI, “Gpt-4v(ision) system card,”OpenAI Technical Report, 2023. [Online]. Available: https://openai.com/research/ gpt-4v-system-card

    2023

  38. [42]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    Google, “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,”Google AI Blog, 2024. [Online]. Available: https://blog.google/technology/ai/google-gemini-1-5/

    2024

  39. [43]

    Sora: Creating video from text,

    OpenAI, “Sora: Creating video from text,”OpenAI Technical Report, 2024. [Online]. Available: https://openai.com/sora

    2024

  40. [44]

    Crash: Crash recognition and anticipation system harnessing with context-aware and temporal focus attentions,

    H. Liao, H. Sun, H. Shen, C. Wang, K. Tam, C. Tian, L. Li, C. Xu, and Z. Li, “Crash: Crash recognition and anticipation system harnessing with context-aware and temporal focus attentions,”

  41. [45]

    When language and vision meet road safety: Leveraging multimodal large language models for video-based traffic accident analysis,

    R. Zhang, B. Wang, J. Zhang, Z. Bian, C. Feng, and K. Ozbay, “When language and vision meet road safety: Leveraging multimodal large language models for video-based traffic accident analysis,”arXiv preprint arXiv:2501.10604, 2025. [Online]. Available: https://arxiv.org/abs/2501.10604

    2025 arXiv

  42. [46]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agar- wal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  43. [47]

    Clip4clip: An empirical study of clip for end-to-end video clip retrieval and captioning,

    H. Luo, L. Ji, M. Zhong, Y . Chen, W. Lei, N. Duan, and T. Li, “Clip4clip: An empirical study of clip for end-to-end video clip retrieval and captioning,”arXiv preprint arXiv:2104.08860, 2021. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. XX, NO. X, MONTH YEAR 23

    2021 arXiv

  44. [48]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  45. [49]

    Quick and robust detection of crash events from video streams,

    J. Wang, J. Li, Y . Wang, and X. He, “Quick and robust detection of crash events from video streams,”IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 8, pp. 5076–5086, 2020

    2020

  46. [50]

    A real-time system for detection of road accidents from video streams,

    D. Singh and M. Singh, “A real-time system for detection of road accidents from video streams,” in2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC). IEEE, 2018, pp. 477–481

    2018

  47. [51]

    Video-based traffic accident detection: A survey,

    J. Tang, Z. Zhang, A. Astolfo, and Y . Li, “Video-based traffic accident detection: A survey,”IEEE Transactions on Intelligent Transportation Systems, 2023

    2023

  48. [52]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,”arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  49. [53]

    Learning to generate text for video-based traffic crash analysis,

    S. Fan, Z. Zhang, Y . Li, J. Tang, and A. Astolfo, “Learning to generate text for video-based traffic crash analysis,”arXiv preprint arXiv:2402.16682, 2024

    2024 arXiv

  50. [54]

    Name and explain your way to action: A video-based framework for anomaly detection,

    J. Wang, Y . Li, Z. Zhang, and A. Astolfo, “Name and explain your way to action: A video-based framework for anomaly detection,” arXiv preprint arXiv:2401.14412, 2024

    2024 arXiv

  51. [55]

    Video anomaly detection with large language models,

    H. Lv, C. Chen, Y . Zhao, and C. Chen, “Video anomaly detection with large language models,”arXiv preprint arXiv:2402.13849, 2024

    2024 arXiv

  52. [56]

    Large language models are temporal and causal reasoners for video question answering,

    D. Zarza, S. M. A. Etesami, and A. Habibian, “Large language models are temporal and causal reasoners for video question answering,”arXiv preprint arXiv:2305.07343, 2023

    2023 arXiv

  53. [58]

    Deep learning for background subtraction: a survey,

    T. Bouwmans, “Deep learning for background subtraction: a survey,”arXiv preprint arXiv:1904.07223, 2019

    1904 arXiv

  54. [60]

    Harnessing the power of large language models for training-free video anomaly detection,

    R. Zanella, C.-H. Li, and F. Zhuang, “Harnessing the power of large language models for training-free video anomaly detection,” arXiv preprint arXiv:2402.12204, 2024

    2024 arXiv

  55. [62]

    Cadp: A novel dataset for car accident detection and prediction from police reports,

    W. Bao, K. Jiang, and G. Yu, “Cadp: A novel dataset for car accident detection and prediction from police reports,” in Proceedings of the 27th ACM International Conference on Multimedia, 2019, pp. 2553–2559

    2019

  56. [63]

    Bdd100k: A diverse driving dataset for heterogeneous multitasking,

    F. Yu, H. Chen, X. Wang, W. Xian, Y . Chen, F. Liu, V . Mad- havan, and T. Darrell, “Bdd100k: A diverse driving dataset for heterogeneous multitasking,”arXiv preprint arXiv:1805.04687, 2020

    2020 arXiv

  57. [64]

    Ucf-crime: A new video dataset of anomalies in crowds,

    W. Soomro, A. R. Zamir, and M. Shah, “Ucf-crime: A new video dataset of anomalies in crowds,”arXiv preprint arXiv:1801.03759, 2018

    2018 arXiv

  58. [65]

    Scvlm: a vision-language model for driving safety critical event understanding,

    L. Shi, B. Jiang, and F. Guo, “Scvlm: a vision-language model for driving safety critical event understanding,”arXiv preprint arXiv:2410.00982, 2024

    2024 arXiv

  59. [66]

    Trafficvlm: A controllable visual language model for traffic video captioning,

    Q. M. Dinh, M. K. Ho, A. Q. Dang, and H. P. Tran, “Trafficvlm: A controllable visual language model for traffic video captioning,” arXiv preprint arXiv:2404.09275, 2024

    2024 arXiv

  60. [67]

    Crash: A context-aware attention-based framework for crash anticipation,

    Y . Liaoet al., “Crash: A context-aware attention-based framework for crash anticipation,” 2024

    2024

  61. [68]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,”arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  62. [69]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” inproceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6202– 6211

    2019

  63. [70]

    Self-supervised anomaly detection: A survey and outlook,

    H. Hojjati, T. K. K. Ho, and N. Armanfard, “Self-supervised anomaly detection: A survey and outlook,”Neural Networks, 2024

    2024

  64. [71]

    Few-shot fast-adaptive anomaly detection,

    Z. Wang, Y . Zhou, R. Wang, T.-Y . Lin, A. Shah, and S.-N. Lim, “Few-shot fast-adaptive anomaly detection,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 1242–1255

    2022

  65. [72]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  66. [73]

    A survey on deep learn- ing techniques for video anomaly detection,

    J. J. P. Suarez and P. C. Naval Jr, “A survey on deep learn- ing techniques for video anomaly detection,”arXiv preprint arXiv:2009.14146, 2020

    2009 arXiv

  67. [74]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  68. [75]

    Convolutional lstm network: A machine learning approach for precipitation nowcasting,

    X. Shi, Z. Chen, H. Wang, D.-Y . Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” inAdvances in neural information processing systems, 2015, pp. 802–810

    2015

  69. [76]

    Video anomaly detection in 10 years: A survey and outlook,

    M. Abdalla, S. Javed, M. Al Radi, A. Ulhaq, and N. Werghi, “Video anomaly detection in 10 years: A survey and outlook,” arXiv preprint arXiv:2405.19387, 2024

    2024 arXiv

  70. [77]

    Vivit: A video vision transformer,

    A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: A video vision transformer,”arXiv preprint arXiv:2103.15691, 2021

    2021 arXiv

  71. [78]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” inProceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318

    2002

  72. [79]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic eval- uation measures for machine translation and/or summarization, 2005, pp. 65–72

    2005

  73. [80]

    Cider: Consensus-based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus-based image description evaluation,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 4566–4575

    2015

  74. [81]

    Sutd-trafficqa: A question answering benchmark and an efficient network for autonomous driving,

    M.-H. Xu, Q. Jin, Y . Cheng, and Z. Wu, “Sutd-trafficqa: A question answering benchmark and an efficient network for autonomous driving,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 16 392–16 401

    2021

  75. [82]

    Causallp: Visual causal question answering with knowledge graphs,

    U. Jaimini, A. Basu, and S. Ghosh, “Causallp: Visual causal question answering with knowledge graphs,” inProceedings of the 21st International Conference on Principles of Knowledge Representation and Reasoning (KR Workshop on Causality), 2024

    2024

  76. [83]

    Causalchaos!: A cartoon-based causal qa dataset,

    K. Lam, Y . Xu, H. Zhang, and S.-C. Zhu, “Causalchaos!: A cartoon-based causal qa dataset,”arXiv preprint arXiv:2404.01299, 2024

    2024 arXiv

  77. [84]

    Bolstering causal reasoning in video question answering with multi-event causal discovery,

    X. Chen, X. Li, Y . Wang, and X. Yang, “Bolstering causal reasoning in video question answering with multi-event causal discovery,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  78. [85]

    Holmes-vad: Towards unbi- ased and explainable video anomaly detection,

    H. Zhang, X. Xu, X. Wang, J. Zuo, C. Han, X. Huang, C. Gao, Y . Wang, and N. Sang, “Holmes-vad: Towards unbi- ased and explainable video anomaly detection,”arXiv preprint arXiv:2406.12235, 2024

    2024 arXiv

  79. [87]

    Deep learning for video anomaly detection: A review,

    P. Wu, C. Pan, Y . Yan, G. Pang, P. Wang, and Y . Zhang, “Deep learning for video anomaly detection: A review,”arXiv preprint arXiv:2409.05383, 2024

    2024 arXiv

  80. [88]

    Edge-based video analytics: A survey,

    J.-k. Zhang, Q.-l. Song, Z.-h. Chen, Z.-l. Zhang, and J.-j. Wu, “Edge-based video analytics: A survey,”arXiv preprint arXiv:2303.14329, 2023

    2023 arXiv

  81. [89]

    Synthetic datasets for autonomous driving: IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. XX, NO. X, MONTH YEAR 24 A survey,

    Z. Song, Z. He, X. Li, Q. Ma, R. Ming, Z. Mao, H. Pei, L. Peng, J. Hu, D. Yaoet al., “Synthetic datasets for autonomous driving: IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. XX, NO. X, MONTH YEAR 24 A survey,”IEEE Transactions on Intelligent Vehicles, vol. 9, ...

    2024

  82. [90]

    Domain general- ization through meta-learning: a survey,

    A. Gholamzadeh Khoee, Y . Yu, and R. Feldt, “Domain general- ization through meta-learning: a survey,”Artificial Intelligence Review, vol. 57, no. 1, p. 285, 2024

    2024

  83. [91]

    Generalized out-of- distribution detection: A survey,

    J. Yang, K. Zhou, Y . Li, and Z. Liu, “Generalized out-of- distribution detection: A survey,”arXiv preprint arXiv:2110.11334, 2021

    2021 arXiv

  84. [92]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” inConference on robot learning. PMLR, 2017, pp. 1–16

    2017

  85. [93]

    Perception, planning, control, and coordination for autonomous vehicles,

    S. D. Pendleton, H. Andersen, X. Du, X. Shen, M. Meghjani, Y . H. Eng, D. Rus, and M. H. Ang, “Perception, planning, control, and coordination for autonomous vehicles,”Machines, vol. 5, no. 1, p. 6, 2017

    2017

  86. [94]

    A survey of the multi-sensor fusion object detection task in autonomous driving,

    H. Wang, J. Liu, H. Dong, and Z. Shao, “A survey of the multi-sensor fusion object detection task in autonomous driving,” Sensors, vol. 25, no. 9, p. 2794, 2025

    2025

  87. [95]

    A survey on efficient vision-language models,

    G. Shinde, A. Ravi, E. Dey, S. Sakib, M. Rampure, and N. Roy, “A survey on efficient vision-language models,”arXiv preprint arXiv:2504.09724, 2025

    2025 arXiv

  88. [96]

    Privacy-preserving video anomaly detection: A survey,

    Y . Liu, S. Liu, X. Zhu, J. Li, H. Yang, L. Teng, J. Guo, Y . Wang, D. Yang, and J. Liu, “Privacy-preserving video anomaly detection: A survey,” 2025. [Online]. Available: https://arxiv.org/abs/2411.14565

    2025 arXiv

  89. [2024]

    Available: https://arxiv.org/abs/2407.17757

    [Online]. Available: https://arxiv.org/abs/2407.17757

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.