Pith. sign in

REVIEW 4 major objections 4 minor 9 cited by

VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VERA claims that a frozen vision-language model can perform explainable video anomaly detection by learning a small set of natural-language guiding questions through a verbal learner–optimizer loop, using only coarse video-level labels…

desk verdict VERA applies verbalized learning to VAD with a genuinely novel idea, but the SOTA claim rests on unfair baselines and single-run evidence. read the letter →

arxiv 2412.01095 v3 pith:GYXLWB4P submitted 2024-12-02 cs.AI cs.CVcs.LG

Muchao Ye , Weiyang Liu , Pan He This is my paper · ORCID
classification cs.AIcs.CVcs.LG
keywords videoanomalydetectionexplainablevision-languagemodelsverbalizedlearninglearnablepromptsweaklysupervisedfrozenpromptoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a frozen vision-language model can integrate description and reasoning for video anomaly detection without instruction tuning or an external reasoning module. The proposed method, VERA, learns a small set of natural-language guiding questions that decompose the vague notion of "anomaly" into concrete, checkable patterns such as people out of place, vehicles misused, or visible damage. These questions are optimized through verbal feedback between a learner VLM and an optimizer VLM, using only coarse video-level labels, and are then embedded in prompts at inference time to produce segment-level and frame-level anomaly scores. If true, explainable anomaly detection becomes much cheaper: no parameter updates, no frame-level annotation, and no extra LLM calls during inference.

What carries the argument

The central object is the learned guiding-question set $Q$, treated as language-based learnable parameters. A learner agent, the frozen VLM prompted with $Q$, makes a binary video-level prediction on uniformly sampled frames; an optimizer agent, the same frozen VLM given a different prompt plus batched frames, predictions, and ground-truth labels, rewrites $Q$. At inference, the best $Q^*$ is inserted into the prompt to score 10-second segments, and those scores are refined by ensembling with scene-similar segments using cosine similarity and ImageBind features, then by Gaussian smoothing and position weighting to yield frame-level scores. The questions carry the argument: they operationalize "anomaly" into patterns the VLM can verify.

What would settle it

Run VERA on UCF-Crime with the optimizer replaced by a non-learning prompt shuffler that paraphrases the current questions without seeing frames or labels; if the paraphrased prompts reach the same 86.55% AUC, the learner–optimizer verbal loop is not what carries the result, and if they fall well short, the data-driven optimization is load-bearing.

Watch

Extended reading notes

Core claim

The paper's central claim is that the bottleneck for frozen VLMs in video anomaly detection is the prompt, not the model. With questions that describe concrete abnormal patterns, a frozen InternVL2-8B reaches 86.55% AUC on UCF-Crime and 88.26% on XD-Violence, reported as the best among explainable methods in a fair comparison, while still being able to answer open-ended follow-up questions. The paper argues that the complex reasoning VAD seems to require can be decomposed into verbal reflections that are learnable, transferable across datasets and models, and sufficient to replace both instruction tuning and external LLM reasoning modules.

Load-bearing premise

The load-bearing premise, stated in the paper's limitations, is that the frozen VLM's visual encoder perceives the fine-grained details needed to distinguish anomaly from normal; if the encoder misses those cues, the learned questions and the frame-level scores inherit the same blind spots.

Editorial extensions

If this is right

  • Explainable VAD can run end-to-end with a frozen VLM, so deployment needs no GPU fine-tuning and keeps the model's general question-answering ability intact.
  • The learned questions transfer across model sizes, architectures, and datasets, so anomaly-detection knowledge can be shipped as text rather than as weights.
  • Coarse video-level labels are sufficient for training, avoiding the frame-level instruction-tuning datasets used by prior explainable pipelines.
  • Anomaly scoring can be made context-aware in three cheap steps, segment prompting, scene-similarity ensembling, and temporal smoothing, each of which adds measurable AUC.
  • Because the backbone stays frozen, VERA can be upgraded simply by swapping in a newer frozen VLM and reusing or re-learning the questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the learner–optimizer verbal loop is a general recipe: any video-understanding task with coarse labels and a checkable visual predicate could be recast as learning a small set of guiding questions, not just anomaly detection.
  • Beyond the paper, the reported transfer asymmetry, in which questions from the smaller model help the larger one but not vice versa, suggests question complexity should be matched to VLM reasoning ability; one could test this by measuring AUC against the syntactic complexity of learned questions.
  • Beyond the paper, a stress test follows from the authors' stated limitation: evaluate VERA on anomalies that CLIP-style encoders are known to miss, such as small-object or fine-grained manipulation anomalies; failure there would confirm the perception bottleneck rather than the question-optimization component.
  • Beyond the paper, the framing suggests a cheap ablation: re-derive the questions with a stronger visual encoder while keeping the same language model, which would localize how much of VERA's lift comes from perception versus verbal prompting.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes VERA, a verbalized-learning framework that adapts a frozen vision-language model to weakly supervised video anomaly detection. The method treats a small set of natural-language "guiding questions" as learnable parameters, refining them through a learner-optimizer loop in which both agents are implemented with the same frozen VLM. At inference, the learned questions are inserted into the prompt, segment-level anomaly scores are generated, and these scores are refined by scene-context retrieval (ImageBind) and temporal Gaussian/position weighting. Experiments on UCF-Crime and XD-Violence report frame-level AUC of 86.55 and 88.26, respectively, together with ablations, transfer experiments, and qualitative examples of explainable predictions.

Significance. If the empirical claims hold, VERA is a meaningful contribution: it demonstrates that a frozen VLM can be adapted to VAD without parameter updates, instruction tuning, or an external reasoning LLM, using only coarse video-level labels, while also providing textual explanations. The training algorithm is described transparently in Algorithm 1, and the ablation study in Section 4.3 gives useful evidence about which components matter (question learning, scene retrieval, temporal refinement). The transfer tables (Tables 7-9) and the sensitivity analysis in Appendix B.4 are informative and suggest the method is not brittle in every hyperparameter. However, the headline state-of-the-art claim is currently underdetermined by the baseline comparison used and by the absence of repeated-run statistics.

major comments (4)
  1. [Section 4.2, Table 2] The claim that VERA achieves the highest AUC among explainable VAD methods on UCF-Crime rests on comparing against Holmes-VAD and VADor "without IT, as reported in their papers." The published versions of these methods are instruction-tuned systems, and their instruction-tuned variants report higher AUC in their own papers. This comparison conflates the proposed architecture with the absence of instruction tuning. The headline claim should be supported either by evaluating the published instruction-tuned checkpoints under the same protocol, or by explicitly softening the claim to "best among explainable methods that do not use instruction tuning or full-frame training."
  2. [Tables 3 and 10] The XD-Violence evaluation is incomplete with respect to the same state-of-the-art claim. Holmes-VAD is omitted from the AUC table (Table 3), while in the AP table (Table 10) Holmes-VAD outperforms VERA by a large margin (84.96 vs 70.54). The AP gap is attributed to full-frame training, but then the "fair comparison" in Table 2 should also apply to the XD-Violence AUC, where the missing Holmes-VAD row makes the claim untestable. Report Holmes-VAD AUC on XD-Violence under both its published and no-IT configurations, with a clear statement of which training frames each configuration uses.
  3. [Section 4 and Appendix B] All experiments are single-run: no seeds, standard deviations, or significance tests are reported for any table. Because training involves random batch sampling and stochastic VLM decoding, the point differences supporting the main claim (e.g., 86.55 vs 85.90 for VADor on UCF-Crime) may lie within run-to-run noise. Please report results over multiple seeds for the main tables, or otherwise provide a statistical significance analysis, and state how Q* and the inference hyperparameters are selected without test-set feedback.
  4. [Appendix B.4] Several inference hyperparameters (K, tau, omega, sigma_1, sigma_2) are set per dataset, and the sensitivity analysis is performed only on UCF-Crime. The final AUC is visibly sensitive to some of these choices (Table 13 shows 85.21 without retrieval and 86.61 at K=0.15h). It is therefore important to specify how these values are chosen for XD-Violence and for future datasets. If any of these choices are made by looking at test-set performance, the reported numbers are optimistic; if not, the validation protocol should be described explicitly.
minor comments (4)
  1. [Table 10 caption] The caption says "No IT is used for Holmes-VAD" but the superscript dagger is only explained after the table; please clarify in the caption which methods use full training frames and which use no instruction tuning, since this distinction is central to the paper's claims.
  2. [Section 4.3, Table 7] There is a grammatical error in the sentence "Additionally, We also evaluate the transferability of Q across different scales and and observe an interesting phenomenon": the doubled "and" should be removed.
  3. [Section 5] The concluding sentence claims VERA works "without additional computation overhead," but inference uses an ImageBind feature extractor and convolutional smoothing on top of the VLM forward passes. Please rephrase to "without additional reasoning modules or parameter updates" to avoid overstating the computational claim.
  4. [Abstract and Introduction] The phrase "state-of-the-art explainable VAD performance" appears in the abstract and contributions before the limitations of the comparison are acknowledged; please qualify it in the abstract to match the actual experimental setup.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: VERA's central claim is an empirical benchmark result, and the only notable self-citation (VML) is non-load-bearing.

full rationale

VERA's load-bearing claim is the UCF-Crime AUC in Table 2 (86.55) and the corresponding explainable-VAD comparison. This is an empirical outcome computed on held-out test frames: the learnable guiding questions Q are optimized on coarse video-level training labels via the learner/optimizer loop (Eqs. 1-2), the best Q* is selected on a 10% validation split, and the reported AUC is measured on test videos. No equation in the paper defines the test AUC as a function of the training objective or of Q*; Step 2 and Step 3 are deterministic post-processing (cosine retrieval, softmax ensembling, Gaussian smoothing, position weighting), not fitted parameters that encode the evaluation labels. Hyperparameters K, tau, omega, sigma_1, and sigma_2 are tuned with sensitivity experiments, which is benchmark tuning rather than a construction-level reduction. The only notable self-citation is VML [45], jointly authored by W. Liu: Section 2 states 'The designed VL framework is inspired by a recent technique called verbalized machine learning (VML) [45]' and Section 3.2 says VERA 'inherits the idea of VML [45] in training.' This is a real prior technique, cited as inspiration rather than as a proof of VERA's effectiveness; the transfer to video VAD is implemented and tested independently, so the citation is not load-bearing in the circularity sense. The paper also honestly flags in Limitations C that performance 'relies heavily on the visual perception capabilities of VLMs'; that is a dependency on the backbone's perception, not a circular reduction of the method's outputs to its inputs. The fairness qualifier in Section 4.2 ('without IT, as reported in their papers') and the AP gap in Table 10 are correctness and benchmarking concerns, not circularity.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The central claim rests on empirical benchmarking. The main free parameters are the learned guiding questions and a collection of per-dataset inference hyperparameters. No new physical or conceptual entities are posited. The key domain assumptions concern VLM perception, frame sampling, ImageBind similarity, and the representativeness of the validation split.

free parameters (10)
  • Learned guiding questions Q* = 5 natural-language questions (e.g., 'Are there any people in the video who are not in their typical positions...')
    Learned via the learner-optimizer verbal loop on coarse labels and selected by validation accuracy; this is the central learnable parameter of the method.
  • Number of guiding questions m = 5
    Chosen by hand based on the ablation in Figure 4; m=5 gives the best AUC, with performance dropping for m=1 or m=7.
  • Sampled frames per video S = 8
    Chosen in trade-off with batch size so that S*n=16; Table 12 shows S=8 and n=2 performs best among tested configurations.
  • Batch size n = 2
    Chosen in trade-off with S; n=2 gives the best validation-guided question learning in Table 12.
  • Retrieval ratio K (as fraction of h segments) = 0.1h for UCF-Crime, 0.15h for XD-Violence
    Set per dataset based on sensitivity analysis in Table 13; larger or random retrieval degrades AUC.
  • Softmax temperature tau = 10 (stated), though sensitivity tests only cover up to tau=1
    Appendix B.4 states tau is set to 10, but the sensitivity table tests tau in [1e-8, +inf) and recommends [0.01, 1]; the reported value is inconsistent with the tested range.
  • Gaussian smoothing filter size omega = 15 for UCF-Crime, 30 for XD-Violence
    Set per dataset; Table 14 shows AUC is fairly insensitive to omega beyond 15.
  • Gaussian smoothing variance sigma_1 = 10 for UCF-Crime, 30 for XD-Violence
    Set per dataset; Table 15 shows robustness to sigma_1.
  • Position weighting variance sigma_2 = floor(F/2) for both datasets
    Chosen to cover the whole video; Table 17 shows sigma_2=0.5F performs best.
  • Training iterations P = Up to 10 epochs (about 5000 mini-batches), early stopping via validation
    The exact number is not fixed; Q* is selected by validation accuracy every 100 iterations, introducing a hand-specified early-stopping criterion.
assumptions (5)
  • domain assumption The frozen VLM (InternVL2-8B) has sufficient visual perception and instruction-following ability to act as both learner and optimizer.
    Invoked throughout Section 3.2; the entire verbalized training loop depends on the VLM producing reliable binary judgments and useful question rewrites. The authors acknowledge in Limitations C that visual perception limits VERA.
  • domain assumption Uniformly sampling 8 frames preserves enough temporal and motion information for video-level anomaly classification.
    Assumed in Section 3.2 and empirically tested in Table 4; uniform sampling outperforms random and TSN sampling, but this is a dataset-specific empirical finding, not a derived guarantee.
  • domain assumption ImageBind cosine similarity captures the scene relevance needed for context ensembling.
    Assumed in Section 3.3, Step 2, following LAVAD [52]; the ensemble weights are similarity-based and were not validated against alternative feature extractors.
  • domain assumption A 10% random validation split drawn from the training set is representative enough to select Q*.
    Assumed in Appendix A.1; the final test AUC depends on this selection, and no cross-validation or repeated splitting is performed.
  • standard math Standard mathematical operations (cosine similarity, softmax, Gaussian convolution, position weighting) behave as expected on the segment scores.
    Used in Eqs. (4) and (5); these are unproblematic but formally assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models." pith.science (2026). https://pith.science/paper/GYXLWB4P

@misc{pith2026241201095,
  author       = {Pith},
  title        = {Pith review of: VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GYXLWB4P}},
  note         = {Machine review of arXiv:2412.01095}
}
read the original abstract

The rapid advancement of vision-language models (VLMs) has established a new paradigm in video anomaly detection (VAD): leveraging VLMs to simultaneously detect anomalies and provide comprehendible explanations for the decisions. Existing work in this direction often assumes the complex reasoning required for VAD exceeds the capabilities of pretrained VLMs. Consequently, these approaches either incorporate specialized reasoning modules during inference or rely on instruction tuning datasets through additional training to adapt VLMs for VAD. However, such strategies often incur substantial computational costs or data annotation overhead. To address these challenges in explainable VAD, we introduce a verbalized learning framework named VERA that enables VLMs to perform VAD without model parameter modifications. Specifically, VERA automatically decomposes the complex reasoning required for VAD into reflections on simpler, more focused guiding questions capturing distinct abnormal patterns. It treats these reflective questions as learnable parameters and optimizes them through data-driven verbal interactions between learner and optimizer VLMs, using coarsely labeled training data. During inference, VERA embeds the learned questions into model prompts to guide VLMs in generating segment-level anomaly scores, which are then refined into frame-level scores via the fusion of scene and temporal contexts. Experimental results on challenging benchmarks demonstrate that the learned questions of VERA are highly adaptable, significantly improving both detection performance and explainability of VLMs for VAD.

Figures

Figures reproduced from arXiv: 2412.01095 by the authors.

Figure 1
Figure 1. VERA renders frozen VLMs to describe and reason with learn [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall training pipeline in VERA aims to optimize VAD guiding questions iteratively. In each iteration [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. VERA computes anomaly scores with Q∗ in three steps. g, e.g., ImageBind [10]. For the u-th segment Vu, its sim￾ilarity with any segment Vw (1 ≤ w ≤ h) is sim(u, w) = cos  eu·ew ||eu||·||ew|| , where cos denotes the cosine function, and eu = g(Vu) and ew = g(Vw) represent their features. Let κu = [κ (1) u , . . . , κ (K) u ] denote the indices of the top-K segments similar to Vu. We refine the anomaly score by \bar… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Effect of the number of guid￾ing questions on AUC. Number of Questions m. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison between VERA and LAVAD. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: VERA can take open-ended questions and interact with humans. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Anomaly scores generated by VERA (with InternVL2-8B) in “Ar￾rest007 x264” from UCF-Crime. coarse-to-fine anomaly scoring, the anomaly score dynamics from VERA well represent the actual real-time anomaly level in this video and gradually increases to nearly 1 when the m…
Figure 10
Figure 10. Figure 10: We take the guiding questions Q learned from the 100th iteration to the 700th iteration for illustration purpose. During the updating process, the optimizer gradually concretizes anomaly patterns that can be applied to different scenarios in a concise expression. • Mo…
Figure 9
Figure 9. Figure 9: The validation accuracy given dif￾ferent learned guiding questions from each it￾eration. The graph is smoothed with moving average (window size 5) for better readability. In training, we assess the quality of the learned guiding questions by the accuracy of the validat…
Figure 11
Figure 11. Figure 11: Given the normal video “Normal Videos 018 x264”, the frozen VLM (InternVL2-8B) can conclude that no anomaly happens in the video under the guidance of Q∗, which is aligned with the ground truth. Since the anomaly scores for all scenes are zeros by VERA, we do not show…
Figure 12
Figure 12. Figure 12: Given the abnormal video “RoadAccidents127 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Anomaly scores generated by VERA (with InternVL2-8B) in “RoadAcci￾dents127 x264” from UCF-Crime. The main anomaly that happens in this video is a traffic accident where a truck crashes into a train from Frame 2160 to Frame 2299, which corresponds to the 5th scene in …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A training-free glance-then-scrutinize pipeline and a reinforcement-learned tool-calling agent both improve joint temporal grounding and semantic understanding of video anomalies on a new benchmark.

  2. VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning

    cs.CV 2025-05 reject novelty 6.0 of 10

    VAU-R1 uses Group Relative Policy Optimization with accuracy, format, and temporal-IoU rewards to improve video anomaly reasoning on a new LLM-generated benchmark, VAU-Bench.

  3. Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A two-stage trained MLLM with a perception-to-cognition chain-of-thought and a self-verification RL reward outperforms prior models on video anomaly detection and reasoning.

  4. Flashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detection

    cs.CV 2025-05 conditional novelty 6.0 of 10

    An offline LLM builds a pseudo-scene caption memory; online embedding retrieval against that memory yields zero-shot, real-time, explainable video anomaly detection with SOTA scores on UCF-Crime and XD-Violence.

  5. VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understanding

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A new benchmark, a training-free framework, and a joint metric for video anomaly detection that combines temporal grounding with semantic understanding.

  6. Uncertainty-Weighted Image-Event Multimodal Fusion for Video Anomaly Detection

    cs.CV 2025-05 conditional novelty 4.0 of 10

    IEF-VAD fuses CLIP image and synthetic-event features via learned inverse-variance weighting with Kalman-style updates and iterative refinement, reporting state-of-the-art AUC/AP on UCF-Crime, XD-Violence, ShanghaiTec...

  7. PiercingEye: Dual-Space Video Violence Detection with Hyperbolic Vision-Language Guidance

    cs.CV 2025-04 conditional novelty 4.0 of 10

    A dual-space (Euclidean and hyperbolic) framework with hyperbolic Dirichlet energy gating and LLM-generated ambiguous text supervision reports 88.82% AP on XD-Violence, a 2.48-point gain over prior hyperbolic methods.

  8. Reasoning Segmentation for Images and Videos: A Survey

    cs.CV 2025-05 conditional novelty 3.0 of 10

    The paper organizes the field of reasoning segmentation into image and video tracks, cataloging 26 methods, 12 metrics, and 29 datasets.

  9. Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlight

    cs.CV 2024-12 conditional novelty 2.0 of 10

    A survey of 13 recent LLM/VLM-based video anomaly detection methods, organized by interpretability, temporal modeling, few-shot learning, and open-world detection.

Reference graph

Works this paper leans on

74 extracted references · 58 canonical work pages · cited by 9 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Anomaly detection in autonomous driving: A survey

    Daniel Bogdoll, Maximilian Nitsche, and J Marius Z ¨ollner. Anomaly detection in autonomous driving: A survey. In CVPR Workshops, 2022. 1

    work page 2022

  3. [3]

    The surprising effectiveness of multimodal large language models for video moment retrieval

    Meinardus Boris, Batra Anil, Rohrbach Anna, and Rohrbach Marcus. The surprising effectiveness of multimodal large language models for video moment retrieval. arXiv preprint arXiv:2406.18113, 2024. 7

    arXiv 2024

  4. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz...

    work page 2020

  5. [5]

    Prompt-enhanced multiple instance learning for weakly supervised video anomaly detection

    Junxi Chen, Liang Li, Li Su, Zheng-jun Zha, and Qing- ming Huang. Prompt-enhanced multiple instance learning for weakly supervised video anomaly detection. In CVPR,

    work page

  6. [6]

    Mgfn: Magnitude- contrastive glance-and-focus network for weakly-supervised video anomaly detection

    Yingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton Fok, Xiaojuan Qi, and Yik-Chung Wu. Mgfn: Magnitude- contrastive glance-and-focus network for weakly-supervised video anomaly detection. In AAAI, 2023. 6, 15

    work page 2023

  7. [7]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In CVPR,

  8. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. 1

    2021

Show all 74 references
  1. [9]

    Mist: Multiple instance self-training framework for video anomaly detection

    Jia-Chang Feng, Fa-Ting Hong, and Wei-Shi Zheng. Mist: Multiple instance self-training framework for video anomaly detection. In CVPR, 2021. 6

    2021

  2. [10]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In CVPR, 2023. 5, 16

    2023

  3. [11]

    Digital image processing

    Rafael C Gonzalez. Digital image processing. Pearson edu- cation india, 2009. 5

    2009

  4. [12]

    Regiongpt: Towards region understanding vision lan- guage model

    Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, and Sifei Liu. Regiongpt: Towards region understanding vision lan- guage model. In CVPR, 2024. 3

    2024

  5. [13]

    Learning temporal reg- ularity in video sequences

    Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit K Roy-Chowdhury, and Larry S Davis. Learning temporal reg- ularity in video sequences. In CVPR, 2016. 6, 12

    2016

  6. [14]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In ICLR, 2021. 8

    2021

  7. [15]

    Clip-tsa: Clip-assisted temporal self-attention for weakly-supervised video anomaly detection

    Hyekang Kevin Joo, Khoa V o, Kashu Yamazaki, and Ngan Le. Clip-tsa: Clip-assisted temporal self-attention for weakly-supervised video anomaly detection. In ICIP, 2023. 6, 15

    2023

  8. [16]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 4

    2014 arXiv

  9. [17]

    Scale-aware spatio-temporal relation learning for video anomaly detection

    Guoqiu Li, Guanxiong Cai, Xingyu Zeng, and Rui Zhao. Scale-aware spatio-temporal relation learning for video anomaly detection. In ECCV, 2022. 6

    2022

  10. [18]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML,

  11. [19]

    Self-training multi- sequence learning with transformer for weakly supervised video anomaly detection

    Shuo Li, Fang Liu, and Licheng Jiao. Self-training multi- sequence learning with transformer for weakly supervised video anomaly detection. In AAAI, 2022. 6, 15

    2022

  12. [20]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR,

  13. [21]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2024. 1, 3

    2024

  14. [22]

    Deep hyperspherical learning

    Weiyang Liu, Yan-Ming Zhang, Xingguo Li, Zhiding Yu, Bo Dai, Tuo Zhao, and Le Song. Deep hyperspherical learning. In NeurIPS, 2017. 5

    2017

  15. [23]

    Fu- ture frame prediction for anomaly detection–a new baseline

    Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Fu- ture frame prediction for anomaly detection–a new baseline. In CVPR, 2018. 3

    2018

  16. [24]

    Parameter-efficient orthogonal finetun- ing via butterfly factorization

    Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, et al. Parameter-efficient orthogonal finetun- ing via butterfly factorization. In ICLR, 2024. 8

    2024

  17. [25]

    Abnormal event de- tection at 150 fps in matlab

    Cewu Lu, Jianping Shi, and Jiaya Jia. Abnormal event de- tection at 150 fps in matlab. In ICCV, 2013. 3, 6

    2013

  18. [26]

    Video anomaly detection and explanation via large language models

    Hui Lv and Qianru Sun. Video anomaly detection and explanation via large language models. arXiv preprint arXiv:2401.05702, 2024. 1, 2, 3, 6

    2024 arXiv

  19. [27]

    Unbiased multiple instance learning for weakly supervised video anomaly detection

    Hui Lv, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, and Hanwang Zhang. Unbiased multiple instance learning for weakly supervised video anomaly detection. InCVPR, 2023. 3

    2023

  20. [28]

    What does a platypus look like? generating customized prompts for zero-shot image classification

    Sarah Pratt, Ian Covert, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts for zero-shot image classification. In ICCV, 2023. 3

    2023

  21. [29]

    Controlling text-to-image diffusion by orthogo- nal finetuning

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Sch¨olkopf. Controlling text-to-image diffusion by orthogo- nal finetuning. In NeurIPS, 2023. 8

    2023

  22. [30]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021. 19 9

    2021

  23. [31]

    Towards total recall in industrial anomaly detection

    Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Sch¨olkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In CVPR, 2022. 1

    2022

  24. [32]

    Real-world anomaly detection in surveillance videos

    Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In CVPR, 2018. 2, 3, 6, 15

    2018

  25. [33]

    Hawk: Learning to understand open-world video anomalies

    Jiaqi Tang, Hao Lu, Ruizheng Wu, Xiaogang Xu, Ke Ma, Cheng Fang, Bin Guo, Jiangbo Lu, Qifeng Chen, and Ying- Cong Chen. Hawk: Learning to understand open-world video anomalies. arXiv preprint arXiv:2405.16886 , 2024. 3

    2024 arXiv

  26. [34]

    Rareanom: A bench- mark video dataset for rare type anomalies

    Kamalakar Vijay Thakare, Debi Prosad Dogra, Heeseung Choi, Haksub Kim, and Ig-Jae Kim. Rareanom: A bench- mark video dataset for rare type anomalies. Pattern Recog- nition, 140:109567, 2023. 6

    2023

  27. [35]

    Dyannet: A scene dynamicity guided self-trained video anomaly detection net- work

    Kamalakar Vijay Thakare, Yash Raghuwanshi, Debi Prosad Dogra, Heeseung Choi, and Ig-Jae Kim. Dyannet: A scene dynamicity guided self-trained video anomaly detection net- work. In WACV, 2023. 6

    2023

  28. [36]

    Weakly-supervised video anomaly detection with robust temporal feature mag- nitude learning

    Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervised video anomaly detection with robust temporal feature mag- nitude learning. In ICCV, 2021. 6, 15

    2021

  29. [37]

    Unsupervised video anomaly detection with dif- fusion models conditioned on compact motion representa- tions

    Anil Osman Tur, Nicola Dall’Asen, Cigdem Beyan, and Elisa Ricci. Unsupervised video anomaly detection with dif- fusion models conditioned on compact motion representa- tions. In International Conference on Image Analysis and Processing, 2023. 3, 6

    2023

  30. [38]

    Gods: Generalized one-class discriminative subspaces for anomaly detection

    Jue Wang and Anoop Cherian. Gods: Generalized one-class discriminative subspaces for anomaly detection. In ICCV,

  31. [39]

    Temporal segment networks: Towards good practices for deep action recogni- tion

    Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks: Towards good practices for deep action recogni- tion. In ECCV, 2016. 7

    2016

  32. [40]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  33. [41]

    Self-supervised sparse representa- tion for video anomaly detection

    Jhih-Ciang Wu, He-Yen Hsieh, Ding-Jie Chen, Chiou-Shann Fuh, and Tyng-Luh Liu. Self-supervised sparse representa- tion for video anomaly detection. In ECCV, 2022. 6, 15

    2022

  34. [42]

    Not only look, but also listen: Learning multimodal violence detection under weak supervision

    Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In ECCV, 2020. 2, 3, 6, 15

    2020

  35. [43]

    Open-vocabulary video anomaly detection

    Peng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun, Jing Liu, Peng Wang, and Yanning Zhang. Open-vocabulary video anomaly detection. In CVPR, pages 18297–18307,

  36. [44]

    Vadclip: Adapting vision-language models for weakly supervised video anomaly detection

    Peng Wu, Xuerong Zhou, Guansong Pang, Lingru Zhou, Qingsen Yan, Peng Wang, and Yanning Zhang. Vadclip: Adapting vision-language models for weakly supervised video anomaly detection. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 6074–6082, 2024. 3

    2024

  37. [45]

    Verbalized machine learning: Revisiting machine learning with language models

    Tim Z Xiao, Robert Bamler, Bernhard Sch ¨olkopf, and Weiyang Liu. Verbalized machine learning: Revisiting machine learning with language models. arXiv preprint arXiv:2406.04344, 2024. 3, 4, 8

    2024 arXiv

  38. [46]

    Follow the rules: reasoning for video anomaly detection with large language models

    Yuchen Yang, Kwonjoon Lee, Behzad Dariush, Yinzhi Cao, and Shao-Yuan Lo. Follow the rules: reasoning for video anomaly detection with large language models. arXiv preprint arXiv:2407.10299, 2024. 1, 3, 6

    2024 arXiv

  39. [47]

    Text prompt with nor- mality guidance for weakly supervised video anomaly detec- tion

    Zhiwei Yang, Jing Liu, and Peng Wu. Text prompt with nor- mality guidance for weakly supervised video anomaly detec- tion. In CVPR, 2024. 3

    2024

  40. [48]

    Anopcn: Video anomaly detection via deep predictive coding network

    Muchao Ye, Xiaojiang Peng, Weihao Gan, Wei Wu, and Yu Qiao. Anopcn: Video anomaly detection via deep predictive coding network. In ACM international conference on multi- media, 2019. 3

    2019

  41. [49]

    Textgrad: Automatic” differentiation” via text

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. Textgrad: Automatic” differentiation” via text. arXiv preprint arXiv:2406.07496, 2024. 3

    2024 arXiv

  42. [50]

    Claws: Clustering assisted weakly supervised learning with normalcy suppression for anoma- lous event detection

    Muhammad Zaigham Zaheer, Arif Mahmood, Marcella Astrid, and Seung-Ik Lee. Claws: Clustering assisted weakly supervised learning with normalcy suppression for anoma- lous event detection. In ECCV, 2020. 6

    2020

  43. [51]

    Generative cooper- ative learning for unsupervised video anomaly detection

    M Zaigham Zaheer, Arif Mahmood, M Haris Khan, Mat- tia Segu, Fisher Yu, and Seung-Ik Lee. Generative cooper- ative learning for unsupervised video anomaly detection. In CVPR, 2022. 6

    2022

  44. [52]

    Harnessing large language mod- els for training-free video anomaly detection

    Luca Zanella, Willi Menapace, Massimiliano Mancini, Yim- ing Wang, and Elisa Ricci. Harnessing large language mod- els for training-free video anomaly detection. In CVPR,

  45. [53]

    Exploiting completeness and uncertainty of pseudo labels for weakly supervised video anomaly detection

    Chen Zhang, Guorong Li, Yuankai Qi, Shuhui Wang, Laiyun Qing, Qingming Huang, and Ming-Hsuan Yang. Exploiting completeness and uncertainty of pseudo labels for weakly supervised video anomaly detection. In CVPR, 2023. 3

    2023

  46. [54]

    Video-llama: An instruction-tuned audio-visual language model for video un- derstanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video un- derstanding. In EMNLP, 2023. 3, 7

    2023

  47. [55]

    Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal llm

    Huaxin Zhang, Xiaohao Xu, Xiang Wang, Jialong Zuo, Chuchu Han, Xiaonan Huang, Changxin Gao, Yuehuan Wang, and Nong Sang. Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal llm. arXiv preprint arXiv:2406.12235, 2024. 1, 2, 3, 6, 15

    2024 arXiv

  48. [56]

    Multi- scale video anomaly detection by multi-grained spatio- temporal representation learning

    Menghao Zhang, Jingyu Wang, Qi Qi, Haifeng Sun, Zirui Zhuang, Pengfei Ren, Ruilong Ma, and Jianxin Liao. Multi- scale video anomaly detection by multi-grained spatio- temporal representation learning. In CVPR, 2024. 3

    2024

  49. [57]

    Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection

    Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H Li, and Ge Li. Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In CVPR, 2019. 6

    2019

  50. [58]

    1. Is there any suspicious person or object that looks unusual in this scene? 2. Is there any behavior that looks unusual in this scene?

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. In ICLR, 2024. 1 10 Appendix Table of Contents A. Training in VERA 12 A.1. Algorithm . . . . . . . . . . . . . . . . ....

    2024

  51. [60]

    Are there any vehicles or people violating traffic rules?

  52. [61]

    Are there any accidents or near-accidents occurring?

  53. [62]

    Are there any objects or people obstructing the normal flow of traffic?

  54. [63]

    Are there any unusual or unexpected behaviors from pedestrians or drivers?

  55. [64]

    Are there any emergency vehicles or personnel present? The second example is another set of guiding questions Qstore that only ask the VLM to identify anomalies in a store setting, which includes questions like:

  56. [65]

    Are there any individuals loitering or behaving suspiciously inside the store?

  57. [66]

    Is there any unusual activity inside the store, such as tampering with items or attempting to enter restricted areas?

  58. [67]

    Are there any signs of forced entry or damage to the store’s entrance?

  59. [68]

    Are there any individuals present who seem to be watching or waiting for something specific inside the store?

  60. [69]

    Are there any interactions between individuals inside the store that appear suspicious or out of the ordinary? 15 Thus, Qtraffic and Qstore focuses on the specific anomalies of traffic accidents and shoplifting, respectively, while theQ∗ that we find focuses on general cases a...

  61. [70]

    Are there any people in the video who are not in their typical positions or engaging in activities that are not consistent with their usual behavior?

  62. [71]

    Are there any vehicles in the video that are not in their typical positions or being used in a way that is not consistent with their usual function?

  63. [72]

    Are there any objects in the video that are not in their typical positions or being used in a way that is not consistent with their usual function?

  64. [73]

    Is there any visible damage or unusual movement in the video that indicates an anomaly?

  65. [74]

    Normal Videos 018 x264

    Are there any unusual sounds or noises in the video that suggest an anomaly? The comparison results of Q∗, Qtraffic, and Qstore in detecting anomalies in general cases (all testing videos on UCF- Crime), traffic scenes (testing videos from the Traffic Accident category on UCF-...

    2000

  66. [2024]

    1, 3, 5, 6, 7, 15, 16

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.