Pith. sign in

REVIEW 3 major objections 7 minor 74 references

From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoning

T0 review · 3 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A plug-and-play symbolic layer turns any open-vocabulary detector into an interpretable event recognizer without extra training.

desk verdict A useful interpretable symbolic layer on top of open-vocabulary detectors, but the paper's 'training-free' claim is contradicted by its own supervised objective and labeled per-event training sets. read the letter →

arxiv 2502.05843 v4 pith:A3RTZP7C submitted 2025-02-09 cs.CV

Yuhui Zeng , Haoxiang Wu , Wenjie Nie , Xiawu Zheng , Guangyao Chen , Yunhang Shen , Jun Peng , Yonghong Tian
show 1 more author
Rongrong Ji
This is my paper · ORCID
classification cs.CV
keywords objectdetectioneventrecognitionsymbolicregressionlargelanguagemodelsneuro-symbolicreasoningtraining-freeinterpretabilityopen-vocabulary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a standard open-vocabulary object detector already contains enough information to recognize complex events, if its outputs are interpreted through short logical expressions. The authors propose SymbolicDet, a plug-and-play layer that searches for such expressions — combining evolutionary symbolic regression with LLM guidance — and applies them to detector outputs for event classification. The point is to get event understanding without task-specific training, with rules a human can read. If right, any off-the-shelf detector can be upgraded to an interpretable event recognizer at the cost of a symbolic search.

What carries the argument

The machinery is a symbolic expression space over arithmetic operators {+, −, ×, ÷, max, min} and logical operators {∧, ∨, ¬}, applied to structured features extracted from detector outputs — chiefly class counts and simple relational features. An evolutionary algorithm (symbolic regression) searches this space for the expression minimizing a fitness loss plus a complexity penalty, while an LLM, prompted with scene context, chain-of-thought, and feedback from past search scores, proposes candidate expressions that steer the population. The LLM's role is not perception but search guidance, keeping the final expression human-readable and the detector untouched.

What would settle it

Take a surveillance event defined purely by motion, such as loitering or intrusion, run SymbolicDet on single frames with the same settings, and check AUROC. If no discovered expression over per-frame object counts and positions exceeds chance level, the claim that detectors' outputs carry enough information for event recognition fails for temporal events; equally, construct a pair of image sets with identical class-count distributions but opposite event labels to show the feature set cannot separate them.

Watch

Extended reading notes

Core claim

The central claim is that detectors are not just entity recognizers but visual sensors whose outputs, when run through a discovered logical rule over object counts and simple relational features, can separate images containing a target event from those that do not. The paper demonstrates this on illegal multi-rod fishing, construction safety helmet compliance, and crowd anomalies, where the discovered rules — e.g., a violation when the count of heads exceeds helmets in a person-and-workshop context — lift AUROC by 8 to 39 points across APE, GLIP, and YOLO-World, all without fine-tuning. On UCSD Ped2 the training-free rules reach 98.7% AUROC, close to specialized trained systems.

Load-bearing premise

The framework assumes the target event can be decided by a short logical expression over object counts and simple spatial features taken from a single static image; events that need motion, temporal context, or fine-grained attributes like which person holds which rod have no representable feature to exploit.

Editorial extensions

If this is right

  • Any open-vocabulary detector can be wrapped with SymbolicDet to recognize a new event from a handful of labeled images or none, because the rule search replaces fine-tuning.
  • Event decisions come with a symbolic explanation, such as 'head count greater than helmet count in a workshop', which makes compliance auditing and failure analysis straightforward.
  • Performance scales with the search budget and with the quality of the underlying detector, so future detector improvements transfer directly to event recognition.
  • On static-frame benchmarks like Helmet-Mac and multi-rod fishing, the training-free rules match or approach fine-tuned detectors, suggesting that much of event semantics is already latent in object class distributions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recipe should transfer to other symbolic targets, such as scene graph relations or action heuristics, wherever the event can be stated as a count or threshold condition over detected entities.
  • A natural test the paper does not run is temporal events such as loitering or intrusion; if a rule over per-frame detections fails there, the claim would narrow to spatial and static events rather than events in general.
  • The observation that LLM guidance mostly accelerates convergence rather than changing the final rule suggests the method could run with weaker LLMs or even keyword-based priors for low-resource deployments.
  • Because the rules are fit on a search set, the method may overfit to dataset-specific object-name vocabularies; robustness across different detector prompt phrasings remains to be checked.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes SymbolicDet, a framework that uses an open-vocabulary object detector to extract object counts and simple relational features, then applies an LLM-guided evolutionary symbolic search to discover logical expressions that classify whether a target event is present in an image. The method is evaluated on event recognition benchmarks including ERA, Helmet-Mac, Multi-rods Fishing, UCSD Ped2, and USED, using APE, GLIP, and YOLO-World as detectors. The authors report AUROC improvements over the raw detectors and claim that the framework is training-free, requires no additional labeled data beyond the detector's pretraining, and provides interpretable symbolic rules.

Significance. The core idea — that pretrained open-vocabulary detectors plus interpretable symbolic search can produce event classifiers — is interesting and potentially useful. The paper's strengths include its detector-agnostic design, the cross-architecture experiments in Table 1, the interpretability of the discovered expressions, the introduction of a new dataset, and the stated intent to release code. However, the central 'training-free / no additional labels' claim is not supported by the method as written, because the symbolic expression is fit to event labels through a supervised objective. If the claims are reframed honestly as 'no detector fine-tuning' and the evaluation is made more rigorous, the contribution could be valuable; in its current form, the significance is contingent on that reframing.

major comments (3)
  1. [Section 3.1, Eq. (5); Section 4.1.1] The central claim that SymbolicDet is 'training-free' and requires 'no additional labeled data beyond what the underlying detector was trained on' (Abstract and Section 1) is contradicted by the method's own objective. Equation (5) explicitly minimizes sum_i L(f(X_i), y_i) over labeled image-event pairs, and Section 4.1.1 reports 15,000 training images for Multi-rods Fishing and 7,571 training images for Helmet-Mac. The evolutionary search is therefore a supervised fit of a symbolic classifier to event labels, not a training-free procedure. This is load-bearing because the abstract and introduction promise plug-and-play generalization to new events without task-specific supervision. The authors should either remove the 'training-free / no additional labels' claims or redefine them precisely as 'no detector fine-tuning' and adjust all downstream statements (including Findings 1 and 2 and Table 2 comparisons) accordingly.
  2. [Section 4.1.2, Section 4.2, Table 3] The evaluation protocol is under-specified for the public benchmarks. For ERA, the paper lists image counts for categories but does not state how the training and test splits are formed, how the symbolic search uses event labels, or whether the reported AUROC numbers are on the same images used for search. For USED, Table 3 reports a single score with no description of the subset, splits, or event definitions, and the supplementary Table 7 reports accuracy on only three subsets without a clear protocol. There are also no error bars or multiple-seed results anywhere in the paper. Without this information, the reported gains cannot be independently assessed or compared with prior work.
  3. [Section 4.1.1, Section 3.1] The paper claims broad applicability to 'complex events' but the experiments do not cover temporal or motion-dependent events, and the feature representation in Eq. (4) and the discovered rules in Eq. (6) are limited to object counts and simple static relations. Section 4.1.1 explicitly lists 'personnel loitering and intrusion' as part of the Multi-Event Dataset, yet no loitering or intrusion results are reported. The absence of any temporal feature or evaluation on such events means the claim that the framework 'unlocks complex visual understanding' is demonstrated only for a narrow class of count- and co-occurrence-based events. The paper should state this limitation explicitly and temper the general claim, or add experiments on at least one temporal event.
minor comments (7)
  1. [Figure 1] The first line of Figure 1 ('PHHWCSHGScaffoldingS >>V') appears to be garbled or corrupted; the symbolic pattern and its explanation should be rendered cleanly.
  2. [Section 3.1, Eq. (6)] Equation (6) uses the symbol E inside the feature functions while the preceding notation uses X; it also leaves the index set {p,d} and the meaning of each feature function undefined in the main text. Please unify the notation.
  3. [Section 3, Eq. (3)] Equation (3) defines the objective with G_LLM and S, but the actual optimization in Eq. (5) does not include these terms. The relationship between the two formulations should be clarified.
  4. [Section 4.1.1] The Multi-Event Dataset description says the multi-rods scenario contains '12,000 images' and then states '15,000 training images' for the same scenario; this inconsistency should be resolved.
  5. [Table 2] The table header and row markers are confusing: the column labeled 'Lora Prompt Our' is not clearly separated, and the checkmarks are not defined in a legend. Please reformat so each configuration is identifiable.
  6. [Supplementary Table 7] The table title says 'computational overhead of each part' but the table reports accuracy values; retitle it appropriately.
  7. [References] Reference [45] for ERA is cited as 'in press' without a year or venue details; please provide the full citation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the symbolic expressions are fitted on labeled event data and tested on held-out splits, so the reported results are genuine evaluations; the 'training-free' wording is an overclaim, not a circular step.

full rationale

Walking the derivation chain: Eq. (5) defines the discovered expression as f* = argmin Σ L(f(X_i), y_i) + λΩ(f), i.e., a supervised fit to event labels, and Section 4.1.1 reports per-event labeled training sets (15,000 images for Multi-rods Fishing, 7,571 for Helmet-Mac). The reported AUROC numbers in Tables 1-3 are then computed on held-out test sets (e.g., 2,283 fishing test images; 4,642 Helmet-Mac test images), so the test performance is not forced by construction. The paper's repeated 'training-free / no additional labeled data' claim (Abstract, Section 1) is contradicted by its own Eq. (5) and dataset descriptions, but that is an accuracy-of-description problem, not a circular reduction: the fitted rule is not being offered as the same quantity it was fit to in the evaluation. The APE detector [56] is a self-citation (Yunhang Shen and Rongrong Ji are co-authors), but APE is only one of three detectors, and Table 1 shows the framework also improves GLIP and YOLO-World, so the self-citation is not load-bearing. The LLM guidance is grounded by the symbolic fitness evaluator S and the objective in Eq. (5), so the search is not defined in terms of the target result. No step in the paper reduces, by the paper's own equations, to its own input; the central derivation is self-contained as a supervised symbolic classifier with held-out evaluation, and the residual issue is an overclaim about training-free operation rather than circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or ontological entities are postulated; the framework's components, symbolic expressions, evolutionary search, and LLM prompts, are methodological constructs. The counted free parameters and assumptions are the items a reader must accept to believe the reported AUROC numbers.

free parameters (4)
  • detection confidence thresholds = APE: 0.05; GLIP/YOLO-World: 0.1
    Set 'based on empirical studies of detector characteristics' in Section 4.1.3; directly controls which detections become symbolic features.
  • symbolic search hyperparameters = population = 2x target categories; crossover 0.5; mutation 0.3; 5000 iterations
    Selected by 'extensive experimental validation' in Section 4.1.3; no sensitivity analysis is reported except for search scale.
  • complexity penalty lambda = not reported
    Introduced in Equation (5) as lambda Omega(f) but its value is never given, so the optimization objective is incompletely specified.
  • LLM choice and prompt templates = qwen series (7B to 72B)
    Ablations show different LLM scales change convergence and accuracy; prompt texts are not included, making results model-dependent.
assumptions (4)
  • domain assumption Detector outputs contain enough information to recognize target events.
    The framework only consumes class labels, boxes, and scores from the detector; no pixels or temporal features are used (Section 3.1).
  • domain assumption Target events are separable by short logical expressions over count and relational features.
    The feature set includes counts and spatial predicates (Equation 6); events requiring motion or fine-grained attributes are outside this hypothesis.
  • domain assumption Event labels are available for the images used in search.
    Equations (1) and (5) require binary event labels y_i to score expressions, which contradicts the 'training-free' narrative in the abstract and Section 1.
  • domain assumption LLM suggestions do not leak test-set information.
    LLM prompts describe scenes and give feedback, but the paper does not analyze whether test-set details could enter the prompt construction process.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoning." pith.science (2026). https://pith.science/paper/A3RTZP7C

@misc{pith2026250205843,
  author       = {Pith},
  title        = {Pith review of: From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A3RTZP7C}},
  note         = {Machine review of arXiv:2502.05843}
}
read the original abstract

Current object detectors excel at entity localization and classification, yet exhibit inherent limitations in event recognition capabilities. This deficiency arises from their architecture's emphasis on discrete object identification rather than modeling the compositional reasoning, inter-object correlations, and contextual semantics essential for comprehensive event understanding. To address this challenge, we present a novel framework that expands the capability of standard object detectors beyond mere object recognition to complex event understanding through LLM-guided symbolic reasoning. Our key innovation lies in bridging the semantic gap between object detection and event understanding without requiring expensive task-specific training. The proposed plug-and-play framework interfaces with any open-vocabulary detector while extending their inherent capabilities across architectures. At its core, our approach combines (i) a symbolic regression mechanism exploring relationship patterns among detected entities and (ii) a LLM-guided strategically guiding the search toward meaningful expressions. These discovered symbolic rules transform low-level visual perception into interpretable event understanding, providing a transparent reasoning path from objects to events with strong transferability across domains.We compared our training-free framework against specialized event recognition systems across diverse application domains. Experiments demonstrate that our framework enhances multiple object detector architectures to recognize complex events such as illegal fishing activities (75% AUROC, +8.36% improvement), construction safety violations (+15.77%), and abnormal crowd behaviors (+23.16%). Code is available at \href{https://github.com/MAC-AutoML/SymbolicDet}{here}.

Figures

Figures reproduced from arXiv: 2502.05843 by the authors.

Figure 1
Figure 1. The radar chart at the top illustrates the comparative per [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed SymbolicDet. SymbolicDet mainly consists of logic search and symbolic reasoning. The former module constructs and explores the search space by leveraging structured entity features extracted from an open-set object detector. The latter module harnesses the symbolic reasoning capabilities of Large Language Models (LLMs) along with their inherent commonsense understanding of visual event p… view at source ↗
Figure 3
Figure 3. Performance on SymbolicDet with or without LLM. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Illustration of the application of symbolic pattern detection in sports environments, showcasing how logical expressions can be [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 45 canonical work pages

  1. [1]

    Kashif Ahmad, Nicola Conci, Giulia Boato, and Francesco G. B. De Natale. Used: a large-scale social event detection dataset. In Proceedings of the 7th International Conference on Multimedia Systems, New York, NY , USA, 2016. Associ- ation for Computing Machinery. 8

    work page 2016

  2. [2]

    Neuro-symbolic visual rea- soning: Disentangling

    Saeed Amizadeh, Hamid Palangi, Alex Polozov, Yichen Huang, and Kazuhito Koishida. Neuro-symbolic visual rea- soning: Disentangling. In International Conference on Ma- chine Learning, pages 279–290. Pmlr, 2020. 3

    work page 2020

  3. [3]

    Cascade r-cnn: Delv- ing into high quality object detection

    Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delv- ing into high quality object detection. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 6154–6162, 2018. 2

    2018

  4. [4]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 2

    2020

  5. [5]

    Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14455–14465,

  6. [6]

    Vtqa: Visual text question answering via entity alignment and cross-media reasoning

    Kang Chen and Xiangqian Wu. Vtqa: Visual text question answering via entity alignment and cross-media reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27218–27227, 2024. 3

    work page 2024

  7. [7]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xing- gang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16901–16911, 2024. 2, 3, 7

    work page 2024

  8. [8]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the asso- ciation for computational linguistics: human language tech- nologies, volume 1 (long and short papers) , pages 4171– 4186, 2019. 7

    2019

Show all 74 references
  1. [9]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  2. [10]

    Flexible visual recognition by evidential modeling of confu- sion and ignorance

    Lei Fan, Bo Liu, Haoxiang Li, Ying Wu, and Gang Hua. Flexible visual recognition by evidential modeling of confu- sion and ignorance. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 1338–1347,

  3. [11]

    Question aware vision transformer for multimodal reasoning

    Roy Ganz, Yair Kittenplon, Aviad Aberdam, Elad Ben Avra- ham, Oren Nuriel, Shai Mazor, and Ron Litman. Question aware vision transformer for multimodal reasoning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13861–13871, 2024. 3

    2024

  4. [12]

    Neural- symbolic computing: An effective methodology for princi- pled integration of machine learning and reasoning

    Artur d’Avila Garcez, Marco Gori, Luis C Lamb, Luciano Serafini, Michael Spranger, and Son N Tran. Neural- symbolic computing: An effective methodology for princi- pled integration of machine learning and reasoning. arXiv preprint arXiv:1905.06088, 2019. 3

    1905 arXiv

  5. [13]

    Recurrent vision transformers for object detection with event cameras

    Mathias Gehrig and Davide Scaramuzza. Recurrent vision transformers for object detection with event cameras. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13884–13893, 2023. 2

    2023

  6. [14]

    A background-agnostic framework with adversarial training for abnormal event detection in video

    Mariana Iuliana Georgescu, Radu Tudor Ionescu, Fa- had Shahbaz Khan, Marius Popescu, and Mubarak Shah. A background-agnostic framework with adversarial training for abnormal event detection in video. IEEE transactions on pattern analysis and machine intelligence, 44(9):4505–4523,

  7. [15]

    Fast r-cnn

    Ross Girshick. Fast r-cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1440–1448,

  8. [16]

    Rich feature hierarchies for accurate object detection and semantic segmentation

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 580–587, 2014. 2

    2014

  9. [17]

    Connecting large language models with evolutionary algo- rithms yields powerful prompt optimizers

    Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algo- rithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532, 2023. 3

    2023 arXiv

  10. [18]

    Ow-detr: Open-world detection transformer

    Akshita Gupta, Sanath Narayan, KJ Joseph, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Ow-detr: Open-world detection transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9235–9244, 2022. 2

    2022

  11. [19]

    Llms as visual explainers: Advancing image classification with evolving vi- sual descriptions

    Songhao Han, Le Zhuo, Yue Liao, and Si Liu. Llms as visual explainers: Advancing image classification with evolving vi- sual descriptions. arXiv preprint arXiv:2311.11904, 2023. 3

    2023 arXiv

  12. [20]

    Making anomalies more anomalous: Video anomaly detection using a novel generator and destroyer

    Seungkyun Hong, Sunghyun Ahn, Youngwan Jo, and Sanghyun Park. Making anomalies more anomalous: Video anomaly detection using a novel generator and destroyer. IEEE Access, 2024. 8

    2024

  13. [21]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 8

    2021 arXiv

  14. [22]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6700–6709, 2019. 3

    2019

  15. [23]

    Detrs with hybrid matching

    Ding Jia, Yuhui Yuan, Haodi He, Xiaopei Wu, Haojun Yu, Weihong Lin, Lei Sun, Chao Zhang, and Han Hu. Detrs with hybrid matching. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 19702–19712, 2023. 2

    2023

  16. [24]

    Consistency and uncertainty: Iden- tifying unreliable responses from black-box vision-language models for selective visual question answering

    Zaid Khan and Yun Fu. Consistency and uncertainty: Iden- tifying unreliable responses from black-box vision-language models for selective visual question answering. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10854–10863, 2024. 3

    2024

  17. [25]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of compute...

    2017

  18. [26]

    The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021. 8

    2021 arXiv

  19. [27]

    Sodformer: Streaming object detection with transformer using events and frames

    Dianze Li, Yonghong Tian, and Jianing Li. Sodformer: Streaming object detection with transformer using events and frames. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):14020–14037, 2023. 2

    2023

  20. [28]

    Asynchronous spatio-temporal memory net- work for continuous event-based object detection

    Jianing Li, Jia Li, Lin Zhu, Xijie Xiang, Tiejun Huang, and Yonghong Tian. Asynchronous spatio-temporal memory net- work for continuous event-based object detection. IEEE Transactions on Image Processing, 31:2975–2987, 2022. 2

    2022

  21. [29]

    Neural- logic human-object interaction detection

    Liulei Li, Jianan Wei, Wenguan Wang, and Yi Yang. Neural- logic human-object interaction detection. Advances in Neu- ral Information Processing Systems, 36:21158–21171, 2023. 3

    2023

  22. [30]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, page...

    2022

  23. [31]

    From pixels to graphs: Open-vocabulary scene graph generation with vision-language models

    Rongjie Li, Songyang Zhang, Dahua Lin, Kai Chen, and Xuming He. From pixels to graphs: Open-vocabulary scene graph generation with vision-language models. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28076–28086, 2024. 3

    2024

  24. [32]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pag...

    2023

  25. [33]

    Complex event recognition via spatial-temporal relation graph reasoning

    Huan Lin, Hongtian Zhao, and Hua Yang. Complex event recognition via spatial-temporal relation graph reasoning. In 2021 International Conference on Visual Communications and Image Processing (VCIP), pages 1–5. IEEE, 2021. 3

    2021

  26. [34]

    Ru-net: Regularized unrolling network for scene graph generation

    Xin Lin, Changxing Ding, Jing Zhang, Yibing Zhan, and Dacheng Tao. Ru-net: Regularized unrolling network for scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19457–19466, 2022. 3

    2022

  27. [35]

    Mo- tion robust high-speed light-weighted object detection with event camera

    Bingde Liu, Chang Xu, Wen Yang, Huai Yu, and Lei Yu. Mo- tion robust high-speed light-weighted object detection with event camera. IEEE Transactions on Instrumentation and Measurement, 72:1–13, 2023. 2

    2023

  28. [36]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. 1

    2023

  29. [37]

    Dab-detr: Dynamic anchor boxes are better queries for detr

    Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201.12329, 2022. 2

    2022 arXiv

  30. [38]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  31. [39]

    Diversity-measurable anomaly detection

    Wenrui Liu, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Diversity-measurable anomaly detection. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 12147–12156, 2023. 8

    2023

  32. [40]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3, 7

    2021

  33. [41]

    Openeqa: Embodied question answering in the era of foun- dation models

    Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foun- dation models. In Proceedings of the IEEE/CVF conference on...

    2024

  34. [42]

    The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from nat- ural supervision

    Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from nat- ural supervision. arXiv preprint arXiv:1904.12584, 2019. 3

    1904 arXiv

  35. [43]

    Conditional detr for fast training convergence

    Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, and Jingdong Wang. Conditional detr for fast training convergence. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 3651–3660, 2021. 2

    2021

  36. [44]

    Automatic interaction and activity recog- nition from videos of human manual demonstrations with application to anomaly detection

    Elena Merlo, Marta Lagomarsino, Edoardo Lamon, and Arash Ajoudani. Automatic interaction and activity recog- nition from videos of human manual demonstrations with application to anomaly detection. In 2023 32nd IEEE Inter- national Conference on Robot and Human Interactive Com...

    2023

  37. [45]

    L. Mou, Y . Hua, P. Jin, and X. X. Zhu. ERA: A dataset and deep learning benchmark for event recognition in aerial videos. IEEE Geoscience and Remote Sensing Magazine, in press. 6, 7

  38. [46]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1

    2023 arXiv

  39. [47]

    Plum: Prompt learning using metaheuristic

    Rui Pan, Shuo Xing, Shizhe Diao, Wenhe Sun, Xiang Liu, Kashun Shum, Renjie Pi, Jipeng Zhang, and Tong Zhang. Plum: Prompt learning using metaheuristic. arXiv preprint arXiv:2311.08364, 2023. 3

    2023 arXiv

  40. [48]

    Fastano: Fast anomaly detection via spatio- temporal patch transformation

    Chaewon Park, MyeongAh Cho, Minhyeok Lee, and Sangy- oun Lee. Fastano: Fast anomaly detection via spatio- temporal patch transformation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2249–2259, 2022. 8

    2022

  41. [49]

    Detgpt: Detect what you need via reasoning

    Renjie Pi, Jiahui Gao, Shizhe Diao, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, et al. Detgpt: Detect what you need via reasoning. arXiv preprint arXiv:2305.14167, 2023. 3

    2023 arXiv

  42. [50]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  43. [51]

    You only look once: Unified, real-time object de- tection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016. 2

    2016

  44. [52]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information process- ing systems, 28, 2015. 2

    2015

  45. [53]

    Self-distilled masked auto-encoders are efficient video anomaly detectors

    Nicolae-C Ristea, Florinel-Alin Croitoru, Radu Tudor Ionescu, Marius Popescu, Fahad Shahbaz Khan, Mubarak Shah, et al. Self-distilled masked auto-encoders are efficient video anomaly detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  46. [54]

    Learning to detect every thing in an open world

    Kuniaki Saito, Ping Hu, Trevor Darrell, and Kate Saenko. Learning to detect every thing in an open world. InEuropean Conference on Computer Vision , pages 268–284. Springer,

  47. [55]

    Deepunseen: Unpredicted event recognition through integrated vision- language models

    Hidetomo Sakaino, Natnapat Gaviphat, Louie Zamora, Ali- vanh Insisiengmay, and Dwi Fetiria Ningrum. Deepunseen: Unpredicted event recognition through integrated vision- language models. In 2023 IEEE Conference on Artificial Intelligence (CAI), pages 48–50. IEEE, 2023. 3

    2023

  48. [56]

    Aligning and prompting everything all at once for univer- sal visual perception

    Yunhang Shen, Chaoyou Fu, Peixian Chen, Mengdan Zhang, Ke Li, Xing Sun, Yunsheng Wu, Shaohui Lin, and Rongrong Ji. Aligning and prompting everything all at once for univer- sal visual perception. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recogn...

    2024

  49. [57]

    Explainable and explicit visual reasoning over scene graphs

    Jiaxin Shi, Hanwang Zhang, and Juanzi Li. Explainable and explicit visual reasoning over scene graphs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8376–8384, 2019. 3

    2019

  50. [58]

    Vald-gan: video anomaly detection using latent discriminator augmented gan

    Rituraj Singh, Anikeit Sethi, Krishanu Saini, Sumeet Saurav, Aruna Tiwari, and Sanjay Singh. Vald-gan: video anomaly detection using latent discriminator augmented gan. Signal, Image and Video Processing, 18(1):821–831, 2024. 8

    2024

  51. [59]

    Eva-clip: Improved training techniques for clip at scale

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15389, 2023. 1, 3, 7

    2023 arXiv

  52. [60]

    Dual memory aggregation network for event-based ob- ject detection with learnable representation

    Dongsheng Wang, Xu Jia, Yang Zhang, Xinyu Zhang, Yaoyuan Wang, Ziyang Zhang, Dong Wang, and Huchuan Lu. Dual memory aggregation network for event-based ob- ject detection with learnable representation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pages 2...

    2023

  53. [61]

    Anomaly detection in crowd scene

    Shu Wang and Zhenjiang Miao. Anomaly detection in crowd scene. In IEEE 10th International Conference on Signal Pro- cessing Proceedings, pages 1220–1223. IEEE, 2010. 2, 6, 8

    2010

  54. [62]

    Scene graph disentanglement and composition for generalizable complex image generation

    Yunnan Wang, Ziqiang Li, Wenyao Zhang, Zequn Zhang, Baao Xie, Xihui Liu, Wenjun Zeng, and Xin Jin. Scene graph disentanglement and composition for generalizable complex image generation. Advances in Neural Information Process- ing Systems, 37:98478–98504, 2025. 3

    2025

  55. [63]

    Gps: Genetic prompt search for efficient few-shot learning

    Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yang- gang Wang, Haiyu Li, and Zhilin Yang. Gps: Genetic prompt search for efficient few-shot learning. arXiv preprint arXiv:2210.17041, 2022. 3

    2022 arXiv

  56. [64]

    Neural-symbolic vqa: Disentangling reasoning from vision and language under- standing

    Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Push- meet Kohli, and Josh Tenenbaum. Neural-symbolic vqa: Disentangling reasoning from vision and language under- standing. Advances in neural information processing sys- tems, 31, 2018. 3

    2018

  57. [65]

    Bridging knowledge graphs to generate scene graphs

    Alireza Zareian, Svebor Karaman, and Shih-Fu Chang. Bridging knowledge graphs to generate scene graphs. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII 16, pages 606–623. Springer, 2020. 3

    2020

  58. [66]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 2, 3

    2022 arXiv

  59. [67]

    Spiking transform- ers for event-based single object tracking

    Jiqing Zhang, Bo Dong, Haiwei Zhang, Jianchuan Ding, Fe- lix Heide, Baocai Yin, and Xin Yang. Spiking transform- ers for event-based single object tracking. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 8801–8810, 2022. 2

    2022

  60. [68]

    Groundhog: Grounding large language models to holistic segmentation

    Yichi Zhang, Ziqiao Ma, Xiaofeng Gao, Suhaila Shakiah, Qiaozi Gao, and Joyce Chai. Groundhog: Grounding large language models to holistic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14227–14238, 2024. 3

    2024

  61. [69]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 2

    2010 arXiv

  62. [70]

    Prob: Probabilistic objectness for open world object detection

    Orr Zohar, Kuan-Chieh Wang, and Serena Yeung. Prob: Probabilistic objectness for open world object detection. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 11444–11453, 2023. 2

    2023

  63. [71]

    Segment everything everywhere all at once

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. Advances in Neural Information Processing Systems, 36, 2024. 3 From Objects to Events: Unlocking Complex Visual Under...

    2024

  64. [72]

    Ablation Study Analysis of Component Contributions. To understand the contribution of each component in our framework, we con- duct comprehensive ablation studies examining the individ- ual and combined effects of LLM reasoning and symbolic regression. Starting with a baseline...

  65. [73]

    Detailed results can be found in Table 4

    Extra experiments We conducted additional experiments by transferring sev- eral representative neuro-symbolic methods to be evaluated on our benchmark dataset. Detailed results can be found in Table 4. In addition, we conducted experiments to com- pare the computational overhe...

  66. [74]

    Table 8 summarizes the key mathematical symbols and their definitions used in this work

    Notation and Results This section provides a comprehensive list of mathematical symbols and notations used throughout this paper, followed by visual demonstration of our proposed method’s perfor- mance. Table 8 summarizes the key mathematical symbols and their definitions used...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.