Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Context-Enhanced Vulnerability Detection Based on Large Language Model

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Feeding large language models compact program-analysis summaries of security-sensitive APIs such as malloc, free, open, and close significantly improves their ability to detect inter-procedural memory-safety vulnerabilities.

desk verdict A sensible abstraction-granularity study is undermined by a missing majority-class baseline: every reported F1 falls below an always-vulnerable classifier, so the central claim of significant enhancement is unverified. read the letter →

arxiv 2504.16877 v1 pith:7PZWPRYS submitted 2025-04-23 cs.SE

Yixin Yang , Bowen Xu , Xiang Gao , Hailong Sun This is my paper · ORCID
classification cs.SE
keywords vulnerabilitydetectionlargelanguagemodelsprimitiveAPIabstractionprogramanalysismemorysafetyinter-proceduralvulnerabilitiespromptengineeringcontextenhancement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vulnerability detectors built on large language models often miss inter-procedural memory bugs because they see only the patched function. This paper claims that adding a compact program-analysis summary of security-sensitive primitive APIs—such as malloc, free, open, and close—as context exposes the root cause without flooding the model with irrelevant repository code. In experiments across several LLMs, prompt strategies, and 623 C/C++ vulnerability samples, every API-abstraction level improved detection over no context, and the level that adds concrete branch conditions plus call counts gave the best overall balance, with top configurations beating raw-callee baselines by as much as 12.77 accuracy points. The paper further finds that the ideal abstraction depth depends on the model and the vulnerability class: reasoning-oriented models prefer high-level summaries, code-specialized models need more detail, and resource-management bugs need richer context than boundary errors. If these findings hold, choosing what context to extract and how abstractly to present it becomes a central design decision for any LLM-based code analysis.

What carries the argument

The central object is the primitive API abstraction: a compact, program-analysis-derived description of how security-sensitive application programming interfaces are used across a target function and its callees. It is defined along four dimensions—fuzzy branch coverage, concrete branch conditions, call counts, and the key variables operated on—which are combined into four abstraction levels. This object carries the argument because it is what converts noisy inter-procedural code into a small prompt that keeps the root-cause information the LLM needs; the empirical comparison across abstraction levels is precisely a test of which combination of these dimensions gives the best signal-to-noise ratio.

What would settle it

Construct or collect an inter-procedural memory-safety vulnerability whose root cause flows through a custom function not on the primitive API list—for example, a hand-rolled memory pool or a reference-count macro—then run PacVD with and without the abstraction. If a strong LLM detects it from raw callee code but not from the API abstraction, while no listed API appears in the call chain, the claim that primitive APIs are an adequate context lens for memory-safety vulnerabilities fails for that class.

Watch

Extended reading notes

Core claim

The paper's central claim is that primitive API abstraction—a program-analysis summary of what security-sensitive API operations a function and its callees perform—substantially improves LLM-based vulnerability detection. For each target function, the method analyzes callees up to three call layers deep and summarizes four dimensions of API usage: whether a primitive API is invoked in all, some, or none of the control-flow branches (fuzzy branches); the exact branch conditions under which it is invoked (concrete branches); how many times it is invoked; and which variables it operates on. Different combinations of these dimensions form abstraction levels A1 through A4, and these summaries are appended to the source code in the prompt. The paper reports that all abstraction levels beat the no-context baseline, with A3 (concrete branches plus call counts) the most consistently effective, reaching F1 scores near 69 percent, while adding variable-level information at A4 yields diminishing returns. It also reports that resource-management vulnerabilities need the richer abstraction levels, boundary-related memory errors can be caught with minimal abstraction, and different models require different abstraction depths, with a reasoning-oriented model performing robustly across levels and a code-specialized model peaking at the more detailed level.

Load-bearing premise

The load-bearing premise is that a hand-picked list of security-sensitive primitive APIs (malloc, free, open, close, and similar) plus summaries of their branch conditions and call counts is enough to expose the root cause of the targeted vulnerabilities; if the dangerous behavior flows through an API outside that list, or through semantics the summaries discard, the added context contributes nothing.

Editorial extensions

If this is right

  • Bare patched functions are a weak input: even the coarsest API summary improved F1 by roughly 5–8 points over the no-context baseline, so LLM detectors should include some abstracted callee context.
  • When a single configuration must be defaulted, concrete branch conditions plus call counts (A3) is the best-documented choice: it is the most consistently effective level across models, prompt strategies, and CWE types.
  • Abstraction level and model type should be chosen jointly: reasoning-oriented models work well with high-level context, while code-specialized models peak at more detailed abstraction; richer prompts such as chain-of-thought only pay off when the API context is already rich.
  • The approach improves the precision–recall profile where it matters operationally: it keeps high recall while increasing precision relative to raw-callee baselines, meaning fewer false positives without losing detections.
  • Resource-management CWEs such as double free, resource leak, and uncontrolled consumption push the needed abstraction level upward, while boundary-error CWEs such as out-of-bounds read and write are detectable even at the coarsest level.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the mechanism is noise filtering, the same primitive API abstraction should transfer to other LLM code-analysis tasks, such as bug localization or API-misuse detection; a direct test would use the exact A1–A4 summaries as prompts in those tasks and compare against raw code.
  • Editorial extension: the diminishing returns at A4 raise a testable question the paper leaves open—whether the drop comes from alias-analysis imprecision in key-variable extraction or from exceeding a general information-saturation point; replacing static variable association with a lightweight taint summary would separate the two.
  • Editorial extension: the primitive list is C/POSIX resource oriented, so extending it to non-memory classes such as locks, integer conversions, and authentication checks would reveal whether the method generalizes beyond memory-safety CWEs to other vulnerability families.
  • Editorial extension: because abstraction level and prompt strategy interact, an adaptive selector that chooses level and prompt per model and per CWE could plausibly outperform the fixed A3 default, a consequence the paper suggests in its discussion but does not implement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes PacVD, a context-enhanced vulnerability detection approach that combines static program analysis with large language models (LLMs). The method extracts primitive-API information (fuzzy branches, concrete branches, call counts, and key variables) from a target function and its callees up to three call layers, and feeds this abstraction together with the source code to ChatGPT-4o, DeepSeek, and CodeLLaMA under several prompting strategies. The authors report four findings: (1) API abstraction improves detection accuracy and reduces false positives; (2) different vulnerability types benefit from different abstraction levels; (3) larger models prefer higher-level abstractions while CodeLLaMA benefits from more detail; and (4) reasoning-oriented prompting matches higher abstraction levels. The evaluation uses a 623-sample C/C++ dataset (340 vulnerable, 283 non-vulnerable) drawn from NVD-New and reports accuracy, precision, recall, F1, and MCC across six prompts and four abstraction levels, with the best F1 being 69.09% for DeepSeek-R1 + CoT at API Level 3.

Significance. If the central claim were established, the idea of using program-analysis-derived API context to improve LLM vulnerability detection would be a useful contribution to the software engineering literature. The paper covers a broad experimental space: three model families, six prompting strategies, four abstraction levels, and a real CVE-derived dataset. The abstraction scheme itself is concrete and could be replicated from the description. However, the evaluation as presented does not support the headline claim. The dataset is imbalanced, and a trivial classifier that always predicts 'vulnerable' achieves F1 = 70.6% on this data; every configuration in Tables 4 and 5 reports an F1 below that value. No majority-class baseline, no significance tests, no confidence intervals, and no repeated runs are reported, and the few-shot examples are drawn from the same dataset without a train/test split. Because the load-bearing evidence is missing or contradicted by the data, the current result does not constitute a verified empirical finding.

major comments (4)
  1. [Section 4.2 (Table 3) and Tables 4-5] The evaluation omits the trivial majority-class baseline. The dataset contains 340 vulnerable and 283 non-vulnerable samples; a classifier that labels every sample as vulnerable has precision 340/623 = 54.6%, recall 100%, and F1 = 70.6%. All reported F1 scores in Tables 4 and 5, including the headline DeepSeek-R1 + CoT result at API Level 3 (69.09%), are below this value. The paper's central claim that the approach 'significantly enhances' detection is therefore not supported by the reported numbers; no configuration in the paper outperforms a predictor that ignores the code entirely. The authors should add this baseline and report statistical tests (e.g., paired tests with multiple runs or bootstrap confidence intervals) before any enhancement claim can be evaluated.
  2. [Section 5.4 (Table 5) and Section 1] The claimed improvement of 'up to 12.77% in accuracy' is computed against the weakest baseline in Table 5 (accuracy 47.37 for SS-C vs. 53.42 for PacVD with DeepSeek-R1 + CoT), while the best baseline in the same row, RS-C, achieves 53.26% accuracy, a difference of only 0.16 percentage points. Many of the reported F1 gaps between the proposed method and baselines are between 1 and 6 points, and no significance testing or repeated sampling is described. These differences are within the range that could be produced by sampling noise or prompt randomness, so the conclusion that PacVD 'consistently outperforms' baselines is not established.
  3. [Section 3.3.2] The few-shot learning prompts use examples that are 'randomly selected from our dataset' with no indication that they are drawn from a separate training set or excluded from the test samples. This creates a leakage risk because a few-shot example may contain the very function being evaluated or a near duplicate from the same CVE, which would artificially inflate the few-shot conditions. The authors should specify how the few-shot examples were selected, confirm that they are disjoint from the test instances, and rerun the affected configurations with a clean split.
  4. [Section 3.2.1 (Table 1)] The primitive API list is explicitly chosen because those APIs are the locus of the targeted vulnerability types (malloc/free for memory errors, open/close for resource leaks, etc.). Since the dataset is restricted to those CWE families, the abstraction directly injects the vocabulary that encodes the expected root cause. The observed gains relative to the no-API condition therefore partially reflect that the hand-picked features are label-aligned by construction, rather than demonstrating a general capability of the abstraction methodology. The claims should be scoped to the memory-safety CWE families considered, and the method should be tested with an expanded API set or with control abstractions built from non-security-sensitive APIs to separate genuine contextual value from direct feature-label correspondence.
minor comments (5)
  1. [Section 4.3 vs. Tables 4-5] The model configuration section mentions 'DeepSeek V2.5', but the experiments report results for DeepSeek-V3 and DeepSeek-R1; the text should be updated to name the exact models used.
  2. [Table 4, Figure 2, Figure 4] The notation for prompt strategies is inconsistent: the table note uses 'CT', 'FR', and 'FC', while the text and figure legends use 'CoT', 'FSR', and 'FSC'. Please unify the abbreviations.
  3. [Section 5.4 and Table 5] The model is called 'CodeLLaMA-34b' in Sections 4.3 and 5.3 but 'CodeLLaMA-32b' in Section 5.4 and Table 5. This needs to be corrected.
  4. [Section 5.2 and Figure 3] The per-CWE F1 scores are presented on small subsets without reporting the number of samples per CWE or any uncertainty measure. Adding sample counts and confidence bounds would help the reader judge whether the per-type gains are meaningful.
  5. [Section 9] The data availability link (https://github.com/DoeSEResearch/PacVD.git) cannot be verified from the manuscript. Please provide a DOI or an accessible repository snapshot and indicate whether the dataset and code are publicly available at the time of publication.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: API-feature vocabulary and test-set screening are coupled to the label set; 'optimal' A3 is selected on the same test data.

  1. self definitional [Section 3.2.1, Table 1; Section 4.2 Dataset]
    "Common primitive APIs are summarized in Table 1, which highlights the different types of vulnerabilities associated with these APIs. … In this step, we screen out specific memory safety vulnerabilities that are applicable to our method."

    Table 1 maps the input vocabulary directly to the label set (e.g., malloc/free -> Memory Leak, Use-After-Free, Double Free), and Table 3's dataset contains exactly those CWE types (CWE-401/415/416). The dataset is then filtered to vulnerabilities 'applicable to our method,' i.e., those whose mechanism is already encoded in the primitive-API feature set. The reported gain from adding API abstraction is therefore partly a restatement of how the dataset and features were constructed, rather than an independent test of whether the abstraction reveals unknown vulnerability structure.

  2. fitted input called prediction [Section 5.1.3]
    "Experimental results consistently demonstrate that the A3 level (i.e., specific branches + call frequency) provides the optimal cost-performance ratio and performance equilibrium point in vulnerability detection."

    A3 is not pre-registered as a hypothesis; it is identified as the best-performing abstraction level after inspecting F1 scores on the same test set in Section 5.1.1. RQ4 then uses the best level per model as 'PacVD' and reports improvement over baselines. Thus the 'prediction' that A3 is optimal and that PacVD outperforms baselines is a post-hoc selection on the evaluation data, not an out-of-sample prediction; the conclusion is forced by the selection criterion rather than independently tested.

full rationale

The central claim is empirical rather than a formal derivation, and the LLM component does provide independent reasoning capacity, so this is not a fully circular paper. However, the evaluation is partially circular by construction: the primitive-API feature vocabulary is defined in Table 1 by the same vulnerability types that appear as labels in the screened dataset (Section 4.2), so observing that API abstraction helps on that dataset is partly baked into the dataset-construction choices. Additionally, the 'optimal' A3 level is selected post-hoc from the same test data and then presented as a finding, which is a fitted-input-called-prediction pattern. The Song et al. [39] citation for the API list is a self-citation (co-author Xiang Gao), but because the paper empirically tests the resulting list on an external benchmark, this citation is not itself load-bearing circularity. The absence of a majority-class baseline, lack of significance testing, and few-shot example leakage are serious correctness risks but are not circularity per se.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are postulated; Primitive API abstraction is a feature-engineering method, not an invented entity in the sense of a new particle or force.

free parameters (4)
  • Maximum call depth = 3
    Set to 3 layers by default based on Li et al. [25] statistics; this bounds how much callee context is included and directly affects the abstraction content.
  • LLM sampling temperature = 0.1
    Chosen to reduce randomness in CodeLLaMA outputs (Section 4.3); affects determinism of all model responses.
  • Top-p = 0.95
    Nucleus sampling parameter for CodeLLaMA (Section 4.3); a generation hyperparameter affecting outputs.
  • Max tokens = 512
    Token limit for reasoning outputs in CoT/ICL (Section 4.3); constrains the amount of reasoning the model can produce.
assumptions (3)
  • domain assumption NVD-New dataset labels (before-fix vulnerable vs after-fix non-vulnerable) are correct and representative
    The dataset is taken from Li et al. [28] and Zhang et al. [59]; the paper relies on CVE annotations and commit-level labels without independent verification.
  • domain assumption Joern code property graphs correctly resolve callees, control flow, and data flow for the C/C++ projects
    Section 3.1 states CPGs are constructed with Joern; the abstraction's correctness depends on the precision of this static analysis.
  • domain assumption The primitive API list (malloc/free, open/close, etc.) is sufficient to capture root causes of the targeted vulnerability types
    Section 3.2.1 adopts the API list from Song et al. [39] and claims vulnerabilities originate from improper use of these APIs; if other APIs are needed, the abstraction is incomplete.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Context-Enhanced Vulnerability Detection Based on Large Language Model." pith.science (2026). https://pith.science/paper/7PZWPRYS

@misc{pith2026250416877,
  author       = {Pith},
  title        = {Pith review of: Context-Enhanced Vulnerability Detection Based on Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7PZWPRYS}},
  note         = {Machine review of arXiv:2504.16877}
}
read the original abstract

Vulnerability detection is a critical aspect of software security. Accurate detection is essential to prevent potential security breaches and protect software systems from malicious attacks. Recently, vulnerability detection methods leveraging deep learning and large language models (LLMs) have garnered increasing attention. However, existing approaches often focus on analyzing individual files or functions, which limits their ability to gather sufficient contextual information. Analyzing entire repositories to gather context introduces significant noise and computational overhead. To address these challenges, we propose a context-enhanced vulnerability detection approach that combines program analysis with LLMs. Specifically, we use program analysis to extract contextual information at various levels of abstraction, thereby filtering out irrelevant noise. The abstracted context along with source code are provided to LLM for vulnerability detection. We investigate how different levels of contextual granularity improve LLM-based vulnerability detection performance. Our goal is to strike a balance between providing sufficient detail to accurately capture vulnerabilities and minimizing unnecessary complexity that could hinder model performance. Based on an extensive study using GPT-4, DeepSeek, and CodeLLaMA with various prompting strategies, our key findings includes: (1) incorporating abstracted context significantly enhances vulnerability detection effectiveness; (2) different models benefit from distinct levels of abstraction depending on their code understanding capabilities; and (3) capturing program behavior through program analysis for general LLM-based code analysis tasks can be a direction that requires further attention.

Figures

Figures reproduced from arXiv: 2504.16877 by the authors.

Figure 1
Figure 1. Primitive API Abstraction and Context-Enhanced Vulnerability Detection Framework analysis, the system constructs CPGs not only for the function itself but also callee functions. This multi-level analysis is crucial for capturing cross-function vulnerabilities. The output of these steps is a collection of target functions with their calling relationships, along with corresponding code property graph representations, … view at source ↗
Figure 2
Figure 2. F1 score for vulnerability detection using different API abstraction levels as auxiliary information. The horizontal axis represents different levels of API abstraction. The vertical axis indicates the F1-score of the model for vulnerability detection. The curves in different colors represent distinct prompt engineering strategies. BP denotes basic prompt, RP denotes role-based prompt, CT denotes Chain-of-thought, I… view at source ↗
Figure 3
Figure 3. The effect of different API abstraction levels on different vulnerability types, Bar height represents F1 score. Higher is better. Arrows (↑) indicate significant improvements over No API baseline. J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018 [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: F1-score comparison across different vulnerability detection models.Star symbols (⋆) denote the top-performing model for each API abstraction level. 5.3.1 Large Language Model Performance Analysis. As shown in [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection

    cs.CR 2026-07 conditional novelty 6.0 of 10

    DREA improves repository-level vulnerability detection by coupling an LLM planner that forms hypotheses with a cheap local explorer that gathers cross-file evidence, lifting paired accuracy from 19-26% to 30-42% at mu...

Reference graph

Works this paper leans on

65 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018. Context-Enhanced Vulnerability Detection Based on Large Langua...

    arXiv 2023

  2. [2]

    Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. 2019. code2vec: Learning distributed representations of code. Proceedings of the ACM on Programming Languages 3, POPL (2019), 1–29

    work page 2019

  3. [3]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    arXiv 2020

  4. [4]

    Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2022. Deep Learning Based Vulnerability Detection: Are We There Yet? IEEE Transactions on Software Engineering 48, 9 (2022), 3280–3296. https://doi.org/10. 1109/TSE.2021.3087402

    arXiv 2022

  5. [5]

    Lei Chen and Raymond Ng. 2004. On the marriage of lp-norms and edit distance. In Proceedings of the Thirtieth international conference on Very large data bases-Volume 30 . 792–803

    work page 2004

  6. [6]

    Yujia Chen, Cuiyun Gao, Zezhou Yang, Hongyu Zhang, and Qing Liao. 2024. Bridge and Hint: Extending Pre-trained Language Models for Long-Range Code. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 274–286

    work page 2024

  7. [7]

    Zhe Chen, Chong Wang, Junqi Yan, Yulei Sui, and Jingling Xue. 2021. Runtime detection of memory errors with smart status. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (Virtual, Denmark) (ISSTA 2021). Association for Computing Machinery, New York, NY, USA, 296–308. https://doi.org/10.1145/ 3460319.3464807

    arXiv 2021

  8. [8]

    Xiao Cheng, Haoyu Wang, Jiayi Hua, Miao Zhang, Guoai Xu, Li Yi, and Yulei Sui. 2019. Static detection of control- flow-related vulnerabilities using graph embedding. In 2019 24th International Conference on Engineering of Complex Computer Systems (ICECCS). IEEE, 41–50

    work page 2019

Show all 65 references
  1. [9]

    CVE. 2021. NVD. https://cve.mitre.org/

    2021

  2. [10]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025 arXiv

  3. [11]

    Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, and Yizheng Chen. 2024. Vulnerability detection with code language models: How far are we? arXiv preprint arXiv:2403.18624 (2024)

    2024 arXiv

  4. [12]

    Xiaoning Du, Bihuan Chen, Yuekang Li, Jianmin Guo, Yaqin Zhou, Yang Liu, and Yu Jiang. 2019. Leopard: Identifying vulnerable code for vulnerability assessment through program metrics. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 60–71

    2019

  5. [13]

    Xiaohu Du, Ming Wen, Jiahao Zhu, Zifan Xie, Bin Ji, Huijun Liu, Xuanhua Shi, and Hai Jin. 2024. Generalization- Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, And...

    2024 doi

  6. [14]

    Xueying Du, Geng Zheng, Kaixin Wang, Jiayi Feng, Wentai Deng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, and Yiling Lou. 2024. Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG. arXiv:2406.11147 [cs.SE] https://arxiv.org/abs/2406.11147 J. ACM, Vol. ...

    2024 arXiv

  7. [15]

    Fenil Fadadu, Anand Handa, Nitesh Kumar, and Sandeep Kumar Shukla. 2020. Evading API call sequence based malware classifiers. In Information and Communications Security: 21st International Conference, ICICS 2019, Beijing, China, December 15–17, 2019, Revised Selected Papers 21...

    2020

  8. [16]

    Angelo Furno, Nour-Eddin El Faouzi, Rajesh Sharma, Valerio Cammarota, and Eugenio Zimeo. 2018. A graph-based framework for real-time vulnerability assessment of road networks. In 2018 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 234–241

    2018

  9. [17]

    Mahmoud Ghorbanzadeh and Hamid Reza Shahriari. 2020. ANOVUL: Detection of logic vulnerabilities in annotated programs via data and control flow analysis. IET Information Security 14, 3 (2020), 352–364

    2020

  10. [18]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey. arXiv:2403.14608 [cs.LG] https://arxiv.org/abs/2403.14608

    2024 arXiv

  11. [19]

    Zhonghao Jiang, Weifeng Sun, Xiaoyan Gu, Jiaxin Wu, Tao Wen, Haibo Hu, and Meng Yan. 2024. DFEPT: Data Flow Embedding for Enhancing Pre-Trained Model Based Vulnerability Detection. In Proceedings of the 15th Asia-Pacific Symposium on Internetware. 95–104

    2024

  12. [20]

    Hongping Li and Li Shan. 2023. LLM-based Vulnerability Detection. In2023 International Conference on Human-Centered Cognitive Systems (HCCS). IEEE, 1–4

    2023

  13. [21]

    Qiang Li, Jinke Song, Dawei Tan, Haining Wang, and Jiqiang Liu. 2021. Pdgraph: a large-scale empirical study on project dependency of security vulnerabilities. In 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) . IEEE, 161–173

    2021

  14. [22]

    Yi Li, Shaohua Wang, and Tien N Nguyen. 2021. Vulnerability detection with fine-grained interpretations. InProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 292–303

    2021

  15. [24]

    Ziyang Li, Saikat Dutta, and Mayur Naik. 2025. IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities. arXiv:2405.17238 [cs.CR] https://arxiv.org/abs/2405.17238

    2025 arXiv

  16. [25]

    Zhen Li, Ning Wang, Deqing Zou, Yating Li, Ruqian Zhang, Shouhuai Xu, Chao Zhang, and Hai Jin. 2024. On the Effectiveness of Function-Level Vulnerability Detectors for Inter-Procedural Vulnerabilities. In Proceedings of the IEEE/ACM 46th International Conference on Software En...

    2024

  17. [26]

    Zhen Li, Deqing Zou, Jing Tang, Zhihao Zhang, Mingqian Sun, and Hai Jin. 2019. A comparative study of deep learning-based vulnerability detection system. IEEE Access 7 (2019), 103184–103197

    2019

  18. [27]

    Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. 2021. Sysevr: A framework for using deep learning to detect software vulnerabilities. IEEE Transactions on Dependable and Secure Computing 19, 4 (2021), 2244–2258

    2021

  19. [28]

    Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen. 2022. SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities. IEEE Transactions on Dependable and Secure Computing 19, 4 (2022), 2244–2258. https://doi.org/10.1109/TDSC.2021.3051525

    2022

  20. [29]

    Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018. Vuldeepecker: A deep learning-based system for vulnerability detection. arXiv preprint arXiv:1801.01681 (2018)

    2018 arXiv

  21. [30]

    Qiheng Mao, Zhenhao Li, Xing Hu, Kui Liu, Xin Xia, and Jianling Sun. 2024. Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models. https://doi.org/10.48550/arXiv.2406.09701

    2024 doi arXiv

  22. [31]

    National Vulnerability Database. 2021. NVD. https://nvd.nist.gov/

    2021

  23. [32]

    Moumita Das Purba, Arpita Ghosh, Benjamin J Radford, and Bill Chu. 2023. Software vulnerability detection using large language models. In 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 112–119

    2023

  24. [33]

    Komang Rinartha and Wayan Suryasa. 2017. Comparative study for better result on query suggestion of article searching with MySQL pattern matching and Jaccard similarity. In 2017 5th International Conference on Cyber and IT Service Management (CITSM) . IEEE, 1–4

    2017

  25. [34]

    Stephen Robertson, Hugo Zaragoza, et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389

    2009

  26. [35]

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models. Nature 623, 7987 (2023), 493–498

    2023

  27. [36]

    Madhu K Shankarapani, Subbu Ramamoorthy, Ram S Movva, and Srinivas Mukkamala. 2011. Malware detection using assembly and API call sequences. Journal in computer virology 7 (2011), 107–119

    2011

  28. [37]

    Aleksei Shestov, Rodion Levichev, Ravil Mussabayev, and Evgeny Maslov. 2024. Finetuning Large Language Models for Vulnerability Detection. https://doi.org/10.13140/RG.2.2.26099.54560

    2024

  29. [38]

    Qingkai Shi, Xiao Xiao, Rongxin Wu, Jinguo Zhou, Gang Fan, and Charles Zhang. 2018. Pinpoint: Fast and Precise Sparse Value Flow Analysis for Million Lines of Code. In Proceedings of the 39th ACM SIGPLAN Conference on Programming J. ACM, Vol. 37, No. 4, Article 111. Publicatio...

    2018

  30. [39]

    Yahui Song, Xiang Gao, Wenhua Li, Wei-Ngan Chin, and Abhik Roychoudhury. 2024. ProveNFix: Temporal Property- Guided Program Repair. Proceedings of the ACM on Software Engineering 1, FSE (2024), 226–248

    2024

  31. [40]

    Benjamin Steenhoek, Hongyang Gao, and Wei Le. 2024. Dataflow analysis-inspired deep learning for efficient vulnerability detection. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering . 1–13

    2024

  32. [41]

    Barr, and Wei Le

    Benjamin Steenhoek, Md Mahbubur Rahman, Monoshi Kumar Roy, Mirza Sanjida Alam, Earl T. Barr, and Wei Le

  33. [42]

    Benjamin Steenhoek, Md Mahbubur Rahman, Monoshi Kumar Roy, Mirza Sanjida Alam, Earl T Barr, and Wei Le. 2024. A Comprehensive Study of the Capabilities of Large Language Models for Vulnerability Detection. arXiv preprint arXiv:2403.17218 (2024)

    2024 arXiv

  34. [43]

    Jingyi Su and Yan Wu. 2023. Optimizing Pre-trained Language Models for Efficient Vulnerability Detection in Code Snippets. In 2023 9th International Conference on Computer and Communications (ICCC) . IEEE, 2139–2143

    2023

  35. [44]

    Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, and Mohit Iyyer. 2023. How does in-context learning help prompt tuning? arXiv preprint arXiv:2302.11521 (2023)

    2023 arXiv

  36. [45]

    Karl Tamberg and Hayretdin Bahsi. 2025. Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study. IEEE Access 13 (2025), 29698–29717. https://doi.org/10.1109/access.2025.3541146

    2025

  37. [46]

    T. J. Team. 2024. Joern. Online. Available: https://joern.io

    2024

  38. [47]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  39. [48]

    Ziliang Wang, Ge Li, Jia Li, Yingfei Xiong, Jia Li, Meng Yan, and Zhi Jin. 2024. M2CVD: Enhancing Vulnerability Semantic through Multi-Model Collaboration for Code Vulnerability Detection. arXiv:2406.05940 [cs.SE] https: //arxiv.org/abs/2406.05940

    2024

  40. [49]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al . 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  41. [50]

    Xin-Cheng Wen, Xinchen Wang, Yujia Chen, Ruida Hu, David Lo, and Cuiyun Gao. 2024. Vuleval: Towards repository- level evaluation of software vulnerability detection. arXiv preprint arXiv:2404.15596 (2024)

    2024 arXiv

  42. [51]

    Bolun Wu and Futai Zou. 2022. Code vulnerability detection based on deep sequence and graph models: A survey. Security and Communication Networks 2022, 1 (2022), 1176898

    2022

  43. [52]

    Fabian Yamaguchi, Nico Golde, Daniel Arp, and Konrad Rieck. 2014. Modeling and discovering vulnerabilities with code property graphs. In 2014 IEEE symposium on security and privacy . IEEE, 590–604

    2014

  44. [53]

    Yanjing Yang, Xin Zhou, Runfeng Mao, Jinwei Xu, Lanxin Yang, Yu Zhang, Haifeng Shen, and He Zhang. 2024. Dlap: A deep learning augmented large language model prompting framework for software vulnerability detection. Journal of Systems and Software (2024), 112234

    2024

  45. [54]

    Yanjing Yang, Xin Zhou, Runfeng Mao, Jinwei Xu, Lanxin Yang, Yu Zhang, Haifeng Shen, and He Zhang. 2025. DLAP: A Deep Learning Augmented Large Language Model Prompting framework for software vulnerability detection. J. Syst. Softw. 219, C (Jan. 2025), 15 pages. https://doi.org...

    2025

  46. [55]

    Chenyuan Zhang, Hao Liu, Jiutian Zeng, Kejing Yang, Yuhong Li, and Hui Li. 2024. Prompt-Enhanced Software Vulnerability Detection Using ChatGPT. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (Lisbon, Portugal) ...

    2024

  47. [56]

    Chenyuan Zhang, Hao Liu, Jiutian Zeng, Kejing Yang, Yuhong Li, and Hui Li. 2024. Prompt-enhanced software vulnerability detection using chatgpt. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings. 276–277

    2024

  48. [57]

    Jie Zhang, Wei Ma, Qiang Hu, Shangqing Liu, Xiaofei Xie, Yves Le Traon, and Yang Liu. 2023. A Black-Box Attack on Code Models via Representation Nearest Neighbor Search. In Findings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kal...

    2023 doi

  49. [58]

    Yulin Zhang, Yong Hu, and Xiao Chen. 2024. Context and Multi-Features-Based Vulnerability Detection: A Vulnerability Detection Frame Based on Context Slicing and Multi-Features. Sensors 24, 5 (2024), 1351

    2024

  50. [59]

    Yuting Zhang, Jiahao Zhu, Yixin Yang, Ming Wen, and Hai Jin. 2023. Comparing the Performance of Different Code Representations for Learning-based Vulnerability Detection. In Proceedings of the 14th Asia-Pacific Symposium on Internetware (Hangzhou, China) (Internetware ’23). As...

    2023

  51. [60]

    Minmin Zhou, Jinfu Chen, Yisong Liu, Hilary Ackah-Arthur, Shujie Chen, Qingchen Zhang, and Zhifeng Zeng. 2019. A method for software vulnerability detection based on improved control flow graph. Wuhan University Journal of Natural Sciences 24, 2 (2019), 149–160

    2019

  52. [61]

    Xin Zhou, Duc-Manh Tran, Thanh Le-Cong, Ting Zhang, Ivana Clairine Irsan, Joshua Sumarlin, Bach Le, and David Lo. 2024. Comparison of static application security testing tools and large language models for repo-level vulnerability detection. arXiv preprint arXiv:2407.16235 (2024)

    2024 arXiv

  53. [62]

    Xin Zhou, Ting Zhang, and David Lo. 2024. Large Language Model for Vulnerability Detection: Emerging Results and Future Directions. In Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results (Lisbon, Portugal) (ICS...

    2024

  54. [63]

    Xin Zhou, Ting Zhang, and David Lo. 2024. Large language model for vulnerability detection: Emerging results and future directions. In Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results. 47–51

    2024

  55. [64]

    Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems 32 (2019)

    2019

  56. [65]

    Thomas Zimmermann, Rahul Premraj, and Andreas Zeller. 2007. Predicting defects for eclipse. In Third International Workshop on Predictor Models in Software Engineering (PROMISE’07: ICSE Workshops 2007) . IEEE, 9–9. J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018

    2007

  57. [2024]

    CoRR abs/2403.17218 (2024)

    A Comprehensive Study of the Capabilities of Large Language Models for Vulnerability Detection. CoRR abs/2403.17218 (2024). https://doi.org/10.48550/arXiv.2403.17218

    2024 doi arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.