Pith. sign in

REVIEW 11 cited by

Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.11147 v3 pith:5LBZH3EQ submitted 2024-06-17 cs.SE cs.AI

Xueying Du , Geng Zheng , Kaixin Wang , Yi Zou , Yujia Wang , Wentai Deng , Jiayi Feng , Mingwei Liu
show 4 more authors
Bihuan Chen Xin Peng Tao Ma Yiling Lou
This is my paper
classification cs.SEcs.AI
keywords vulnerabilitydetectionllmsaccuracyvul-ragcodeenhancingknowledge
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Although LLMs have shown promising potential in vulnerability detection, this study reveals their limitations in distinguishing between vulnerable and similar-but-benign patched code (only 0.06 - 0.14 accuracy). It shows that LLMs struggle to capture the root causes of vulnerabilities during vulnerability detection. To address this challenge, we propose enhancing LLMs with multi-dimensional vulnerability knowledge distilled from historical vulnerabilities and fixes. We design a novel knowledge-level Retrieval-Augmented Generation framework Vul-RAG, which improves LLMs with an accuracy increase of 16% - 24% in identifying vulnerable and patched code. Additionally, vulnerability knowledge generated by Vul-RAG can further (1) serve as high-quality explanations to improve manual detection accuracy (from 60% to 77%), and (2) detect 10 previously-unknown bugs in the recent Linux kernel release with 6 assigned CVEs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection

    cs.CR 2026-07 conditional novelty 6.0 of 10

    DREA improves repository-level vulnerability detection by coupling an LLM planner that forms hypotheses with a cheap local explorer that gathers cross-file evidence, lifting paired accuracy from 19-26% to 30-42% at mu...

  2. Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization

    cs.CR 2026-03 conditional novelty 6.0 of 10

    Knowdit links abstract DeFi semantics to vulnerability patterns in a knowledge graph and drives an agentic specify–harness–fuzz–reflect loop that finds all high-severity and most medium-severity bugs on held-out Code4...

  3. Multi-Modal Requirements Data-based Acceptance Criteria Generation using LLMs

    cs.SE 2025-08 conditional novelty 6.0 of 10

    Multi-modal RAG (text plus UI screenshots) with reward-based polishing generates acceptance criteria from user stories that three industry experts rated near 4/5 on relevance, correctness, and understandability.

  4. Improving LLM Reasoning for Vulnerability Detection via Group Relative Policy Optimization

    cs.CR 2025-07 conditional novelty 6.0 of 10

    GRPO fine-tuning with a dynamic formatting, reasoning, and correctness reward helps small LLMs detect vulnerabilities more accurately and reason more coherently than supervised fine-tuning.

  5. iPanda: An LLM-based Agent for Automated Conformance Testing of Communication Protocols

    cs.SE 2025-07 conditional novelty 6.0 of 10

    An LLM-based agent that parses protocol RFCs into test cases, synthesizes executable test programs with RAG and chain-of-thought, and iteratively debugs them, reports 4.7x to 10.8x Pass@1 improvements over a pure-LLM ...

  6. LLM Embedding-based Attribution (LEA): Quantifying Source Contributions to Generative Model's Response for Vulnerability Analysis

    cs.CR 2025-06 reject novelty 6.0 of 10

    LEA uses rank-based linear dependence of layer-0 hidden states to attribute each response token to query, retrieved context, or internal knowledge, and distinguishes valid from generic retrieval with over 95% accuracy.

  7. Minimizing False Positives in Static Bug Detection via LLM-Enhanced Path Feasibility Analysis

    cs.SE 2025-06 conditional novelty 6.0 of 10

    LLM4PFA combines iterative LLM-based constraint extraction with Z3 solving to reduce false positives in static analysis, achieving 72-96% false positive reduction and 93% recall on a new benchmark.

  8. Enhancing Semantic Understanding in Pointer Analysis using Large Language Models

    cs.SE 2025-08 conditional novelty 5.0 of 10

    LMPA is a proposed, unevaluated framework for using LLMs to improve pointer analysis through API abstraction, parameter inference, and natural-language summaries.

  9. Identifying Helpful Context for LLM-based Vulnerability Repair: A Preliminary Study

    cs.SE 2025-06 conditional novelty 5.0 of 10

    Using CVE descriptions and manually selected code context in prompts, and combining the best prompts, GPT-4o fixed 26 of 42 Java vulnerabilities at least once, up from 19 with its baseline prompt.

  10. Large Language Models for Security Operations Centers: A Comprehensive Survey

    cs.CR 2025-09 conditional novelty 4.0 of 10

    A systematic review of 138 papers classifying LLM applications in SOC workflows by phase, model family, datasets, and maturity.

  11. WALL: A Web Application for Automated Quality Assurance using Large Language Models

    cs.SE 2025-09 reject novelty 4.0 of 10

    WALL combines SonarQube with GPT-3.5 Turbo and GPT-4o to automatically fix detected code issues, reporting 71.6% and 85.5% revision rates plus cost reductions from a hybrid workflow.

Pith tools