REVIEW 13 cited by
To Err is Machine: Vulnerability Detection Challenges LLM Reasoning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
To Err is Machine: Vulnerability Detection Challenges LLM Reasoning
read the original abstract
In this paper, we present a challenging code reasoning task: vulnerability detection. Large Language Models (LLMs) have shown promising results in natural-language and math reasoning, but state-of-the-art (SOTA) models reported only 54.5% Balanced Accuracy in our vulnerability detection evaluation, even those models pre-trained on large amounts of source code. Our error analysis on LLM responses shows that the models struggle to reason about the code semantics relevant to identifying vulnerabilities, especially subtle semantic differences caused by small textual changes. We explored prominent models and training settings to understand their effects on vulnerability detection performance -- including better prompts, larger models, more pre-training data, and fine-tuning -- but none led to significant improvements. This raises the question of whether simply scaling training data and model size will allow us to "solve" complex code reasoning tasks like vulnerability detection, or if a fundamental shift in modeling and training techniques is required. We also explored adding domain knowledge to prompts; although it helped certain models understand some code semantics, vulnerability detection requires multi-step reasoning, and these models still failed in steps, such as reasoning about variable relations. Our results suggest that new models, new training methods, or more execution-specific pretraining data may be needed to conquer vulnerability detection. We speculate that auto-regressive pre-training on source code may not effectively extract code semantics, especially on the current pretraining mixtures, in which execution data is scarce. Success on vulnerability detection as a code reasoning task can benefit many areas of software engineering such as debugging, test input generation, and program repair. Our code and data are available at https://doi.org/10.6084/m9.figshare.27368025.
Forward citations
Cited by 13 Pith papers
-
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?
SEC-bench Pro benchmark with 183 real vulnerabilities shows frontier LLM coding agents achieve at most 38.8% success on SpiderMonkey and 32% on V8.
-
Veritas: Grounding LLM Agents for Reliable Vulnerability Reasoning over Stripped Binaries
Veritas detects binary out-of-bounds vulnerabilities via witness-backed IR flows, dual-view LLM claims, and multi-agent runtime validation, reaching 90% recall and a confirmed Apple CVE.
-
Geometric quantification for nonlinear deformation in knitted fabrics
A geometric quantification framework reconstructs yarn centerlines and fabric surfaces from sparse data to quantify how nonlinear deformation distributes among stitch reorientation, loop bending, surface bending, and ...
-
DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection
DREA improves repository-level vulnerability detection by coupling an LLM planner that forms hypotheses with a cheap local explorer that gathers cross-file evidence, lifting paired accuracy from 19-26% to 30-42% at mu...
-
SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?
On 183 validated V8/SpiderMonkey vulnerabilities, the best coding agents verify at most 38.8% of instances, and crash-only grading would inflate judged successes by 43.6%.
-
Three Heads Are Better Than One: A Multi-perspective Reasoning Framework for Enhanced Vulnerability Detection
ReasonVul deploys three LLM agents with independent analysis and structured debate to achieve 40% PairAcc and 72.52% F1 on PrimeVul, outperforming baselines by 81% in PairAcc.
-
Veritas: Grounding LLM Agents for Reliable Vulnerability Reasoning over Stripped Binaries
Veritas detects memory corruption vulnerabilities in stripped binaries by combining static value-flow slicing, dual-view LLM reasoning, and multi-agent runtime validation, reporting 90% recall, zero false positives on...
-
Veritas: Grounding LLM Agents for Reliable Vulnerability Reasoning over Stripped Binaries
Veritas detects out-of-bounds vulnerabilities in stripped binaries at 90% recall by grounding LLM reasoning in static witness-backed flows and runtime validation.
-
SAGE: Signal-Amplified Guided Embeddings for LLM-based Vulnerability Detection
SAGE uses sparse autoencoders to boost vulnerability signals in LLMs, raising internal SNR 12.7x and delivering up to 318% MCC gains on vulnerability detection benchmarks.
-
Do Fine-Tuned LLMs Understand Vulnerabilities? An Investigation into the Semantic Trap
Fine-tuned decoder-only LLMs fall into a Semantic Trap on vulnerability detection, achieving high scores on unpaired normal code but failing on paired vulnerable-patched code, semantic perturbations, and gap analysis,...
-
Geometric quantification for nonlinear deformation in knitted fabrics
A geometric quantification framework reconstructs yarn centerlines and fabric surfaces from sparse knit data and partitions large deformation into stitch reorientation, loop bending, surface bending, and dilation.
-
What You Code Is What We Prove: Translating BLE App Logic into Formal Models with LLMs for Vulnerability Detection
An LLM-based pipeline that converts Android BLE app code into ProVerif models finds that only about 10% of 1,050 apps implement encryption, nonces, and authentication.
-
Case Study: Fine-tuning Small Language Models for Accurate and Private CWE Detection in Python Code
Fine-tuning codegen-mono on 500 synthetic-plus-reviewed Python examples produces 99% accuracy, 98.08% precision, 100% recall and 99.04% F1 for CWE detection.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.