Empirical study of 400 LLM attack runs finds exploitation success rates of 25-85% across four models against a fixed multi-service honeypot, with model-distinctive failure modes and p<0.001 differences.
Autopenbench: Benchmarking generative agents for penetration testing
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
citation-role summary
background 2
citation-polarity summary
roles
background 2representative citing papers
CyberEvolver introduces a four-layer self-evolving agent architecture with trace-to-diagnosis and population beam search that raises seed agent success rates by 13.6% on CTF, exploitation, and penetration tasks across four LLMs.
An evaluation protocol for AI pentesting agents that scores validated vulnerability discovery using LLM-based semantic matching and bipartite resolution.
RSA prompting enables LLMs to automatically create functional exploits for CVEs in Odoo ERP, succeeding on all tested cases in 3-5 rounds and removing the need for manual effort.
A tiered server benchmark with 300 targets shows current LLMs achieve autonomous penetration success rates of 10.7-69.3% using only general cybersecurity tools and no target-specific knowledge.
Expert-defined action plans for LLM agents achieve higher task completion in lateral-movement scenarios than fully autonomous or self-scaffolded modes, but failures remain common due to brittle commands and state handling.
A systematic review of neuro-symbolic AI in cybersecurity finds that deeper integration and causal reasoning improve performance across intrusion detection and vulnerability tasks, while identifying barriers and a research roadmap.
citing papers explorer
-
How Reliable Are AI Attackers Against a Fixed Vulnerable Target? A 400-Run Empirical Study of LLM Penetration Testing Consistency
Empirical study of 400 LLM attack runs finds exploitation success rates of 25-85% across four models against a fixed multi-service honeypot, with model-distinctive failure modes and p<0.001 differences.
-
CyberEvolver: Structured Self-Evolution for Cybersecurity Agents On the Fly
CyberEvolver introduces a four-layer self-evolving agent architecture with trace-to-diagnosis and population beam search that raises seed agent success rates by 13.6% on CTF, exploitation, and penetration tasks across four LLMs.
-
From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World
An evaluation protocol for AI pentesting agents that scores validated vulnerability discovery using LLM-based semantic matching and bipartite resolution.
-
From Rookie to Expert: Manipulating LLMs for Automated Vulnerability Exploitation in Enterprise Software
RSA prompting enables LLMs to automatically create functional exploits for CVEs in Odoo ERP, succeeding on all tested cases in 3-5 rounds and removing the need for manual effort.
-
The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems
A tiered server benchmark with 300 targets shows current LLMs achieve autonomous penetration success rates of 10.7-69.3% using only general cybersecurity tools and no target-specific knowledge.
-
Autonomous Adversary: Red-Teaming in the age of LLM
Expert-defined action plans for LLM agents achieve higher task completion in lateral-movement scenarios than fully autonomous or self-scaffolded modes, but failures remain common due to brittle commands and state handling.
-
Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities
A systematic review of neuro-symbolic AI in cybersecurity finds that deeper integration and causal reasoning improve performance across intrusion detection and vulnerability tasks, while identifying barriers and a research roadmap.