REVIEW 10 cited by
PentestAgent: Incorporating LLM Agents to Automated Penetration Testing
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
PentestAgent: Incorporating LLM Agents to Automated Penetration Testing
read the original abstract
Penetration testing is a critical technique for identifying security vulnerabilities, traditionally performed manually by skilled security specialists. This complex process involves gathering information about the target system, identifying entry points, exploiting the system, and reporting findings. Despite its effectiveness, manual penetration testing is time-consuming and expensive, often requiring significant expertise and resources that many organizations cannot afford. While automated penetration testing methods have been proposed, they often fall short in real-world applications due to limitations in flexibility, adaptability, and implementation. Recent advancements in large language models (LLMs) offer new opportunities for enhancing penetration testing through increased intelligence and automation. However, current LLM-based approaches still face significant challenges, including limited penetration testing knowledge and a lack of comprehensive automation capabilities. To address these gaps, we propose PentestAgent, a novel LLM-based automated penetration testing framework that leverages the power of LLMs and various LLM-based techniques like Retrieval Augmented Generation (RAG) to enhance penetration testing knowledge and automate various tasks. Our framework leverages multi-agent collaboration to automate intelligence gathering, vulnerability analysis, and exploitation stages, reducing manual intervention. We evaluate PentestAgent using a comprehensive benchmark, demonstrating superior performance in task completion and overall efficiency. This work significantly advances the practical applicability of automated penetration testing systems.
Forward citations
Cited by 10 Pith papers
-
How Reliable Are AI Attackers Against a Fixed Vulnerable Target? A 400-Run Empirical Study of LLM Penetration Testing Consistency
Empirical study of 400 LLM attack runs finds exploitation success rates of 25-85% across four models against a fixed multi-service honeypot, with model-distinctive failure modes and p<0.001 differences.
-
Honeyval: A Comprehensive Evaluation Framework for LLM-powered HTTP Honeypots
Honeyval evaluates LLM HTTP honeypots with AI attackers and shows they produce longer interactions, lower detection rates, and cost advantages over rule-based baselines.
-
Dynamic Cyber Ranges
Dynamic Cyber Ranges with LLM defender agents reduce attacker success to 0-55% and preserve evaluation headroom as models advance by using comparable capabilities on both sides.
-
APT-Agent: Automated Penetration Testing using Large Language Models
APT-Agent automates penetration testing with LLMs using rectification and memory modules, achieving 84.29% end-to-end success on Metasploitable 2 versus lower rates for baselines.
-
Self-Adaptive Multi-Agent LLM-Based Security Pattern Selection for IoT Systems
ASPO combines multi-agent LLM proposals with deterministic enforcement in a MAPE-K loop to select conflict-free, resource-feasible security patterns for IoT, delivering 100% safety invariants and 21-23% tail latency/e...
-
Benchmarking LLM-Assisted Blue Teaming via Standardized Threat Hunting
Standardized modular threat-hunting workflows (CyberTeam) improve LLM performance on blue team tasks compared to open-ended ICL, CoT, and ToT prompting across 30 tasks and 452k samples.
-
Synthetic APTs: the Collapse of TTP-Based Attribution
AI-emulated APTs compromise enterprise hosts and weaponize defender tools in 8/10 trials while military ranges resist, indicating TTP attribution fails when agents can be scaffolded to mimic threat actors.
-
Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities
A systematic review of neuro-symbolic AI in cybersecurity finds that deeper integration and causal reasoning improve performance across intrusion detection and vulnerability tasks, while identifying barriers and a res...
-
MultiFuzz: A Dense Retrieval-based Multi-Agent System for Network Protocol Fuzzing
MultiFuzz combines retrieval-augmented generation and multiple LLM agents within the ChatAFL protocol fuzzer, reporting marginal and statistically unsupported gains in branch coverage and state exploration for RTSP.
-
Pixels to Play: A Foundation Model for 3D Gameplay
Pixels2Play-0.1 is a decoder-only transformer trained via behavior cloning and inverse-dynamics-imputed actions to play 3D games from pixels, with only qualitative results reported.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.