PenTest2.0: Towards Autonomous Privilege Escalation Using GenAI

Chris J. Mitchell; Haitham S. Al-Sinani

REVIEW 5 major objections 5 minor 19 references

PenTest2.0: Towards Autonomous Privilege Escalation Using GenAI

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM can escalate a low-privileged Linux account to root on a deliberately vulnerable target, the paper reports.

desk verdict A credible supervised feasibility demo of LLM privilege escalation on one easy target, but the 'multi-turn adaptive' headline claim is not supported by the reported runs. read the letter →

arxiv 2507.06742 v1 pith:ZZ7SESBS submitted 2025-07-09 cs.CR

Haitham S. Al-Sinani , Chris J. Mitchell This is my paper

classification cs.CR

keywords privilegeescalationLLMagentspenetrationtestingethicalhackingretrieval-augmentedgenerationchain-of-thoughtpromptinghuman-in-the-loopLinuxsecurity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a large language model, embedded in an execute-and-feedback loop, can carry out the privilege-escalation phase of ethical hacking—moving from a low-privileged account to root on a Linux target—with only light human supervision. The authors present PenTest2.0, a system that builds a fresh prompt each turn, asks the LLM for the next command and its rationale, runs the approved command over SSH, and feeds the output back into the next prompt. In tests against one deliberately vulnerable Linux virtual machine, all seven configurations (with and without Chain-of-Thought reasoning, retrieval-augmented generation, task-tree tracking, and human hints) reached root; four configurations also detected root automatically, usually within one or two turns. The paper argues that this shows the feasibility of scalable, semi-autonomous AI-driven penetration testing, while the failures in root auto-detection and repeated ineffective commands show why a human-in-the-loop remains necessary.

What carries the argument

The load-bearing mechanism is the turn-based prompt–execute–parse loop: each turn the system assembles a prompt from modular blocks (system facts, a capped command history, the goal to reach root, and togglable enhancements), requires the LLM to answer with a single JSON object containing a non-interactive command, an interactive variant, and a rationale, then executes the approved command over SSH and feeds a summary of the output into the next turn. Three named enhancements do specific work: Chain-of-Thought prompting adds a step-by-step reasoning instruction; retrieval-augmented generation injects relevant snippets from a local or online knowledge base of Unix command-escaping techniques; and the PenTest Task Tree keeps a persistent list of pending, done, and skipped subtasks so the model does not lose its plan across turns. A regex-based root detector watches for markers such as uid=0(root), and the system enforces non-interactive command variants so interactive shells do not hang the automation.

What would settle it

Repeat the same seven configurations with the same ten-turn cap and the same LLM against a current, fully patched Linux system with no sudo misconfigurations, no writable SUID binaries, and no exploitable services; if none of the configurations reaches root, or if the LLM repeatedly issues the same ineffective command even with Chain-of-Thought and hints enabled, the general feasibility claim would be shown to hold only for the paper's specially prepared target.

Watch

Extended reading notes

Core claim

The central claim is that privilege escalation, normally a manual and highly skilled step in penetration testing, can be automated by an LLM that reasons over live command output and adapts its next move. The evidence is a prototype that repeatedly sends a structured JSON prompt containing a system summary, a capped command history, and optional reasoning aids, then parses the LLM's JSON reply, executes the suggested non-interactive command on the target via SSH, and checks the output for a root marker such as uid=0. On the paper's benchmark VM, the model exploited a sudo rule for awk to execute commands as root, and all seven system configurations eventually achieved root; the configurations that combined Chain-of-Thought prompting with human hints succeeded in one or two turns and auto-detected the result, while the remaining three reached root but failed to auto-detect it because the model spawned interactive shells, so the loop ran to the ten-turn cap and root was confirmed manually. The authors interpret this as evidence for practical, scalable, semi-autonomous AI-driven penetration testing, with the caveat that LLM sensitivity to prompt structure and execution context, plus a tendency to repeat failed commands, keeps full autonomy out of reach.

Load-bearing premise

All conclusions about the system's capability depend on the evaluation target being one deliberately vulnerable Linux virtual machine pre-configured with easy privilege-escalation vectors, such as a sudo rule for awk; a hardened or unfamiliar system could produce very different success rates.

Editorial extensions

If this is right

A human-approved LLM loop can perform the privilege-escalation phase of a penetration test on a vulnerable Linux host, reducing the need for a highly skilled operator to type each command.
The cheapest tested setup, Chain-of-Thought plus one human hint, reached root in one turn at a cost of roughly half a thousandth of a dollar, making wide scanning economically plausible.
Deployments that want unattended operation must enforce non-interactive command variants and robust root detection, because configurations without those features reached root but did not auto-detect it, so the loop ran to the ten-turn cap and needed manual confirmation.
Operators can start with the minimal prompt and switch on retrieval-augmented generation and task-tree tracking only when the LLM stalls, since the richer modes did not improve success on this target.
If the result reproduces on other targets, it extends the path from AI-assisted penetration testing to AI-executed post-exploitation while keeping a human-in-the-loop safety checkpoint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the same loop met a hardened system, the cheap Chain-of-Thought-plus-hint configuration would likely lose its advantage; retrieval-augmented generation and task-tree tracking, which added cost without helping here, may become load-bearing when the root path is not a single obvious sudo misconfiguration.
The paper leaves implicit that the interactive-shell blind spot is a monitoring gap rather than merely a parsing detail: an LLM that spawns an interactive shell is doing what a real attacker would do, so full autonomy claims require a session-aware monitor rather than output regexes alone.
A testable extension is to add repetition-detection logic that counts identical failed commands and forces the LLM to propose a new vector, which would directly attack the wasted-turn behaviour observed in the retrieval-augmented, task-tree, and no-flags runs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read it. The architecture is the thing: PenTest2.0 actually runs LLM-suggested commands over SSH, enforces non-interactive variants, checks for root, tracks costs, and toggles CoT/RAG/PTT/hints. That is a real step beyond PentestGPT, which only advises a human. The authors also report failures honestly—interactive shells defeating root detection, command repetition, RAG not helping—and the cost data is a nice touch.

What is new is narrow but genuine: a concrete scaffold for supervised LLM privilege escalation, plus evidence that on one deliberately vulnerable Debian box, gpt-4o-mini can find a sudo-awk vector in a turn or two when prompted well. The feasibility claim, as a proof of concept, holds.

The soft spots are mostly about scope, not execution. The headline 'multi-turn, adaptive' claim is not actually supported. The four auto-root configurations succeed in 1–2 turns on a single known vector. The three configurations that run to the 10-turn limit fail to auto-detect root, and the paper itself says they repeat the same ineffective strategy. So there is no observed success case where the loop turned a failure into an auto-confirmed root by adaptation. That is a fair target for the stress-test note.

The evaluation is also thinner than the 'comprehensive' framing: one target VM, no code or data release, no per-run table, and for configs 5–7 'root achieved' is manual verification, often on turn 1, while auto-detection—the system's own success criterion—never fires. The authors acknowledge the single-target limitation and the need for HITL, which mitigates but does not repair the gap.

Who gets value: anyone working on LLM agents for offensive security will want this as a baseline and a catalog of failure modes. It deserves peer review, but the authors should be asked to reposition the claims, release the prototype and logs, broaden the target set, and report run-level statistics. I'd cite it as an honest feasibility data point and a caution that multi-turn adaptivity needs direct evidence.

Recommendation: engage, with major revision.

Referee Report

5 major / 5 minor

Summary. This paper presents PenTest2.0, a system that extends the authors' earlier PenTest++ by automating the privilege escalation (PrivEsc) phase of penetration testing using large language model (LLM) reasoning. The system operates as a multi-turn loop: it gathers system context via reconnaissance commands, constructs a prompt for the LLM, executes the suggested command over SSH, and repeats until root is detected or a turn limit is reached. Optional modules include retrieval-augmented generation (RAG) from GTFOBins, chain-of-thought (CoT) prompting, PenTest Task Trees (PTTs), and human-injected hints. The paper reports an evaluation on a single deliberately vulnerable Linux VM across seven configurations, claiming all seven achieve root, with four achieving automatic root detection within 1-2 turns. It also reports a cost analysis and discusses observed LLM failure modes.

Significance. The system is a well-engineered proof-of-concept that combines several LLM reasoning techniques with safety controls and cost estimation, and the paper candidly documents the LLM's failure modes (command repetition, interactive shell issues, hallucinated commands). If the claims are confirmed, this is a useful step toward LLM-assisted post-exploitation automation. However, the evaluation is limited to one intentionally vulnerable target and a single sudo-awk exploit vector; the evidence does not substantiate the advertised 'multi-turn, adaptive' capability, since successful auto-detected runs complete in 1-2 turns with a one-liner, while the runs that actually span 10 turns are described as non-adaptive. The paper also does not report the number of runs per configuration consistently, making the feasibility claims difficult to verify.

major comments (5)

[Abstract, §4.2, §7.2, Table 1] The central claim that PenTest2.0 performs 'multi-turn, adaptive privilege escalation' is not supported by the reported data. In Table 1, Configurations 1-4 auto-detect root in 1-2 turns by issuing a single known GTFOBins escape (e.g., 'sudo awk ...'), while Configurations 5-7 run to the 10-turn limit without auto-detection; Section 7.3 describes these runs as exhibiting 'command repetition and non-adaptive output'. No successful run is reported in which the multi-turn loop turned an initial failure into an auto-confirmed root. Present per-turn traces of at least one successful multi-turn run, or restrict the claim to feasibility of LLM-guided PrivEsc.
[§6.2, §7] The evaluation is conducted on a single 'deliberately vulnerable Linux VM pre-configured with PrivEsc vectors'. The paper acknowledges this limits generalisability (Sections 9.3 and 11), but it does not address the consequence: all auto-detected successes use the same sudo awk vector, which the initial reconnaissance prompt exposes via 'sudo -l'. This makes the target a best-case scenario. The authors should either test on additional targets with different vectors or clearly state that the contribution is a demonstration on a toy example, not evidence of general capability.
[§7.2, Table 1] The statement 'All seven configurations successfully achieved root access, confirmed either automatically or manually' conflates manual verification with the system's own success criterion. Configurations 5-7 did not auto-detect root, so by the paper's own 'Auto Root Detected' metric they did not achieve autonomous success. The abstract and conclusions should be revised to say that four of seven configurations achieved auto-detected root, and three only achieved root with manual confirmation.
[§7.3] The number of runs per configuration is not systematically reported. For instance, Configuration 3 is described as 'executed once', Configuration 5 as 'in all 10 runs', and Configuration 7 as 'in all ten runs', but no run counts are given for Configurations 1, 2, 4, and 6. Without run counts, claims such as 'consistently by Turn 2' cannot be verified. Provide a table listing the number of runs, the number of successes, and the distribution of success turns for each configuration.
[§5.4] The example of a 'Turn 7' rationale is presented as evidence of multi-turn correction, but no corresponding full run trace is provided, and it is unclear whether this run belongs to one of the seven configurations in Table 1 or to a preliminary test. If this is meant to demonstrate self-healing, include the full turn-by-turn log for that run.

minor comments (5)

[§3.4] The spelling 'F AISS' contains spurious spaces; it should be 'FAISS'.
[§8.1] The cost tracking uses a word-to-token approximation (1 word ≈ 1.33 tokens) and estimates completion as 40% of prompt size; clarify whether the reported costs are based on these estimates or on actual API usage counters.
[Abstract, §1] The phrase 'driven entirely by Large Language Model reasoning' is contradicted by the optional RAG and human-hint mechanisms described later; qualify this to 'LLM reasoning plus optional external knowledge and human hints'.
[Table 3] Row labels 'CoT + HumanHint' and 'HumanHint' are inconsistent with the flag names used in Table 1 ('--cot --hint', '--hint'); unify the notation.
[§3.3] The example human hint ('Use the id command instead of /bin/sh for root automated verification') is implementation-specific and would not generalise; consider providing a more generic example.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claim is an empirical demonstration against an external target and external knowledge sources; self-citations are background context only.

full rationale

PenTest2.0 makes an empirical, not derivational, claim: that an LLM-driven system can gain root on a deliberately vulnerable Linux VM. The evaluation is anchored to an external target ('a deliberately vulnerable Linux VM pre-configured with PrivEsc vectors', Section 7) and external knowledge sources (GTFOBins, FAISS-indexed content, Section 3.4). The system's own prior work, PenTest++, is presented as a predecessor whose limitation (no privilege-escalation support) motivated the new system; it is not used to justify the success of PenTest2.0. The RAG corpus and the target VM are outside the paper's fitted values, so the observed root access is not equivalent to the input by construction. Human hints are an explicitly tested optional feature, disclosed as such, and the paper even reports configurations without hints succeeding. The CoT examples are stated to be 'curated from prior successful root escalation attempts' but are disclosed as prompt scaffolding rather than hidden fitted parameters, and the no-flags configuration still achieved manual root, so the result does not reduce to the examples. The skeptical concern that 'multi-turn, adaptive' behavior is not fully evidenced by 1-2-turn successes is a legitimate correctness/evidence-sufficiency criticism, not a circularity: the paper reports failures to adapt in RAG, PTT, and no-flags runs, which is consistent with an honest empirical study. Overall, no circular step reduces the central claim to its own inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim of feasibility depends on a small number of design choices (turn limit, cost-estimation factors) and on domain assumptions about the target environment. There are no fitted parameters in the usual sense, and no new entities are postulated.

free parameters (3)

max_turns = 10
The maximum number of LLM turns per session is set to 10 in the configuration file. This cutoff determines whether a run counts as a failure when root is not achieved, so it shapes all reported turn counts and the 'root achieved' outcome.
token_estimate_factor = 1.33 tokens per word
The cost analysis converts word counts to token counts using 1 word approximately 1.33 tokens (Section 8.1), an approximation from OpenAI's tokenizer. It affects cost estimates but not the central security claim.
completion_estimate_ratio = 40% of prompt size
Before each prompt submission, PenTest2.0 estimates the completion size as 40% of the prompt size for cost preview purposes (Section 8.1). This is a hand-set estimate for pre-run cost display.

assumptions (3)

domain assumption The attacker already has a low-privileged foothold on the target (assumed-breach scenario).
The system begins after exploitation has succeeded, per Section 4.2, and this assumption defines the scope of the evaluation.
domain assumption The target VM is configured with known, exploitable privilege escalation vectors (e.g., sudo awk from GTFOBins).
Section 7 states all experiments used 'a deliberately vulnerable Linux VM pre-configured with PrivEsc vectors', which makes successful escalation highly likely for a model that has seen these techniques in training or via RAG.
domain assumption LLM-suggested commands can be screened for safety by a blacklist and human approval before execution.
The system's safety and the experimental integrity depend on the assumption that dangerous commands are filtered and that human review is sufficient, as described in Sections 5.9 and 6.4.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PenTest2.0: Towards Autonomous Privilege Escalation Using GenAI." pith.science (2026). https://pith.science/paper/ZZ7SESBS

@misc{pith2026250706742,
  author       = {Pith},
  title        = {Pith review of: PenTest2.0: Towards Autonomous Privilege Escalation Using GenAI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZZ7SESBS}},
  note         = {Machine review of arXiv:2507.06742}
}

read the original abstract

Ethical hacking today relies on highly skilled practitioners executing complex sequences of commands, which is inherently time-consuming, difficult to scale, and prone to human error. To help mitigate these limitations, we previously introduced 'PenTest++', an AI-augmented system combining automation with generative AI supporting ethical hacking workflows. However, a key limitation of PenTest++ was its lack of support for privilege escalation, a crucial element of ethical hacking. In this paper we present 'PenTest2.0', a substantial evolution of PenTest++ supporting automated privilege escalation driven entirely by Large Language Model reasoning. It also incorporates several significant enhancements: 'Retrieval-Augmented Generation', including both one-line and offline modes; 'Chain-of-Thought' prompting for intermediate reasoning; persistent 'PenTest Task Trees' to track goal progression across turns; and the optional integration of human-authored hints. We describe how it operates, present a proof-of-concept prototype, and discuss its benefits and limitations. We also describe application of the system to a controlled Linux target, showing it can carry out multi-turn, adaptive privilege escalation. We explain the rationale behind its core design choices, and provide comprehensive testing results and cost analysis. Our findings indicate that 'PenTest2.0' represents a meaningful step toward practical, scalable, AI-automated penetration testing, whilst highlighting the shortcomings of generative AI systems, particularly their sensitivity to prompt structure, execution context, and semantic drift, reinforcing the need for further research and refinement in this emerging space. Keywords: AI, Ethical Hacking, Privilege Escalation, GenAI, ChatGPT, LLM (Large Language Model), HITL (Human-in-the-Loop)

Figures

Figures reproduced from arXiv: 2507.06742 by the authors.

**Figure 2.** Raw LLM response [PITH_FULL_IMAGE:figures/full_fig_p036_2.png] view at source ↗

**Figure 3.** Human hinting feature enabled [PITH_FULL_IMAGE:figures/full_fig_p036_3.png] view at source ↗

Figures from the paper (20 more)

**Figure 4.** Figure 4: Example of a human-injected hint embedded in the LLM prompt [PITH_FULL_IMAGE:figures/full_fig_p037_4.png]

**Figure 5.** Figure 5: ‘Root’ achieved with the HumanHint feature enabled [PITH_FULL_IMAGE:figures/full_fig_p037_5.png]

**Figure 6.** Figure 6: The LLM interprets and responds appropriately to a human-injected hint [PITH_FULL_IMAGE:figures/full_fig_p037_6.png]

**Figure 7.** Figure 7: CoT + HumanHint features enabled [PITH_FULL_IMAGE:figures/full_fig_p038_7.png]

**Figure 8.** Figure 8: ‘Root’ achieved and auto-detected in Turn 2 using full configuration; cost [PITH_FULL_IMAGE:figures/full_fig_p038_8.png]

**Figure 9.** Figure 9: LLM response in Turn 2 showing correct reasoning, format compliance, [PITH_FULL_IMAGE:figures/full_fig_p039_9.png]

**Figure 10.** Figure 10: Token cost distribution across failed RAG-enabled runs [PITH_FULL_IMAGE:figures/full_fig_p039_10.png]

**Figure 11.** Figure 11: RAG-enabled Configuration [PITH_FULL_IMAGE:figures/full_fig_p039_11.png]

**Figure 12.** Figure 12: Comparison of repeating vs non-repeating commands during RAG runs [PITH_FULL_IMAGE:figures/full_fig_p040_12.png]

**Figure 13.** Figure 13: The LLM is instructed to return a RAG search query [PITH_FULL_IMAGE:figures/full_fig_p040_13.png]

**Figure 14.** Figure 14: Turn 1 summary showing system context, command history, and sug [PITH_FULL_IMAGE:figures/full_fig_p041_14.png]

**Figure 15.** Figure 15: LLM response with structured output in PTT-mode [PITH_FULL_IMAGE:figures/full_fig_p041_15.png]

**Figure 16.** Figure 16: Token cost summary in PTT mode runs [PITH_FULL_IMAGE:figures/full_fig_p041_16.png]

**Figure 17.** Figure 17: Command repetition vs variation in PTT; high repetition observed [PITH_FULL_IMAGE:figures/full_fig_p042_17.png]

**Figure 18.** Figure 18: Prompt cost preview shown to user for approval before LLM submission [PITH_FULL_IMAGE:figures/full_fig_p042_18.png]

**Figure 19.** Figure 19: Success vs failure of executed commands [PITH_FULL_IMAGE:figures/full_fig_p043_19.png]

**Figure 20.** Figure 20: LLM repeating commands despite clear instructions [PITH_FULL_IMAGE:figures/full_fig_p043_20.png]

**Figure 21.** Figure 21: OpenAI pricing for GPT models as of July 2025 [PITH_FULL_IMAGE:figures/full_fig_p044_21.png]

**Figure 22.** Figure 22: Total cost vs number of turns per configuration [PITH_FULL_IMAGE:figures/full_fig_p045_22.png]

**Figure 23.** Figure 23: Prompt cost preview with no optional flags enabled [PITH_FULL_IMAGE:figures/full_fig_p045_23.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 13 canonical work pages

[1]

In: The 4th international conference on natural language computing, Copen- hagen, Denmark

Abu-Dabaseh, F., Alshammari, E.: Automated penetration testing: An overview. In: The 4th international conference on natural language computing, Copen- hagen, Denmark. pp. 121–129 (2018),https://airccj.org/CSCP/vol8/ csit88610.pdf

work page 2018
[2]

Technical report, Royal Holloway, University of London (2024),https://pure.royalholloway.ac.uk/files/58692091/ TechReport_UnleashingAIinEthicalHacking.pdf

Al-Sinani, H., Mitchell, C.: Unleashing AI in ethical hacking: A prelim- inary experimental study. Technical report, Royal Holloway, University of London (2024),https://pure.royalholloway.ac.uk/files/58692091/ TechReport_UnleashingAIinEthicalHacking.pdf

work page arXiv 2024
[4]

AI-Augmented Ethical Hacking: A Practical Examination of Manual Exploitation and Privilege Escalation in Linux Environments

Al-Sinani, H.S., Mitchell, C.J.: AI-augmented ethical hacking: A practical exam- ination of manual exploitation and privilege escalation in Linux environments. CoRRabs/2411.17539(Nov 2024).https://doi.org/10.48550/arXiv. 2411.17539

work page Pith review arXiv doi:10.48550/arxiv.2411.17539 2024
[5]

CoRRabs/2502.09484(Feb 2025).https://doi.org/10

Al-Sinani, H.S., Mitchell, C.J.: PenTest++: Elevating ethical hacking with AI and automation. CoRRabs/2502.09484(Feb 2025).https://doi.org/10. 48550/ARXIV.2502.09484

arXiv doi:10.48550/arxiv.2502.09484 2025
[6]

In: Martinelli, F., Rios, R

Al-Sinani, H.S., Mitchell, C.J., Sahli, N., Al-Siyabi, M.: Unleashing AI in ethical hacking. In: Martinelli, F., Rios, R. (eds.) Security and Trust Management — 20th International Workshop, STM’24, Bydgoszcz, Poland, September 19–20, 2024, Proceedings. Lecture Notes in Computer Science, vol. 15235, pp. 140–151. Springer (2024),https://doi.org/10.1007/978-...

work page doi:10.1007/978-3-031-76371-7_10 2024
[7]

In: Costa, G., Montanari, R., Carmi- nati, M., Sciarretta, G

Al-Sinani, H.S., Sahli, N., Mitchell, C.J., Al-Siyabi, M.: Advancing ethical hacking with AI: A Linux-based experimental study. In: Costa, G., Montanari, R., Carmi- nati, M., Sciarretta, G. (eds.) Proceedings of the Joint National Conference on Introducing PenTest2.0 33 Cybersecurity (ITASEC & SERICS 2025), February 03–08, 2025, Bologna, Italy. vol. 3962....

work page 2025
[8]

In: AI Magazine

Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: The role of humans in interactive machine learning. In: AI Magazine. vol. 35, pp. 105–120 (2014).https://doi.org/10.1609/aimag.v35i4.2513,https:// ojs.aaai.org/index.php/aimagazine/article/view/2513

doi:10.1609/aimag.v35i4.2513 2014
[9]

In: 33rd USENIX Security Sympo- sium (USENIX Security 24)

Deng, G., Liu, Y., Mayoral-Vilches, V., Liu, P., Li, Y., Xu, Y., Zhang, T., Liu, Y., Pinzger, M., Rass, S.: PentestGPT: Evaluating and harnessing large language models for automated penetration testing. In: 33rd USENIX Security Sympo- sium (USENIX Security 24). pp. 847–864. USENIX Association, Philadelphia, PA (Aug 2024),https://www.usenix.org/conference/...

work page 2024

Show all 19 references

[10]

In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Happe, A., Cito, J.: Getting pwn’d by AI: Penetration testing with large language models. In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. pp. 2082–
[11]

arXiv preprint arXiv:2405.14487 (2024),https://arxiv.org/abs/2405.14487

Hassanin, M., Moustafa, N.: A comprehensive overview of large language mod- els (LLMs) for cyber defences: Opportunities and directions. arXiv preprint arXiv:2405.14487 (2024),https://arxiv.org/abs/2405.14487

2024 arXiv
[12]

Computers & Secu- rity154, 104399 (2025).https://doi.org/https://doi.org/10.1016/j

Lazarov, W., Seda, P., Martinasek, Z., Kummel, R.: Penterep: Comprehensive penetration testing with adaptable interactive checklists. Computers & Secu- rity154, 104399 (2025).https://doi.org/https://doi.org/10.1016/j. cose.2025.104399

2025
[13]

In: Advances in Neural Information Processing Systems

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Kuksa, P., Polosukhin, I., Riedel, S., Kiela, D., et al.: Retrieval- augmented generation for knowledge-intensive nlp tasks. In: Advances in Neural Information Processing Systems. vol. 33, pp. 9459–9474 (...

2020
[14]

Li, Z., Calvo-Bartolom´ e, L., Hoyle, A., Xu, P., Dima, A., Fung, J.F., Boyd-Graber, J.: Large language models struggle to describe the haystack without human help: Human-in-the-loop evaluation of topic models (2025),https://arxiv.org/ abs/2502.14748

2025 arXiv
[15]

benefits and drawbacks

Stefinko, Y., Piskozub, A., Banakh, R.: Manual and automated penetration test- ing. benefits and drawbacks. Modern tendency. In: 13th International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science. pp. 488–491. IEEE (2016).https://doi...

2016 doi
[16]

Special Publica- tion 800-115, National Institute of Standards and Technology (2008),https: //csrc.nist.gov/publications/detail/sp/800-115/final

Swanson, M., Bartol, N., Sabato, J., Hash, J., Graffo, L.: Technical guide to in- formation security testing and assessment (NIST SP 800-115). Special Publica- tion 800-115, National Institute of Standards and Technology (2008),https: //csrc.nist.gov/publications/detail/sp/800...

2008
[17]

In: International Confer- ence on Learning Representations (2022),https://openreview.net/forum? id=gEZrGCozdqR

Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V.: Finetuned language models are zero-shot learners. In: International Confer- ence on Learning Representations (2022),https://openreview.net/forum? id=gEZrGCozdqR

2022
[18]

Al-Sinani & C

Wei, J., Wang, X., Schuurmans, D., Bosma, M., ichter, b., Xia, F., Chi, E., Le, Q.V., Zhou, D.: Chain-of-thought prompting elicits reasoning in 34 H. Al-Sinani & C. Mitchell large language models. In: Koyejo, S., Mohamed, S., Agarwal, A., Bel- grave, D., Cho, K., Oh, A. (eds.)...

2022
[19]

In: 8th International Conference on PST: Privacy, Security and Trust

Xiong, P., Peyton, L.: A model-driven penetration test framework for web appli- cations. In: 8th International Conference on PST: Privacy, Security and Trust. pp. 173–180. IEEE (2010).https://doi.org/10.1109/PST.2010.5593250 Introducing PenTest2.0 35 PenT est2.0 (Prompt Builde...

2010
[2086]

ESEC/FSE, ACM: Association for Computing Machinery, New York, NY, USA (2023).https://doi.org/10.1145/3611643.3613083

2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

In: The 4th international conference on natural language computing, Copen- hagen, Denmark

Abu-Dabaseh, F., Alshammari, E.: Automated penetration testing: An overview. In: The 4th international conference on natural language computing, Copen- hagen, Denmark. pp. 121–129 (2018),https://airccj.org/CSCP/vol8/ csit88610.pdf

work page 2018

[2] [2]

Technical report, Royal Holloway, University of London (2024),https://pure.royalholloway.ac.uk/files/58692091/ TechReport_UnleashingAIinEthicalHacking.pdf

Al-Sinani, H., Mitchell, C.: Unleashing AI in ethical hacking: A prelim- inary experimental study. Technical report, Royal Holloway, University of London (2024),https://pure.royalholloway.ac.uk/files/58692091/ TechReport_UnleashingAIinEthicalHacking.pdf

work page arXiv 2024

[3] [4]

AI-Augmented Ethical Hacking: A Practical Examination of Manual Exploitation and Privilege Escalation in Linux Environments

Al-Sinani, H.S., Mitchell, C.J.: AI-augmented ethical hacking: A practical exam- ination of manual exploitation and privilege escalation in Linux environments. CoRRabs/2411.17539(Nov 2024).https://doi.org/10.48550/arXiv. 2411.17539

work page Pith review arXiv doi:10.48550/arxiv.2411.17539 2024

[4] [5]

CoRRabs/2502.09484(Feb 2025).https://doi.org/10

Al-Sinani, H.S., Mitchell, C.J.: PenTest++: Elevating ethical hacking with AI and automation. CoRRabs/2502.09484(Feb 2025).https://doi.org/10. 48550/ARXIV.2502.09484

arXiv doi:10.48550/arxiv.2502.09484 2025

[5] [6]

In: Martinelli, F., Rios, R

Al-Sinani, H.S., Mitchell, C.J., Sahli, N., Al-Siyabi, M.: Unleashing AI in ethical hacking. In: Martinelli, F., Rios, R. (eds.) Security and Trust Management — 20th International Workshop, STM’24, Bydgoszcz, Poland, September 19–20, 2024, Proceedings. Lecture Notes in Computer Science, vol. 15235, pp. 140–151. Springer (2024),https://doi.org/10.1007/978-...

work page doi:10.1007/978-3-031-76371-7_10 2024

[6] [7]

In: Costa, G., Montanari, R., Carmi- nati, M., Sciarretta, G

Al-Sinani, H.S., Sahli, N., Mitchell, C.J., Al-Siyabi, M.: Advancing ethical hacking with AI: A Linux-based experimental study. In: Costa, G., Montanari, R., Carmi- nati, M., Sciarretta, G. (eds.) Proceedings of the Joint National Conference on Introducing PenTest2.0 33 Cybersecurity (ITASEC & SERICS 2025), February 03–08, 2025, Bologna, Italy. vol. 3962....

work page 2025

[7] [8]

In: AI Magazine

Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: The role of humans in interactive machine learning. In: AI Magazine. vol. 35, pp. 105–120 (2014).https://doi.org/10.1609/aimag.v35i4.2513,https:// ojs.aaai.org/index.php/aimagazine/article/view/2513

doi:10.1609/aimag.v35i4.2513 2014

[8] [9]

In: 33rd USENIX Security Sympo- sium (USENIX Security 24)

Deng, G., Liu, Y., Mayoral-Vilches, V., Liu, P., Li, Y., Xu, Y., Zhang, T., Liu, Y., Pinzger, M., Rass, S.: PentestGPT: Evaluating and harnessing large language models for automated penetration testing. In: 33rd USENIX Security Sympo- sium (USENIX Security 24). pp. 847–864. USENIX Association, Philadelphia, PA (Aug 2024),https://www.usenix.org/conference/...

work page 2024