Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the research community building autonomous offensive-LLM agents systematically acknowledges dual-use risk without implementing or reporting mitigations: recognition in 39% of papers versus concrete mitigation in 7%…

desk verdict The abstract's headline gap is not in the paper; the body is a different, smaller study—reject this version, but the underlying ethics-audit question is worth pursuing. read the letter →

arxiv 2506.08693 v3 pith:UZNRFSES submitted 2025-06-10 cs.CR

Andreas Happe , Jürgen Cito This is my paper
classification cs.CR
keywords EthicsOffensivesecurityLargelanguagemodelsLLMagentsAutonomouspenetrationtestingDual-useriskCoordinateddisclosuremandates
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the research community building autonomous offensive-LLM penetration-testing agents systematically acknowledges dual-use risk without implementing or reporting protective measures. Auditing 54 prototypes from 2023 to 2026, it finds dual-use risk reported as recognized in 39% of papers but a concrete mitigation reported in only 7%, roughly a 5:1 gap. It also finds that 17% of papers are anti-safeguard, reporting the defeat of model safety controls with no countermeasure, and that the near-universal safeguards reported are research-integrity controls protecting the experiment rather than the public. Institutional review appears in only 2% of papers and coordinated disclosure in 6%, both confined to a small subset of the corpus. Measured against the 2025-26 venue ethics mandates, the paper argues, the corpus defines a pre-regulation baseline that current practice does not meet.

What carries the argument

The load-bearing mechanism is the paper's nine-dimension ethics-and-risk scoring instrument, built top-down from a security-research ethics framework and bottom-up from the 2025-26 venue ethics mandates, and applied to a corpus assembled from a pre-registered bibliographic database query plus forward snowballing through a citation graph. Its central analytic move is separating recognition from mitigation: one dimension codes whether a paper acknowledges dual-use risk, another codes whether it reports a concrete protective measure. That split produces the headline ratio, and the instrument's further distinction between research-integrity controls, which protect the experiment, and public-protection controls, such as institutional review and coordinated disclosure, is what converts the ratio into an ethical finding rather than a counting exercise.

What would settle it

A replication that builds the corpus by different means, such as full-text keyword search across preprint and peer-reviewed venues plus direct author surveys of whether a mitigation was implemented, and finds a recognition-to-mitigation ratio far from five to one would settle whether the gap is real. The most direct check is to publish the nine-dimension instrument and per-paper scores; if many papers coded as 'no mitigation' actually contain a concrete protective measure, the headline ratio collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that the literature on autonomous LLM penetration-testing agents exhibits a recognition-without-mitigation gap: authors state that their work is dual-use, but they almost never translate that recognition into concrete protective measures. Across 54 prototypes, 39% report recognizing dual-use risk while only 7% report a concrete mitigation; 17% explicitly report defeating model safety controls without offering a countermeasure. The safeguards that are reported are mostly research-integrity controls, such as sandboxes, monitoring, and artifact hygiene, which protect the researchers' own experiment rather than third parties. Institutional review board involvement and coordinated disclosure are almost absent, and measured against the 2025-26 ethics-section mandates of top security venues, the corpus defines a pre-regulation baseline: current practice does not satisfy the substantive requirements those venues now demand.

Load-bearing premise

The numbers stand on two assumptions: the bibliographic query plus forward snowballing captures essentially the whole population of autonomous offensive-LLM agent papers, and the nine-dimension scoring instrument, which the body of the paper does not define, validly operationalizes ethical responsibility; if either fails, the 39%-vs-7% ratio could be a selection or measurement artifact rather than a property of the literature.

Editorial extensions

If this is right

  • If the audit is correct, current autonomous offensive-LLM research does not meet the substantive ethics requirements that top security venues began mandating in 2025-26, so future papers will need to report concrete mitigations rather than merely include ethics statements.
  • The 17% share of anti-safeguard papers means a nontrivial fraction of the literature demonstrates how to bypass model safety controls without accompanying countermeasures, a risk profile venues may want to screen for explicitly.
  • The near-absence of coordinated disclosure and institutional review implies the field has not imported standard vulnerability-disclosure practice into agent research.
  • Because recognition currently outpaces mitigation by roughly five to one, the paper positions its containment checklist as a low-cost way for future work to close the gap, and the audit itself as defensive intelligence on the offensive-agent ecosystem.
  • The 5:1 ratio supplies a quantitative pre-regulation baseline against which later compliance with the new venue ethics mandates can be measured.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: a large part of the gap is likely a reporting incentive problem, since venues reward an ethics section but rarely require evidence of mitigation; requiring a mitigation paragraph tied to the specific experiment could close much of the 5:1 gap without changing research practice.
  • My inference: the paper's pre-regulation baseline is directly testable; a 2026-2028 replication of the same corpus protocol should show the recognition-without-mitigation ratio shrinking if the venue mandates are being enforced.
  • My inference: the audit method could be extended to adjacent literatures, such as autonomous social-engineering agents or LLM-driven malware generation, where the same recognition-without-mitigation pattern may hold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is presented as a systematic audit of ethics-and-risk reporting in the literature on autonomous LLM-based offensive-security agents. The abstract claims a pre-registered Scopus query (Channel A, n=35) plus a Semantic Scholar forward-snowball (Channel B, n=19), yielding 54 prototypes scored on a nine-dimension instrument derived from the Menlo Report and 2025-26 venue ethics mandates, with a headline 'recognition-without-mitigation' gap: dual-use risk recognized in 39% of papers but a concrete mitigation reported in only 7%. The full text, however, reports a different study: a thematic analysis of 15 prototypes collected via Google Scholar forward/backward referencing from wintermute and pentestGPT, with different statistics (86.6% mention ethical considerations, 53% describe sandboxed environments, 66.6% release artifacts) and no trace of the Scopus query, the Semantic Scholar citation graph, the nine-dimension instrument, the Menlo Report, or the venue mandates. The body and abstract are therefore describing different studies, and the central quantitative claim is not derivable from the submitted manuscript.

Significance. If a rigorous audit of ethics reporting in offensive-LLM-agent research existed, it would be timely and important: the 2025-26 venue mandates cited in the abstract make the topic practically relevant, and a 5:1 recognition-to-mitigation gap would be a useful, falsifiable finding for the community. The full text also contains a few thoughtful qualitative observations, such as the discussion of the inconsistency between stated capability and ethical claims in Singer et al. (Section 5.1) and the analysis of artifact-disclosure practices in Section 5.5. However, as submitted, the paper's headline result exists only in the abstract; the body is a smaller, differently designed thematic analysis. For that reason the manuscript cannot currently be evaluated as supporting its central claim, and its significance is undermined by the mismatch between the claimed and reported studies.

major comments (3)
  1. [Abstract vs. Section 3 and Table 1] The abstract describes a pre-registered Scopus query, a Semantic Scholar forward snowball over two seed papers, a corpus of 54 prototypes, and a nine-dimension coding instrument derived from the Menlo Report and 2025-26 venue mandates, with the central 39%-vs-7% recognition-without-mitigation gap. The full text describes a thematic analysis of 15 prototypes assembled via Google Scholar forward/backward referencing from wintermute and pentestGPT; it includes no Scopus query, no pre-registration, no Semantic Scholar citation graph, no nine-dimension instrument, and no mention of the Menlo Report or venue mandates. The body's reported numbers (86.6% ethical mention, 53% sandboxing, 66.6% artifact release) cannot produce the abstract's 39%/7% ratio. The headline claim is therefore stated only in the abstract and is not supported by the submitted artifact.
  2. [Table 1 and Section 4] Even setting aside the corpus-size mismatch, Table 1 contains no columns for 'risk recognized' or 'concrete mitigation,' and Section 4 reports no quantities from which a 39%/7% split could be reconstructed. The variables actually reported are availability, motivation, presence of an ethics statement, and the presence of mitigations such as sandboxing, remediation, monitoring, or regulation. The body's analysis is thematically interesting but it is not the audit promised in the abstract, and the central ratio is not verifiable from the presented evidence.
  3. [Section 3 and Table 1 (corpus construction)] Three of the 15 reviewed prototypes are papers authored by the present auditors (Happe and Cito [11] and [12]; Happe et al. [13]), and the corpus was seeded from one of those papers (wintermute [11]). The manuscript nowhere discloses this overlap or discusses how it might influence the qualitative conclusions about the field's ethics culture. At minimum, the authorship overlap should be reported and the analysis should indicate whether the findings are robust to excluding the auditors' own papers.
minor comments (5)
  1. [Section 3] The phrase 'with bothwintermute[ 11] andpentestGPT[ 5]' is missing spacing; it should read 'with wintermute [11] and pentestGPT [5].'
  2. [Table 1] The row labeled 'Autonomously Exploit One-day Vulns. [9]' cites reference [9], which in the reference list is the zero-day paper; the one-day vulnerability paper is reference [7]. Please correct the citation.
  3. [Section 5.1] The text refers to 'Muszai et al. [28]', but the corresponding reference is 'Muzsai et al.'; the spelling should be corrected.
  4. [Section 2.2] The quotation formatting in 'We fully agree with the analysis by Zhang et al. [44], stating: For instance, Silic [36] surveys...' is ambiguous: it is unclear whether the quoted passage is from Zhang et al. or is an unattributed secondary quotation from Silic. Please clarify the source and formatting.
  5. [Entire manuscript] The abstract's numbers (54 prototypes, 39%, 7%, 17%, 2%, 6%) appear nowhere in the body. If this submission is intended as a revised version, the abstract and full text must be synchronized; as submitted, they describe two different studies.

Circularity Check

2 steps flagged · score 4.0 of 10

The abstract's 5:1 recognition-without-mitigation gap is unverifiable from the full-text study, and the corpus is partly self-seeded from the authors' own wintermute paper; the body's thematic claims are not definitionally circular.

  1. other [Abstract (headline result) vs. Section 3 Methodology and Section 4 Results]
    "Our central result is a recognition-without-mitigation gap: dual-use risk is reported as recognized in 39% of papers but a concrete mitigation is reported in only 7%, roughly a 5:1 gap. ... 13 of the 15 reviewed prototypes (86.6%) contained a mention of ethical considerations."

    The abstract describes a pre-registered Scopus query (n=35), a Semantic Scholar snowball (n=19), and a nine-dimension Menlo/venue-mandate instrument that do not appear in the body. The body reports a 15-prototype thematic analysis whose Table 1 has no columns for 'risk recognized' or 'concrete mitigation,' so the 39%/7% ratio cannot be recomputed or checked from the presented artifact. The headline result is therefore asserted rather than derived in the submitted text; the derivation chain breaks at the abstract/body boundary. This is a missing-evidence inconsistency rather than a definitional tautology, but it makes the claimed central result unfalsifiable within the paper.

  2. self citation load bearing [Section 3 Methodology, corpus construction]
    "Using Google Scholar, we performed forward-referencing by adding papers that cited either pentestGPT or wintermute and fit our initial selection criteria."

    One of the two forward-snowball seeds, wintermute [11], is the present authors' own paper, and three of the fifteen reviewed prototypes [11,12,13] are authored by the auditors. Because the sample is partly generated from the authors' own citation neighborhood, the measured culture of ethics awareness is not fully independent of the measuring authors. The connection 'paper cites wintermute' does not by itself imply 'paper mentions ethics,' so this is a sampling bias rather than a logical reduction; however, it is a self-referential load on the corpus that underlies every percentage in the paper.

full rationale

The full-text thematic analysis is an empirical coding exercise and does not contain equations or fitted parameters, so the body's own 86.6% awareness figure is not circular: it is a summary of the authors' coding of the 15-paper corpus. The circularity concern is concentrated in two places. First, the abstract's headline 39%-vs-7% result depends on a 54-paper corpus and a nine-dimension instrument that are absent from the body, so the claimed result cannot be independently recomputed; this is best characterized as missing evidence rather than a definitional circle. Second, corpus construction is partly self-seeded from the authors' own wintermute paper and includes three of their own prototypes, which weakens the independence of the measurement. These issues justify a moderate score, but nothing in the submitted text reduces a prediction to its input by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The audit relies on the representativeness of its corpus and the validity of its coding instrument. No free parameters or invented entities are introduced, but the measurement assumptions are load-bearing for the central gap claim.

assumptions (4)
  • domain assumption The Scopus query plus forward snowball captures the population of autonomous offensive-LLM agent papers.
    In the abstract's Channel A/B description; if the corpus is incomplete or biased, the gap ratio is not generalizable.
  • domain assumption The nine-dimension instrument is a valid and complete operationalization of ethical responsibility grounded in the Menlo Report and venue mandates.
    Stated in the abstract but the instrument is not defined in the full text, so its validity cannot be checked.
  • domain assumption Authors' ethics sections and statements accurately reflect their actual practices and intentions.
    Used implicitly throughout the coding; a paper may omit or embellish mitigations in prose.
  • domain assumption The two coders' thematic labels are reliable.
    Methodology notes dual coding and discussion, but no inter-rater reliability statistic is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research." pith.science (2026). https://pith.science/paper/UZNRFSES

@misc{pith2026250608693,
  author       = {Pith},
  title        = {Pith review of: Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UZNRFSES}},
  note         = {Machine review of arXiv:2506.08693}
}
read the original abstract

Large language models have moved from advising on offensive security to autonomously conducting it. A growing literature presents agents that execute reconnaissance, exploitation, and privilege escalation against real or simulated targets. Such an agent is a deployable, re-pointable capability whose harm potential scales with the underlying model. The papers that introduce it therefore carry an unusual ethical burden, which top security venues have begun to encode as hard policy in 2025-2026 ethics-section mandates. We present a systematic, reproducible audit of ethics-and-risk reporting in this literature. From a pre-registered Scopus query (Channel A, n=35) plus a reproducible forward-snowball of two seed papers via the Semantic Scholar citation graph (Channel B, n=19, all Scopus-absent) we assemble 54 autonomous offensive-LLM penetration-testing prototypes (2023-2026). We score each against a nine-dimension instrument derived both top-down from the Menlo Report, and bottom-up from the 2025-26 venue mandates. Our central result is a recognition-without-mitigation gap: dual-use risk is reported as recognized in 39% of papers but a concrete mitigation is reported in only 7%, roughly a 5:1 gap. Of the papers, 17% are anti-safeguard, reporting the defeat of model safety controls with no countermeasure. The near-universal safeguards reported are research-integrity controls that protect the experiment, not the public; institutional-review (2%) and coordinated-disclosure (6%) practice is almost absent and confined to Channel B. Measured against the new mandates, the corpus defines a pre-regulation baseline: current practice does not meet the substantive requirements. We argue this audit is itself defensive intelligence on the offensive-agent ecosystem, and we distill a minimal containment checklist for future work.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Ethics of Autonomous AI Agents for Offensive Security

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Autonomous AI hacking tools combine three kinds of indeterminacy—action, impact, and users—making moral responsibility diffuse and giving attackers a short-term advantage under current cost asymmetries.

  2. On the Surprising Efficacy of LLMs for Penetration-Testing

    cs.CR 2025-07 conditional novelty 3.0 of 10

    A critical review arguing that LLMs are surprisingly effective for penetration testing because the task is largely pattern-matching, while noting serious reliability, safety, and cost barriers to autonomous use.

Reference graph

Works this paper leans on

45 extracted references · 32 canonical work pages · cited by 2 Pith papers

  1. [11]

    Getting pwn’d by ai: Penetration testing with large language models

    Andreas Happe and Jürgen Cito. Getting pwn’d by ai: Penetration testing with large language models. InProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 2082–2086, 2023

    work page 2023

  2. [12]

    Can llms hack enterprise networks? autonomous assumed breach penetration-testing active directory networks.arXiv preprint arXiv:2502.04227, 2025

    Andreas Happe and Jürgen Cito. Can llms hack enterprise networks? autonomous assumed breach penetration-testing active directory networks.arXiv preprint arXiv:2502.04227, 2025

    arXiv 2025

  3. [13]

    Llms as hackers: Autonomous linux privilege escalation attacks.arXiv preprint arXiv:2310.11409, 2024

    Andreas Happe, Aaron Kaplan, and Juergen Cito. Llms as hackers: Autonomous linux privilege escalation attacks.arXiv preprint arXiv:2310.11409, 2024

    arXiv 2024

  4. [1]

    https://en.wikipedia.org/wiki/ Coordinated_vulnerability_disclosure

    Coordinated vulnerability disclosure. https://en.wikipedia.org/wiki/ Coordinated_vulnerability_disclosure. Accessed: 2025-02-19

    work page 2025

  5. [2]

    https://c3.unu.edu/blog/large-language-models-in-vulnerability-research- opportunities-and-responsibilities

    Large language models in vulnerability research: Opportunities and responsibili- ties. https://c3.unu.edu/blog/large-language-models-in-vulnerability-research- opportunities-and-responsibilities. Accessed: 2025-02-20

    work page 2025

  6. [3]

    Llms for intelli- gent software testing: A comparative study

    Mohamed Boukhlif, Nassim Kharmoum, and Mohamed Hanine. Llms for intelli- gent software testing: A comparative study. InProceedings of the 7th International Conference on Networking, Intelligent Systems and Security, NISS ’24, New York, NY, USA, 2024. Association for Computing Machinery

    work page 2024

  7. [4]

    Using thematic analysis in psychology

    Virginia Braun and Victoria Clarke. Using thematic analysis in psychology. Qualitative research in psychology, 3(2):77–101, 2006

    2006

  8. [5]

    {PentestGPT}: Evaluating and harnessing large language models for automated penetration testing

    Gelei Deng, Yi Liu, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. {PentestGPT}: Evaluating and harnessing large language models for automated penetration testing. In33rd USENIX Security Symposium (USENIX Security 24), pages 847–864, 2024

    2024

Show all 45 references
  1. [6]

    Large language models in information security research: A january 2024 survey.ResearchGate preprint RG, 2(20107.26404), 2024

    Rohit Dube. Large language models in information security research: A january 2024 survey.ResearchGate preprint RG, 2(20107.26404), 2024

    2024

  2. [7]

    Llm agents can autonomously exploit one-day vulnerabilities, 2024

    Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. Llm agents can autonomously exploit one-day vulnerabilities, 2024

    2024

  3. [8]

    Llm agents can autonomously hack websites, 2024

    Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. Llm agents can autonomously hack websites, 2024

    2024

  4. [9]

    Teams of llm agents can exploit zero-day vulnerabilities, 2024

    Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. Teams of llm agents can exploit zero-day vulnerabilities, 2024

    2024

  5. [10]

    Autopenbench: Benchmarking generative agents for penetration testing, 2024

    Luca Gioacchini, Marco Mellia, Idilio Drago, Alexander Delsanto, Giuseppe Siracusano, and Roberto Bifulco. Autopenbench: Benchmarking generative agents for penetration testing, 2024

    2024

  6. [14]

    A comprehensive overview of large language models (llms) for cyber defences: Opportunities and directions, 2024

    Mohammed Hassanin and Nour Moustafa. A comprehensive overview of large language models (llms) for cyber defences: Opportunities and directions, 2024. Ethics Statements in Autonomous Penetration-Testing Agent Research

    2024

  7. [15]

    Penheal: a two-stage llm framework for auto- mated pentesting and optimal remediation

    Junjie Huang and Quanyan Zhu. Penheal: a two-stage llm framework for auto- mated pentesting and optimal remediation. InProceedings of the Workshop on Autonomous Cybersecurity, pages 11–22, 2023

    2023

  8. [16]

    Disclosure policy

    Zero Day Initiative. Disclosure policy. https://www.zerodayinitiative.com/ advisories/disclosure_policy/. Accessed: 2025-02-19

    2025

  9. [17]

    Towards auto- mated penetration testing: Introducing llm benchmark, analysis, and improve- ments.arXiv preprint arXiv:2410.17141, 2024

    Isamu Isozaki, Manil Shrestha, Rick Console, and Edward Kim. Towards auto- mated penetration testing: Introducing llm benchmark, analysis, and improve- ments.arXiv preprint arXiv:2410.17141, 2024

    2024 arXiv

  10. [18]

    A survey on large language models for code generation.arXiv preprint arXiv:2406.00515, 2024

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation.arXiv preprint arXiv:2406.00515, 2024

    2024 arXiv

  11. [19]

    From llms to llm-based agents for software engineering: A survey of current, challenges and future, 2024

    Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, and Huaming Chen. From llms to llm-based agents for software engineering: A survey of current, challenges and future, 2024

    2024

  12. [20]

    A capture the flag (ctf) platform and exercises for an intro to computer security class

    Zack Kaplan, Ning Zhang, and Stephen V Cole. A capture the flag (ctf) platform and exercises for an intro to computer security class. InProceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 2, pages 597–598, 2022

    2022

  13. [21]

    An analysis and evaluation of open source capture the flag platforms as cy- bersecurity e-learning tools

    Stylianos Karagiannis, Elpidoforos Maragkos-Belmpas, and Emmanouil Magkos. An analysis and evaluation of open source capture the flag platforms as cy- bersecurity e-learning tools. InIFIP World Conference on Information Security Education, pages 61–77. Springer, 2020

    2020

  14. [22]

    Large language models versus natural language un- derstanding and generation

    Nikitas Karanikolas, Eirini Manga, Nikoletta Samaridi, Eleni Tousidou, and Michael Vassilakopoulos. Large language models versus natural language un- derstanding and generation. InProceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics, pages 2...

    2023

  15. [23]

    From vulnerability to defense: The role of large language models in enhancing cybersecurity.Computation, 13(2):30, 2025

    Wafaa Kasri, Yassine Himeur, Hamzah Ali Alkhazaleh, Saed Tarapiah, Shadi Atalla, Wathiq Mansoor, and Hussain Al-Ahmad. From vulnerability to defense: The role of large language models in enhancing cybersecurity.Computation, 13(2):30, 2025

    2025

  16. [24]

    Vulnbot: Autonomous penetration testing for a multi-agent collaborative framework

    He Kong, Die Hu, Jingguo Ge, Liangxiong Li, Tong Li, and Bingzhen Wu. Vulnbot: Autonomous penetration testing for a multi-agent collaborative framework. arXiv preprint arXiv:2501.13411, 2025

    2025 arXiv

  17. [25]

    When llm-based code generation meets the software development process.arXiv preprint arXiv:2403.15852, 2024

    Feng Lin, Dong Jae Kim, et al. When llm-based code generation meets the software development process.arXiv preprint arXiv:2403.15852, 2024

    2024 arXiv

  18. [26]

    Mavikumbure, Victor Cobilean, Chathurika S

    Harindra S. Mavikumbure, Victor Cobilean, Chathurika S. Wickramasinghe, Devin Drake, and Milos Manic. Generative ai in cyber security of cyber physical systems: Benefits and threats. In2024 16th International Conference on Human System Interaction (HSI), pages 1–8, 2024

    2024

  19. [27]

    Large language models in cybersecurity: State-of-the-art, 2024

    Farzad Nourmohammadzadeh Motlagh, Mehrdad Hajizadeh, Mehryar Majd, Pejman Najafi, Feng Cheng, and Christoph Meinel. Large language models in cybersecurity: State-of-the-art, 2024

    2024

  20. [28]

    Hacksynth: Llm agent and evaluation framework for autonomous penetration testing, 2024

    Lajos Muzsai, David Imolai, and András Lukács. Hacksynth: Llm agent and evaluation framework for autonomous penetration testing, 2024

    2024

  21. [29]

    National Academies Press, 2019

    National Academies of Sciences, Policy, Global Affairs, Board on Research Data, Information, Division on Engineering, Physical Sciences, Committee on Applied, Theoretical Statistics, Board on Mathematical Sciences, et al.Reproducibility and replicability in science. National A...

    2019

  22. [30]

    The sword and the shield: Hacking tools as offensive weapons and defensive tools.Geo

    Tiffany S Rad. The sword and the shield: Hacking tools as offensive weapons and defensive tools.Geo. J. Int’l Aff., 16:123, 2015

    2015

  23. [31]

    Reproducibility and research integrity

    David B Resnik and Adil E Shamoo. Reproducibility and research integrity. Accountability in research, 24(2):116–123, 2017

    2017

  24. [32]

    Real world research, 2002

    Collin Robson. Real world research, 2002

    2002

  25. [33]

    Schneier: Full disclosure of security vulnerabilities a ’damned good idea’

    Bruce Schneier. Schneier: Full disclosure of security vulnerabilities a ’damned good idea’. https://www.schneier.com/essays/archives/2007/01/schneier_full_ disclo.html, 2007. Accessed: 2025-02-19

    2007

  26. [34]

    An empirical evaluation of llms for solving offensive security challenges, 2024

    Minghao Shao, Boyuan Chen, Sofija Jancheska, Brendan Dolan-Gavitt, Siddharth Garg, Ramesh Karri, and Muhammad Shafique. An empirical evaluation of llms for solving offensive security challenges, 2024

    2024

  27. [35]

    Nyu ctf dataset: A scalable open-source benchmark dataset for evaluating llms in offensive security, 2024

    Minghao Shao, Sofija Jancheska, Meet Udeshi, Brendan Dolan-Gavitt, Haoran Xi, Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krish- namurthy, Farshad Khorrami, Ramesh Karri, and Muhammad Shafique. Nyu ctf dataset: A scalable open-source benchmark dataset for ...

    2024

  28. [36]

    Dual-use open source security software in organizations–dilemma: help or hinder?Computers & Security, 39:386–395, 2013

    Mario Silic. Dual-use open source security software in organizations–dilemma: help or hinder?Computers & Security, 39:386–395, 2013

    2013

  29. [37]

    On the feasibility of using llms to execute multistage network attacks

    Brian Singer, Keane Lucas, Lakshmi Adiga, Meghna Jain, Lujo Bauer, and Vyas Sekar. On the feasibility of using llms to execute multistage network attacks. arXiv preprint arXiv:2501.16466, 2025

    2025

  30. [38]

    Benefits and pitfalls of using capture the flag games in university courses

    Jan Vykopal, Valdemar Švábensk`y, and Ee-Chien Chang. Benefits and pitfalls of using capture the flag games in university courses. InProceedings of the 51st ACM Technical symposium on computer science education, pages 752–758, 2020

    2020

  31. [39]

    Large language models for cyber security: A systematic literature review, 2024

    Hanxiang Xu, Shenao Wang, Ningke Li, Kailong Wang, Yanjie Zhao, Kai Chen, Ting Yu, Yang Liu, and Haoyu Wang. Large language models for cyber security: A systematic literature review, 2024

    2024

  32. [40]

    Autoattacker: A large language model guided system to implement automatic cyber-attacks.arXiv preprint arXiv:2403.01038, 2024

    Jiacen Xu, Jack W Stokes, Geoff McDonald, Xuesong Bai, David Marshall, Siyue Wang, Adith Swaminathan, and Zhou Li. Autoattacker: A large language model guided system to implement automatic cyber-attacks.arXiv preprint arXiv:2403.01038, 2024

    2024 arXiv

  33. [41]

    A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.High-Confidence Computing, 4(2):100211, 2024

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.High-Confidence Computing, 4(2):100211, 2024

    2024

  34. [42]

    Review of generative ai methods in cybersecurity, 2024

    Yagmur Yigit, William J Buchanan, Madjid G Tehrani, and Leandros Maglaras. Review of generative ai methods in cybersecurity, 2024

    2024

  35. [43]

    Vulnerability disclosure policy

    Google Project Zero. Vulnerability disclosure policy. https://googleprojectzero. blogspot.com/p/vulnerability-disclosure-policy.html. Accessed: 2025-02-19

    2025

  36. [44]

    Cybench: A framework for evaluating cybersecurity capabilities and risks of language models

    Andy K Zhang, Neil Perry, Riya Dulepet, Joey Ji, Justin W Lin, Eliot Jones, Celeste Menders, Gashon Hussein, Samantha Liu, Donovan Jasper, et al. Cybench: A framework for evaluating cybersecurity capabilities and risks of language models. arXiv preprint arXiv:2408.08926, 2024

    2024 arXiv

  37. [45]

    When llms meet cybersecurity: A systematic literature review, 2024

    Jie Zhang, Haoyu Bu, Hui Wen, Yu Chen, Lun Li, and Hongsong Zhu. When llms meet cybersecurity: A systematic literature review, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.