SoK: On the Offensive Potential of AI

Ali Sunyaev; Ashley Shen; Aurore Fass; Ben Nassi; Edward W. N. Bernroider; Fabio Roli; Gang Wang; Giovanni Apruzzese; Hyrum S. Anderson; Isabel Wagner

REVIEW 3 major objections 4 minor 1 cited by

SoK: On the Offensive Potential of AI

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Offensive AI spans systems, humans, and society; this paper gives the first cross-source systematization—95 papers, 38 briefings, a 549-person survey, and 12 expert statements.

desk verdict A genuinely broader SoK on offensive AI that mostly earns its 'holistic' claim, except the 'laypeople' survey turns out to be a convenience sample of tech-literate Westerners. read the letter →

arxiv 2412.18442 v4 pith:GL5S7XQX submitted 2024-12-24 cs.CR cs.AIcs.CYcs.LG

Saskia Laura Schröer , Giovanni Apruzzese , Soheil Human , Pavel Laskov , Hyrum S. Anderson , Edward W. N. Bernroider , Aurore Fass , Ben Nassi

show 8 more authors

Vera Rimmer Fabio Roli Samer Salam Ashley Shen Ali Sunyaev Tim Wadhwa-Brown Isabel Wagner Gang Wang

This is my paper · ORCID

classification cs.CRcs.AIcs.CYcs.LG

keywords offensiveAIAI-enabledcyberattackssystematizationofknowledgesecurityandprivacythreatsusersurveyexpertelicitationcost-benefitanalysishuman-centric

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that no existing work has drawn a holistic picture of the offensive use of artificial intelligence, and that such a picture can be built by pooling four sources of knowledge: academic papers, industrial security briefings, the opinions of laypeople, and the views of experts. The authors analyse 95 research papers and 38 briefings with a common checklist, survey 549 non-experts, and collect statements from 12 experts, which they condense into ten open problems. The central claim is that offensive AI targets systems, humans, and society in ways that a single attack-tactic taxonomy cannot capture, so a reusable assessment checklist is needed. If the paper is right, future work on offensive AI gains a shared vocabulary and a way to compare attacks, costs, and countermeasures across otherwise incommensurable sources.

What carries the argument

The OAI Assessment Checklist, a reusable set of criteria built around three questions: What is the use case? What is the target? What is the cost/benefit? It maps each work to a standardized attack taxonomy as its first step, adds an original target dimension distinguishing humans, systems, and society (and real versus toy systems), and records whether benefits, costs, and non-AI baselines are quantified. The checklist is the alignment device that lets the authors compare a peer-reviewed paper, an industrial briefing, a survey answer, and an expert statement on the same footing.

What would settle it

Run the same four-question survey on a probability sample of the general population; if the share concerned about offensive AI falls well below the paper's 84% or the open-ended answers no longer centre on misinformation and deepfakes, the survey pillar's general claims about laypeople would be contradicted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that the offensive potential of AI is heterogeneous across three target classes—systems, humans, and society—and that each knowledge source reveals a different slice of that heterogeneity. Academic technical papers mostly propose novel attacks against toy systems and neglect attacker costs, while industrial security briefings demonstrate attacks against real systems and humans but rarely discuss cost. Non-technical academic work emphasises warfare and society, laypeople are broadly concerned yet hold some concerns orthogonal to actual offensive use, and experts, after reading the draft, shifted their stated priorities toward privacy and cost. From these observations the paper derives a three-question OAI Assessment Checklist—use case, target, cost/benefit—and uses it to show that existing mappings miss privacy attacks, attacks on society, and autonomous agents. The paper further claims to be the first systematization of offensive AI to combine all four knowledge sources and to report expert opinions verbatim.

Load-bearing premise

The load-bearing premise is that the 549 convenience-sampled respondents, mostly from Europe and North America, largely degree-holding and IT-employed, stand in for 'laypeople' closely enough that the survey's concern levels and themes can support the paper's general lessons about public perception.

Editorial extensions

If this is right

Any future work on offensive AI can be classified with the checklist, making the snapshot extendable rather than frozen.
Researchers reporting novel AI attacks should expect to evaluate countermeasures; only about half of the surveyed attack papers do, and few offensive-security tools warn about malicious abuse.
Attacker cost/benefit and non-AI baselines should become standard reporting items, because the surveyed literature mostly omits them and real-world risk is therefore hard to judge.
Attacks targeting humans are under-represented in academic literature yet dominate laypeople's concerns, marking a concrete research gap.
Taxonomies built only on attack tactics miss important offensive AI, so society-level and privacy use cases need separate tracking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the checklist is adopted as a community standard, the same three questions could be asked of future AI security incidents, turning the curated archive into an early-warning signal for new offensive uses.
The survey's demographic skew means the 84% concern figure best describes a highly educated, IT-adjacent population; a representative sample could plausibly yield different levels and themes of concern.
The observed shift in expert priorities toward privacy and cost after reading the draft suggests that systematic reviews can change expert agendas; a controlled pre/post study with a placebo document could test this directly.
The human-targeting attacks the paper finds overlooked—attribute inference and profile matching—may become more central as generative models make personal data easier to exploit; monitoring that gap is a concrete follow-up.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague —

The headline: this SoK delivers more than the usual survey. The genuinely new pieces are the 38 InfoSec briefings systematized with the same checklist as the academic papers, and the 12 expert statements reported verbatim in the appendix. Those two sources reveal real blind spots in prior surveys — attacks on society show up far more in briefings than in academic literature, and experts visibly shift their priorities after reading the paper. The checklist is a practical tool for future classification, and the methodology (dual review, adjudication, grounded-theory coding) is transparent enough to reproduce.

The soft spot is the survey pillar. The paper calls the 549 respondents 'laypeople,' but the sample is 75% IT-affiliated, 80% tertiary-educated, mostly European and North American, only 2% over 65, and 63% male. The 'over 80% are concerned' result and the lessons in §VII-A are descriptive of a tech-literate convenience sample, not the general public. The paper does acknowledge this in §V-A and §VII-B, and the correlation analysis showing IT employment does not predict concern is a fair attempt at mitigation — but that null is computed inside a sample that is 75% IT. The survey is not fatal: the core systematization of 133 works stands on its own, and the survey is one of four knowledge sources. But the 'holistic picture' claim is weakened if the human-perception pillar is this skewed. Post-stratification or a relabeling ('tech-literate public' rather than 'laypeople') would largely fix it.

The expert elicitation has a mild circularity — open problems were written after reading the draft — but the paper is upfront about it and explicitly checks for influence (privacy, cost). That is acceptable for a research agenda. Search limitations are admitted. These are minor issues.

Bottom line: this deserves serious refereeing and a cite. The survey needs caveats, not a rebuild. I would bring it to a reading group to discuss the multi-source method — and the sample problem is a good cautionary tale for anyone planning a similar design.

Referee Report

3 major / 4 minor

Summary. The paper presents a systematization of offensive AI (OAI) that combines four knowledge sources: 95 academic papers, 38 InfoSec briefings from BlackHat/DefCon, a 549-person survey, and statements from 12 experts. The authors introduce an OAI Assessment Checklist, an online tool for applying it, and derive ten open problems from the expert statements. The central claim is that this is the first work to draw a holistic picture of OAI by aligning academic, industrial, lay, and expert perspectives under a common set of criteria.

Significance. If the validity concerns below are resolved, this would be a valuable contribution: the methodology is transparent (published search queries, dual reviewing with adjudication, grounded theory coding), the curated archive of 133 works is reusable, the online checklist tool lowers the barrier for future systematization, and the multi-source design is genuinely novel among SoK papers. The checklist and its operationalization are falsifiable and can be applied by other researchers. The paper also explicitly documents limitations and provides code for the NLP analysis. These strengths are real and should be credited. However, the two pillars that most distinguish this SoK from prior work—the laypeople survey and the expert opinions—currently have significant validity concerns that affect the central 'holistic picture' claim.

major comments (3)

[§II-D and author list (page 1)] The 12 experts whose opinions anchor Section VI and Contribution C3 are, according to the author list, co-authors of this paper (the 12 names after the four corresponding authors). Nowhere in §II-D or elsewhere is this disclosed; the text says 'we reached out to 12 experts' and describes them as external sources. This compromises the independence of the expert pillar, and the pre/post comparison in §VI-C is especially affected because the 'post' statements were written by people who had already seen and contributed to the draft and its framework. The authors must either explicitly disclose that the experts are co-authors and re-characterize the findings as an internal author elicitation, or recruit independent external experts. Without this, the claim of consolidating 'expert opinions' as a separate knowledge source is not substantiated.
[§V-A and Appendix B, Table V] The survey sample is described in the abstract and Section V as representing 'laypeople' with 'diverse backgrounds and expertise,' but the demographic data show 70% European, 21% North American, 80% with at least a Bachelor's degree, 75% in IT-related work, and 77% self-rating at least intermediate cybersecurity knowledge. The paper acknowledges in §VII-B that it 'cannot claim representativeness,' yet the headline '84% of respondents are concerned' and the lessons learned in §V-B2 and §VII-A are presented without this caveat. The non-IT subgroup (n≈137) is too small and not randomly selected to support generalizable claims about the general public. The abstract and the survey sections should be reframed to describe this as a convenience sample of a tech-literate, educated, Western-centric population, and ideally supplemented with post-stratified estimates or a sensitivity analysis.
[§VI-C] The comparison of expert opinions before and after reading the draft is interpreted as evidence that the paper raised awareness of topics such as privacy and cost. Because the experts are co-authors who had already collaborated on the systematization, this comparison does not measure the effect of the paper on independent experts; it measures a change in the authors' own framing after being exposed to the paper's checklist and findings. This is circular with respect to Contribution C3, which presents the ten open problems as if they were distilled from external expert input. The authors should remove the causal interpretation or reframe this subsection as a design artifact of the elicitation process.

minor comments (4)

[§II-E.2] The 'social perspective' metric counts occurrences of the strings 'society,' 'social,' 'societal,' and 'socio' and describes this as an objective measure of a paper's social focus. A raw word count is a crude proxy (for instance, 'social' appears in routine technical phrases such as 'social engineering'), and the paper would benefit from acknowledging this limitation or supplementing the count with a manual check.
[§II-A and Fig. 2] The screening funnel shows 3311 papers, then '128 papers' after preliminary screening, then 95 final papers, but the figure and text do not clearly explain how many papers were excluded during each of the two screening rounds. Please clarify the intermediate counts in the figure or in the captions.
[Tables I–III] The custom icon set (♂shield-alt, ⋆, †, /user, ♂server, /coins, /calcula◎or, /commen◎, etc.) is dense and may be difficult to parse; adding a plain-text legend with short definitions in each caption or in a single table of symbols would improve readability.
[References] Several references are informal or volatile (e.g., [47] is a LinkedIn post, [48] is a bare URL, [212] is an archived news page). For a SoK aimed at long-term utility, please replace these with archival or stable citations where possible, or clearly mark them as online resources.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the SoK's systematization is transparent and its findings are recomputed from the corpus rather than imported from its own definitions.

full rationale

This paper is a systematization, not a derivation. Its central claims—gaps in the literature, heterogeneity of OAI use cases, neglect of cost/benefit analysis, and the expert-derived research agenda—are each based on applying an explicitly stated checklist (§II-E) to 133 independently collected primary sources (95 papers and 38 briefings) and to 549 survey responses. The checklist's categories are defined a priori and are transparently inspired by prior work ([20], [62]); even where [62] is a self-citation by overlapping authors, the quantitative findings (e.g., 73% of technical papers omit cost analysis, 78% target systems) are tabulated directly from the reviewed corpus (Tables I–III), so the citation is inspirational rather than load-bearing. The expert open problems (C3) are explicitly a synthesis of 12 experts' statements, and the paper fully discloses the design whereby experts read the draft before writing their statements (§II-D, §VI-C); this is a measured intervention used to study awareness, not a hidden circular validation. The survey sample's representativeness is a genuine validity limitation, acknowledged in §V-A and §VII-B, but it is not a circularity. No equation-level or by-construction reduction, and no fitted parameter renamed as a prediction, appears anywhere in the manuscript. The paper is therefore self-contained against external benchmarks and merits a score of 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

This is a meta-analysis, so there are no fitted parameters or invented physical entities. The checklist itself is a methodological instrument. The axioms listed are the scoping assumptions that determine which works are included and how they are classified.

assumptions (5)

domain assumption Offensive AI is defined as the use of AI to accomplish a task that violates security and privacy objectives, by an attacker deliberately causing harm.
This scoping choice determines which 95 papers are included and excludes harms from negligence or misconfiguration. Stated in Section I (Scope).
domain assumption MITRE ATT&CK (Enterprise, Mobile, ICS) is an appropriate and sufficient mapping baseline for OAI use cases; use cases not mappable are categorized ad hoc.
Used throughout Section II-E and III to label use cases; the paper's finding that many use cases are outside MITRE depends on this choice.
domain assumption The four knowledge sources (95 papers, 38 briefings, 549 respondents, 12 experts) constitute a meaningful 'snapshot' of offensive AI.
Load-bearing for contribution C1; the paper itself acknowledges under-representation in Section VII-B.
domain assumption Convenience-sampled user study (mostly Europe/North America, highly educated, 75% IT-related) provides valid insights into laypeople perceptions.
Section V-A acknowledges the sample is not representative of the world population, but the paper still generalizes survey findings to 'laypeople' in its lessons learned.
ad hoc to paper Counting occurrences of 'society', 'social', 'societal', or 'socio' is an objective measure of a paper's social perspective.
Used in checklist step 2 (III), Section II-E. This is a somewhat arbitrary proxy for societal focus.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SoK: On the Offensive Potential of AI." pith.science (2026). https://pith.science/paper/GL5S7XQX

@misc{pith2026241218442,
  author       = {Pith},
  title        = {Pith review of: SoK: On the Offensive Potential of AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GL5S7XQX}},
  note         = {Machine review of arXiv:2412.18442}
}

read the original abstract

Our society increasingly benefits from Artificial Intelligence (AI). Unfortunately, more and more evidence shows that AI is also used for offensive purposes. Prior works have revealed various examples of use cases in which the deployment of AI can lead to violation of security and privacy objectives. No extant work, however, has been able to draw a holistic picture of the offensive potential of AI. In this SoK paper we seek to lay the ground for a systematic analysis of the heterogeneous capabilities of offensive AI. In particular we (i) account for AI risks to both humans and systems while (ii) consolidating and distilling knowledge from academic literature, expert opinions, industrial venues, as well as laypeople -- all of which being valuable sources of information on offensive AI. To enable alignment of such diverse sources of knowledge, we devise a common set of criteria reflecting essential technological factors related to offensive AI. With the help of such criteria, we systematically analyze: 95 research papers; 38 InfoSec briefings (from, e.g., BlackHat); the responses of a user study (N=549) entailing individuals with diverse backgrounds and expertise; and the opinion of 12 experts. Our contributions not only reveal concerning ways (some of which overlooked by prior work) in which AI can be offensively used today, but also represent a foothold to address this threat in the years to come.

Figures

Figures reproduced from arXiv: 2412.18442 by the authors.

**Figure 1.** Targets and knowledge sources related to offensive AI. In this SoK, we seek to establish a foundation for understanding and mitigating the (current and future) offensive potential of AI. To this end, we make three high-level contributions: C1: We present a snapshot of the current landscape of offensive AI, by accounting for its three crucial stakeholders: systems, humans, and society. This contribution serves to r… view at source ↗

**Figure 2.** Systematic Literature Review. We collect over 3000 papers from various repositories. After filtering, screening and inter-researcher discussions, we coalesce 95 papers on OAI which we consider in this SoK. Search Queries. We began our literature review by asking ourselves “how has prior work envisioned offensive AI?” To systematically encompass a broad spectrum of prior art, we search for related papers indexed by f… view at source ↗

**Figure 3.** Questionnaire. Depending on the answers, participants have to respond to up to four broad questions (e.g., no specific time frame is given). Some questions expect open answers, ensuring freedom to share any concern. related to OAI, and the questionnaire did not provide any specific information about OAI. Indeed, our goal was to collect the genuine opinion of each participant about their own vision of OAI. Moreover, … view at source ↗

Figures from the paper (14 more)

**Figure 4.** Figure 4: OAI use cases not covered by MITRE ATT&CK (technical papers). Some focus on society and privacy (OSN=online social networks). Some works focusing on novel attacks (†) consider potential defenses against the proposed attacks: e.g., [75] considers Web Application Firewal…

**Figure 5.** Figure 5: Quantitative results (laypeople). Sankey chart of the closed questions. 2) Qualitative Analysis: We review the answers to the open questions wherein we ask participants to provide reasons why they are (or are not) concerned about OAI. Here, we present the results of ou…

**Figure 9.** Figure 9: Employment. The majority of the participants are employed (68%). 15% are students and 10% are self-employed. 2% are retired or not employed. TABLE V: Comparison of our sample to OECD population. The data is reported in %. Compared to the OECD reference values, our samp…

**Figure 6.** Figure 6: Age range. 51% of the participants are below 35, and overall 90% of the participants are between 18 and 54. Only 2% are older than 65. 0 50 100 150 200 250 300 350 400 450 Europe North America Asia Australia Africa South America Number of participants [PITH_FULL_IMAGE…

**Figure 7.** Figure 7: Residence based on continent. Most participants are from Europe (70%) and North America (21%). 7% are from Asia, while overall only 2% of all participants are from Australia, South America, or Africa. 0 50 100 150 200 250 Master's degree Bachelor's degree Completed sec…

**Figure 8.** Figure 8: Education. 20% of the participants do not hold a university degree, while 31% hold a Bachelor’s degree, and 50% a Master’s degree or higher. Expertise. Figs. 10 and 11 provide details on the participants’ expertise in AI and cybersecurity. We asked the users to rate t…

**Figure 10.** Figure 10: AI Expertise. 17% of the participants have little or no knowledge of AI, while around half of the participants have a basic understanding of AI, and only very few are experts. Beginner 23% Intermediate 43% Advanced 24% Expert 10% [PITH_FULL_IMAGE:figures/full_fig_p02…

**Figure 11.** Figure 11: Cybersecurity Expertise. Only 23% of the participants have little or no knowledge of cybersecurity, while the majority of participants have at least an intermediate understanding of cybersecurity. dependent variables. First, we transform the qualitative responses int…

**Figure 12.** Figure 12: Topics identified by BERTopic. We use BERTopic to analyze our expert statements and output the 8 most relevant topics. Each plot in the figure refers to a topic (title), wherein the y-axis shows the six most relevant words in the topic, and the x-axis denotes the weig…

**Figure 13.** Figure 13: Word Clouds of the topics identified by BERTopic. Each subfigure reports the word cloud of each of the 8 topics identified by BERTopic. 0 0.2 0.4 0.6 0.8 1 7-Risk Assessment and Quantification for Offensive AI Tools 2-Risks and Challenges in Offensive AI Research 3-Co…

**Figure 14.** Figure 14: Hierarchical clustering of the topics identified by BERTopic. We visualize the 8 topics identified by BERTopic to discern similarities. • We take the 8 topics provided by BERTopic, remove noisy stopwords (the same we considered for the keyword extraction), and use LL…

**Figure 16.** Figure 16: The evolution of the term “offensive AI.” The term offensive AI (and similar related terms) has substantially evolved over time. what follows, we will describe the temporal evolution of our activities, describing also some challenges we encountered. We began with a li…

**Figure 15.** Figure 15: All works on OAI per year. We present the yearly distribution of the works on OAI considered in this SoK, distinguishing technical and nontechnical academic publications (§III) from InfoSec briefings (§IV). B. History of the term “offensive AI” In [PITH_FULL_IMAGE:f…

**Figure 17.** Figure 17: Timeline of our Research. We began working on this SoK at the beginning of 2023. Throughout the entire activities shown in the figure, we have also had frequent meetings to steer the direction of our research and also to revise the paper. For instance, gaps in the “ex…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

LLM Cyber Evaluations Don't Capture Real-World Risk
cs.CR 2025-01 conditional novelty 4.0 of 10

The paper argues and demonstrates with a 100-prompt case study that LLM cyber risk evaluations need to include threat actor adoption and impact, not just model capability.

Reference graph

Works this paper leans on

271 extracted references · 72 canonical work pages · cited by 1 Pith paper

[1]

Girasa, Artificial Intelligence as a Disruptive Technology: Economic Transformation and Government Regulation

R. Girasa, Artificial Intelligence as a Disruptive Technology: Economic Transformation and Government Regulation . Springer, 2020

2020
[2]

Artificial intelligence as a disruptive technology—a systematic literature review,

V .-D. P˘av˘aloaia and S.-C. Necula, “Artificial intelligence as a disruptive technology—a systematic literature review,” Electronics, 2023

2023
[3]

Artificial intelligence, machine learning and deep learning in advanced robotics, a review,

M. Soori, B. Arezoo, and R. Dastres, “Artificial intelligence, machine learning and deep learning in advanced robotics, a review,” Cognitive Robotics, 2023. 14

2023
[4]

Machine learning and ai in business intelligence: Trends and opportunities,

J. P. Bharadiya, “Machine learning and ai in business intelligence: Trends and opportunities,” International Journal of Computer , 2023

2023
[5]

What factors contribute to the acceptance of artificial intelligence? a systematic review,

S. Kelly, S.-A. Kaye, and O. Oviedo-Trespalacios, “What factors contribute to the acceptance of artificial intelligence? a systematic review,” Telematics and Informatics, 2023

2023
[6]

The role of machine learning in cybersecurity,

G. Apruzzese, P. Laskov, E. Montes de Oca, W. Mallouli, L. Brdalo Rapa, A. V . Grammatopoulos, and F. Di Franco, “The role of machine learning in cybersecurity,” ACM Digital Threats: Research and Practice, 2023

2023
[7]

Artificial intelligence and national security,

K. M. Sayler, “Artificial intelligence and national security,” Congres- sional Research Service , 2020

2020
[8]

Trusting artificial intel- ligence in cybersecurity is a double-edged sword,

M. Taddeo, T. McCutcheon, and L. Floridi, “Trusting artificial intel- ligence in cybersecurity is a double-edged sword,” Nature Machine Intelligence, 2019

2019

Show all 271 references

[9]

Deepcase: Semi-supervised contextual analysis of security events,

T. Van Ede, H. Aghakhani, N. Spahn, R. Bortolameotti, M. Cova, A. Continella, M. van Steen, A. Peter, C. Kruegel, and G. Vigna, “Deepcase: Semi-supervised contextual analysis of security events,” in IEEE Symposium on Security and Privacy , 2022

2022
[10]

Wild patterns: Ten years after the rise of adversarial machine learning,

B. Biggio and F. Roli, “Wild patterns: Ten years after the rise of adversarial machine learning,” Pattern Recognition, 2018

2018
[11]

Sok: Security and privacy in machine learning,

N. Papernot, P. McDaniel, A. Sinha, and M. P. Wellman, “Sok: Security and privacy in machine learning,” in IEEE European Symposium on Security and Privacy , 2018

2018
[12]

Towards evaluating the robustness of neural networks,

N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in IEEE Symposium on Security and Privacy , 2017

2017
[13]

Deep models under the gan: information leakage from collaborative deep learning,

B. Hitaj, G. Ateniese, and F. Perez-Cruz, “Deep models under the gan: information leakage from collaborative deep learning,” inACM SIGSAC Conference on Computer and Communications Security , 2017

2017
[14]

Stealing machine learning models via prediction APIs,

F. Tram `er, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction APIs,” in USENIX Security Symposium, 2016

2016
[15]

Phishing faster: Implementing chatgpt into phishing campaigns,

T. Langford and B. Payne, “Phishing faster: Implementing chatgpt into phishing campaigns,” in Future Technologies Conference, 2023

2023
[16]

Devising and detecting phishing emails using large language models,

F. Heiding, B. Schneier, A. Vishwanath, J. Bernstein, and P. S. Park, “Devising and detecting phishing emails using large language models,” IEEE Access, 2024

2024
[17]

Ai tools such as chatgpt are generating a mammoth increase in malicious phishing emails,

V . Bob, “Ai tools such as chatgpt are generating a mammoth increase in malicious phishing emails,” CNBC,
[18]

The state of phishing,

Slashnext, “The state of phishing,” https://slashnext .com/wp-content/ uploads/2023/10/SlashNext-The-State-of-Phishing-Report-2023 .pdf, Slashnext, Tech. Rep., 2023

2023
[19]

A systematic review of defensive and offensive cybersecurity with machine learning,

I. D. Aiyanyo, H. Samuel, and H. Lim, “A systematic review of defensive and offensive cybersecurity with machine learning,” Applied Sciences, 2020

2020
[20]

The Threat of Offensive AI to Organizations,

Y . Mirsky, A. Demontis, J. Kotak, R. Shankar, D. Gelei, L. Yang, X. Zhang, M. Pintor, W. Lee, Y . Eloviciet al., “The Threat of Offensive AI to Organizations,” Computers & Security , 2023

2023
[21]

Long-term Collection (and classification) of works on Offensive AI (website of this paper),

“Long-term Collection (and classification) of works on Offensive AI (website of this paper),” https://sok-offensive-ai .github.io/, 2024

2024
[22]

You are who you know and how you behave: Attribute inference attacks via users’ social friends and behaviors,

N. Z. Gong and B. Liu, “You are who you know and how you behave: Attribute inference attacks via users’ social friends and behaviors,” in USENIX Security Symposium , 2016

2016
[23]

Predicting personality from twitter,

J. Golbeck, C. Robles, M. Edmondson, and K. Turner, “Predicting personality from twitter,” in IEEE International Conference on Privacy, Security, Risk and Trust & IEEE International Conference on Social Computing, 2011

2011
[25]

Generating adversarial malware examples for black-box attacks based on GAN,

W. Hu and Y . Tan, “Generating adversarial malware examples for black-box attacks based on GAN,” in International Conference on Data Mining and Big Data , 2022

2022
[26]

Idsgan: Generative adversarial networks for attack generation against intrusion detection,

Z. Lin, Y . Shi, and Z. Xue, “Idsgan: Generative adversarial networks for attack generation against intrusion detection,” in Pacific-asia Con- ference on Knowledge Discovery and Data Mining , 2022

2022
[27]

Practical evasion of a learning-based clas- sifier: A case study,

N. ˇSrndi´c and P. Laskov, “Practical evasion of a learning-based clas- sifier: A case study,” in IEEE Symposium on Security and Privacy , 2014

2014
[28]

Machine learning: Trends, perspec- tives, and prospects,

M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspec- tives, and prospects,” Science, 2015

2015
[29]

The AI-based cyber threat landscape: A survey,

N. Kaloudi and J. Li, “The AI-based cyber threat landscape: A survey,” ACM Computing Surveys , 2020

2020
[30]

The emerging threat of ai-driven cyber attacks: A review,

B. Guembe, A. Azeta, S. Misra, V . C. Osamor, L. Fernandez-Sanz, and V . Pospelova, “The emerging threat of ai-driven cyber attacks: A review,” Applied Artificial Intelligence , 2022

2022
[31]

Guidelines for performing systematic literature reviews in software engineering,

“Guidelines for performing systematic literature reviews in software engineering,” School of Computer Science and Mathematics, Keele University & Department of Computer Science, University of Durham, Tech. Rep., 2007

2007
[32]

Sok: Taxonomy of attacks on open-source software supply chains,

P. Ladisa, H. Plate, M. Martinez, and O. Barais, “Sok: Taxonomy of attacks on open-source software supply chains,” in IEEE Symposium on Security and Privacy , 2023

2023
[33]

Simulating SQL injection vulnerability exploitation using Q-learning reinforcement learning agents,

L. Erd ˝odi, ˚A. ˚A. Sommervoll, and F. M. Zennaro, “Simulating SQL injection vulnerability exploitation using Q-learning reinforcement learning agents,” Journal of Information Security and Applications , 2021

2021
[34]

Modelling penetration testing with reinforcement learning using capture-the-flag challenges: Trade-offs between model-free learning and a priori knowledge,

F. M. Zennaro and L. Erd ˝odi, “Modelling penetration testing with reinforcement learning using capture-the-flag challenges: Trade-offs between model-free learning and a priori knowledge,” IET Information Security, 2023

2023
[35]

Towards pentesting automation using the metasploit framework,

O. Valea and C. Opris ¸a, “Towards pentesting automation using the metasploit framework,” inIEEE International Conference on Intelligent Computer Communication and Processing , 2020

2020
[36]

SOFIA: An automated security oracle for black-box testing of SQL-injection vulnerabilities,

M. Ceccato, C. D. Nguyen, D. Appelt, and L. C. Briand, “SOFIA: An automated security oracle for black-box testing of SQL-injection vulnerabilities,” in IEEE/ACM International Conference on Automated Software Engineering, 2016

2016
[37]

Automated adversary emulation for cyber-physical systems via reinforcement learning,

A. Bhattacharya, T. Ramachandran, S. Banik, C. P. Dowling, and S. D. Bopardikar, “Automated adversary emulation for cyber-physical systems via reinforcement learning,” in IEEE International Conference on Intelligence and Security Informatics , 2020

2020
[38]

Outcomes from health information exchange: systematic review and future research needs,

W. R. Hersh, A. M. Totten, K. B. Eden, B. Devine, P. Gorman, S. Z. Kassakian, S. S. Woods, M. Daeges, M. Pappas, and M. S. McDonagh, “Outcomes from health information exchange: systematic review and future research needs,” JMIR Medical Informatics , 2015

2015
[39]

Appmine: Behavioral analytics for web appli- cation vulnerability detection,

I. Jana and A. Oprea, “Appmine: Behavioral analytics for web appli- cation vulnerability detection,” in ACM SIGSAC Conference on Cloud Computing Security Workshop, 2019

2019
[40]

Spacephish: the evasion-space of adversarial attacks against phishing website detectors using machine learning,

G. Apruzzese, M. Conti, and Y . Yuan, “Spacephish: the evasion-space of adversarial attacks against phishing website detectors using machine learning,” in Annual Computer Security Applications Conference, 2022

2022
[41]

Wavenet: A generative model for raw audio,

A. Van Den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, K. Kavukcuoglu et al. , “Wavenet: A generative model for raw audio,” arXiv preprint arXiv:1609.03499, 2016

2016 arXiv
[42]

A systematic literature review and meta-analysis on artificial intelligence in penetration testing and vulnerability assessment,

D. R. McKinnel, T. Dargahi, A. Dehghantanha, and K.-K. R. Choo, “A systematic literature review and meta-analysis on artificial intelligence in penetration testing and vulnerability assessment,” Computers & Electrical Engineering, 2019

2019
[43]

Can artificial intelligence power future malware,

O. Kubovi ˇc, P. Ko ˇsin´ar, and J. J ´anoˇs´ık, “Can artificial intelligence power future malware,” ESET white paper , 2018

2018
[44]

Forbes, https://www .forbes.com/sites/forbestechcouncil/2023/07/17/ mitigating-ai-based-cyberattacks/, 2023

2023
[45]

Economist, https://www .economist.com/biometrics-pod, 2023

2023
[46]

CNN, https://edition .cnn.com/videos/business/2023/05/19/exp-ai- signifyd-intv-051909aseg1-cnni-business .cnn

2023
[47]

https://www .linkedin.com/posts/ben-nassi-phd-68a743115 i-have- recently-received-a-few-emails-from-activity-7161264634505703424- -NYk?utm source=share&utm medium=member deskto, 2024

2024
[48]

Wilkin, https://www .blackhat.com/us-18/arsenal.html#jacob-wilkin, 2018

J. Wilkin, https://www .blackhat.com/us-18/arsenal.html#jacob-wilkin, 2018

2018
[49]

Machine learning attacks against the Asirra CAPTCHA,

P. Golle, “Machine learning attacks against the Asirra CAPTCHA,” in ACM SIGSAC Conference on Computer and Communications Security, 2008

2008
[50]

Hacking desire: Reverse-engineering what people wan,

I. Clarke, “Hacking desire: Reverse-engineering what people wan,” in DefCon, 2008

2008
[51]

Sok: Hate, harassment, and the changing landscape of online abuse,

K. Thomas, D. Akhawe, M. Bailey, D. Boneh, E. Bursztein, S. Con- solvo, N. Dell, Z. Durumeric, P. G. Kelley, D. Kumar et al. , “Sok: Hate, harassment, and the changing landscape of online abuse,” inIEEE Symposium on Security and Privacy , 2021

2021
[52]

Sok: Authentication in augmented and virtual reality,

S. Stephenson, B. Pal, S. Fan, E. Fernandes, Y . Zhao, and R. Chatter- jee, “Sok: Authentication in augmented and virtual reality,” in IEEE Symposium on Security and Privacy , 2022

2022
[53]

The Menlo report,

M. Bailey, D. Dittrich, E. Kenneally, and D. Maughan, “The Menlo report,” IEEE Security & Privacy , 2012

2012
[54]

Our repository,

“Our repository,” https://github .com/hihey54/sok oai/, 2024

2024
[55]

Convenience sampling, random sampling, and snow- 15 ball sampling: How does sampling affect the validity of research?

R. W. Emerson, “Convenience sampling, random sampling, and snow- 15 ball sampling: How does sampling affect the validity of research?” Journal of Visual Impairment & Blindness , 2015

2015
[56]

Comparisons of online recruitment strategies for convenience samples: Craigslist, google adwords, facebook, and amazon mechanical turk,

C. Antoun, C. Zhang, F. G. Conrad, and M. F. Schober, “Comparisons of online recruitment strategies for convenience samples: Craigslist, google adwords, facebook, and amazon mechanical turk,” Field meth- ods, 2016

2016
[57]

Charmaz, Constructing grounded theory: A practical guide through qualitative analysis

K. Charmaz, Constructing grounded theory: A practical guide through qualitative analysis. SAGE Publications Ltd, 2006

2006
[58]

“So what if ChatGPT wrote it?

Y . K. Dwivedi, N. Kshetri, L. Hughes, E. L. Slade, and A. Jeyaraj et al., ““So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversa- tional AI for research, practice and policy,” International Journal o...

2023
[59]

Whatever next? predictive brains, situated agents, and the future of cognitive science,

A. Clark, “Whatever next? predictive brains, situated agents, and the future of cognitive science,” Behavioral and brain sciences , 2013

2013
[60]

Advances and open problems in federated learning,

P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, and M. Bennis et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning , 2021

2021
[61]

Interviewing the investigator: Strategies for addressing instrumentation and researcher bias concerns in qualitative research

R. J. Chenail, “Interviewing the investigator: Strategies for addressing instrumentation and researcher bias concerns in qualitative research.” Qualitative report, 2011

2011
[62]

“Real Attackers Don’t Compute Gradients

G. Apruzzese, H. S. Anderson, S. Dambra, D. Freeman, F. Pierazzi, and K. Roundy, ““Real Attackers Don’t Compute Gradients”: Bridging the Gap Between Adversarial ML Research and Practice,” in IEEE Conference on Secure and Trustworthy Machine Learning , 2023

2023
[63]

https://attack .mitre.org/
[64]

https://attack .mitre.org/matrices/enterprise/
[65]

https://attack .mitre.org/matrices/mobile/
[66]

https://attack .mitre.org/matrices/ics/
[67]

Attribute inference attacks in online multiplayer video games: A case study on Dota2,

P. P. Tricomi, L. Facciolo, G. Apruzzese, and M. Conti, “Attribute inference attacks in online multiplayer video games: A case study on Dota2,” in ACM Conference on Data and Application Security and Privacy, 2023

2023
[68]

A low-cost approach to crack python captchas using ai-based chosen-plaintext attack,

N. Yu and K. Darling, “A low-cost approach to crack python captchas using ai-based chosen-plaintext attack,” Applied Sciences, 2019

2019
[69]

”Do users fall for real adversarial phishing?

A. Draganovic, S. Dambra, J. A. Iuit, K. Roundy, and G. Apruzzese, “”Do users fall for real adversarial phishing?” Investigating the human response to evasive webpages,” in APWG Symposium on Electronic Crime Research, 2023

2023
[70]

My privacy my decision: Control of photo sharing on online social networks,

K. Xu, Y . Guo, L. Guo, Y . Fang, and X. Li, “My privacy my decision: Control of photo sharing on online social networks,”IEEE Transactions on Dependable and Secure Computing , 2015

2015
[71]

Keyword occurrences and journal specialization,

G. Sampagnaro, “Keyword occurrences and journal specialization,” Scientometrics, 2023

2023
[72]

“dirclustering

D. Antonelli, R. Cascella, A. Schiano, G. Perrone, and S. P. Romano, ““dirclustering”: a semantic clustering approach to optimize website structure discovery during penetration testing,” Journal of Computer Virology and Hacking Techniques, 2024

2024
[73]

Vulnerability exploitation using reinforcement learning,

A. AlMajali, L. Al-Abed, R. Mutleq, Z. Samamah, A. A. Shhadeh, B. J. Mohd, and K. M. A. Yousef, “Vulnerability exploitation using reinforcement learning,” in IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology , 2023

2023
[74]

GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learning,

J. Chen, S. Hu, H. Zheng, C. Xing, and G. Zhang, “GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learning,” Computers & Security , 2023

2023
[75]

Generative Adversarial Network (GAN)-Based Autonomous Penetration Testing for Web Applications,

A. Chowdhary, K. Jha, and M. Zhao, “Generative Adversarial Network (GAN)-Based Autonomous Penetration Testing for Web Applications,” Sensors, 2023

2023
[76]

Generative neural networks as a tool for web applications penetration testing,

P. Gallus, M. ˇStˇep´anek, T. R ´aˇcil, and P. Franti ˇs, “Generative neural networks as a tool for web applications penetration testing,” in IEEE Communication and Information Technologies , 2023

2023
[77]

Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks,

M. C. Ghanem, T. M. Chen, and E. G. Nepomuceno, “Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks,” Journal of Intelligent Information Systems , 2023

2023
[78]

Getting pwn’d by ai: Penetration testing with large language models,

A. Happe and J. Cito, “Getting pwn’d by ai: Penetration testing with large language models,” in ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineer- ing, 2023

2023
[79]

When ChatGPT goes rogue: exploring the potential cybersecurity threats of AI-powered conversational chatbots,

F. Iqbal, F. Samsom, F. Kamoun, and ´A. MacDermott, “When ChatGPT goes rogue: exploring the potential cybersecurity threats of AI-powered conversational chatbots,” Frontiers in Communications and Networks , 2023

2023
[80]

Working with ai to persuade: Examining a large language model’s ability to generate pro-vaccination messages,

E. Karinshak, S. X. Liu, J. S. Park, and J. T. Hancock, “Working with ai to persuade: Examining a large language model’s ability to generate pro-vaccination messages,” Proceedings of the ACM on Human-Computer Interaction, 2023

2023
[81]

New tricks to old codes: can ai chatbots replace static code analysis tools?

O. S. Ozturk, E. Ekmekcioglu, O. Cetin, B. Arief, and J. Hernandez- Castro, “New tricks to old codes: can ai chatbots replace static code analysis tools?” in European Interdisciplinary Cybersecurity Confer- ence, 2023

2023
[82]

An attacker’s dream? exploring the capabilities of chatgpt for developing malware,

Y . M. Pa Pa, S. Tanizaki, T. Kou, M. Van Eeten, K. Yoshioka, and T. Matsumoto, “An attacker’s dream? exploring the capabilities of chatgpt for developing malware,” in Cyber Security Experimentation and Test Workshop, 2023

2023
[83]

An automated approach to web offensive security,

N. Auricchio, A. Cappuccio, F. Caturano, G. Perrone, and S. P. Ro- mano, “An automated approach to web offensive security,” Computer Communications, 2022

2022
[84]

Combining variational au- toencoders and transformer language models for improved password generation,

D. Biesner, K. Cvejoski, and R. Sifa, “Combining variational au- toencoders and transformer language models for improved password generation,” in International Conference on Availability, Reliability and Security, 2022

2022
[85]

Discovering exfiltra- tion paths using reinforcement learning with attack graphs,

T. Cody, A. Rahman, C. Redino, L. Huang, R. Clark, A. Kakkar, D. Kushwaha, P. Park, P. Beling, and E. Bowen, “Discovering exfiltra- tion paths using reinforcement learning with attack graphs,” in IEEE Conference on Dependable and Secure Computing , 2022

2022
[86]

Reinforcing Penetration Testing Using AI,

A. Confido, E. V . Ntagiou, and M. Wallum, “Reinforcing Penetration Testing Using AI,” in IEEE Aerospace Conference, 2022

2022
[87]

Using cyber terrain in re- inforcement learning for penetration testing,

R. Gangupantulu, T. Cody, P. Park, A. Rahman, L. Eisenbeiser, D. Radke, R. Clark, and C. Redino, “Using cyber terrain in re- inforcement learning for penetration testing,” in IEEE International Conference on Omni-layer Intelligent Systems , 2022

2022
[88]

Penetration testing procedure using machine learning,

R. S. Jagamogan, S. A. Ismail, N. H. Hassan, and H. Abas, “Penetration testing procedure using machine learning,” in International Conference on Smart Sensors and Application , 2022

2022
[89]

My tweets bring all the traits to the yard: Predicting personality and relational traits in online social networks,

D. Karanatsiou, P. Sermpezis, D. Gruda, K. Kafetsios, I. Dimitriadis, and A. Vakali, “My tweets bring all the traits to the yard: Predicting personality and relational traits in online social networks,” ACM Transactions on the Web , 2022

2022
[90]

Link: Black-box detection of cross-site scripting vulnerabilities using reinforcement learning,

S. Lee, S. Wi, and S. Son, “Link: Black-box detection of cross-site scripting vulnerabilities using reinforcement learning,” in The ACM Web Conference 2022, 2022

2022
[91]

Deep reinforcement learning for penetra- tion testing of cyber-physical attacks in the smart grid,

Y . Li, J. Yan, and M. Naili, “Deep reinforcement learning for penetra- tion testing of cyber-physical attacks in the smart grid,” in International Joint Conference on Neural Networks , 2022

2022
[92]

Leveraging deep reinforcement learning for automating penetration testing in reconnaissance and exploitation phase,

N. X. Nhu, T. T. Nghia, N. H. Quyen, V .-H. Pham, P. T. Duy et al. , “Leveraging deep reinforcement learning for automating penetration testing in reconnaissance and exploitation phase,” inIEEE International Conference on Computing and Communication Technologies , 2022

2022
[93]

Passflow: guessing passwords with generative flows,

G. Pagnotta, D. Hitaj, F. De Gaspari, and L. V . Mancini, “Passflow: guessing passwords with generative flows,” in Annual IEEE/IFIP International Conference on Dependable Systems and Networks , 2022

2022
[94]

Cascaded reinforcement learning agents for large action spaces in autonomous penetration testing,

K. Tran, M. Standen, J. Kim, D. Bowman, T. Richer, A. Akella, and C.-T. Lin, “Cascaded reinforcement learning agents for large action spaces in autonomous penetration testing,” Applied Sciences, 2022

2022
[95]

Intelligent penetration testing in dynamic defense environment,

Q. Yao, Y . Wang, X. Xiong, and Y . Li, “Intelligent penetration testing in dynamic defense environment,” in International Conference on Cyber Security, 2022

2022
[96]

Discovering reflected cross-site scripting vulnerabilities using a multiobjective reinforcement learning environment,

F. Caturano, G. Perrone, and S. P. Romano, “Discovering reflected cross-site scripting vulnerabilities using a multiobjective reinforcement learning environment,” Computers & Security , 2021

2021
[97]

Crown jewels analysis using reinforcement learning with at- tack graphs,

R. Gangupantulu, T. Cody, A. Rahma, C. Redino, R. Clark, and P. Park, “Crown jewels analysis using reinforcement learning with at- tack graphs,” inIEEE Symposium Series on Computational Intelligence, 2021

2021
[98]

Offensive AI: Unification of email generation through GPT-2 with a game-theoretic approach for spear-phishing attacks,

H. Khan, M. Alam, S. Al-Kuwari, and Y . Faheem, “Offensive AI: Unification of email generation through GPT-2 with a game-theoretic approach for spear-phishing attacks,” Competitive Advantage in the Digital Economy, 2021

2021
[99]

Automating privilege escalation with deep reinforcement learning,

K. Kujanp ¨a¨a, W. Victor, and A. Ilin, “Automating privilege escalation with deep reinforcement learning,” in ACM Workshop on Artificial Intelligence and Security , 2021

2021
[100]

Offensive security of keyboard data using machine learning for password authentication in iot,

K. Lee, J. Lee, C. Choi, and K. Yim, “Offensive security of keyboard data using machine learning for password authentication in iot,” IEEE Access, 2021

2021
[101]

Automating post-exploitation with deep reinforcement learning,

R. Maeda and M. Mimura, “Automating post-exploitation with deep reinforcement learning,” Computers & Security , 2021

2021
[102]

Reinforcement 16 learning based penetration testing of a microgrid control algorithm,

C. Neal, H. Dagdougui, A. Lodi, and J. M. Fernandez, “Reinforcement 16 learning based penetration testing of a microgrid control algorithm,” in IEEE Annual Computing and Communication Workshop and Confer- ence, 2021

2021
[103]

Regulation tl; dr: Adversarial text summarization of federal register articles,

F. Sharevski, P. Jachim, and E. Pieroni, “Regulation tl; dr: Adversarial text summarization of federal register articles,” in Workshop on Cyber- Security Arms Race , 2021

2021
[104]

CybORG: A Gym for the Development of Autonomous Cyber Agents,

M. Standen, M. Lucas, D. Bowman, T. Richer, J. Kim, and D. Mar- riott, “CybORG: A Gym for the Development of Autonomous Cyber Agents,” in International Workshop on Adaptive Cyber Defense (co- located with IJCAI) , 2021

2021
[105]

Catch Me If You GAN: Using Artificial Intelligence for Fake Log Generation,

C. Toemmel, “Catch Me If You GAN: Using Artificial Intelligence for Fake Log Generation,” arXiv:2112.12006, 2021

2021 arXiv
[106]

Deep hierarchical reinforcement agents for automated penetration testing,

K. Tran, A. Akella, M. Standen, J. Kim, D. Bowman, T. Richer, and C.-T. Lin, “Deep hierarchical reinforcement agents for automated penetration testing,” arXiv:2109.06449, 2021

2021 arXiv
[107]

Man-in-the-middle attacks to detect and identify services in encrypted network flows using machine learn- ing,

A. Al-Hababi and S. C. Tokgoz, “Man-in-the-middle attacks to detect and identify services in encrypted network flows using machine learn- ing,” in IEEE International Conference on Advanced Communication Technologies and Networking, 2020

2020
[108]

Autonomous security analysis and penetration testing,

A. Chowdhary, D. Huang, J. S. Mahendran, D. Romo, Y . Deng, and A. Sabur, “Autonomous security analysis and penetration testing,” in International Conference on Mobility, Sensing and Networking , 2020

2020
[109]

Efficient quantification of profile matching risk in social networks using belief propagation,

A. Halimi and E. Ayday, “Efficient quantification of profile matching risk in social networks using belief propagation,” in European Sympo- sium on Research in Computer Security , 2020

2020
[110]

Automated penetration testing using deep reinforcement learning,

Z. Hu, R. Beuran, and Y . Tan, “Automated penetration testing using deep reinforcement learning,” in IEEE European Symposium on Secu- rity and Privacy Workshops , 2020

2020
[111]

Cybersecurity threats based on machine learning- based offensive technique for password authentication,

K. Lee and K. Yim, “Cybersecurity threats based on machine learning- based offensive technique for password authentication,” Applied Sci- ences, 2020

2020
[112]

Improved practical vulnerability analysis of mouse data according to offensive security based on machine learning in image-based user authentication,

K. Lee and S.-Y . Lee, “Improved practical vulnerability analysis of mouse data according to offensive security based on machine learning in image-based user authentication,” Entropy, 2020

2020
[113]

Deepsqli: Deep semantic learning for testing sql injection,

M. Liu, K. Li, and T. Chen, “Deepsqli: Deep semantic learning for testing sql injection,” in ACM SIGSOFT International Symposium on Software Testing and Analysis , 2020

2020
[114]

Machine learning for offensive security: sandbox classification using decision trees and artificial neural networks,

W. Pearce, N. Landers, and N. Fulda, “Machine learning for offensive security: sandbox classification using decision trees and artificial neural networks,” in Computing Conference, 2020

2020
[115]

Wikipediabot: Machine learning assisted adversarial manipulation of wikipedia articles,

F. Sharevski, P. Jachim, and E. Pieroni, “Wikipediabot: Machine learning assisted adversarial manipulation of wikipedia articles,” in Workshop on DYnamic and Novel Advances in Machine Learning and Intelligent Cyber Security , 2020

2020
[116]

Generative adversarial attacks against intrusion detection systems using active learning,

D. Shu, N. O. Leslie, C. A. Kamhoua, and C. S. Tucker, “Generative adversarial attacks against intrusion detection systems using active learning,” in ACM Workshop on Wireless Security and Machine Learn- ing, 2020

2020
[117]

Mab- malware: A reinforcement learning framework for blackbox generation of adversarial malware,

W. Song, X. Li, S. Afroz, D. Garg, D. Kuznetsov, and H. Yin, “Mab- malware: A reinforcement learning framework for blackbox generation of adversarial malware,” in ACM Asia Conference on Computer and Communications Security, 2022

2022
[118]

Ai-powered gui attack and its defensive methods,

N. Yu, Z. Tuttle, C. J. Thurnau, and E. Mireku, “Ai-powered gui attack and its defensive methods,” in ACM Southeast Conference , 2020

2020
[119]

Generating targeted e-mail at scale using neural machine translation,

C. Basu, S. Venkatesan, C.-Y . J. Chiang, N. Leslie, and C. Kamhoua, “Generating targeted e-mail at scale using neural machine translation,” in Workshop on DYnamic and Novel Advances in Machine Learning and Intelligent Cyber Security , 2019

2019
[120]

Skype & type: Keyboard eavesdropping in voice-over-ip,

S. Cecconello, A. Compagno, M. Conti, D. Lain, and G. Tsudik, “Skype & type: Keyboard eavesdropping in voice-over-ip,” ACM Transactions on Privacy and Security , 2019

2019
[121]

Availability attacks on computing systems through alteration of environmental control: smart malware approach,

K. Chung, Z. T. Kalbarczyk, and R. K. Iyer, “Availability attacks on computing systems through alteration of environmental control: smart malware approach,” in ACM/IEEE International Conference on Cyber- Physical Systems, 2019

2019
[122]

X-DeepSCA: Cross-device deep learning side channel attack,

D. Das, A. Golder, J. Danial, S. Ghosh, A. Raychowdhury, and S. Sen, “X-DeepSCA: Cross-device deep learning side channel attack,” in Annual Design Automation Conference , 2019

2019
[123]

Reinforcement learning for efficient network penetration testing,

M. C. Ghanem and T. M. Chen, “Reinforcement learning for efficient network penetration testing,” Information, 2019

2019
[124]

Har-search: A method to discover hidden affinity relationships in online communities,

J. M. Tshimula, B. Chikhaoui, and S. Wang, “Har-search: A method to discover hidden affinity relationships in online communities,” in IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining , 2019

2019
[125]

Using ai to attack va: a stealthy spyware against voice assistances in smart phones,

R. Zhang, X. Chen, S. Wen, X. Zheng, and Y . Ding, “Using ai to attack va: a stealthy spyware against voice assistances in smart phones,” IEEE Access, 2019

2019
[126]

Keyboard emanations in remote voice calls: Password leakage and noise (less) masking defenses,

S. A. Anand and N. Saxena, “Keyboard emanations in remote voice calls: Password leakage and noise (less) masking defenses,” in ACM Conference on Data and Application Security and Privacy , 2018

2018
[127]

Deep- phish: simulating malicious ai,

A. C. Bahnsen, I. Torroledo, L. D. Camacho, and S. Villegas, “Deep- phish: simulating malicious ai,” in APWG Symposium on Electronic Crime Research, 2018

2018
[128]

Discovering software vulnerabilities using data-flow analysis and machine learning,

J. Kronjee, A. Hommersom, and H. Vranken, “Discovering software vulnerabilities using data-flow analysis and machine learning,” in In- ternational Conference on Availability, Reliability and Security , 2018

2018
[129]

Bringing a GAN to a knife-fight: Adapting malware communication to avoid detection,

M. Rigaki and S. Garcia, “Bringing a GAN to a knife-fight: Adapting malware communication to avoid detection,” in IEEE Security and Privacy Workshops, 2018

2018
[130]

Deeplink: A deep learning approach for user identity linkage,

F. Zhou, L. Liu, K. Zhang, G. Trajcevski, J. Wu, and T. Zhong, “Deeplink: A deep learning approach for user identity linkage,” inIEEE Conference on Computer Communications , 2018

2018
[131]

Automated crowdturfing attacks and defenses in online review systems,

Y . Yao, B. Viswanath, J. Cryan, H. Zheng, and B. Y . Zhao, “Automated crowdturfing attacks and defenses in online review systems,” in ACM SIGSAC Conference on Computer and Communications Security, 2017

2017
[132]

DeepDGA: Adversarially-tuned domain generation and detection,

H. S. Anderson, J. Woodbridge, and B. Filar, “DeepDGA: Adversarially-tuned domain generation and detection,” in ACM Work- shop on Artificial Intelligence and Decurity , 2016

2016
[133]

Toward large-scale vulnerability discovery using machine learning,

G. Grieco, G. L. Grinblat, L. Uzal, S. Rawat, J. Feist, and L. Mounier, “Toward large-scale vulnerability discovery using machine learning,” in ACM Conference on Data and Application Security and Privacy , 2016

2016
[134]

Reverse en- gineering socialbot infiltration strategies in twitter,

C. Freitas, F. Benevenuto, S. Ghosh, and A. Veloso, “Reverse en- gineering socialbot infiltration strategies in twitter,” in IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2015

2015
[135]

The end is nigh: Generic solving of text-based CAPTCHAs,

E. Bursztein, J. Aigrain, A. Moscicki, and J. C. Mitchell, “The end is nigh: Generic solving of text-based CAPTCHAs,” in USENIX Workshop on Offensive Technologies, 2014

2014
[136]

Predicting personality with social behav- ior,

S. Adali and J. Golbeck, “Predicting personality with social behav- ior,” in IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining , 2012

2012
[137]

Studying user footprints in different online social networks,

A. Malhotra, L. Totti, W. Meira Jr, P. Kumaraguru, and V . Almeida, “Studying user footprints in different online social networks,” in IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining , 2012

2012
[138]

Predicting dark triad personality traits from twitter usage and a linguistic analysis of tweets,

C. Sumner, A. Byers, R. Boochever, and G. J. Park, “Predicting dark triad personality traits from twitter usage and a linguistic analysis of tweets,” in International Conference on Machine Learning and Applications, 2012

2012
[139]

Vulnerability extrapolation: Assisted discovery of vulnerabilities using machine learning,

F. Yamaguchi, K. Rieck et al. , “Vulnerability extrapolation: Assisted discovery of vulnerabilities using machine learning,” in USENIX Work- shop on Offensive Technologies , 2011

2011
[140]

Decaptcha: breaking 75% of ebay audio captchas,

E. Bursztein and S. Bethard, “Decaptcha: breaking 75% of ebay audio captchas,” in USENIX Conference on Offensive technologies , 2009

2009
[141]

Adversarially adapting deceptive views and reconnaissance scans on a software defined network,

J. Kelly, M. DeLaus, E. Hemberg, and U.-M. O’Reilly, “Adversarially adapting deceptive views and reconnaissance scans on a software defined network,” in IFIP/IEEE Symposium on Integrated Network and Service Management, 2019

2019
[142]

Autonomous Weapons Open Letter: AI & Robotics Re- searchers,

Future of Life, “Autonomous Weapons Open Letter: AI & Robotics Re- searchers,” https://futureoflife .org/open-letter/open-letter-autonomous- weapons-ai-robotics/, 2016

2016
[143]

Detecting al- gorithmically generated domain-flux attacks with DNS traffic analysis,

S. Yadav, A. K. K. Reddy, A. N. Reddy, and S. Ranjan, “Detecting al- gorithmically generated domain-flux attacks with DNS traffic analysis,” IEEE/ACM Transactions on Networking , 2012

2012
[144]

From Throw-Away traffic to bots: Detecting the rise of DGA-Based malware,

M. Antonakakis, R. Perdisci, Y . Nadji, N. Vasiloglou, S. Abu-Nimeh, W. Lee, and D. Dagon, “From Throw-Away traffic to bots: Detecting the rise of DGA-Based malware,” in USENIX Security Symposium , 2012

2012
[145]

Artificial intelligence and the future of warfare: The usa, china, and strategic stability,

A. C. Dall’Agnol, “Artificial intelligence and the future of warfare: The usa, china, and strategic stability,” Journal of Strategic Studies , 2023

2023
[146]

ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health,

L. De Angelis, F. Baglivo, G. Arzilli, G. P. Privitera, P. Ferragina, A. E. Tozzi, and C. Rizzo, “ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health,” Frontiers in Public Health , 2023

2023
[147]

Security-informed safety analysis of autonomous trans- port systems considering ai-powered cyberattacks and protection,

O. Illiashenko, V . Kharchenko, I. Babeshko, H. Fesenko, and F. Di Gi- andomenico, “Security-informed safety analysis of autonomous trans- port systems considering ai-powered cyberattacks and protection,” 17 Entropy, 2023

2023
[148]

Destabilization of unstable dynamic social equilibriums and the malicious use of artificial intelligence in high-tech strategic psychological warfare,

E. Pashentsev, “Destabilization of unstable dynamic social equilibriums and the malicious use of artificial intelligence in high-tech strategic psychological warfare,” in The Palgrave Handbook of Malicious Use of AI and Psychological Security . Springer, 2023

2023
[149]

Artificial intelligence in warfare: Military uses of ai and their international security implications,

J.-M. Rickli and F. Mantellassi, “Artificial intelligence in warfare: Military uses of ai and their international security implications,” in The AI Wave in Defence Innovation . Routledge, 2023

2023
[150]

Intelligent penetration and attack simulation system based on attack chain,

W. Hao, C. Shen, X. Yang, and C. Wang, “Intelligent penetration and attack simulation system based on attack chain,” in International Symposium on Computational Intelligence and Design , 2022

2022
[151]

Cybersecurity as a Tic-Tac-Toe Game Using Autonomous Forwards (Attacking) And Backwards (Defending) Penetration Testing in a Cyber Adversarial Artificial Intelligence System,

S. Kasim, N. Valliani, N. K. K. Wong, S. Samadi, L. Watkins, and A. Rubin, “Cybersecurity as a Tic-Tac-Toe Game Using Autonomous Forwards (Attacking) And Backwards (Defending) Penetration Testing in a Cyber Adversarial Artificial Intelligence System,” in IEEE Inter- national C...

2022
[152]

Mimetic models: Ethical implications of ai that acts like you,

R. McIlroy-Young, J. Kleinberg, S. Sen, S. Barocas, and A. Anderson, “Mimetic models: Ethical implications of ai that acts like you,” in AAAI/ACM Conference on AI, Ethics, and Society , 2022

2022
[153]

Using weaponized machine learning in cyber offensive operations,

C. Nica and T. T ˘anase, “Using weaponized machine learning in cyber offensive operations,” in International Conference: The Knowledge- based Organization, 2020

2020
[154]

Informational friction as a lens for studying algorithmic aspects of privacy,

P. Skeba and E. P. Baumer, “Informational friction as a lens for studying algorithmic aspects of privacy,” Proceedings of the ACM on Human- Computer Interaction, 2020

2020
[155]

A methodological approach to weaponizing machine learning,

C. Easttom, “A methodological approach to weaponizing machine learning,” in International Conference on Artificial Intelligence and Advanced Manufacturing, 2019

2019
[156]

Autonomous distributed electronic warfare system of systems,

I. Burton and J. Straub, “Autonomous distributed electronic warfare system of systems,” in Annual Conference System of Systems Engi- neering, 2019

2019
[157]

Understanding the strategic implications of the weaponization of artificial intelligence,

J. Burton and S. R. Soare, “Understanding the strategic implications of the weaponization of artificial intelligence,” in International Con- ference on Cyber Conflict , 2019

2019
[158]

Community targeted phishing: A middle ground between massive and spear phishing through natural language generation,

A. Giaretta and N. Dragoni, “Community targeted phishing: A middle ground between massive and spear phishing through natural language generation,” in International Conference in Software Engineering for Defence Applications, 2019

2019
[159]

Decoding, hacking, and optimizing societies: Exploring potential applications of human data analytics in sociological engi- neering, both internally and as offensive weapons,

G. Maus, “Decoding, hacking, and optimizing societies: Exploring potential applications of human data analytics in sociological engi- neering, both internally and as offensive weapons,” in IEEE Science and Information Conference , 2015

2015
[160]

Autonomous intelligent agents in cyber offence,

A. Guarino, “Autonomous intelligent agents in cyber offence,” in International Conference on Cyber Conflict , 2013

2013
[161]

”Get in Researchers; We’re Measuring Reproducibility

D. Olszewski, A. Lu, C. Stillman, K. Warren, C. Kitroser, A. Pascual, D. Ukirde, K. Butler, and P. Traynor, “”Get in Researchers; We’re Measuring Reproducibility”: A Reproducibility Study of Machine Learning Papers in Tier 1 Security Conferences,” in ACM SIGSAC Conference on C...

2023
[162]

Synthetic trust: Exploiting biases at scale,

E. Scheiner and B. Bivu, “Synthetic trust: Exploiting biases at scale,” in BlackHat, 2023

2023
[163]

Me and my evil digital twin: The psychology of human exploitation by ai assistants,

M. Canham and B. Sawyer, “Me and my evil digital twin: The psychology of human exploitation by ai assistants,” in BlackHat, 2023

2023
[164]

Devising and Detecting Phishing: Large Language Models (GPT3, GPT4) vs. Smaller Human Models (V-Triad, Generic Emails),

F. Heiding, B. Schneier, A. Vishwanath, and J. Bernstein, “Devising and Detecting Phishing: Large Language Models (GPT3, GPT4) vs. Smaller Human Models (V-Triad, Generic Emails),” inBlackHat, 2023

2023
[165]

Devising and detecting phishing: Large language models vs. smaller human models,

——, “Devising and detecting phishing: Large language models vs. smaller human models,” in BlackHat, 2023

2023
[166]

How NOT to Train Your Hack Bot: Dos and Dont’s of Building Offensive GPTs,

A. Herbert-V oss and S. Caldwell, “How NOT to Train Your Hack Bot: Dos and Dont’s of Building Offensive GPTs,” in BlackHat, 2023

2023
[167]

How we taught ChatGPT-4 to break Mbed TLS AES with side-channel attacks,

W. Walig ´ora, “How we taught ChatGPT-4 to break Mbed TLS AES with side-channel attacks,” in BlackHat, 2023

2023
[168]

Kidnapping without hostages: Virtual kidnapping and the dark road ahead,

C. Gibson, V . Kropotov, and F. Yarochkin, “Kidnapping without hostages: Virtual kidnapping and the dark road ahead,” in BlackHat, 2023

2023
[169]

Look ma i’m the ceo! real-time video and audio deep-fake!

G. Zror, “Look ma i’m the ceo! real-time video and audio deep-fake!” in DefCon, 2023

2023
[170]

Human or not: Can you really detect the fake voices?

L. Xin and T. Yuan, “Human or not: Can you really detect the fake voices?” in BlackHat, 2022

2022
[171]

Bridging the gap between research and practice in intelligently bypassing waf,

C. Chi, “Bridging the gap between research and practice in intelligently bypassing waf,” in BlackHat, 2022

2022
[172]

Hacking humans with ai as a service,

E. Lim, G. Tan, T. K. Hock, and T. Lee, “Hacking humans with ai as a service,” in DefCon, 2021

2021
[173]

Disinformation at scale: Using gpt-3 mali- ciously for information operations,

A. Lohn and M. Musser, “Disinformation at scale: Using gpt-3 mali- ciously for information operations,” in BlackHat, 2021

2021
[174]

Repurposing neural networks,

P. Tully and L. Foster, “Repurposing neural networks,” in BlackHat, 2020

2020
[175]

How i clone myself using ai - next gen social engineering,

T. Basu, “How i clone myself using ai - next gen social engineering,” in BlackHat, 2020

2020
[176]

Effective vulnerability discovery with machine learning,

A. Sharma and A. M. Yi, “Effective vulnerability discovery with machine learning,” in BlackHat, 2020

2020
[177]

Automated identifica- tion of libraries from vulnerability data,

Y . Chen, A. E. Santosa, A. Sharma, and D. Lo, “Automated identifica- tion of libraries from vulnerability data,” in ACM/IEEE International Conference on Software Engineering: Software Engineering in Prac- tice, 2020

2020
[178]

Gyoithon,

I. Takaesu, M. Masuya, and T. Yoneyama, “Gyoithon,” in BlackHat, 2019

2019
[179]

Cotopaxi iot protocols testing toolkit,

J. Botwicz, “Cotopaxi iot protocols testing toolkit,” in DefCon, 2019

2019
[180]

Deep learning revolutionizing side channel cryptanaly- sis,

E. Bursztein, “Deep learning revolutionizing side channel cryptanaly- sis,” in DefCon, 2019

2019
[181]

Automated rest api endpoint identification for security testing at scale,

L. Ding, A. Benameur, J. Jacob, J. Chen, and S. Pham, “Automated rest api endpoint identification for security testing at scale,” in BlackHat, 2019

2019
[182]

Playing offense and defense with deepfakes,

M. Price and M. Price, “Playing offense and defense with deepfakes,” in BlackHat, 2019

2019
[183]

Deepphish simulating malicious ai,

A. C. Bahnsen, “Deepphish simulating malicious ai,” in BlackHat, 2018

2018
[184]

De-anonymizing programmers from source code and binaries,

R. Greenstadt and A. Caliskan, “De-anonymizing programmers from source code and binaries,” in DefCon, 2018

2018
[185]

When coding style survives compilation: De-anonymizing programmers from executable binaries,

A. Caliskan, F. Yamaguchi, E. Dauber, R. Harang, K. Rieck, R. Greenstadt, and A. Narayanan, “When coding style survives compilation: De-anonymizing programmers from executable binaries,” arXiv:1512.08546, 2015

2015 arXiv
[186]

Deeplocker: Concealing targeted attacks with ai locksmithing,

D. Kirat, J. Jang, and M. P. Stoecklin, “Deeplocker: Concealing targeted attacks with ai locksmithing,” in BlackHat, 2018

2018
[187]

Lowering the bar: Deep learning for side channel analysis,

G. Perin, B. Ege, and J. V . Woudenberg, “Lowering the bar: Deep learning for side channel analysis,” in BlackHat, 2018

2018
[188]

Video killed the text star: Osint approach,

F. Gomez and C. Jimenez, “Video killed the text star: Osint approach,” in BlackHat, 2018

2018
[189]

Evading next-gen av using a.i

H. Anderson, “Evading next-gen av using a.i.” in DefCon, 2017

2017
[190]

Learning to evade static PE machine learning malware models via reinforcement learning,

H. S. Anderson, A. Kharkar, B. Filar, D. Evans, and P. Roth, “Learning to evade static PE machine learning malware models via reinforcement learning,” arXiv:1801.08917, 2018

2018 arXiv
[191]

Skype & type: Keystroke leakage over voip,

D. Lain, M. Conti, G. Tsudik, and A. Compagno, “Skype & type: Keystroke leakage over voip,” in BlackHat, 2017

2017
[192]

Weaponizing machine learning: Humanity was overrated anyway,

B. Morris and D. Petro, “Weaponizing machine learning: Humanity was overrated anyway,” in DefCon, 2017

2017
[193]

A picture is worth a thousand words, literally: Deep neural networks for social stego,

P. Tully and M. Raggo, “A picture is worth a thousand words, literally: Deep neural networks for social stego,” in DefCon, 2017

2017
[194]

Wire me through machine learning,

A. Singh and V . Thaware, “Wire me through machine learning,” in BlackHat, 2017

2017
[195]

I’m not a human: Breaking the google recaptcha,

I. Polakis and S. Sivakorn, “I’m not a human: Breaking the google recaptcha,” in BlackHat, 2016

2016
[196]

I am robot:(deep) learn- ing to break semantic image captchas,

S. Sivakorn, I. Polakis, and A. D. Keromytis, “I am robot:(deep) learn- ing to break semantic image captchas,” in IEEE European Symposium on Security and Privacy , 2016

2016
[197]

Another brick off the wall: Deconstructing web application firewalls using automata learning,

G. Argyros and I. Stais, “Another brick off the wall: Deconstructing web application firewalls using automata learning,” in BlackHat, 2016

2016
[198]

Sfadiff: Automated evasion attacks and fingerprinting using black- box differential automata learning,

G. Argyros, I. Stais, S. Jana, A. D. Keromytis, and A. Kiayias, “Sfadiff: Automated evasion attacks and fingerprinting using black- box differential automata learning,” in ACM SIGSAC Conference on Computer and Communications Security , 2016

2016
[199]

Weaponizing data science for social engi- neering: Automated e2e spear phishing on twitter,

J. Seymour and P. Tully, “Weaponizing data science for social engi- neering: Automated e2e spear phishing on twitter,” in DefCon, 2016

2016
[200]

Applied machine learning for data exfiltration and other fun topics,

M. Wolff, B. Wallace, S. Researcher, and X. Zhao, “Applied machine learning for data exfiltration and other fun topics,” in BlackHat, 2016

2016
[201]

I am a legend: Hacking hearthstone with machine learning,

C. Bursztein and E. Bursztein, “I am a legend: Hacking hearthstone with machine learning,” in DefCon, 2014

2014
[202]

My google glass sees your passwords!

X. Fu, Q. Yue, and Z. Ling, “My google glass sees your passwords!” in BlackHat, 2014

2014
[203]

Blind recognition of touched keys on mobile devices,

Q. Yue, Z. Ling, X. Fu, B. Liu, K. Ren, and W. Zhao, “Blind recognition of touched keys on mobile devices,” in ACM SIGSAC Conference on Computer and Communications Security , 2014

2014
[204]

Evolving exploits through genetic algorithms,

S. Vanned, “Evolving exploits through genetic algorithms,” in DefCon, 2013

2013
[205]

Using online activity as digital fingerprints to create a better spear phisher,

J. Espinhara and U. Albuquerque, “Using online activity as digital fingerprints to create a better spear phisher,” in BlackHat, 2013. 18

2013
[206]

Hacking desire: Reverse-engineering what people want,

I. Clarke, “Hacking desire: Reverse-engineering what people want,” in DefCon, 2008

2008
[207]

Global cybersecurity outlook,

“Global cybersecurity outlook,” https://www3 .weforum.org/docs/ WEF Global Cybersecurity Outlook 2024.pdf, World Economic Forum, Tech. Rep., 2024

2024
[208]

Keybert: Minimal keyword extraction with bert

M. Grootendorst, “Keybert: Minimal keyword extraction with bert.”
[209]

Bertopic: Neural topic modeling with a class-based tf-idf procedure,

——, “Bertopic: Neural topic modeling with a class-based tf-idf procedure,” arXiv:2203.05794, 2022

2022 arXiv
[210]

Automated website fingerprinting through deep learning,

V . Rimmer, D. Preuveneers, M. Juarez, T. Van Goethem, and W. Joosen, “Automated website fingerprinting through deep learning,” in Network and Distributed Systems Security Symposium , 2018

2018
[211]

Frontier safety framework (v1.0),

A. Dragan, H. King, and A. Dafoe, “Frontier safety framework (v1.0),” Google DeepMind, Tech. Rep., 2024. [Online]. Available: https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/ introducing-the-frontier-safety-framework/fsf-technical-report .pdf

2024
[212]

RSA Conference 2024 goes beyond AI-powered security to securing AI itself,

SiliconAngle, “RSA Conference 2024 goes beyond AI-powered security to securing AI itself,” https://web .archive.org/web/ 20240601053332/https://siliconangle.com/2024/05/11/rsa-conference- 2024-goes-beyond-ai-powered-security-securing-ai/, 2024

2024
[213]

Variation in number of hits for complex searches in google scholar,

W. M. Bramer, “Variation in number of hits for complex searches in google scholar,” Journal of the Medical Library Association , 2016

2016
[214]

99% False Positives: A Qualitative Study of SOC Analysts’ Perspectives on Security Alarms,

B. A. Alahmadi, L. Axon, and I. Martinovic, “99% False Positives: A Qualitative Study of SOC Analysts’ Perspectives on Security Alarms,” in USENIX Security Symposium , 2022

2022
[215]

SoK: Pragmatic Assess- ment of Machine Learning for Network Intrusion Fetection,

G. Apruzzese, P. Laskov, and J. Schneider, “SoK: Pragmatic Assess- ment of Machine Learning for Network Intrusion Fetection,” in IEEE European Symposium on Security and Privacy , 2023

2023
[216]

Everybody’s got ML, tell me what else you have: Practitioners’ perception of ML-based security tools and explanations,

J. Mink, H. Benkraouda, L. Yang, A. Ciptadi, A. Ahmadzadeh, D. V otipka, and G. Wang, “Everybody’s got ML, tell me what else you have: Practitioners’ perception of ML-based security tools and explanations,” in IEEE Symposium on Security and Privacy , 2023

2023
[217]

Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies,

A. Madill, A. Jordan, and C. Shirley, “Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies,” British journal of psychology , 2000

2000
[218]

A review of techniques to detect the GAN- generated fake images,

T. Arora and R. Soni, “A review of techniques to detect the GAN- generated fake images,” Generative Adversarial Networks for Image- to-Image Translation, 2021

2021
[219]

Artificial intelligence (AI) cybersecurity dimensions: a comprehensive framework for understanding adversarial and offensive AI,

M. Malatji and A. Tolah, “Artificial intelligence (AI) cybersecurity dimensions: a comprehensive framework for understanding adversarial and offensive AI,” AI and Ethics , 2024

2024
[220]

Shujun Li’s Bibliography of SoK Papers,

S. Li, “Shujun Li’s Bibliography of SoK Papers,” https:// www.hooklee.com/Research/SoK/SoK.html, 2023

2023
[221]

Sok: The impact of unlabelled data in cyberthreat detection,

G. Apruzzese, P. Laskov, and A. Tastemirova, “Sok: The impact of unlabelled data in cyberthreat detection,” inIEEE European Symposium on Security and Privacy , 2022

2022
[222]

Nettleton, Commercial data mining: processing, analysis and mod- eling for predictive analytics projects

D. Nettleton, Commercial data mining: processing, analysis and mod- eling for predictive analytics projects . Elsevier, 2014

2014
[223]

Producing fake information is getting easier,

A. Gerace, “Producing fake information is getting easier,” Economist,
[224]

The Cyber Kill Chain,

“The Cyber Kill Chain,” https://www .lockheedmartin.com/en-us/ capabilities/cyber/cyber-kill-chain.html
[225]

Membership in- ference attacks against machine learning models,

R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership in- ference attacks against machine learning models,” in IEEE Symposium on Security and Privacy , 2017

2017
[226]

De-fake: Detection and attribution of fake images generated by text-to-image generation models,

Z. Sha, Z. Li, N. Yu, and Y . Zhang, “De-fake: Detection and attribution of fake images generated by text-to-image generation models,” in ACM SIGSAC Conference on Computer and Communications Security, 2023

2023
[227]

Facilitating threat modeling by leveraging large language models,

I. Elsharef, Z. Zeng, and Z. Gu, “Facilitating threat modeling by leveraging large language models,” in Workshop on AI Systems with Confidential Computing, 2024

2024
[228]

Forecasting cyber security threats landscape and associated technical trends in telehealth using bidirectional encoder representations from transformers (bert),

U. H. Govindarajan, D. K. Singh, and H. A. Gohel, “Forecasting cyber security threats landscape and associated technical trends in telehealth using bidirectional encoder representations from transformers (bert),” Computers & Security , 2023

2023
[229]

C-pack: Packaged resources to advance general chinese embedding,

S. Xiao, Z. Liu, P. Zhang, and N. Muennighoff, “C-pack: Packaged resources to advance general chinese embedding,” 2023

2023
[230]

Llama 2: Open foundation and fine-tuned chat models,

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv:2307.09288, 2023

2023 arXiv
[231]

Ethical Requirements,

IEEE, “Ethical Requirements,” https://web .archive.org/web/ 20241202060241/https://books.ieeeauthorcenter.ieee.org/book- publishing-at-ieee/publishing-ethics/ethical-requirements/, 2024

2024
[232]

CRediT author statement,

Elsevier, “CRediT author statement,” https://web .archive.org/web/ 20241130183511/https://www.elsevier.com/researcher/author/policies- and-guidelines/credit-author-statement, 2024

2024
[233]

Scimago Journal & Country Rank,

“Scimago Journal & Country Rank,” https://www .scimagojr.com/
[234]

Proceedings of the 2023 European Interdisciplinary Cybersecurity Conference,

“Proceedings of the 2023 European Interdisciplinary Cybersecurity Conference,” https://dl.acm.org/doi/proceedings/10.1145/3590777

2023 doi
[235]

Midjourney V3 release,

“Midjourney V3 release,” https://en .wikipedia.org/wiki/Midjourney
[236]

Stable Diffusion release,

“Stable Diffusion release,” https://stability .ai/news/stable-diffusion- announcement
[237]

ChatGPT release,

“ChatGPT release,” https://openai .com/index/chatgpt/
[238]

ElevenLabs release,

“ElevenLabs release,” https://elevenlabs .io/blog/elevenlabs-raises- 2m-pre-seed-and-announces-ai-speech-platform-promising-to- revolutionize-audio-storytelling
[239]

Google Bard release,

“Google Bard release,” https://blog .google/technology/ai/bard-google- ai-search-updates/
[240]

Claude launch,

“Claude launch,” https://www .anthropic.com/news/claude-3-family
[241]

Adobe Firefly launch,

“Adobe Firefly launch,” https://news .adobe.com/news/2024/10/ 101424-adobe-launches-firefly-video-model

2024
[242]

OpenAI DALLE3 launch,

“OpenAI DALLE3 launch,” https://openai .com/index/new-models- and-developer-products-announced-at-devday/
[243]

Google Gemini launch,

“Google Gemini launch,” https://blog .google/technology/ai/google- gemini-ai/
[244]

OpenAI Sora announcement,

“OpenAI Sora announcement,” https://openai .com/index/sora/
[245]

Mirai launch,

“Mirai launch,” https://mistral .ai/news/mixtral-of-experts/
[246]

ChatGPT-4o launch,

“ChatGPT-4o launch,” https://openai .com/index/gpt-4o-and-more- tools-to-chatgpt-free/
[247]

GPT-4 launch,

“GPT-4 launch,” https://openai .com/index/gpt-4-research/
[248]

Dos and don’ts of machine learning in computer security,

D. Arp, E. Quiring, F. Pendlebury, A. Warnecke, F. Pierazzi, C. Wress- negger, L. Cavallaro, and K. Rieck, “Dos and don’ts of machine learning in computer security,” in USENIX Security Symposium , 2022

2022
[249]

Why so toxic? measuring and triggering toxic behavior in open-domain chatbots,

W. M. Si, M. Backes, J. Blackburn, E. De Cristofaro, G. Stringhini, S. Zannettou, and Y . Zhang, “Why so toxic? measuring and triggering toxic behavior in open-domain chatbots,” in ACM SIGSAC Conference on Computer and Communications Security , 2022. APPENDIX A CHECKLIST FOR A...

2022
[253]

First, we scrutinize whether the applied algorithm/tech- nique is publicly available (e.g., ChatGPT) and/or can be easily re-used (such as in [98]) If so, we mark the column with a “yes.” Otherwise, if we do not mark the column, this means that the attacker has to develop the ...
[254]

I have little or no knowledge of AI/cyberse- curity

If the algorithm needs to be developed from scratch, we review the availability of training data. We distinguish: • publicly available data ( /desk◎op), e.g., Biesner et al [84] use publicly available data sets of leaked passwords; • data collected by the authors ( /user-cog),...

2024
[255]

One of the greatest problems in this field is the lack of understanding regarding the real potential and limitations of AI on the offensive side

Lack of understanding regarding the potential and lim- itations of Offensive AI tools. One of the greatest problems in this field is the lack of understanding regarding the real potential and limitations of AI on the offensive side. The vast majority of the published research ...
[256]

In some usecases, AI has reached the maturity required to orchestrate cyber attacks that could lead to financial losses (e.g., audio deep fakes)

Lack of Effective Countermeasures. In some usecases, AI has reached the maturity required to orchestrate cyber attacks that could lead to financial losses (e.g., audio deep fakes). However, despite the rapid advancement made by the community on the offensive side, one of the g...
[257]

One of the greatest problems in this field is the inability to determine whether the risk posed to systems by the advancements published by new research is real

Lack of effective TARA standards for systems against Offensive AI. One of the greatest problems in this field is the inability to determine whether the risk posed to systems by the advancements published by new research is real. This happens because the practicality and the re...
[258]

We have already clear evidence that AI can be used to hack human by performing contextualization and personalization in phishing attacks

Cognitive biases and offensive AI. We have already clear evidence that AI can be used to hack human by performing contextualization and personalization in phishing attacks. It is therefore easy to conjecture that offensive AI could exploit cognitive biases of individuals to ac...
[259]

Offensive AI and behavioral economics. The field of behavioral economics pointed out well how cognitive biases strongly affect the financial decisions of individuals and how these decisions deviate from those predicted by classical economic theory. Given the above conjecture t...
[260]

social media took the cost of distribution to zero, and generative AI takes the cost of generation to zero

Human-AI Teaming against Offensive AI exploiting cog- nitive biases. I do believe that as offensive AI will become more sophisticated in exploiting cognitive biases such as confirmation bias and loss aversion, there will be a press- ing need for defence measures that leverage ...
[261]

ground truth

Measuring AI usage in real-world attacks. An impor- tant open question is how to reliably measure the use of offensive AI in real-world attacks. So far, the community has studied a range of theoretically possible AI attacks, but we don’t have sufficient understanding of whethe...
[262]

deepfakes

Defending against AI use in disinformation and online deception. Disinformation is a major threat to our society today, and it is difficult to address this threat with technical means only. Disinformation has been a problem even before the take-off of generative AI (e.g., with...
[263]

Offensive AI has the potential to be used positively to improve our defense

Using offensive AI to enhance existing defense. Offensive AI has the potential to be used positively to improve our defense. A concrete example is to use AI methods (e.g., Large Language Models or “LLM”) to scan software code bases to detect bugs/vulnerabilities and augment tr...
[264]

Detection of AI-based attacks. Detecting ongoing attacks and correctly attributing their source is particularly important for attacks that target humans, such as misinformation or phishing, which rely on AI-generated content. If we can attribute this content to AI in general, ...
[265]

The effectiveness of attacks and defenses is commonly quantified with traditional machine learning met- rics such as precision and recall

Quantification. The effectiveness of attacks and defenses is commonly quantified with traditional machine learning met- rics such as precision and recall. While these metrics are useful to compare the effectiveness of new attacks/defenses with existing attacks/defenses in cont...
[266]

In many offensive AI ap- plications common evaluation practices have been established for the sake of simplicity, reproducibility, and compatibility with prior work

Realistic attack simulation. In many offensive AI ap- plications common evaluation practices have been established for the sake of simplicity, reproducibility, and compatibility with prior work. The consequences are as follows: (i) most of the studies do not or cannot aim for ...
[267]

Offensive GenAI is a novel research direction that presents an unprece- dented level of urgency due to the wide-spread adoption

Threat of offensive generative AI (GenAI). Offensive GenAI is a novel research direction that presents an unprece- dented level of urgency due to the wide-spread adoption. The quality of AI-generated content has recently surpassed any expectations, demonstrating blasting perfo...
[268]

offensive ai

N-grams analysis: As a preliminary check, we process the entire statements and extract the 20 most common bi- grams/trigrams. The results are as follows: as we expected, the most common n-gram is “offensive ai” (61 occurrences); the second most common is “use ai” (11 occurrenc...
[269]

offensive

Keyword Extraction. : We then analyze each statement individually by extracting the most relevant keywords, using KeyBERT [208], a popular text-mining technique (used also, e.g., in [227]). Specifically, KeyBERT takes some text as input, and returns a list of keywords, each pr...

1979
[270]

submitted’ version of this paper for SaTML25; and (iii) assisting in the rebuttal phase of SaTML25, including the preparation of this “revised

Topic Modeling : Finally, we use topic modeling to extract the most relevant “topics” envisaged by our experts. To this end, we rely on BERTopic [209] (used in, e.g., [228]). Specifically, BERTopic takes as input a collection of docu- ments, and returns as output a finite numb...

2020
[271]

The third activity we have carried out is the analysis of InfoSec briefings—which spanned between Jan

We then analyzed the collected responses and derived our codebook (which we knew would be used also later for analyzing the experts’ input). The third activity we have carried out is the analysis of InfoSec briefings—which spanned between Jan. and March
[272]

practical

We posited that these venues could provide a comple- mentary perspective on the “practical” use cases of OAI in the real world—especially given that not many research papers showcased real-world demonstrations of OAI. This procedure was not trivial—despite the existence of a m...

2024
[2020]

Available: https://doi .org/10.5281/zenodo.4461265

[Online]. Available: https://doi .org/10.5281/zenodo.4461265

doi
[2023]

Available: https://www .cnbc.com/2023/11/28/ai-like- chatgpt-is-creating-huge-increase-in-malicious-phishing-email .html

[Online]. Available: https://www .cnbc.com/2023/11/28/ai-like- chatgpt-is-creating-huge-increase-in-malicious-phishing-email .html

2023
[2024]

Available: https://www .economist.com/science-and- technology/2024/05/01/producing-fake-information-is-getting-easier

[Online]. Available: https://www .economist.com/science-and- technology/2024/05/01/producing-fake-information-is-getting-easier

2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Girasa, Artificial Intelligence as a Disruptive Technology: Economic Transformation and Government Regulation

R. Girasa, Artificial Intelligence as a Disruptive Technology: Economic Transformation and Government Regulation . Springer, 2020

2020

[2] [2]

Artificial intelligence as a disruptive technology—a systematic literature review,

V .-D. P˘av˘aloaia and S.-C. Necula, “Artificial intelligence as a disruptive technology—a systematic literature review,” Electronics, 2023

2023

[3] [3]

Artificial intelligence, machine learning and deep learning in advanced robotics, a review,

M. Soori, B. Arezoo, and R. Dastres, “Artificial intelligence, machine learning and deep learning in advanced robotics, a review,” Cognitive Robotics, 2023. 14

2023

[4] [4]

Machine learning and ai in business intelligence: Trends and opportunities,

J. P. Bharadiya, “Machine learning and ai in business intelligence: Trends and opportunities,” International Journal of Computer , 2023

2023

[5] [5]

What factors contribute to the acceptance of artificial intelligence? a systematic review,

S. Kelly, S.-A. Kaye, and O. Oviedo-Trespalacios, “What factors contribute to the acceptance of artificial intelligence? a systematic review,” Telematics and Informatics, 2023

2023

[6] [6]

The role of machine learning in cybersecurity,

G. Apruzzese, P. Laskov, E. Montes de Oca, W. Mallouli, L. Brdalo Rapa, A. V . Grammatopoulos, and F. Di Franco, “The role of machine learning in cybersecurity,” ACM Digital Threats: Research and Practice, 2023

2023

[7] [7]

Artificial intelligence and national security,

K. M. Sayler, “Artificial intelligence and national security,” Congres- sional Research Service , 2020

2020

[8] [8]

Trusting artificial intel- ligence in cybersecurity is a double-edged sword,

M. Taddeo, T. McCutcheon, and L. Floridi, “Trusting artificial intel- ligence in cybersecurity is a double-edged sword,” Nature Machine Intelligence, 2019

2019