REVIEW 5 cited by
Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Generative, multimodal artificial intelligence (GenAI) offers transformative potential across industries, but its misuse poses significant risks. Prior research has shed light on the potential of advanced AI systems to be exploited for malicious purposes. However, we still lack a concrete understanding of how GenAI models are specifically exploited or abused in practice, including the tactics employed to inflict harm. In this paper, we present a taxonomy of GenAI misuse tactics, informed by existing academic literature and a qualitative analysis of approximately 200 observed incidents of misuse reported between January 2023 and March 2024. Through this analysis, we illuminate key and novel patterns in misuse during this time period, including potential motivations, strategies, and how attackers leverage and abuse system capabilities across modalities (e.g. image, text, audio, video) in the wild.
Forward citations
Cited by 5 Pith papers
-
Can We End the Cat-and-Mouse Game? Simulating Self-Evolving Phishing Attacks with LLMs and Genetic Algorithms
A closed-loop LLM simulation with genetic algorithms suggests that phishing strategies can evolve to bypass simulated victims' defenses, but the result has not been validated against real humans.
-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.
-
Model Immunization from a Condition Number Perspective
A new regularizer increases the condition number of the linear-probing Hessian on harmful tasks, making gradient-descent fine-tuning slower, but the theoretical analysis contains a false claim.
-
Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI
A qualitative study of 21 U.S. GenAI users reveals that existing security and privacy transparency is perceived as ineffective and lacking credibility, leading users to rely on proxies like popularity and constraining...
-
The Ethics of Generative AI in Anonymous Spaces: A Case Study of 4chan's /pol/ Board
A case study of 66 AI-generated images from 4chan's /pol/ board finds 28.8% contain racist content and 28.8% anti-Semitic content, but the sample is small and likely skewed.
Discussion (0). Sign in to comment.