REVIEW 28 cited by
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Safety alignment of Large Language Models (LLMs) can be compromised with manual jailbreak attacks and (automatic) adversarial attacks. Recent studies suggest that defending against these attacks is possible: adversarial attacks generate unlimited but unreadable gibberish prompts, detectable by perplexity-based filters; manual jailbreak attacks craft readable prompts, but their limited number due to the necessity of human creativity allows for easy blocking. In this paper, we show that these solutions may be too optimistic. We introduce AutoDAN, an interpretable, gradient-based adversarial attack that merges the strengths of both attack types. Guided by the dual goals of jailbreak and readability, AutoDAN optimizes and generates tokens one by one from left to right, resulting in readable prompts that bypass perplexity filters while maintaining high attack success rates. Notably, these prompts, generated from scratch using gradients, are interpretable and diverse, with emerging strategies commonly seen in manual jailbreak attacks. They also generalize to unforeseen harmful behaviors and transfer to black-box LLMs better than their unreadable counterparts when using limited training data or a single proxy model. Furthermore, we show the versatility of AutoDAN by automatically leaking system prompts using a customized objective. Our work offers a new way to red-team LLMs and understand jailbreak mechanisms via interpretability.
Forward citations
Cited by 28 Pith papers
-
OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents
OTora provides the first unified framework for reasoning-level denial-of-service attacks on LLM agents, achieving up to 10x more reasoning tokens and order-of-magnitude latency increases while preserving task accuracy...
-
Agent Security Needs Redefinition through a Holistic Framework
Agent security should be redefined around four contextual authorization properties instead of the content of the action performed.
-
DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail
A 4B LLM safety guardrail trained with reasoning supervision but deployed with reasoning-free inference outperforms 8B baselines on safety benchmarks.
-
AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Models
Obfuscation alone reaches 76% success against intent-aware defenses; pairing it with emotional manipulation yields 97.6% success under a modeled composite evaluation of 250 prompts.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization
RAF, a two-stage token-optimization attack, creates brief natural-sounding text injections that reliably boost a target item's rank in LLM reranking outputs, beating state-of-the-art baselines in effectiveness, stealt...
-
Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
Coding agents jailbroken with simple prompts produced executable malicious code in 27–32% of attempts, and single/multi-file scaffolds drove compliance to roughly 100% for frontier models.
-
Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World
Adversarial audio noise, optimized with audio augmentations, can trigger and distort the behavior of audio-based LLMs both digitally and when played through the air.
-
Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking
Across 2M+ in-the-wild LLM conversations, jailbreak attempts show no higher complexity than normal chats, and assistant toxicity has declined over time, suggesting bounded attack sophistication.
-
Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space
A component-based, genetically optimized jailbreak framework reports over 90% success on Claude-3.5 and strong cross-model transferability.
-
GloSS over Toxicity: Understanding and Mitigating Toxicity in LLMs via Global Toxic Subspace
Detoxifying LLMs by deleting a global, cross-layer 'toxic subspace' from feed-forward weights reduces toxic outputs more than layer-local subspace methods.
-
JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation
JBShield detects jailbreaks by checking whether a prompt activates both a toxic concept and a jailbreak concept inside an LLM, then steers those concepts to produce a safe refusal.
-
Rerouting LLM Routers
Adversarially optimized, query-independent token prefixes can reroute nearly all queries to the expensive strong model in both open-source and commercial LLM routers.
-
DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak
A diffusion-based prompt rewriter that pushes rewritten prompts toward harmless regions of a target model's hidden states achieves higher jailbreak success than existing suffix and template attacks.
-
No Free Lunch for Defending Against Prefilling Attack by In-Context Learning
Adversative in-context examples ('Sure... However...') defend many LLMs against prefilling jailbreaks but cause over-refusal, so the defense trades safety for helpfulness.
-
From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection
A two-stage adversarial tool injection attack achieves up to 91.67% privacy theft and 100% denial-of-service and unscheduled tool-calling success across several LLM tool-calling systems.
-
A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation
Introduces a multi-role red teaming framework using attacker and jury models that increases attack success rates by up to 7.9% on LLM faithfulness in question-answering tasks.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
Innocence in the Crossfire: Roles of Skip Connections in Jailbreaking Visual Language Models
The paper reports higher harmful-output rates in three open-source VLMs from detailed image descriptions, in-context examples, and positive openings, and from a skip connection between internal layers, with memes riva...
-
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
JailFlipBench and JailFlip attacks show that leading LLMs can be made to answer benign-looking questions with plausible but factually wrong and dangerous responses.
-
BadNAVer: Exploring Jailbreak Attacks On Vision-and-Language Navigation
Prompt-level jailbreak attacks against MLLM-driven vision-and-language navigation agents achieve high success rates in Matterport3D and on a physical robot.
-
KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMs
A distilled open-source attacker, KDA, imitates three jailbreak methods to write diverse attack prompts, and reports higher success and efficiency than each teacher.
-
SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Models
Across 510 HarmBench behaviors and seven attack methods, GPT-4 models show more consistent jailbreak resilience than DeepSeek models, whose vulnerability grows with scale.
-
Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attacks
Greedy Coordinate Gradient-optimized suffixes appended to one candidate answer flip Qwen2.5-3B and Falcon3-3B judge verdicts in over 30% of MT-Bench pairwise comparisons.
-
Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration
Injecting a fine-tuned BERT classifier's category label into LLM prompts improves accuracy on a 150-question automotive jailbreak benchmark, but the evaluation is self-referential and lacks external validation.
-
Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context
Human-readable adversarial insertions placed inside movie-summary prompts can jailbreak several open and closed LLMs, but the paper's measured attack rates are not statistically supported.
-
From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Models
The paper argues hallucinations and jailbreaks share the same optimization dynamics and shows that defenses for one also reduce the other, but the theoretical support is largely circular.
-
Large Language Model Safety: A Holistic Survey
A broad survey of LLM safety that groups the literature into four risk areas and four related areas, with a taxonomy and a public repository of papers, but no new empirical results.
Discussion (0). Continue with ORCID to comment.