REVIEW 21 cited by
Attacking Large Language Models with Projected Gradient Descent
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Current LLM alignment methods are readily broken through specifically crafted adversarial prompts. While crafting adversarial prompts using discrete optimization is highly effective, such attacks typically use more than 100,000 LLM calls. This high computational cost makes them unsuitable for, e.g., quantitative analyses and adversarial training. To remedy this, we revisit Projected Gradient Descent (PGD) on the continuously relaxed input prompt. Although previous attempts with ordinary gradient-based attacks largely failed, we show that carefully controlling the error introduced by the continuous relaxation tremendously boosts their efficacy. Our PGD for LLMs is up to one order of magnitude faster than state-of-the-art discrete optimization to achieve the same devastating attack results.
Forward citations
Cited by 21 Pith papers
-
On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
Pre-LayerNorm transformers and linear attention are almost always surjective, so any target output has an input that produces it in the continuous embedding space.
-
Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Prompts
Adversarial images optimized to map harmless text prefixes to toxic tokens jailbreak vision-language models more effectively than continuing toxic text.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generation
PromptSan reduces NSFW image generation by sanitizing input prompts through classifier-guided token edits or a learned safety suffix, with quantitative validation limited to nudity on the I2P benchmark.
-
FORTRESS: Frontier Risk Evaluation for National Security and Public Safety
A new benchmark with instance-specific rubrics measures frontier LLMs' willingness to assist with national security and public safety threats, alongside a paired over-refusal test.
-
SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
A security-aware prompt compressor that reveals the hidden intent of jailbreak prompts and injects it into the system prompt reduces average attack success from 35% to 1% with negligible overhead.
-
Lifelong Safety Alignment for Language Models
A co-evolutionary attacker-defender loop, warmed up by strategies extracted from jailbreak papers, reduces jailbreak success rate on a robust model from 73% to 7% in two iterations.
-
OET: Optimization-based prompt injection Evaluation Toolkit
OET is an optimization-based evaluation toolkit that benchmarks prompt injection attacks and defenses across eight datasets and shows current defenses remain vulnerable in several domains.
-
DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak
A diffusion-based prompt rewriter that pushes rewritten prompts toward harmless regions of a target model's hidden states achieves higher jailbreak success than existing suffix and template attacks.
-
Improving the Robustness of the Projected Gradient Descent Method for Nonlinear Constrained Optimization Problems in Topology Optimization
An enhanced inertial projected gradient descent algorithm with Schur-complement projection and constraint-aware step splitting matches or exceeds MMA on a 3D heat-sink topology optimization problem with an overhang co...
-
Rethinking the Intermediate Features in Adversarial Attacks: Misleading Robotic Models via Adversarial Distillation
An adversarial text prefix optimized on continuous action and self-attention features misleads VIMA language-conditioned robotic policies across 13 manipulation tasks, beating GCG-based baselines.
-
Position: It's Time to Optimize LLMs for Self-Consistency
The paper proposes self-consistency, a mathematical framework that treats relationships between model outputs across related inputs as the primary training target, unifying many existing alignment and robustness methods.
-
Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.
-
MPLinker: Multi-template Prompt-tuning with Adversarial Training for Issue-commit Link Recovery
MPLinker reframes issue-commit link recovery as a masked-language-model cloze task with multi-template averaging and adversarial training, reporting an average F1 of 96.10% on six projects.
-
Trojan Detection Through Pattern Recognition for Large Language Models
A logits-only, black-box pipeline with token filtration, greedy or beam-search trigger inversion, and perturbation-based verification detects Trojan triggers in TrojAI and RLHF-poisoned LLMs.
-
Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks
CoopGuard's defer-tempt-analyze-coordinate agents cut reported jailbreak success and raise attacker token costs on the new EMRA benchmark, but the deceptive-rate metric is partly defined by the paper's own scoring rubric.
-
SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Models
Across 510 HarmBench behaviors and seven attack methods, GPT-4 models show more consistent jailbreak resilience than DeepSeek models, whose vulnerability grows with scale.
-
Edge-Based Learning for Improved Classification Under Adversarial Noise
Training on Canny edge maps improves robustness to FGSM adversarial noise compared to training on raw images, though retraining on clean plus noisy raw images recovers more accuracy.
-
Safety Reasoning with Guidelines
Training LLMs to reason through explicit safety guidelines reduces out-of-distribution jailbreak success rates compared to standard refusal training.
-
On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMs
Applying CVSS, DREAD, OWASP, and SSVC to 56 adversarial LLM attacks via three LLM judges yields near-constant factor scores, which the authors take as evidence that these metrics cannot differentiate LLM attacks.
-
LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Seconds
LIAR shows that best-of-N sampling of suffixes from a GPT-2 model jailbreaks several aligned LLMs with low-perplexity prompts and far faster time-to-attack than training-based attacks.
Discussion (0). Continue with ORCID to comment.