REVIEW 24 cited by
AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As large language models (LLMs) become increasingly prevalent and integrated into autonomous systems, ensuring their safety is imperative. Despite significant strides toward safety alignment, recent work GCG~\citep{zou2023universal} proposes a discrete token optimization algorithm and selects the single suffix with the lowest loss to successfully jailbreak aligned LLMs. In this work, we first discuss the drawbacks of solely picking the suffix with the lowest loss during GCG optimization for jailbreaking and uncover the missed successful suffixes during the intermediate steps. Moreover, we utilize those successful suffixes as training data to learn a generative model, named AmpleGCG, which captures the distribution of adversarial suffixes given a harmful query and enables the rapid generation of hundreds of suffixes for any harmful queries in seconds. AmpleGCG achieves near 100\% attack success rate (ASR) on two aligned LLMs (Llama-2-7B-chat and Vicuna-7B), surpassing two strongest attack baselines. More interestingly, AmpleGCG also transfers seamlessly to attack different models, including closed-source LLMs, achieving a 99\% ASR on the latest GPT-3.5. To summarize, our work amplifies the impact of GCG by training a generative model of adversarial suffixes that is universal to any harmful queries and transferable from attacking open-source LLMs to closed-source LLMs. In addition, it can generate 200 adversarial suffixes for one harmful query in only 4 seconds, rendering it more challenging to defend.
Forward citations
Cited by 24 Pith papers
-
Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors
Eyes-on-Me makes RAG data poisoning reusable: a transferable attention-steering attractor is optimized once, then combined with different attack payloads at near-zero cost.
-
On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
Pre-LayerNorm transformers and linear attention are almost always surjective, so any target output has an input that produces it in the continuous embedding space.
-
GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models
GhostPrompt is a universal adversarial text suffix that, after one optimization, steers VLMs to attacker-chosen outputs across diverse unseen images, reporting >30% ASR gains over prior prompt attacks.
-
Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Per-bias selection of a cross-family LLM auditor lifts biased-judgment accuracy from 0.805/0.824 baselines to 0.884.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models
CAMO hides harmful instructions across text and image using masked keywords and math-puzzle clues, making several LVLMs answer banned queries while evading common safety filters.
-
Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning
A three-stage RL framework (cold start, diversity warm-up, curriculum jailbreak) trains a 7B red-team model that reports SOTA jailbreak ASR and diversity on HarmBench, though the evaluation is compromised by training-...
-
Adversarial Suffix Filtering: a Defense Pipeline for LLMs
ASF detects and removes adversarial suffixes by segmenting prompts and classifying each segment with a fine-tuned BERT model, reducing jailbreak success on several LLMs while leaving normal-task performance roughly intact.
-
Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions
Speak Easy, a simple multi-step and multilingual jailbreak framework, raises attack success by 0.319 and harmfulness by 0.426 on average across three LLMs and four safety benchmarks.
-
Adversarial Reasoning at Jailbreaking Time
A loss-guided 'reason, verify, search' loop with three LLM modules surpasses prior semantic jailbreak methods on several defended models.
-
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Universal adversarial suffixes can shift any text's embedding toward a model's biased mean direction, breaking embedding-based LLM safety classifiers.
-
Rethinking the Intermediate Features in Adversarial Attacks: Misleading Robotic Models via Adversarial Distillation
An adversarial text prefix optimized on continuous action and self-attention features misleads VIMA language-conditioned robotic policies across 13 manipulation tasks, beating GCG-based baselines.
-
SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks
SafeLLM detects unsafe outputs, traces them to specific feedforward-network components, and applies constrained optimization to unlearn harmful generation while preserving general capability.
-
Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning
A jailbreak defense that reasons about hidden manipulations in attack prompts, trained with supervised fine-tuning plus entropy-guided reinforcement learning, generalizes to attacks never seen in training.
-
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
JailFlipBench and JailFlip attacks show that leading LLMs can be made to answer benign-looking questions with plausible but factually wrong and dangerous responses.
-
One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs
ArrAttack fine-tunes a judge on the SmoothLLM defense, uses it to filter rewriting-attack data, and trains a generator that produces jailbreak prompts transferring across defenses.
-
Adversarial Attack on Large Language Models using Exponentiated Gradient Descent
Exponentiated gradient descent over relaxed one-hot token encodings finds adversarial suffixes that jailbreak several open-source LLMs with higher success rate and lower runtime than GCG, PGD, and SoftPromptThreats.
-
Fast Proxies for LLM Robustness Evaluation
Simple prompt-based and embedding-space attacks predict, with rank correlations up to 0.94, how open-source LLMs fare against a six-attack red-teaming ensemble, at roughly one thousandth of the compute.
-
KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMs
A distilled open-source attacker, KDA, imitates three jailbreak methods to write diverse attack prompts, and reports higher success and efficiency than each teacher.
-
Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attacks
LATPC combines variance-based selection of refusal features for adversarial training with an inference-time embedding calibrator, reducing jailbreak success while curbing over-refusal across several LLMs.
-
Adversarial Preference Learning for Robust LLM Alignment
APL iteratively trains an attacker to generate adversarial prompt rewrites and a defender to resist them, using the defender's own preference probabilities as the attack signal.
-
LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Seconds
LIAR shows that best-of-N sampling of suffixes from a GPT-2 model jailbreaks several aligned LLMs with low-perplexity prompts and far faster time-to-attack than training-based attacks.
-
The Resurgence of GCG Adversarial Attacks on Large Language Models
The paper reports that GCG attack success drops with model size, prefix-based metrics overestimate harm, and coding prompts are more vulnerable, but these findings rest on a small model set and a GPT-4o pipeline that ...
-
Large Language Model Safety: A Holistic Survey
A broad survey of LLM safety that groups the literature into four risk areas and four related areas, with a taxonomy and a public repository of papers, but no new empirical results.
Discussion (0). Continue with ORCID to comment.