REVIEW 14 cited by
AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As large language models (LLMs) become increasingly prevalent and integrated into autonomous systems, ensuring their safety is imperative. Despite significant strides toward safety alignment, recent work GCG~\citep{zou2023universal} proposes a discrete token optimization algorithm and selects the single suffix with the lowest loss to successfully jailbreak aligned LLMs. In this work, we first discuss the drawbacks of solely picking the suffix with the lowest loss during GCG optimization for jailbreaking and uncover the missed successful suffixes during the intermediate steps. Moreover, we utilize those successful suffixes as training data to learn a generative model, named AmpleGCG, which captures the distribution of adversarial suffixes given a harmful query and enables the rapid generation of hundreds of suffixes for any harmful queries in seconds. AmpleGCG achieves near 100\% attack success rate (ASR) on two aligned LLMs (Llama-2-7B-chat and Vicuna-7B), surpassing two strongest attack baselines. More interestingly, AmpleGCG also transfers seamlessly to attack different models, including closed-source LLMs, achieving a 99\% ASR on the latest GPT-3.5. To summarize, our work amplifies the impact of GCG by training a generative model of adversarial suffixes that is universal to any harmful queries and transferable from attacking open-source LLMs to closed-source LLMs. In addition, it can generate 200 adversarial suffixes for one harmful query in only 4 seconds, rendering it more challenging to defend.
Forward citations
Cited by 14 Pith papers
-
Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors
Eyes-on-Me makes RAG data poisoning reusable: a transferable attention-steering attractor is optimized once, then combined with different attack payloads at near-zero cost.
-
On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
Pre-LayerNorm transformers and linear attention are almost always surjective, so any target output has an input that produces it in the continuous embedding space.
-
GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models
GhostPrompt is a universal adversarial text suffix that, after one optimization, steers VLMs to attacker-chosen outputs across diverse unseen images, reporting >30% ASR gains over prior prompt attacks.
-
Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Per-bias selection of a cross-family LLM auditor lifts biased-judgment accuracy from 0.805/0.824 baselines to 0.884.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models
CAMO hides harmful instructions across text and image using masked keywords and math-puzzle clues, making several LVLMs answer banned queries while evading common safety filters.
-
Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning
A three-stage RL framework (cold start, diversity warm-up, curriculum jailbreak) trains a 7B red-team model that reports SOTA jailbreak ASR and diversity on HarmBench, though the evaluation is compromised by training-...
-
SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks
SafeLLM detects unsafe outputs, traces them to specific feedforward-network components, and applies constrained optimization to unlearn harmful generation while preserving general capability.
-
Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning
A jailbreak defense that reasons about hidden manipulations in attack prompts, trained with supervised fine-tuning plus entropy-guided reinforcement learning, generalizes to attacks never seen in training.
-
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
JailFlipBench and JailFlip attacks show that leading LLMs can be made to answer benign-looking questions with plausible but factually wrong and dangerous responses.
-
One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs
ArrAttack fine-tunes a judge on the SmoothLLM defense, uses it to filter rewriting-attack data, and trains a generator that produces jailbreak prompts transferring across defenses.
-
Fast Proxies for LLM Robustness Evaluation
Simple prompt-based and embedding-space attacks predict, with rank correlations up to 0.94, how open-source LLMs fare against a six-attack red-teaming ensemble, at roughly one thousandth of the compute.
-
Adversarial Preference Learning for Robust LLM Alignment
APL iteratively trains an attacker to generate adversarial prompt rewrites and a defender to resist them, using the defender's own preference probabilities as the attack signal.
-
The Resurgence of GCG Adversarial Attacks on Large Language Models
The paper reports that GCG attack success drops with model size, prefix-based metrics overestimate harm, and coding prompts are more vulnerable, but these findings rest on a small model set and a GPT-4o pipeline that ...
Discussion (0). Sign in to comment.