REVIEW 11 cited by
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies. Limited studies have been conducted to formalize and analyze these attacks and their mitigations. We bridge this gap by proposing a formalism and a taxonomy of known (and possible) jailbreaks. We survey existing jailbreak methods and their effectiveness on open-source and commercial LLMs (such as GPT-based models, OPT, BLOOM, and FLAN-T5-XXL). We further discuss the challenges of jailbreak detection in terms of their effectiveness against known attacks. For further analysis, we release a dataset of model outputs across 3700 jailbreak prompts over 4 tasks.
Forward citations
Cited by 11 Pith papers
-
Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interface
Attackers can use the loss signal from a remote LLM fine-tuning API to optimize adversarial prefix and suffix tokens, turning existing prompt injections into high-success attacks on closed-weight Gemini models.
-
AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Models
Obfuscation alone reaches 76% success against intent-aware defenses; pairing it with emotional manipulation yields 97.6% success under a modeled composite evaluation of 250 prompts.
-
Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels
Indirect prompt injection through ads, webviews, and notifications reliably diverts mobile LLM agents into leaking data and installing malware across eight evaluated agents.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring
JADES judges jailbreak success by decomposing harmful prompts into weighted sub-questions and scoring each part, claiming 98.5% human agreement and showing prior attack success rates are inflated.
-
Agents Are All You Need for LLM Unlearning
A four-agent pipeline, Vanilla, AuditErase, Critic, and Composer, filters target references out of LLM responses, claiming robust and scalable inference-time unlearning without weight updates.
-
On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
A PRISMA-based survey of 85 papers shows agentic LLM security research is attack-heavy and perception-focused, leaving action-layer and code-execution risks understudied.
-
PromptShield: Deployable Detection for Prompt Injection Attacks
PromptShield reports a 65.3% true positive rate at 0.1% false positive rate for prompt injection detection, more than six times the best prior model, on its own out-of-distribution evaluation split.
-
Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration
Injecting a fine-tuned BERT classifier's category label into LLM prompts improves accuracy on a 150-question automotive jailbreak benchmark, but the evaluation is self-referential and lacks external validation.
-
Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs
A survey categorizing prompt-based attacks on LLMs into four classes and proposing aspirational goals of un-distillable, un-finetunable, and un-editable models.
-
SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation
A systematization-of-knowledge survey that categorizes LLM privacy risks into training data, prompts, outputs, and agents, and reviews limitations of current mitigations.
Discussion (0). Continue with ORCID to comment.