REVIEW 8 cited by
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent explorations with commercial Large Language Models (LLMs) have shown that non-expert users can jailbreak LLMs by simply manipulating their prompts; resulting in degenerate output behavior, privacy and security breaches, offensive outputs, and violations of content regulator policies. Limited studies have been conducted to formalize and analyze these attacks and their mitigations. We bridge this gap by proposing a formalism and a taxonomy of known (and possible) jailbreaks. We survey existing jailbreak methods and their effectiveness on open-source and commercial LLMs (such as GPT-based models, OPT, BLOOM, and FLAN-T5-XXL). We further discuss the challenges of jailbreak detection in terms of their effectiveness against known attacks. For further analysis, we release a dataset of model outputs across 3700 jailbreak prompts over 4 tasks.
Forward citations
Cited by 8 Pith papers
-
AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Models
Obfuscation alone reaches 76% success against intent-aware defenses; pairing it with emotional manipulation yields 97.6% success under a modeled composite evaluation of 250 prompts.
-
Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels
Indirect prompt injection through ads, webviews, and notifications reliably diverts mobile LLM agents into leaking data and installing malware across eight evaluated agents.
-
SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses
A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.
-
JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring
JADES judges jailbreak success by decomposing harmful prompts into weighted sub-questions and scoring each part, claiming 98.5% human agreement and showing prior attack success rates are inflated.
-
Runtime-Structured Task Decomposition for Agentic Coding Systems
Runtime-structured task decomposition reduces retry costs in agentic coding systems by up to 51.7% versus monolithic prompts by rerunning only failed subtasks on two software engineering workloads.
-
TrustLLM: Trustworthiness in Large Language Models
TrustLLM defines eight trustworthiness principles, creates a six-dimension benchmark, and evaluates 16 LLMs showing proprietary models generally lead but some open-source ones are close while over-calibration can hurt...
-
Jailbreak Attacks and Defenses Against Large Language Models: A Survey
A survey that creates taxonomies for jailbreak attacks and defenses on LLMs, subdivides them into sub-classes, and compares evaluation approaches.
-
Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs
A survey categorizing prompt-based attacks on LLMs into four classes and proposing aspirational goals of un-distillable, un-finetunable, and un-editable models.
Discussion (0). Sign in to comment.