REVIEW 13 cited by
EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Jailbreak attacks are crucial for identifying and mitigating the security vulnerabilities of Large Language Models (LLMs). They are designed to bypass safeguards and elicit prohibited outputs. However, due to significant differences among various jailbreak methods, there is no standard implementation framework available for the community, which limits comprehensive security evaluations. This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against LLMs. It builds jailbreak attacks using four components: Selector, Mutator, Constraint, and Evaluator. This modular framework enables researchers to easily construct attacks from combinations of novel and existing components. So far, EasyJailbreak supports 11 distinct jailbreak methods and facilitates the security validation of a broad spectrum of LLMs. Our validation across 10 distinct LLMs reveals a significant vulnerability, with an average breach probability of 60% under various jailbreaking attacks. Notably, even advanced models like GPT-3.5-Turbo and GPT-4 exhibit average Attack Success Rates (ASR) of 57% and 33%, respectively. We have released a wealth of resources for researchers, including a web platform, PyPI published package, screencast video, and experimental outputs.
Forward citations
Cited by 13 Pith papers
-
IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement
IntentionReasoner adds four-level intent classification and targeted query rewriting to LLM guardrails, reporting state-of-the-art harm detection and near-zero jailbreak success.
-
Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks
IMAGINE pre-synthesizes intent-concealed jailbreak-like instructions via iterative latent-space expansion, and DPO with that data reduces jailbreak attack success rates on Qwen2.5, Llama3.1 and Llama3.2.
-
PUZZLED: Jailbreaking LLMs through Word-Based Puzzles
PUZZLED masks harmful keywords as word-search, anagram, or crossword puzzles and achieves a reported 88.8% average attack success rate across five leading LLMs.
-
DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification
Targeted attention-head modification, tuned on jailbreak data, lowers attack success rates across models and attacks without fine-tuning.
-
Lexical Hints of Accuracy in LLM Reasoning Chains
Hesitation words in reasoning chains are claimed to flag incorrect LLM answers, but the manuscript body is a different paper and contains no such study.
-
Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignment
LARF ranks fine-tuning samples by how close their hidden representations lie to unsafe versus safe reference responses, and removing the top-ranked samples preserves safety alignment.
-
Safety Features for a Centralised AGI Project
A policy proposal for seven safety features, including bottom-up pause authority, congressional-chartered board oversight, risk monitoring, and verification technology, to reduce catastrophic risks in a centralized US...
-
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense
Layer-AdvPatcher edits 'toxic' transformer layers using self-generated harmful examples to block jailbreaks, but its reported attack-success rates worsen on several benchmarks.
-
SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Models
Across 510 HarmBench behaviors and seven attack methods, GPT-4 models show more consistent jailbreak resilience than DeepSeek models, whose vulnerability grows with scale.
-
PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
PandaBench, a 3-billion-token benchmark built on PandaGuard, evaluates 49 LLMs under 19 attacks and 12 defenses and finds persistent model vulnerabilities, defense trade-offs, and low judge agreement.
-
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
A structured survey of LLM safety evaluation that proposes a why/what/where/how taxonomy and catalogs metrics, datasets, benchmarks, evaluators, and frameworks.
-
100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models
A survey of DeepSeek-R1 replication studies covering SFT, RLVR, algorithm variants, reward design, and future directions.
-
Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspective
A structured survey of jailbreak prompts and layered defenses for large language models, with six illustrative case studies and no empirical evaluation.
Discussion (0). Continue with ORCID to comment.