pith:PQSCZURR
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
JailbreakBench supplies an open repository of adversarial prompts, a 100-behavior dataset, a fixed evaluation framework, and a public leaderboard to make jailbreak comparisons reproducible across models.
arxiv:2404.01318 v5 · 2024-03-28 · cs.CR · cs.LG
Add to your LaTeX paper
\usepackage{pith}
\pithnumber{PQSCZURRQSYBR5I56YTFM325O6}
Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge
Record completeness
Claims
To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (2) a jailbreaking dataset comprising 100 behaviors; (3) a standardized evaluation framework; and (4) a leaderboard.
That the selected 100 behaviors, threat model, system prompts, and scoring functions sufficiently capture real-world jailbreaking risks and success without introducing systematic bias in evaluation.
JailbreakBench supplies an evolving set of jailbreak prompts, a 100-behavior dataset aligned with usage policies, a standardized evaluation framework, and a leaderboard to enable comparable assessments of attacks and defenses on LLMs.
References
Formal links
Cited by
Receipt and verification
| First computed | 2026-05-17T23:38:53.302991Z |
|---|---|
| Builder | pith-number-builder-2026-05-17-v1 |
| Signature | Pith Ed25519
(pith-v1-2026-05) · public key |
| Schema | pith-number/v1.0 |
Canonical hash
7c242cd23184b018f51df626566f5d77a643f2c1653587b310e29909b38bfe48
Aliases
· · · · ·Agent API
Verify this Pith Number yourself
curl -sH 'Accept: application/ld+json' https://pith.science/pith/PQSCZURRQSYBR5I56YTFM325O6 \
| jq -c '.canonical_record' \
| python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 7c242cd23184b018f51df626566f5d77a643f2c1653587b310e29909b38bfe48
Canonical record JSON
{
"metadata": {
"abstract_canon_sha256": "705445e4a0882b24468b88e0d56f75d406be1010542225a2e711fbe7e30a8ec4",
"cross_cats_sorted": [
"cs.LG"
],
"license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
"primary_cat": "cs.CR",
"submitted_at": "2024-03-28T02:44:02Z",
"title_canon_sha256": "567337dfe199e91f48ff0e9b7da157d811d7b1d3e7f9d6d2aef3b5f19080f0e0"
},
"schema_version": "1.0",
"source": {
"id": "2404.01318",
"kind": "arxiv",
"version": 5
}
}