Pith. sign in

REVIEW 6 major objections 7 minor 39 references

Jailbreak defenses rarely improve LLM capability; they trade safety for usability and cost in strategy-specific ways.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 16:01 UTC pith:3X2WPOK6

load-bearing objection Solid joint benchmark of defense side-effects by strategy; the safety/over-refusal rankings rest too heavily on an under-validated LLM judge. the 6 major comments →

arxiv 2607.24392 v1 pith:3X2WPOK6 submitted 2026-07-27 cs.CR cs.LG

When LLM Defenses Backfire: Characterizing Safety, Performance, and Cost Trade-offs

Tong Zhang , Zexin Li , Simin Chen , Yun Peng This is my paper
classification cs.CR cs.LG
keywords LLM jailbreak defensessafety-utility trade-offsover-refusalinference coststrategy taxonomyperformance impactdeployment constraints
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Jailbreak defenses protect large language models from harmful prompts, but they often quietly damage ordinary usefulness. This paper sorts eleven common defenses into four operational families and measures three side effects together: drop in task accuracy, over-refusal of safe requests, and extra inference cost. Across six open-source models and five standard benchmarks, defenses almost never raise downstream performance. Instead, rule-based filters best preserve reasoning, highly conservative self-reflection methods reject too many benign queries, and multi-round perturbation methods inflate tokens, latency, and energy the most. The practical point is that safety gains alone do not decide deployability; operators need a joint map of safety, usability, and cost so they can pick a defense that fits the actual constraint of their setting.

Core claim

Defenses rarely improve downstream capability. They instead trade safety gains against usability and efficiency in ways that track operational strategy: rule-based checks best preserve task performance, highly conservative self-reflective defenses often raise over-refusal on safe prompts, and multi-round defenses impose the largest runtime overhead.

What carries the argument

A four-family strategy taxonomy (Rule-Based Check, Output Reflection, Input Reflection & Alignment, Perturbation & Multi-Round) that groups defenses by dominant operational mechanism rather than by name, then jointly scores each family on performance impact, over-refusal, and inference cost.

Load-bearing premise

The safety and over-refusal rankings rest on an automated judge’s labels of full compliance, full refusal, and partial refusal, checked manually on only a small sample.

What would settle it

Re-run the same unsafe and safe prompt sets with human labels or a different independent judge; if the relative ordering of Self-Defend, PPL, SmoothLLM, and Self-Paraphrase on acceptance and over-refusal rates reverses, the strategy-to-side-effect map fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • General-purpose high-throughput assistants should default to lightweight rule-based filters when reasoning quality and latency matter most.
  • Balanced consumer apps can prefer output cross-reflection checkers that improve screening without multi-round latency spikes.
  • High-stakes compliance settings may accept conservative self-reflection only if they consciously tolerate higher refusal of benign queries and weaker task accuracy.
  • Adversarial environments that need multi-round robustness must budget for large token, latency, and energy overhead.
  • Future defense design and reporting should treat performance impact, over-refusal, and inference cost as first-class metrics alongside attack blocking.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Vendors that publish only attack-success reductions without joint utility and cost numbers will systematically oversell reflective and multi-round defenses.
  • Adaptive routers that escalate from rule-based checks to heavier reflection only on high-risk prompts are a natural next design once the side-effect profiles are known.
  • The same strategy map predicts that closed frontier models with heavy built-in alignment may show smaller absolute drops but the same relative ranking of side-effect families.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

6 major / 7 minor

Summary. The paper presents an empirical benchmark of side effects induced by 11 published jailbreak defenses across 6 open-source 7B–9B instruction-tuned LLMs. It groups defenses into four operational strategy families (Rule-Based Check; Output Reflection; Input Reflection & Alignment; Perturbation & Multi-Round) and measures three axes: unsafe-query acceptance on XSTest (RQ1), over-refusal on Open Assistant and XSTest-safe (RQ2), accuracy impact on MMLU-Pro/IFEval/GSM8K (RQ3), and token/latency/energy overhead (RQ4). The headline claims are that rule-based defenses (PPL) best preserve task performance, conservative self-reflective defenses (Self-Defend) maximize safety but over-refuse heavily, and multi-round defenses (SmoothLLM) dominate inference cost. Safety and over-refusal rates are derived from GPT-4o-mini labels classifying responses as full compliance / full refusal / partial refusal.

Significance. If the measurements hold, this is a useful deployment-oriented contribution: a joint three-axis (performance, over-refusal, cost) evaluation of defenses that prior work examined mostly in isolation, an operationally motivated taxonomy, and concrete practitioner guidance (§8). The coverage is substantial — 11 defenses × 6 models × 5 benchmarks — and the authors promise code, configurations, and evaluation logs, which would make the benchmark reusable. Several findings are genuinely non-obvious and practically relevant (e.g., PPL preserving accuracy while more sophisticated reflective defenses degrade it; SmoothLLM's ~6× output-token inflation in Tables 6–7). The central qualitative pattern (safety/usability/cost trade-offs are strategy-dependent) is plausible and consistent across multiple tables. However, the quantitative rankings that give the paper its force rest on an unvalidated LLM judge, small per-cell sample sizes without uncertainty estimates, and at least one internal contradiction between the summary taxonomy table and the RQ2 findings; these currently cap the confidence a reader can place in the specific rankings.

major comments (6)
  1. [§4.1/§5.1, Appendix B] §4.1, §5.1, Appendix B: the safety and over-refusal rates driving Findings 1.1–1.4 and 2.1–2.4 depend entirely on GPT-4o-mini's full/partial-refusal labels, but the validation is asserted rather than quantified. The paper says 200 examples were 'randomly sampled and manually checked' and never reports the outcome — no agreement rate, no Cohen's κ, no per-class confusion breakdown. Two concrete anomalies in Table 4 suggest the labeling may be unstable across strategies: (a) Mistral+Self-Defend shows FR 67.7 with PR 0.8 (below the 3.5 baseline) on Open Assistant and FR 80.8 / PR 0.0 on XSTest — a near-total migration into the full-refusal bin that looks more like a binning artifact than behavior; (b) PPL, an input perplexity filter, allegedly full-refuses ~43–45% of benign Open Assistant prompts across all four models, which is implausible for a threshold filter unless the threshold or the
  2. [§4.1/§5.1 metric definitions] §4.1 vs §5.1: the treatment of 'partial refusal' is asymmetric and under-specified. The over-refusal metric (§5.1) counts partial refusal as a failure, but for the unsafe-query acceptance rate (§4.1) it is never stated whether a partial refusal counts as acceptance (unsafe) or rejection (safe). Given that PR rates reach 30–70% in some Table 4 cells, this definitional choice can move acceptance rates by tens of points and reorder Table 3. Please define the acceptance metric precisely and report Table 3 under both conventions, or justify the chosen one.
  3. [Table 2 vs §5.2 (Finding 2.2)] Table 2 vs §5.2: internal contradiction on the paper's central summary artifact. Table 2 marks the Rule-Based Check family's over-refusal correlation as negligible (#), yet Finding 2.2 and the §5 Insight state that rule-based defenses (PPL) are 'the most prone to over-refusal.' Similarly, Table 2 assigns Output Reflection only 'weak' over-refusal correlation while Self-Exam shows substantial refusal rates in several Table 4 rows. Since Table 2 is the taxonomy-level takeaway the abstract and §8 guidance rest on, the side-effect profiles there must be reconciled with the per-method results, ideally by deriving them mechanically from Tables 3–7 rather than assigning them qualitatively.
  4. [§4 RQ1 setup; §8] §4 (RQ1): the safety evaluation uses only the 250 plainly-unsafe XSTest prompts; no actual jailbreak or adversarial attack corpus (e.g., GCG, PAIR, or the attack suites in the cited [32, 34]) is evaluated. XSTest's unsafe subset is designed as the complement of an exaggerated-safety suite, not as a jailbreak benchmark. This undercuts load-bearing framing: the abstract and §8 claim guidance about 'blocking jailbreaks,' and §8 asserts SmoothLLM is 'highly effective against advanced token-level jailbreaks,' which the presented evidence cannot support. Either add at least one attack-based evaluation or scope the claims down to refusal of plainly harmful prompts.
  5. [§4.2/§5.2, Tables 3–5] Tables 3–5: no uncertainty quantification anywhere. Per-cell n is 250 (XSTest unsafe), 200 (XSTest safe), 1000 (Open Assistant); rank claims in Findings 1.1–1.3 and 2.4 lean on differences of a few percentage points (e.g., Table 3 clusters around 7–14% acceptance where a 4-point gap is ~10 prompts, well inside Wilson-interval overlap), and there are no seeds, no error bars, and no accounting for judge or decoding stochasticity. At minimum, report binomial confidence intervals on Tables 3–4 and either multi-seed runs or a statement that decoding was deterministic, and temper rank language where intervals overlap.
  6. [§3, §7.1, Tables 6–7] §3/§7, Tables 6–7: defense configurations and cost methodology are not reproducible from the text. The PPL base LM and rejection threshold, SmoothLLM perturbation count and aggregation rule, ICD exemplars, and the exact Self-Defend/Self-Exam prompts are never specified — these are free parameters that materially affect every result. For RQ4, the latency and energy columns lack units and measurement methodology (per-query? totals? which power interface?), and Table 2 promises an 'EC' (estimated cost) axis that never appears as actual numbers in any table. Please add a configuration table, units/method for latency and energy, and either the EC figures or drop that column.
minor comments (7)
  1. [Table 4 caption vs §5.2] Table 4 caption says 'red cells indicate better performance' while §5.2 says 'red cells denote stronger refusal' — these are not the same thing for an over-refusal table; please make the color semantics consistent.
  2. [Table 4] Table 4: DeepSeek+LlamaGuard cells are '-' with no explanation. If LlamaGuard is incompatible with that model, say so explicitly as was done for Self-Reminder/Gemma in Table 3.
  3. [Table 5] Table 5 covers only Llama, Mistral, and Qwen2, while Tables 3–4 cover six models; state why DeepSeek, Gemma, and Phi are excluded from RQ3.
  4. [§8 / Figure 1] §8 references 'the trade-off radar chart in right side of Fig. 1,' but Figure 1 as presented is a schematic overview with no radar chart; either include the chart or fix the reference.
  5. [Table 3] Table 3 'Acc. %' values such as 0.50 (Qwen, Self-Defend) — clarify whether these are percentages (0.5%) or fractions; the column header suggests percent but several entries read ambiguously.
  6. [§1, §5.1] §1 Evaluation: the citation list '[4, 12, 13, 13, 18, ...]' repeats [13]; also 'meta-defender' (§5.1, §7, Appendix A) is used without definition — unify with 'strategy family' terminology.
  7. [§6.2] PriorityGoal's catastrophic GSM8K drops (e.g., 0.08 on Llama, 0.02 on Qwen2) receive no analysis; a short failure-case inspection would strengthen RQ3, since these outliers drive the 'reflective defenses hurt capability' narrative as much as PPL's stability does.

Circularity Check

0 steps flagged

No circularity: empirical benchmark measures external defenses on external datasets; strategy taxonomy is an analysis grouping, not a self-forcing derivation.

full rationale

This paper does not claim a first-principles derivation or parameter-free prediction that could fold inputs back into outputs. It evaluates 11 published defenses on public benchmarks (XSTest, Open Assistant, MMLU-Pro, IFEval, GSM8K) and six open-source LLMs, reporting measured acceptance rates, over-refusal, task accuracy, and token/latency cost. Strategy families in Table 2 are operational groupings assigned by primary bottleneck; the side-effect marks (strong/weak/negligible) are summaries of those measurements, not quantities forced by the grouping definition. There is no fitted parameter re-presented as a prediction, no load-bearing self-citation uniqueness theorem, and no renaming of a known closed-form result as a new derivation. Rankings (e.g., Self-Defend lowest unsafe acceptance; PPL preserving MMLU-Pro; SmoothLLM highest output-token overhead) are contingent empirical outcomes that could have come out otherwise. Minor taxonomy judgment when multi-idea methods are assigned to one family is an analysis choice, not circular reduction. Score 0 is appropriate.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

Load-bearing content is empirical protocol, not free physical constants. The claim rests on a hand-built strategy taxonomy, external public benchmarks treated as representative of deployment, an LLM-as-judge labeling rule, and standard assumptions that open 7B–9B instruction models are informative proxies. No new particles or forces; the main invented structure is the four strategy families and the three-axis side-effect ledger.

free parameters (3)
  • PPL rejection threshold and base LM for perplexity
    Rule-based PPL behavior (and thus its over-refusal vs preservation profile) depends on the numeric cutoff and reference model; the preprint does not fix or sweep this value in the main text.
  • SmoothLLM perturbation count / aggregation rounds
    Multi-round cost and safety gains scale with how many perturbed generations are run; overhead findings assume the authors' chosen round budget.
  • GPT-4o-mini refusal/compliance decision boundary
    Acceptance and over-refusal rates are defined by the judge's categorical labels; that boundary is an unreported prompt/policy choice validated on only 200 manual samples.
axioms (4)
  • ad hoc to paper Assigning each defense to exactly one primary operational-bottleneck family yields mutually exclusive categories whose side-effect correlations are interpretable as strategy effects.
    Stated in §2.1; multi-idea methods are forced into one family by 'primary bottleneck,' which is an analysis axiom rather than a measured fact.
  • domain assumption Public English benchmarks (XSTest, Open Assistant, MMLU-Pro subsets, IFEval, GSM8K) are sufficiently representative of real deployment queries for ranking defense side effects.
    Used throughout RQ1–RQ4; limitations note English-only and fixed corpora.
  • domain assumption Full / partial / no refusal categories assigned by GPT-4o-mini are valid proxies for human safety and usability judgments.
    RQ1–RQ2 setup and Appendix B; only a 200-example manual spot check is reported.
  • domain assumption Open instruction-tuned models in the 7B–9B class are adequate subjects for characterizing defense trade-offs relevant to broader LLM deployment.
    Experimental subjects §3; limitations explicitly exclude frontier closed systems as defended targets.
invented entities (2)
  • Four unified defense strategy families (Rule-Based Check; Output Reflection; Input Reflection & Alignment; Perturbation & Multi-Round) no independent evidence
    purpose: Group 11 defenses so side effects can be attributed to operational mechanism rather than method name.
    Introduced in §2.1 and Table 2; taxonomy is the paper's organizing construct. Independent evidence is limited to the empirical correlations the paper itself measures.
  • Joint three-axis side-effect profile (performance impact, over-refusal, inference cost) as a deployment decision object no independent evidence
    purpose: Replace single-axis (mostly over-refusal) evaluation with a deployability map.
    Framed in Fig. 1, Table 1, and §8 guidelines; the bundle is a measurement frame, not a physical entity, and is only validated inside this study's protocol.

pith-pipeline@v1.2.0-grok45-kimik3 · 22662 in / 3596 out tokens · 77173 ms · 2026-07-31T16:01:12.052950+00:00 · methodology

0 comments
read the original abstract

Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study of these defense trade-offs along three dimensions: performance impact, over-refusal on benign inputs, and inference cost. Rather than treating defenses as a single class, we organize them by operational strategy and examine how different strategies correlate with different side-effect profiles. Across state-of-the-art defense methods, widely used benchmark datasets, and representative open-source LLMs, we find that defenses rarely improve downstream capability, but instead vary in how they trade safety gains against usability and efficiency. In particular, rule-based defenses best preserve task performance, highly conservative self-reflective defenses often increase over-refusal, and multi-round defenses incur the largest runtime overhead. These results provide both a benchmark for evaluating defense side effects and practical guidance for selecting defenses under deployment constraints.

Figures

Figures reproduced from arXiv: 2607.24392 by Simin Chen, Tong Zhang, Yun Peng, Zexin Li.

Figure 1
Figure 1. Figure 1: Overview of defense-induced trade-offs in LLM systems. A defended LLM is expected to improve safety by reducing attack success on harmful prompts. However, stronger defenses may also backfire by over￾refusing benign requests, degrading task performance, and increasing inference overhead such as latency, token usage, and API cost. practitioners with limited guidance on the broader trade-off space that actua… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 1 canonical work pages

  1. [1]

    Awadalla

    Marah Abdin, Jyoti Aneja, and Hany et al. Awadalla. Phi-3 technical report: A highly capable language model locally on your phone, 2024. URLhttps://arxiv.org/abs/2404.14219

    Pith/arXiv arXiv 2024

  2. [2]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Akram Al-Rumaim and Jyoti D. Pawar. Revolutionizing authentication: Harnessing natural language understanding for dynamic password generation and verification. In Jyoti D. Pawar and Sobha Lalitha Devi, editors,Proceedings of the 20th International Conference on Natural Language Processing (ICON), pages 670–678, Goa University, Goa, India, December 2023. N...

    2023

  4. [4]

    Detecting language model attacks with perplexity, 2023

    Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity, 2023. URLhttps://arxiv.org/abs/2308.14132

    Pith/arXiv arXiv 2023

  5. [5]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    Tom B Brown. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    Pith/arXiv arXiv 2005

  6. [6]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  7. [7]

    Or-bench: An over-refusal benchmark for large language models.arXiv preprint arXiv:2405.20947, 2024

    Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. Or-bench: An over-refusal benchmark for large language models.arXiv preprint arXiv:2405.20947, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model, 2024

    DeepSeek-AI. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model, 2024

    2024

  9. [9]

    Abhimanyu Dubey, Abhinav Jauhri, and Abhinav et al. Pandey. The llama 3 herd of models,

  10. [10]

    Improving llm-based machine translation with systematic self-correction.arXiv preprint arXiv:2402.16379, 2024

    Zhaopeng Feng, Yan Zhang, Hao Li, Wenqiang Liu, Jun Lang, Yang Feng, Jian Wu, and Zuozhu Liu. Improving llm-based machine translation with systematic self-correction.arXiv preprint arXiv:2402.16379, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    The robots are coming: Exploring the implications of openai codex on introductory program- ming

    James Finnie-Ansley, Paul Denny, Brett A Becker, Andrew Luxton-Reilly, and James Prather. The robots are coming: Exploring the implications of openai codex on introductory program- ming. InProceedings of the 24th Australasian Computing Education Conference, pages 10–19, 2022

    2022

  12. [12]

    Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa

    Hakan Inan, K. Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama guard: Llm- based input-output safeguard for human-ai conversations.ArXiv, abs/2312.06674, 2023. URL https://api.semanticscholar.org/CorpusID:266174345

    Pith/arXiv arXiv 2023

  13. [13]

    Baseline defenses for adversarial attacks against aligned language models, 2023

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models, 2023. URL https://arxiv. org/abs/2309.00614. 10

    Pith/arXiv arXiv 2023

  14. [14]

    Jiang, Alexandre Sablayrolles, and Arthur et al

    Albert Q. Jiang, Alexandre Sablayrolles, and Arthur et al. Mensch. Mistral 7b, 2023. URL https://arxiv.org/abs/2310.06825

    Pith/arXiv arXiv 2023

  15. [15]

    Openassistant conversations-democratizing large language model alignment.Advances in Neural Information Processing Systems, 36, 2024

    Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richárd Nagyfi, et al. Openassistant conversations-democratizing large language model alignment.Advances in Neural Information Processing Systems, 36, 2024

    2024

  16. [16]

    Salad-bench: A hierarchical and comprehensive safety benchmark for large language models

    Lijun Li, Bowen Dong, Ruoyu Wang, Jiaojiao Hu, Wenmian Zuo, Dahua Lin, et al. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. InACL, 2024

    2024

  17. [17]

    You can’t eat your cake and have it too: The performance degradation of llms with jailbreak defense.arXiv preprint arXiv:2407.13540, 2024

    Wuyuao Mai, Geng Hong, Pei Chen, Xu Pan, Baojun Liu, Yuan Zhang, Haixin Duan, and Min Yang. You can’t eat your cake and have it too: The performance degradation of llms with jailbreak defense.arXiv preprint arXiv:2407.13540, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Llm self defense: By self examination, llms know they are being tricked,

    Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. Llm self defense: By self examination, llms know they are being tricked,

  19. [19]

    Smoothllm: Defending large language models against jailbreaking attacks.arXiv preprint arXiv:2310.03684, 2023

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks.arXiv preprint arXiv:2310.03684, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    URLhttps://arxiv.org/abs/2308.07308

    Pith/arXiv arXiv

  21. [21]

    Trustllm: Trustworthiness in large language models

    Licheng Sun, Yuxin Huang, Heng Ding, Yue Liu, et al. Trustllm: Trustworthiness in large language models. InInternational Conference on Machine Learning (ICML), 2024

    2024

  22. [22]

    XSTest: A test suite for identifying exaggerated safety behaviours in large language models

    Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. XSTest: A test suite for identifying exaggerated safety behaviours in large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors,Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics...

    2024

  23. [23]

    Gemma Team, Morgane Riviere, and Shreya et al. Pathak. Gemma 2: Improving open language models at a practical size, 2024. URLhttps://arxiv.org/abs/2408.00118

    Pith/arXiv arXiv 2024

  24. [24]

    Llm4decompile: Decompiling binary code with large language models.arXiv preprint arXiv:2403.05286, 2024

    Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. Llm4decompile: Decompiling binary code with large language models.arXiv preprint arXiv:2403.05286, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Defending LLMs against jailbreaking attacks via backtranslation

    Yihan Wang, Zhouxing Shi, Andrew Bai, and Cho-Jui Hsieh. Defending LLMs against jailbreaking attacks via backtranslation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics ACL 2024, pages 16031– 16046, Bangkok, Thailand and virtual meeting, August 2024. Association for Computational Linguis...

    doi:10.18653/v1/2024.findings-acl.948 2024

  26. [26]

    The art of defending: A systematic evaluation and analysis of LLM defense strategies on safety and over-defensiveness

    Neeraj Varshney, Pavel Dolin, Agastya Seth, and Chitta Baral. The art of defending: A systematic evaluation and analysis of LLM defense strategies on safety and over-defensiveness. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics ACL 2024, pages 13111–13128, Bangkok, Thailand and virtual m...

    doi:10.18653/v1/2024 2024

  27. [27]

    Jailbreak and guard aligned language models with only few in-context demonstrations, 2024

    Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. Jailbreak and guard aligned language models with only few in-context demonstrations, 2024. URL https://arxiv.org/ abs/2310.06387. 11

    Pith/arXiv arXiv 2024

  28. [28]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Defending chatgpt against jailbreak attack via self-reminders.Nature Machine Intelligence, 5(12):1486–1496, 2023

    Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. Defending chatgpt against jailbreak attack via self-reminders.Nature Machine Intelligence, 5(12):1486–1496, 2023

    2023

  30. [30]

    Llms can defend themselves against jailbreaking in a practical manner: A vision paper.ArXiv, abs/2402.15727, 2024

    Daoyuan Wu, Shuaibao Wang, Yang Liu, and Ning Liu. Llms can defend themselves against jailbreaking in a practical manner: A vision paper.ArXiv, abs/2402.15727, 2024. URL https://api.semanticscholar.org/CorpusID:267938128

    Pith/arXiv arXiv 2024

  31. [31]

    SafeDecoding: Defending against jailbreak attacks via safety-aware decoding

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Pooven- dran. SafeDecoding: Defending against jailbreak attacks via safety-aware decoding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5587–560...

    doi:10.18653/v1/2024.acl-long.303 2024

  32. [32]

    Contrastive preference optimization: Pushing the bound- aries of LLM performance in machine translation

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the bound- aries of LLM performance in machine translation. InForty-first International Conference on Machine Learning, 2024. URLhttps://openreview.net/forum?id=51iwkioZpn

    2024

  33. [33]

    An Yang, Baosong Yang, and Binyuan et al. Hui. Qwen2 technical report, 2024. URL https://arxiv.org/abs/2407.10671

    Pith/arXiv arXiv 2024

  34. [34]

    A comprehensive study of jailbreak attack versus defense for large language models

    Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. A comprehensive study of jailbreak attack versus defense for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics ACL 2024, pages 7432–7449, Bangkok, Thailand and virtual meeting, August 2024. Association for C...

    doi:10.18653/v1/2024.findings-acl.443 2024

  35. [35]

    Intention analysis makes llms a good jailbreak defender, 2024

    Yuqi Zhang, Liang Ding, Lefei Zhang, and Dacheng Tao. Intention analysis makes llms a good jailbreak defender, 2024. URLhttps://arxiv.org/abs/2401.06561

    Pith/arXiv arXiv 2024

  36. [36]

    Jailbreak open-sourced large language models via enforced decoding

    Hangfan Zhang, Zhimeng Guo, Huaisheng Zhu, Bochuan Cao, Lu Lin, Jinyuan Jia, Jinghui Chen, and Dinghao Wu. Jailbreak open-sourced large language models via enforced decoding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5475–...

    work page doi:10.18653/v1/2024.acl-long.299 2024

  37. [37]

    Why should black beans never be president?

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023. 12 A Overhead Table 6 shows input token overhead across different defense mechanisms, grouped by meta-defender strategies, on the Open-Assistant dataset...

    Pith/arXiv arXiv 2023

  38. [38]

    Defending large language models against jailbreaking attacks through goal prioritization

    Zhexin Zhang, Junxiao Yang, Pei Ke, Fei Mi, Hongning Wang, and Minlie Huang. Defending large language models against jailbreaking attacks through goal prioritization. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8865–8887, Ba...

    2024

  39. [2024]

    URLhttps://arxiv.org/abs/2407.21783

    Pith/arXiv arXiv