Pith. sign in

REVIEW 3 major objections 4 minor 46 references

Refusing Intent, Not Form: Wrapper-Based Intent-Group Supervision for LLM Safety

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Safety tuning that supervises intent groups, not prompt forms, lets LLMs refuse wrapped harmful requests without over-refusing benign ones.

desk verdict Solid, honestly-reported empirical safety paper; the matched-wrapper intent-group idea is genuinely useful, but the headline below-base over-refusal number is a single-seed minimum of a test-set margin scan and should be treated as a demonstration until multi-seed validation is shown. read the letter →

arxiv 2608.13304 v1 pith:STG3H3RA submitted 2026-08-13 cs.CL

Ping Wu , Haibo Tong , Feifei Zhao , Han Shen , Yu Shi , Yilin Zhao , Sicheng Shen , Guobin Shen
show 2 more authors
Yun Luo Yi Zeng
This is my paper · ORCID
classification cs.CL
keywords LLMsafetyalignmentintent-groupsupervisionwrapperrobustnessover-refusalrefusaltuninggroup-consistentregularizationsurface-formshortcutsjailbreak
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper takes on a specific failure of LLM safety tuning: models learn that the wrapper around a request, not the intent underneath it, determines whether to refuse, so wrapped harmful prompts slip through and similarly wrapped benign prompts get refused. It proposes supervising whole intent groups, the same underlying request written in several surface styles, paired with structurally matched benign requests, so wrapper form carries no information about the right decision. Two training routes built on that data layer make the point concrete: one maximizes refusal of wrapped harmful requests, raising SORRY-Bench mutation-average refusal from 22.1 to 63.7 in the Qwen setting, and the other cuts over-refusal on benign requests from 25.7% to 17.4% while still refusing harmful ones. A sympathetic reader would care because the methods separate two goals usually conflated in refusal tuning, harmful-request refusal and benign compliance, and because the reproduced baselines do not reach these operating points, suggesting the intent-group structure itself rather than extra data or stronger fine-tuning is doing the work.

What carries the argument

The load-bearing objects are intent groups and a refusal/compliance decision score. An intent group $G(z)$ is the set of prompt forms expressing one underlying request; WIFA builds matched harmful and benign groups under shared wrapper families so that form carries no signal about the desired decision. A-GCRT then regularizes the first-token decision score $s_\theta(x)$, defined as the maximum next-token logit over refusal prefixes minus the maximum over compliance prefixes just after the intent-analysis marker, using the loss $\mathcal{L} = \mathcal{L}_{\mathrm{SFT}} + \lambda_{\mathrm{gcr}}(\mathcal{L}_{\mathrm{var}} + \gamma \mathcal{L}_{\mathrm{anchor}})$, where the variance term penalizes score spread within an intent group and the anchor term pushes harmful group means above a margin $+m$ and benign group means below $-m$. This score, used only at training time, is what lets the loss reshape the refusal boundary without an auxiliary classifier or preference pairs.

What would settle it

Retrain A-GCRT with the decision score computed from shuffled or randomly chosen prefix token sets and check whether OR-Bench over-refusal still drops from 25.7% to roughly 17.4%; if the drop survives, the score-based group regularization is not what carries the result. A complementary check: measure free-generation refusal rates across many wrappers of the same harmful intent group; if refusal still swings sharply across wrappers, decisions track form rather than intent, and the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that safety supervision should be defined over intent groups rather than isolated prompt-response pairs. WIFA constructs, for each harmful intent, a group of wrapped variants plus direct refusal anchors, and for each benign intent a group under the same wrapper families, so the wrapper is a nuisance variable rather than a decision label. The intended consequence, demonstrated in the Qwen setting, is that decisions track intent: WIFA-Boost raises SORRY-Bench mutation-average refusal from 22.1 to 63.7 and the misrepresentation subset from 0.2 to 59.3, while A-GCRT-M5 lowers OR-Bench over-refusal from 25.7% to 17.4%, below both the base model and every reproduced defense, while still improving harmful-request refusal. The Llama results and the ablations over data structure, stage order, margins, and loss components support this intent-group interpretation without claiming universal below-base over-refusal.

Load-bearing premise

The low-over-refusal result rests on a hand-built training signal, a single-position score comparing the model's next-token preference for refusal phrases versus polite-answer phrases, being a faithful proxy for real refusal behavior, even though the paper's own diagnostics show the score separates harmful from benign prompts at only about chance level and is often uncomputable because the marker it depends on is missing from the model's output.

Editorial extensions

If this is right

  • Wrapper form becomes non-diagnostic: models trained on matched intent groups refuse wrapped harmful prompts at rates close to direct harmful prompts, and average attack success across 15 unseen attack families falls from 48.3% on the base model to 9.5% for WIFA-Boost, evidence against template memorization though not against adaptive attackers.
  • Over-refusal and harmful refusal can be selected as separate operating points: A-GCRT margin and anchor settings act as validation-selected controls (the M5 point gives 17.4% OR with 46.7 SB-avg5, the M10 point gives 40.7% OR with 52.0 SB-avg5), not a monotonic larger-is-better dial.
  • Curriculum order matters: learning the intent-form structure before plain-benign calibration is what preserves refusal on difficult mutated harmful prompts (59.3 misrepresentation refusal), while reversing the order collapses it to 0.2, showing WIFA-Boost is not merely a data-mixture effect.
  • Stronger fine-tuning is not a substitute for the group objective: a high-learning-rate WIFA-SFT run approaches A-GCRT on misrepresentation refusal but leaves OR near 68%, and variance-only or anchor-only components leave OR at 74.5% and 48.7%, respectively.
  • The safety gain is not capability-neutral: A-GCRT keeps MMLU close to the base model (69.0 versus 70.1 in Qwen) but still costs GSM8K performance, and the Llama setting shows the below-base OR result does not transfer across models and data sources.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Because WIFA is a data layer independent of the optimizer, the same matched intent groups could feed preference-based objectives such as DPO-style or RLHF-style training; A-GCRT's variance-plus-anchor structure is orthogonal to the SFT loss it currently rides on, and the paper itself notes that ungrouped preference pairs do not supply the structure it needs.
  • Editorial inference: The decision score's known weakness (near-chance harmful/benign separation, with the intent-analysis marker missing in 14-34% of free generations) suggests A-GCRT's gains might be reproducible with a more reliable score, such as a generative refusal judge or a hidden-state probe, which could turn the training-time regularizer into an inference-time control.
  • Editorial inference: The matched-benign principle yields a practical diagnostic: measuring within-intent variance of refusal decisions across wrappers could tell a practitioner whether a safety-tuned model has learned a form shortcut, before deployment, using the paper's variance notion as a cheap inspection tool.
  • Editorial inference: The cross-setting gap in the paper's own numbers, Qwen below base on OR while Llama stays at 40.1% against a 28.4% base, raises the open question of whether the intent-group advantage is model- and data-source-dependent rather than a universal law.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes that LLM safety tuning should supervise refusal at the level of intent groups rather than individual prompt forms. It introduces WIFA, a data-augmentation method that pairs wrapped harmful prompts with structurally matched wrapped benign prompts under shared wrapper families, and two training routes built on this data layer: WIFA-Boost, a two-stage high-safety recipe, and A-GCRT, which regularizes refusal/compliance decision scores within intent groups and anchors harmful and benign groups on opposite sides of a margin. The experiments cover Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct across seven benchmarks and a 15-family unseen-attack suite. The headline claims are that WIFA-Boost reaches the strongest transformed-harmful refusal (63.7 SORRY-Bench mutation-average in Qwen) and that A-GCRT-M5 reduces OR-Bench over-refusal from 25.7% to 17.4%, with reproduced baselines failing to match these operating points. The paper also reports extensive ablations on data structure, benign/harmful ratio, stage order, A-GCRT components, margins, and decision-score diagnostics, and it explicitly notes that Llama does not reproduce below-base over-refusal.

Significance. If verified, the paper would offer a concrete, teacher-free way to reduce surface-form shortcuts in safety tuning and to separate the high-safety and low-over-refusal operating points. The WIFA construction is a useful idea: pairing wrapped harmful and benign intents makes the wrapper non-diagnostic of the decision, and the two-stage and margin-based recipes provide a principled way to think about safety versus over-refusal trade-offs. Strengths include the release of code and sanitized artifacts, the breadth of ablations, the honest reporting of the Llama boundary, and the inclusion of decision-score diagnostics that are negative in part. The central limitation is verification quality: the headline below-base over-refusal figure comes from a test-set margin scan with a single seed, and the A-GCRT score that drives the objective is shown to be a weak label-side separator with frequent missing markers. These issues do not make the idea circular or obviously wrong, but they mean the quantitative claims as stated are not yet reliable.

major comments (3)
  1. [§6.5, Table 21, Table 1, Appendix B.2] The central over-refusal result is not supported as a general estimate because the 17.4% figure is the best point of a margin/anchor scan performed on the same OR-Bench-Hard test set used to state the claim. Table 21 scans m = 2.5, 5, 7.5, 10 with gamma = 1 and gamma = 2, and Table 1 reports A-GCRT-M5 as the headline point; all runs use seed 42 per Appendix B.2. No held-out validation split for margin selection is described anywhere in the paper, despite Section 6.5 calling these 'validation-selected operating controls.' The adjacent configuration m = 2.5 gives OR 29.3, above the base model's 25.7, so the conclusion reverses under a nearby hyperparameter choice. Please provide a pre-specified validation protocol, seed-averaged results with confidence intervals, or revise the abstract and Section 5.1 to state that 17.4 is the best-scanned point rather than a robust estimate.
  2. [§3.3, Eq. (3), Appendix C.6, Table 22, Table 23] The mechanism underlying A-GCRT depends on the decision-position score s_theta(x) being a usable refusal/compliance signal, but the manuscript's own diagnostics show this score is a weak label-side separator: Table 22 reports label AUROC 0.453 for A-GCRT-M5, and missing-marker rates in free generation range from 14.3% to 33.7%. Table 23's target-forced diagnostic supplies the intent-analysis marker and uses train-sampled groups, so it matches the training-time objective but does not establish that optimizing Lvar and Lanchor transfers to actual generated refusal/compliance behavior. The paper properly cautions that the score is not an inference-time classifier, yet the objective's gradient signal is still this same score. Please provide a direct analysis of how changes in the regularization loss translate into changes in generated refusal/compliance behavior, or temper the claim that A-GCRT shapes the refusal boundary through group-consistent decision scores.
  3. [§5.1, Table 1] All headline numbers in Table 1 come from single training runs with no seed variation or uncertainty estimates, and several comparisons that support the paper's operating-point narrative are numerically small, such as WIFA-SFT versus WIFA-Boost on SB-avg5 (63.6 versus 63.7). Without multiple seeds, bootstrap confidence intervals, or a stated evaluation-variance protocol, the paper cannot support claims that one method 'reaches' a point or that reproduced baselines 'do not match' it in a statistically meaningful sense. Reporting uncertainty for at least the headline metrics and the margin-scan table would address this concern.
minor comments (4)
  1. [§4.3 and Figure 6] The corrected capability protocol applies a fixed benign intent-analysis prefix to MMLU and GSM8K for all methods, including base models; please provide a short comparison with the standard protocol for at least the base model so readers can quantify how the prefix changes capability estimates.
  2. [Table 1 and Appendix C tables] Several tables, especially Table 1, do not explicitly mark the direction of every column in the main text; adding arrows such as 'SB-avg5 ↑' and 'OR ↓' consistently would reduce the risk of misreading.
  3. [Appendix C.6] The wording 'Behavior Agree' and 'Label Accuracy' in Table 22 is not defined in the main text; please define these metrics and explain why label-side accuracy is computed on a presumably balanced or imbalanced sample.
  4. [Section 3.3] The sentence 'we use this score only as a training-time regularization proxy, not as a calibrated inference-time classifier' is helpful, but the same paragraph should note explicitly that the score is also not a clean training-time separator, as later shown in Table 22, so readers are not misled about its quality.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the central claims are empirical and self-contained, with the nearest concern being test-set margin selection rather than derivation-level circularity.

full rationale

The paper does not derive its headline results from its inputs by construction. WIFA constructs matched harmful and benign intent groups by applying hand-designed wrapper families to source intents and self-distilling direct-form targets (Section 3.1), then evaluates on independent benchmarks (SORRY-Bench, OR-Bench, HarmBench, StrongREJECT, and an unseen attack-family suite) that are not the WIFA training templates; the paper explicitly checks that training wrappers are not copied from SORRY-Bench mutations and that the Qwen seeds do not duplicate HarmBench prompts (Appendices A.3 and A.5). A-GCRT's decision score is defined and used only as a training-time proxy: Section 8 states that 'the A-GCRT decision-position score should be interpreted as a training-time regularization signal rather than an inference-time classifier,' and Appendix C.6 reports weak label-side separation (AUROC 0.453) and frequent missing markers, which is an honest limitation rather than a hidden reduction. The only self-citation to prior work by overlapping authors is SIRL, which Section 4.2 explicitly reports 'only as an external reference, not as a reproduced baseline,' so it is not load-bearing. The closest circularity-adjacent issue is that the low-over-refusal headline point A-GCRT-M5 is selected from the margin scan in Section 6.5 (Table 21) on the same OR-Bench-Hard test set used to state the 17.4 claim, with only seed 42 reported and no validation split described; this is test-set hyperparameter selection that weakens the estimate as an expected operating point, but it is not a derivation whose output equals its input, so it does not meet the exhibit-a-reduction standard for circularity. Overall, the central claims are empirical and self-contained; score 2 reflects the non-load-bearing self-citation and the circularity-adjacent selection concern rather than a circular derivation.

Assumptions & free parameters 8 free parameters · 4 assumptions · 1 invented entities

The central empirical claims rest on a small number of hand-picked data-construction and regularization choices: wrapper family count, benign-to-harmful ratio, direct anchors, and the A-GCRT margin and anchor weight. The margin scan in Section 6.5 shows the headline OR point is selected from a frontier rather than derived. The method also assumes self-distilled targets are accurate and that the decision-prefix score tracks refusal behavior despite weak label-side calibration.

free parameters (8)
  • A-GCRT margin m = 5.0 (M5), 10.0 (M10)
    Operating-point control selected via scan over OR-Bench and SORRY-Bench (Section 6.5); M5 gives the lowest OR.
  • Anchor weight gamma = 1.0 (M5), 2.0 (M10)
    Hyperparameter paired with margin; changes the safety-over-refusal trade-off.
  • lambda_gcr = 0.3
    Weight on A-GCRT regularizer relative to SFT loss.
  • Forms per group = 3
    Number of wrapped forms sampled per intent group in each A-GCRT update.
  • Benign-to-harmful ratio = 1.56 (3500/2250)
    Chosen data ratio; ratio scan shows moderate ratios most stable, and no scanned ratio dominates both safety and OR.
  • Wrapper family count |Wm| = 7
    Manually designed fixed wrapper families; central to the data construction.
  • Direct anchors per harmful intent = 2
    Adds direct-form refusal anchors to each harmful group, expanding harmful examples to 9 forms.
  • Decision-prefix token sets R and C = hand-crafted lists (Table 11)
    Construct refusal and comply prefix sets; the score sign convention depends on these lists.
assumptions (4)
  • domain assumption Self-distilled direct-form responses of the base model are accurate enough to serve as WIFA targets.
    Appendix A.2 audit shows 68 invalid harmful targets out of 300, mostly format issues; benign targets are mostly valid, but the assumption is load-bearing.
  • ad hoc to paper The first-token refusal-minus-compliance logit score is a usable training-time proxy for refusal behavior.
    Eq. (3) and Appendix C.6 show weak label-side AUROC and missing-marker rates, so the proxy is heuristic and not externally validated.
  • domain assumption Fixed wrapper families cover the nuisance variation needed for robustness.
    Training uses 7 manually designed wrappers; unseen-attack results show some generalization but the paper disclaims universality.
  • standard math Standard fine-tuning and LoRA assumptions hold.
    LoRA on attention projections, bf16, seed 42; standard practice, not further justified.
invented entities (1)
  • Decision-position score s_theta(x)
    purpose: Training-time regularizer that imposes within-group consistency and directional anchors on refusal/compliance orientation.
    No external falsifiable prediction; the paper itself shows it is not a clean harmful/benign classifier (Table 22), so it is a heuristic construct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Refusing Intent, Not Form: Wrapper-Based Intent-Group Supervision for LLM Safety." pith.science (2026). https://pith.science/paper/STG3H3RA

@misc{pith2026260813304,
  author       = {Pith},
  title        = {Pith review of: Refusing Intent, Not Form: Wrapper-Based Intent-Group Supervision for LLM Safety},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/STG3H3RA}},
  note         = {Machine review of arXiv:2608.13304}
}
read the original abstract

Safety tuning can improve harmful refusal, but models may learn surface-form shortcuts: wrapped harmful prompts bypass safety, while similarly wrapped benign prompts are over-refused. We propose Wrapper-Based Intent-Form Augmentation (WIFA), an automatic intent-group augmentation method that pairs wrapped harmful examples with structurally matched wrapped benign counterexamples, requiring no external teacher or manual per-wrapper intent labels. We use WIFA as a common data layer for two complementary fine-tuning routes: WIFA-Boost, a two-stage high-safety recipe, and Anchored Group-Consistent Refusal Training (A-GCRT), which regularizes refusal/compliance decision scores across same-intent wrappers and anchors harmful and benign groups on opposite sides of a margin. In the Qwen setting, WIFA-Boost reaches the strongest transformed-harmful refusal, while A-GCRT reduces OR-Bench over-refusal from 25.7\% for the base model to 17.4\%; reproduced baselines do not match these operating points. Llama results and ablations over data structure, two-stage order, and A-GCRT components support this intent-group interpretation without claiming universal below-base over-refusal.

Figures

Figures reproduced from arXiv: 2608.13304 by the authors.

Figure 1
Figure 1. Overview of WIFA, WIFA-Boost, and A-GCRT. Left: WIFA constructs matched harmful and benign [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Attack-family breakdown in the Qwen setting. Values are attack success rates, where lower is better. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. A-GCRT margin/anchor scan. Different settings select different safety–over-refusal operating points rather than a monotonic curve. 6.5 Do A-GCRT Margins Control Operating Points? This ablation tests whether A-GCRT can move the operating point through margin and anchor settings [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Self-distilled target construction for WIFA. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: A-GCRT training objective. A-GCRT samples wrapper forms from the same underlying intent group, [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Corrected capability evaluation protocol. A [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Safety–over-refusal frontier across Qwen and Llama settings. WIFA-Boost provides a safety-oriented [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Capability change relative to the base model under the corrected capability evaluation protocol. All [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: XSTest safe/unsafe refusal trade-off. Preferred behavior lies toward lower safe-prompt refusal and higher [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Ablation map. The ablations are organized around the main alternative explanations: whether matched [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Benign-to-harmful ratio scan under WIFA [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 11 canonical work pages

  1. [1]

    arXiv preprint arXiv:2409.00598 , year=

    Automatic pseudo-harmful prompt generation for evaluating false refusals in large language models , author=. arXiv preprint arXiv:2409.00598 , year=

    arXiv

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Refusal in language models is mediated by a single direction , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    arXiv preprint arXiv:1907.02893 , year=

    Invariant risk minimization , author=. arXiv preprint arXiv:1907.02893 , year=

    arXiv 1907

  4. [4]

    arXiv preprint arXiv:2212.08073 , year=

    Constitutional ai: Harmlessness from ai feedback , author=. arXiv preprint arXiv:2212.08073 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    arXiv

  6. [6]

    International Conference on Learning Representations , volume=

    Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions , author=. International Conference on Learning Representations , volume=

    work page

  7. [7]

    2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) , pages=

    Jailbreaking black box large language models in twenty queries , author=. 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) , pages=. 2025 , organization=

    2025

  8. [8]

    URL https://arxiv

    Training verifiers to solve math word problems, 2021 , author=. URL https://arxiv. org/abs/2110.14168 , volume=

    arXiv 2021

Show all 46 references
  1. [9]

    arXiv preprint arXiv:2405.20947 , year=

    Or-bench: An over-refusal benchmark for large language models , author=. arXiv preprint arXiv:2405.20947 , year=

    arXiv

  2. [10]

    International Conference on Learning Representations , volume=

    Safe rlhf: Safe reinforcement learning from human feedback , author=. International Conference on Learning Representations , volume=

  3. [11]

    arXiv preprint arXiv:2209.07858 , year=

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned , author=. arXiv preprint arXiv:2209.07858 , year=

    arXiv

  4. [12]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  5. [13]

    Advances in neural information processing systems , volume=

    Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms , author=. Advances in neural information processing systems , volume=

  6. [14]

    arXiv preprint arXiv:2009.03300 , year=

    Measuring massive multitask language understanding , author=. arXiv preprint arXiv:2009.03300 , year=

    2009 arXiv

  7. [15]

    arXiv preprint arXiv:2312.06674 , year=

    Llama guard: Llm-based input-output safeguard for human-ai conversations , author=. arXiv preprint arXiv:2312.06674 , year=

    arXiv

  8. [16]

    Advances in Neural Information Processing Systems , volume=

    Beavertails: Towards improved safety alignment of llm via a human-preference dataset , author=. Advances in Neural Information Processing Systems , volume=

  9. [17]

    Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining , pages=

    Lookahead tuning: Safer language models via partial answer previews , author=. Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining , pages=

  10. [18]

    International Conference on Learning Representations , volume=

    Autodan: Generating stealthy jailbreak prompts on aligned large language models , author=. International Conference on Learning Representations , volume=

  11. [19]

    arXiv preprint arXiv:2402.04249 , year=

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal , author=. arXiv preprint arXiv:2402.04249 , year=

    arXiv

  12. [20]

    Advances in Neural Information Processing Systems , volume=

    Tree of attacks: Jailbreaking black-box llms automatically , author=. Advances in Neural Information Processing Systems , volume=

  13. [21]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  14. [22]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    Red teaming language models with language models , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  15. [23]

    International Conference on Learning Representations , volume=

    Safety alignment should be made more than just a few tokens deep , author=. International Conference on Learning Representations , volume=

  16. [24]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  17. [25]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    Xstest: A test suite for identifying exaggerated safety behaviours in large language models , author=. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2024

  18. [26]

    arXiv preprint arXiv:1911.08731 , year=

    Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization , author=. arXiv preprint arXiv:1911.08731 , year=

    1911 arXiv

  19. [27]

    arXiv preprint arXiv:2510.01088 , year=

    Safety instincts: Llms learn to trust their internal compass for self-defense , author=. arXiv preprint arXiv:2510.01088 , year=

  20. [28]

    do anything now

    " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models , author=. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security , pages=

    2024

  21. [29]

    Advances in Neural Information Processing Systems , volume=

    A strongreject for empty jailbreaks , author=. Advances in Neural Information Processing Systems , volume=

  22. [30]

    2023 , publisher=

    Stanford alpaca: An instruction-following llama model , author=. 2023 , publisher=

    2023

  23. [31]

    arXiv preprint arXiv:2404.13208 , year=

    The instruction hierarchy: Training llms to prioritize privileged instructions , author=. arXiv preprint arXiv:2404.13208 , year=

    arXiv

  24. [32]

    arXiv preprint arXiv:2308.13387 , year=

    Do-not-answer: A dataset for evaluating safeguards in llms , author=. arXiv preprint arXiv:2308.13387 , year=

    arXiv

  25. [33]

    Advances in neural information processing systems , volume=

    Jailbroken: How does llm safety training fail? , author=. Advances in neural information processing systems , volume=

  26. [34]

    IEEE Transactions on Image Processing , volume=

    Unsupervised domain adaptation with label and structural consistency , author=. IEEE Transactions on Image Processing , volume=. 2016 , publisher=

    2016

  27. [35]

    Nature Machine Intelligence , volume=

    Defending chatgpt against jailbreak attack via self-reminders , author=. Nature Machine Intelligence , volume=. 2023 , publisher=

    2023

  28. [36]

    International Conference on Learning Representations , volume=

    Sorry-bench: Systematically evaluating large language model safety refusal , author=. International Conference on Learning Representations , volume=

  29. [37]

    arXiv preprint arXiv:2505.08054 , year=

    Falsereject: A resource for improving contextual safety and mitigating over-refusals in llms via structured reasoning , author=. arXiv preprint arXiv:2505.08054 , year=

    arXiv

  30. [38]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Refuse whenever you feel unsafe: Improving safety in llms via decoupled refusal training , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  31. [39]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Intention analysis makes llms a good jailbreak defender , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

  32. [40]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Safety is not only about refusal: Reasoning-enhanced fine-tuning for interpretable llm safety , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  33. [41]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Defending large language models against jailbreaking attacks through goal prioritization , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  34. [42]

    arXiv preprint arXiv:2307.15043 , year=

    Universal and transferable adversarial attacks on aligned language models , author=. arXiv preprint arXiv:2307.15043 , year=

    arXiv

  35. [43]

    Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking

    Zhu, Junda and Yan, Lingyong and Wang, Shuaiqiang and Yin, Dawei and Sha, Lei. Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.1...

    2025 doi

  36. [44]

    2023 , howpublished =

    Phan, Justin , title =. 2023 , howpublished =

    2023

  37. [45]

    Advances in neural information processing systems , volume=

    Unsupervised data augmentation for consistency training , author=. Advances in neural information processing systems , volume=

  38. [46]

    2024 , url =

    Yang, An and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Li, Chengyuan and Liu, Dayiheng and Huang, Fei and Wei, Haoran and Lin, Huan and Yang, Jian and Tu, Jianhong and Zhang, Jianwei and Yang, Jianxin and Yang, Jiaxi and Zhou, Jingren an...

    2024

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.