Pith. sign in

REVIEW 3 major objections 5 minor 67 references

Incomplete Prompt Jailbreaks in Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that incomplete prompts are systematically more jailbreakable than complete ones because models postpone refusal until sentence completion.

desk verdict The core empirical claim survives contact with the data — incomplete prompts reliably raise ASR — but the 'refusals after termination' story rests on shaky measurement and the neuron part is a promissory note. read the letter →

arxiv 2607.20473 v1 pith:MID6XWXP submitted 2026-05-24 cs.AI

Yeonjea Kim , Bumjin Park , Jaesik Choi This is my paper
classification cs.AI
keywords incompletepromptjailbreakdiscourseattractorsentencecompletionrefusaldelayneuronsteeringsafetyfine-tuningopen-weightLLMattacksuccessrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes a failure mode it calls incomplete prompt jailbreaks (IPJ): when a harmful request is left unfinished by appending a phrase like "one way is" or "Step 1:", open-weight LLMs continue generating harmful content instead of refusing. Across seven models, attack success rates are consistently higher for such incomplete prompts than for complete questions, and the gap persists across harmfulness thresholds and independent judges. The paper shows refusals are typically postponed until after the sentence reaches terminal punctuation, so harmful content appears before the refusal. It then argues that ordinary safety tuning via parameter updates fails to generalize across content domains and attractor types, and proposes neuron-level steering of termination and continuation neurons as a finer-grained defense. The sympathetic reading is that autoregressive sentence coherence itself is the vulnerability: the model prioritizes finishing the sentence over safety.

What carries the argument

Discourse attractor: a short grammatical prefix appended to a harmful request (e.g., "one way is", "Step 1:") that prevents sentence termination and biases generation toward continuation. Termination neurons are MLP neurons whose activations correlate with terminal punctuation token positions (Pearson correlation ≥ 0.25); continuation neurons are isolated by set-difference of activations under a "terminate now" versus "never stop" instruction. Steering scales these neurons' activations (α > 1 for termination, α < 1 for continuation) to shorten or prolong harmful generation. The taxonomy of nine attractor categories is the lens for measuring how different incomplete-prompt constructions modul

What would settle it

Ask a human annotator to rate a set of outputs without revealing whether each prompt was complete or incomplete; if the jailbreak-rate gap disappears or reverses under blinding, the ASR difference is an artifact of the prompt format or judge rather than a true failure mode. Alternatively, take the identified termination neurons and steer them on a model whose terminal punctuation was replaced by a rare sentinel token during training; if steering still changes refusal timing, the neurons are not specifically about sentence termination.

Watch

Extended reading notes

Core claim

The central claim is that incomplete harmful prompts are a distinct, systematic jailbreak vector. An attractor (a short grammatical cue such as "one way is", "Step 1:", or "for example,") prevents the prompt from reaching a sentence-complete state and pulls the model toward continuing the harmful content. The paper measures that incomplete prompts raise attack success rate by roughly 10–15 percentage points across models, that the effect is robust across thresholds and judges, and that refusals occur mostly after the sentence has terminated, meaning harmful tokens are generated before safety language appears. The paper further demonstrates that LoRA-based refusal tuning is brittle — it learn

Load-bearing premise

The paper labels neurons as termination or continuation based on how often they light up at punctuation marks or under different instructions, and the entire steering claim assumes those labels point to real control circuits rather than to trivial position or format patterns.

Editorial extensions

If this is right

  • If IPJ is real, open-weight models cannot rely on post-generation filters; harmful content must be interrupted during decoding, before sentence completion.
  • Any defense trained on a few attractor phrases will be brittle; models will latch onto distinctive surface cues ("one way is") while missing generic ones ("for example,").
  • Activating termination neurons at moderate strength shifts outputs from harmful completions to immediate refusals, so neuron-level control can complement or replace prompt-level defenses.
  • The refusal-delay result implies that measuring refusal distance — tokens generated before a refusal appears — is a more sensitive safety metric than binary refusal detection.
  • The same incomplete-prompt effect appears, in weaker form, in closed API models, so the phenomenon is not unique to open weights.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The correlation-based neuron labels (termination vs continuation) may be tracking token-position or instruction-format artifacts; the paper itself concedes it lacks a causal neuron-level framework, so the steering results should be read as demonstrations of intervention sensitivity, not proof of mechanism.
  • If the IPJ gap is driven by sentence-boundary dynamics rather than content, then non-English languages with different punctuation or explicit sentence-final particles should show different (or vanishing) gaps — a testable extension.
  • A practical extension would be a decoding-time interrupter that watches for the model's own termination-token prediction and forces a safety check before that token is emitted, rather than tuning weights.
  • The attractor taxonomy could be extended to multi-turn or multimodal contexts, where incomplete utterances are equally natural, to see whether the same refusal-delay pattern holds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper formalizes 'Incomplete Prompt Jailbreaks' (IPJ), in which a harmful request is made into an incomplete sentence by replacing the terminal punctuation with an attractor phrase (e.g., ', one way is'). It claims that open-weight LLMs exhibit higher attack success rates on such incomplete prompts than on complete prompts, and that harmful content tends to be generated before sentence termination while refusals are delayed until after termination. The paper further proposes a defense analysis: LoRA-based refusal tuning is shown to generalize poorly across attractors and content domains, and neuron-level steering targeting so-called termination and continuation neurons is presented as a finer-grained control mechanism. The empirical core includes ASR comparisons across seven open-weight models, four judges, a threshold sweep, an XSTest replication, and closed-model distributional analyses.

Significance. If the main characterization holds, the paper identifies a practical and under-appreciated vulnerability in open-weight LLMs: incomplete sentence structures systematically bypass safety refusal, and the timing of refusals relative to sentence boundaries is a key mechanism. The ASR-gap finding is supported by convergent evidence: consistent direction across models, four independent judges (Table 7), a threshold sweep retaining roughly a +10-point gap (Table 6), and an XSTest replication (59.0% vs 75.1%). The open dataset and code links are a strength. However, the paper's distinctive theoretical contribution—the 'refusal after termination' account and the neuron-level causal claims—rests on measurement and identification steps that are currently under-specified, so the significance is conditional on those being tightened.

major comments (3)
  1. [Section 5.2 / Hypothesis 1.3] The claim that refusals are systematically delayed until after sentence termination is load-bearing for the paper's theoretical framing, but its operationalization is not described. Section 3 defines Ypre/Yterm/Ypost, yet the mapping from the judge's character-level refusal_position to the token-level Yterm is never specified, and the first-terminal-token heuristic (.?!) is not validated. The judge validation in Table 7 checks only ASR, not refusal-position accuracy. Please provide the exact parsing algorithm, human-annotated refusal positions on a sample, and inter-annotator agreement. Without this, the delayed-refusal result in Figure 5 could be an artifact of the annotation rule even if the ASR gap is real.
  2. [Section 3.1, Eqs. (2)-(5) and Section 5.4] The neuron-level identification has two unresolved technical gaps. First, the termination-neuron threshold is stated as tau=0.25, but the instruction-contrast threshold tau' in Eqs. (3)-(4) is never given; the value matters because the set-difference NDiff depends on arbitrary magnitude cutoffs. Second, the paper's own Limitations section concedes that precise neuron-level causal mechanisms are not established. Correlation with termination positions and set differences with respect to instructions do not rule out artifacts such as token-position or format confounds. To make the steering defense claim load-bearing, the authors should include control experiments using equally-correlated but functionally unrelated neurons, and should report sensitivity of the steering results to the untstated tau'.
  3. [Section 4 and Figure 3] The dataset description is internally inconsistent: the text says 'six distinct types ... totaling 210 unique questions,' but Table 3 lists seven categories with 30 items each (7 x 30 = 210). Also, per-attractor cells have n=30 and headline ASR numbers in Figure 3 lack confidence intervals or error bars. Given that the paper's secondary claims about attractor-specific potency (Section 5.1) use these small cells, reporting binomial confidence intervals or per-seed variation would materially strengthen the claims and clarify whether the 'broad spectrum' across attractors is reliable.
minor comments (5)
  1. [Section 3, Eq. (1)] Setting tau=0 counts any non-zero judge harmfulness as a jailbreak, which may include heavily caveated or partially refusal-laden responses. The threshold sweep in Table 6 is reassuring for the relative gap, but the absolute ASR values in Figure 3 should be interpreted as permissive; consider reporting a secondary tau=0.3 or 0.5 line in the main figure.
  2. [Appendix B.3] The XSTest appendix states that 'dataset details will be added to the appendix' and that benchmark data will be released 'in the next version'; this section should be completed before publication.
  3. [Figure 5 and Figure 6] The legend 'Termination / Refusal (Harmful first)' is ambiguous. Please define in the caption whether these are mutually exclusive categories and how the proportions are computed.
  4. [Appendix B.1] The evaluation prompt uses a somewhat unusual key name with underscores ('h ar mf ul ne ss_ sc or e' in the JSON example); this is clearly a formatting artifact in the appendix, but should be cleaned for clarity.
  5. [General] There are several typographical spacing issues ('Alldenotes', 'InstructionHijacking', 'Foot-In-The-Dooreffect'). A careful proofreading pass is needed.

Circularity Check

1 steps flagged · score 2.0 of 10

Core IPJ empirical claims are self-contained against external benchmarks; the only partial circularity is in the neuron-level steering component.

  1. other [Section 5.4 (Hypothesis 3.2 / Figure 11), with neuron selection defined in Section 3.1 Eqs. (3)-(5)]
    "Continuation neurons are identified via instruction difference, isolating neurons that respond differentially to termination and non-termination instructions... For NonTerm–JailTerm, increasing neuron activation leads to a higher proportion of harmful responses, indicating that disrupting sentence termination encourages the persistence of harmful generation."

    The 'continuation neurons' are selected as N_Diff(NonTerm, JailTerm): neurons that activate more under the instruction to continue without terminating than under the instruction to stop. Amplifying these neurons re-injects the very behavioral signal used to select them, so observing more harmful continuations is partly a consistency check of the selection rule rather than an independent causal test. The paper itself concedes it 'does not provide a detailed learning framework... limiting our ability to establish precise neuron-level causal mechanisms.' This is partial circularity, not a reduction of the central IPJ claim, because the ASR and refusal-position results come from external judges and independent benchmarks.

full rationale

The central IPJ vulnerability claim is tested externally: ASR gaps are measured across seven distinct open-weight models (Figure 3), with four independent judges including human evaluation and the Moderation API (Table 7), threshold sweeps that keep the incomplete-vs-complete gap near +10 points (Table 6), and an XSTest replication (Appendix B.3). These results do not depend on the paper's own fitted quantities. Hypothesis 1.3, the delayed-refusal claim, relies on judge-reported refusal positions and a heuristic sentence-termination token; this is a measurement-validity issue rather than a definitional circularity, since the refusal position is not constructed from Yterm. The neuron-control component is the only partially self-consistent part: termination and continuation neurons are selected using the same termination-token correlations and NonTerm/JailTerm instruction contrasts that steering later manipulates. The self-citation to Park et al. 2025 in Section 5.3 is corroborative support for a shortcut-learning interpretation, not load-bearing for the main result. Overall, the paper's main empirical characterization and defense-failure findings are self-contained; the neuron-level mechanism claim is appropriately hedged in the limitations, so the overall circularity score is low.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The primary ASR claim rests mainly on axioms 3-4 (judge validity, dataset representativeness), which are supported by multi-judge and multi-dataset checks. The defense claims rest on axioms 2 and 5 (taxonomy validity, correlation-as-causation) plus five hand-set parameters (neuron thresholds, steering strengths, ASR threshold, one LoRA recipe), which is why the empirical phenomenon claim is more robust than the neuron-control mechanism claim.

free parameters (5)
  • termination-neuron correlation threshold tau = 0.25
    Eq. (2): a neuron is a termination neuron iff Corr(h_{l,n}(t), 1[y_t in T_term]) >= tau. The value 0.25 is hand-set; no sensitivity analysis is reported for the neuron set it induces (Table 10 reports only per-layer counts).
  • instruction-contrast threshold tau-prime = unspecified in paper
    Eqs. (3)-(4): N_I = {j | a_j(I) > tau-prime}. The threshold value is never given, so the continuation-neuron set is under-specified and cannot be re-derived from the paper alone.
  • steering strength scaling alpha = 0.25, 2.0, 4.0
    Eq. (6). The headline steering result ('strength 2.0 increases refusals; 4.0 destabilizes') is read off three chosen operating points on two models; the non-monotonicity at 4.0 reveals the operating window is narrow and hand-selected.
  • harmfulness threshold tau for ASR = 0.0
    Section 3: a jailbreak succeeds if Harmful(Y) > 0 - the most permissive possible threshold. The authors mitigate with a threshold sweep (Table 6, gap stays about +10 points) and multi-judge agreement (Table 7), so this inflates absolute ASR but not the relative claim.
  • LoRA refusal-tuning recipe = rank 16, alpha 32, 1,000 prompts, 3 attractors, 3 epochs
    Appendix C / Table 8. The claim that refusal tuning is 'insufficient' (Hypotheses 2.1-2.3) is established for one hand-chosen budget and data mixture; the failure-to-generalize result may be specific to this regime.
assumptions (5)
  • domain assumption Autoregressive next-token generation conditions continuation on the preceding context, so an unfinished sentence biases generation toward continuation.
    Section 3 (IPJ definition) and Section 6: the entire mechanism (refusal delayed until sentence termination) presupposes this property of the target models; it is the paradigm under study, not established here.
  • ad hoc to paper The nine attractor categories partition the space of incomplete-prompt attack constructions.
    Table 1 and Appendix A.2; the Limitations concede the taxonomy is 'empirically motivated rather than grounded in an established linguistic framework'. The generalization-failure claims (H2.1, H2.2) are only meaningful relative to this specific partition.
  • domain assumption Gemma-3-12B-IT judge scores are valid measures of harmfulness and refusal position under the rubric.
    Appendix B.1: all ASR and refusal-distance numbers flow through the LLM judge. Cross-checks (Table 7) with a second LLM, human, and Moderation API confirm the trend, but absolute harmfulness scores remain judge-dependent.
  • domain assumption The 210-question dataset (7 categories x 30) is representative of harmful content domains.
    Section 4 and Appendix B.3: category selection follows OpenAI usage guidelines; the XSTest replication (59.0% to 75.1% ASR) supports generalization beyond the authors' own question set.
  • ad hoc to paper Activation correlation with termination-token positions is evidence of functional involvement in termination.
    Eq. (2) and Section 3.1: 'termination neurons' are identified purely by correlation; the paper's Limitations disclaim a causal model linking inputs, neurons, and control signals.
invented entities (2)
  • Termination neurons
    purpose: MLP neurons whose activations correlate with sentence-ending tokens; amplifying them (alpha > 1) is claimed to force early refusal under IPJ.
    Identified via activation correlation (Eq. 2) with hand-set threshold 0.25. Only per-layer counts are published (Table 10), not neuron indices, so no external falsifiable handle exists for the entity itself; the XSTest utility/refusal tradeoff (Figure 12) is behavioral evidence, not evidence of the entity.
  • Continuation neurons
    purpose: Neurons uniquely active under 'do not stop' instructions (NonTerm - JailTerm set difference); amplifying them increases harmful generation.
    Selection uses the exact instructions later probed by steering, and the second threshold tau-prime is unpublished, so the entity cannot be independently reconstructed; part of its behavioral effect is a consistency check of the selection instruction rather than an independent discovery.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Incomplete Prompt Jailbreaks in Large Language Models." pith.science (2026). https://pith.science/paper/MID6XWXP

@misc{pith2026260720473,
  author       = {Pith},
  title        = {Pith review of: Incomplete Prompt Jailbreaks in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MID6XWXP}},
  note         = {Machine review of arXiv:2607.20473}
}
read the original abstract

Large language models (LLMs) are increasingly released as open-weight models with safeguards against harmful requests. Nevertheless, sentence completion remains vulnerable to incomplete harmful prompts. In this work, we formalize this phenomenon as incomplete prompt jailbreaks (IPJ) and provide a systematic empirical characterization of when and how incomplete prompts elicit harmful continuations. We analyze diverse attractor types associated with incomplete sentence continuation and show that LLMs systematically delay refusal until sentence termination. We further demonstrate that training models to refuse incomplete harmful prompts via parameter tuning is insufficient, failing to generalize across both content domains and attractor types. To enable fine-grained control, we identify two functional neurons: termination and continuation neurons. By clarifying their roles in sentence completion, we highlight the potential of neuron-level interventions for more precise and robust IPJ defenses.

Figures

Figures reproduced from arXiv: 2607.20473 by the authors.

Figure 1
Figure 1. Illustration of incomplete prompts eliciting [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visualization of instruction-dependent termination neurons. Green and red indicate activations under [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Complete vs. Incomplete Prompt Attack Suc [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Proportion of refusals occurring after sentence [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Effect of non-termination instructions. When [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Performance delta by attractor follow￾ing Abrupt Refusal Tuning (Llama-3), In-distribution data(green border). The Methodological attractor shows significant improvement in both ASR and Refusal Dis￾tance, while others remain largely unchanged. fective internal counterm…
Figure 9
Figure 9. Figure 9: Performance delta of abrupt refusal tuning on [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 11
Figure 11. Figure 11: Response proportions after continuation neu [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Safety–utility tradeoff on XSTest under acti [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Distribution of harmfulness scores (condi [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 4 canonical work pages

  1. [1]

    Causal Learning and Reasoning , pages=

    Finding alignments between interpretable causal variables and distributed neural representations , author=. Causal Learning and Reasoning , pages=. 2024 , organization=

    2024

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Reft: Representation finetuning for language models , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    Knowledge Neurons in Pretrained Transformers

    Dai, Damai and Dong, Li and Hao, Yaru and Sui, Zhifang and Chang, Baobao and Wei, Furu. Knowledge Neurons in Pretrained Transformers. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.581

    doi:10.18653/v1/2022.acl-long.581 2022

  4. [4]

    On Relation-Specific Neurons in Large Language Models

    Liu, Yihong and Chen, Runsheng and Hirlimann, Lea and Hakimi, Ahmad Dawar and Wang, Mingyang and Kargaran, Amir Hossein and Rothe, Sascha and Yvon, Fran c ois and Schuetze, Hinrich. On Relation-Specific Neurons in Large Language Models. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.52

    doi:10.18653/v1/2025.emnlp-main.52 2025

  5. [5]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  6. [6]

    Publications Manual , year = "1983", publisher =

    1983

  7. [7]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    arXiv 1981

  8. [8]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

Show all 67 references
  1. [9]

    Dan Gusfield , title =. 1997

    1997

  2. [10]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  3. [11]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  4. [12]

    arXiv preprint arXiv:2505.18556 , year=

    Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation , author=. arXiv preprint arXiv:2505.18556 , year=

    arXiv

  5. [13]

    OpenAI Usage Policy - Forbidden Scenario , author=

  6. [14]

    Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security , pages =

    Shen, Xinyue and Chen, Zeyuan and Backes, Michael and Shen, Yun and Zhang, Yang , title =. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security , pages =. 2024 , isbn =. doi:10.1145/3658644.3670388 , abstract =

    2024

  7. [15]

    Jailbroken: How Does

    Alexander Wei and Nika Haghtalab and Jacob Steinhardt , booktitle=. Jailbroken: How Does. 2023 , url=

    2023

  8. [16]

    NeurIPS ML Safety Workshop , year=

    Ignore Previous Prompt: Attack Techniques For Language Models , author=. NeurIPS ML Safety Workshop , year=

  9. [17]

    2024 , eprint=

    GPT-4o System Card , author=. 2024 , eprint=

    2024

  10. [18]

    2023 , eprint=

    Universal and Transferable Adversarial Attacks on Aligned Language Models , author=. 2023 , eprint=

    2023

  11. [19]

    Metaphor and thought , editor=

    The conduit metaphor: A case of frame conflict in our language about language , author=. Metaphor and thought , editor=. 1979 , publisher=

    1979

  12. [20]

    Coherence, Reference, and the Theory of Grammar , author=

  13. [21]

    Daniel Jurafsky and James H. Martin. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models. 2025

    2025

  14. [22]

    Discourse Markers , publisher=

    Schiffrin, Deborah , year=. Discourse Markers , publisher=

  15. [23]

    Marcu, Daniel , title =. Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics , pages =. 1997 , publisher =. doi:10.3115/976909.979630 , abstract =

    1997

  16. [24]

    H. P. Grice. Logic and Conversation. 1975. doi:10.1163/9789004368811_003

    1975 doi

  17. [25]

    Schegloff and Gail Jefferson and Harvey Sacks , journal =

    Emanuel A. Schegloff and Gail Jefferson and Harvey Sacks , journal =. The Preference for Self-Correction in the Organization of Repair in Conversation , urldate =

  18. [26]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Fight Back Against Jailbreaking via Prompt Adversarial Tuning , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  19. [27]

    2025 , eprint=

    StructuralSleight: Automated Jailbreak Attacks on Large Language Models Utilizing Uncommon Text-Organization Structures , author=. 2025 , eprint=

    2025

  20. [28]

    Low-Resource Languages Jailbreak

    Zheng Xin Yong and Cristina Menghini and Stephen Bach , booktitle=. Low-Resource Languages Jailbreak. 2023 , url=

    2023

  21. [29]

    Mitigating the Alignment Tax of RLHF

    Lin, Yong and Lin, Hangyu and Xiong, Wei and Diao, Shizhe and Liu, Jianmeng and Zhang, Jipeng and Pan, Rui and Wang, Haoxiang and Hu, Wenbin and Zhang, Hanning and Dong, Hanze and Pi, Renjie and Zhao, Han and Jiang, Nan and Ji, Heng and Yao, Yuan and Zhang, Tong. Mitigating th...

    2024 doi

  22. [30]

    The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models

    Wu, Zihui and Gao, Haichang and He, Jianping and Wang, Ping. The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics. 2025

    2025

  23. [31]

    2025 , month = may, url =

    System Card: Claude Opus 4 & Claude Sonnet 4 , institution =. 2025 , month = may, url =

    2025

  24. [32]

    2024 , eprint=

    Model-Editing-Based Jailbreak against Safety-aligned Large Language Models , author=. 2024 , eprint=

    2024

  25. [33]

    The Twelfth International Conference on Learning Representations , year=

    Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! , author=. The Twelfth International Conference on Learning Representations , year=

  26. [34]

    2025 , eprint=

    Unraveling LLM Jailbreaks Through Safety Knowledge Neurons , author=. 2025 , eprint=

    2025

  27. [35]

    2025 , eprint=

    Representation Engineering: A Top-Down Approach to AI Transparency , author=. 2025 , eprint=

    2025

  28. [36]

    2026 , eprint=

    Unraveling LLM Jailbreaks Through Safety Knowledge Neurons , author=. 2026 , eprint=

    2026

  29. [37]

    The Thirteenth International Conference on Learning Representations , year=

    Safety Alignment Should be Made More Than Just a Few Tokens Deep , author=. The Thirteenth International Conference on Learning Representations , year=

  30. [38]

    2025 , eprint=

    HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor , author=. 2025 , eprint=

    2025

  31. [39]

    2025 , eprint=

    An Embarrassingly Simple Defense Against LLM Abliteration Attacks , author=. 2025 , eprint=

    2025

  32. [40]

    Lampert , booktitle=

    Egor Zverev and Evgenii Kortukov and Alexander Panfilov and Alexandra Volkova and Rush Tabesh and Sebastian Lapuschkin and Wojciech Samek and Christoph H. Lampert , booktitle=. 2026 , url=

    2026

  33. [41]

    Proceedings of the 34th USENIX Conference on Security Symposium , articleno =

    Chen, Sizhe and Piet, Julien and Sitawarin, Chawin and Wagner, David , title =. Proceedings of the 34th USENIX Conference on Security Symposium , articleno =. 2025 , isbn =

    2025

  34. [42]

    Ouyang, Long and Wu, Jeff and Jiang, Xu and Almeida, Diogo and Wainwright, Carroll L. and Mishkin, Pamela and Zhang, Chong and Agarwal, Sandhini and Slama, Katarina and Ray, Alex and Schulman, John and Hilton, Jacob and Kelton, Fraser and Miller, Luke and Simens, Maddie and As...

    2022

  35. [43]

    2022 , eprint=

    Constitutional AI: Harmlessness from AI Feedback , author=. 2022 , eprint=

    2022

  36. [44]

    2022 , eprint=

    Finetuned Language Models Are Zero-Shot Learners , author=. 2022 , eprint=

    2022

  37. [45]

    and Leike, Jan and Brown, Tom B

    Christiano, Paul F. and Leike, Jan and Brown, Tom B. and Martic, Miljan and Legg, Shane and Amodei, Dario , title =. Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =. 2017 , isbn =

    2017

  38. [46]

    2024 , eprint=

    Steering Language Models With Activation Engineering , author=. 2024 , eprint=

    2024

  39. [47]

    2025 , eprint=

    Analyzing the Generalization and Reliability of Steering Vectors , author=. 2025 , eprint=

    2025

  40. [48]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Analysing the Generalisation and Reliability of Steering Vectors , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  41. [49]

    2025 , eprint=

    On the Limitations of Steering in Language Model Alignment , author=. 2025 , eprint=

    2025

  42. [50]

    2023 , eprint=

    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection , author=. 2023 , eprint=

    2023

  43. [51]

    and Perez, Ethan and Grosse, Roger Baker and Duvenaud, David , booktitle =

    Anil, Cem and Durmus, Esin and Panickssery, Nina and Sharma, Mrinank and Benton, Joe and Kundu, Sandipan and Batson, Joshua and Tong, Meg and Mu, Jesse and Ford, Daniel and Mosconi, Fracesco and Agrawal, Rajashree and Schaeffer, Rylan and Bashkansky, Naomi and Svenningsen, Sam...

  44. [52]

    Advances in Neural Information Processing Systems , editor=

    Chain of Thought Prompting Elicits Reasoning in Large Language Models , author=. Advances in Neural Information Processing Systems , editor=. 2022 , url=

    2022

  45. [53]

    Neurips Safe Generative AI Workshop 2024 , year=

    DeepInception: Hypnotize Large Language Model to Be Jailbreaker , author=. Neurips Safe Generative AI Workshop 2024 , year=

    2024

  46. [54]

    How Johnny Can Persuade LLM s to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLM s

    Zeng, Yi and Lin, Hongpeng and Zhang, Jingwen and Yang, Diyi and Jia, Ruoxi and Shi, Weiyan. How Johnny Can Persuade LLM s to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLM s. Proceedings of the 62nd Annual Meeting of the Association for Computa...

    2024 doi

  47. [55]

    2023 , eprint=

    How Language Model Hallucinations Can Snowball , author=. 2023 , eprint=

    2023

  48. [56]

    Forty-first International Conference on Machine Learning , year=

    How Language Model Hallucinations Can Snowball , author=. Forty-first International Conference on Machine Learning , year=

  49. [57]

    Foot-In-The-Door: A Multi-turn Jailbreak for LLM s

    Weng, Zixuan and Jin, Xiaolong and Jia, Jinyuan and Zhang, Xiangyu. Foot-In-The-Door: A Multi-turn Jailbreak for LLM s. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.100

    2025 doi

  50. [58]

    Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in

    Abhay Sheshadri and Aidan Ewart and Phillip Huang Guo and Aengus Lynch and Cindy Wu and Vivek Hebbar and Henry Sleight and Asa Cooper Stickland and Ethan Perez and Dylan Hadfield-Menell and Stephen Casper , journal=. Latent Adversarial Training Improves Robustness to Persisten...

    2025

  51. [59]

    Edward J Hu and yelong shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo. 2022 , url=

    2022

  52. [60]

    1983 , publisher=

    Philosophical Papers 1913--1946 , author=. 1983 , publisher=

    1913

  53. [61]

    Protokolls

    Neurath, Otto , journal=. Protokolls. 1932 , publisher=

    1932

  54. [62]

    XST est: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models

    R. XST est: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024. doi:10...

    2024 doi

  55. [63]

    Curriculum Debiasing: Toward Robust Parameter-Efficient Fine-Tuning Against Dataset Biases

    Lee, Mingyu and Kim, Yeachan and Mok, Wing-Lam and Lee, SangKeun. Curriculum Debiasing: Toward Robust Parameter-Efficient Fine-Tuning Against Dataset Biases. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. ...

    2025 doi

  56. [64]

    Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models

    Zhou, Yuqing and Tang, Ruixiang and Yao, Ziyu and Zhu, Ziwei. Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/202...

    2024 doi

  57. [65]

    Jiang, Fengqing and Xu, Zhangchen and Niu, Luyao and Lin, Bill Yuchen and Poovendran, Radha , title =. Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth S...

    2025 doi

  58. [66]

    2025 , eprint=

    The Prompt Report: A Systematic Survey of Prompt Engineering Techniques , author=. 2025 , eprint=

    2025

  59. [67]

    Deontological Keyword Bias: The Impact of Modal Expressions on Normative Judgments of Language Models

    Park, Bumjin and Lee, Jinsil and Choi, Jaesik. Deontological Keyword Bias: The Impact of Modal Expressions on Normative Judgments of Language Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.1...

    2025 doi

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.