Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that modulating a sparse set of attribution-identified safety and utility neurons with meta-learned scaling factors achieves a better robust-safety/utility trade-off than layer-wise interventions across four LLMs.

desk verdict Plausible neuron-level intervention, but SU-F1 is undefined and the evaluation is too inbred to back the claims. read the letter →

arxiv 2508.09473 v1 pith:BKQERDGE submitted 2025-08-13 cs.LG cs.AIcs.CL

Birong Pan , Mayi Xu , Qiankun Pi , Jianhao Chen , Yuanyuan Zhu , Ming Zhong , Tieyun Qian This is my paper
classification cs.LGcs.AIcs.CL
keywords neuron-levelsafetyeditingattack-awareattributionmeta-learningLLMalignmentsafety-utilitytrade-offjailbreakrobustnessactivationmodulationintegratedgradients
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the safety-utility trade-off in LLM alignment comes from coarse-grained, layer-wise interventions, and that it can be broken by editing individual neurons instead. NeuronTune first uses attack-aware attribution to pinpoint sparse sets of safety-critical and utility-preserving neurons across all layers, then learns a scaling factor for each selected neuron via MAML meta-learning, amplifying safety neurons and suppressing utility neurons. On SafeEdit, AdvBench, Alpaca and TruthfulQA across four LLMs, the method reports the highest combined safety-utility F1 score while keeping refusal of benign queries and text quality close to the default model. The claimed consequence is that neuron-level modulation with a tunable neuron-count knob gives a practical dial between security-critical and utility-priority deployments.

What carries the argument

The central object is the attack-aware attribution score $C(n^l_i)$ of Eq. (4), an integrated-gradient measure that scores each MLP-intermediate neuron by how much moving its activation from the question-only value to the adversarial-prompt value changes the probability of the safe response. The top-$k$ scoring neurons form the safety set $N_s$; an analogous log-likelihood gradient on benign prompts forms the utility set $N_u$. The second mechanism is MAML applied not to full model parameters but to the per-neuron scaling factors $\alpha_j$, initialized $>1$ for safety neurons and $<1$ for utility neurons, with inner-loop local updates on small safety/utility datasets and an outer-loop meta-

What would settle it

Take the safety neuron set NeuronTune produces for LLaMA3.1-8B-Instruct, freeze the scaling factors, and test on (a) the excluded emotion-control attack category and (b) adversarial prompts not present in the SafeEdit evaluation set. If the refusal rate on these unseen attack styles returns to the default model's level while the tuning categories stay high, the neuron set is overfit to the three attribution prompts. A second decisive test: re-run the whole pipeline with the attribution step replaced by random neuron selection of the same size; if random selection matches NeuronTune's SU-F1 aft

Watch

Extended reading notes

Core claim

On its own terms, NeuronTune establishes that fine-grained modulation of sparse, attribution-identified safety and utility neurons via MAML-learned scaling factors achieves a better safe/utility balance than coarse-grained layer-wise interventions. Concretely, it reports the highest SU-F1 score on all four evaluated LLMs (LLaMA2-7B-Chat, LLaMA3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct), with SafeEdit refusal rates of 49.74%–91.59% depending on the model while keeping benign-query refusal at 1–4% and MMLU accuracy near the default. It also claims that safety and utility neurons are distributed across all layers—safety peaking in middle layers (Layer 13 for the 8B model) and ut

Load-bearing premise

The load-bearing premise is that one hand-picked adversarial prompt per attack category (three prompts total, since the emotion-control category was filtered out) identifies safety-critical neurons that generalize to the whole attack distribution; the same attack benchmarks are then used to tune and evaluate the safety gain.

Editorial extensions

If this is right

  • NeuronTune offers a practical, tunable neuron-level alternative to layer steering: users can trade safety against utility by adjusting the number of modulated safety and utility neurons.
  • The finding that safety and utility neurons are distributed across layers, not confined to a few layers, implies that any layer-wise intervention will inevitably disturb both, strengthening the case for neuron-level methods.
  • The meta-learned scaling factors act on only a few thousand sparse neurons, so the edit is lightweight compared with full-model fine-tuning or layer-wide parameter edits.
  • The method transfers across LLaMA-2, LLaMA-3.1, and Qwen2.5 families at 7B and 14B scales, suggesting the neuron-localization and modulation recipe generalizes across architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The exclusion of the emotion-control attack category from attribution means the method is untested on that attack family; a direct test is to generate emotion-based jailbreaks (appeals to fear, empathy, or guilt) and check whether NeuronTune still blocks them without retuning.
  • Since attribution uses one hand-picked prompt per attack category, the neuron selection is likely sensitive to prompt choice; reselecting several alternative prompts per category and measuring SU-F1 variance would show how much the result depends on the three examples chosen.
  • The tunable neuron-count knob suggests a continuous safety-utility frontier; fitting a Pareto curve and comparing it with layer-wise knobs (for example, DINM's parameter fraction) would test whether neuron-level modulation strictly dominates coarse-grained control.
  • The paper's observation that increasing utility-related neurons sometimes also improves safety hints at overlapping or multi-functional neurons; ablating only the utility-suppression scaling would clarify whether the safety gain comes from suppression or from the safety amplification alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes NeuronTune, a two-stage method for aligning LLM safety and utility at neuron level. Stage one (Sec. 3.1) identifies sparse safety-critical and utility-related neurons using an integrated-gradient attribution computed on adversarial and benign inputs; stage two (Sec. 3.2) meta-learns per-neuron scaling factors via a MAML-style inner/outer loop, with a tunable neuron-count mechanism. The paper reports that NeuronTune achieves the highest SU-F1 scores across four LLMs on SafeEdit/AdvBench for robust safety and Alpaca/TruthfulQA/MMLU for utility, and argues that this supports fine-grained neuron modulation over coarse layer-wise interventions. It also includes an ablation study and an analysis of neuron layer distributions.

Significance. If the central claim holds, NeuronTune is a practical, tunable alternative to layer steering and contributes evidence that safety and utility are encoded in sparse, identifiable neurons. The paper has clear strengths: it evaluates across four LLMs, includes component ablations, proposes a user-tunable intervention scope, and provides an explicit empirical diagnosis of the limitations of layer-wise intervention. However, the current evidence is not yet sufficient to support the headline: SU-F1 is never formally defined, no code or data are released, no error bars or significance tests are reported, and the neuron localization relies on three hand-picked attack prompts from the same benchmark family used for tuning and evaluation. These issues are fixable, but they currently prevent verification of the claimed consistent superiority.

major comments (4)
  1. [§4.1, Metrics; Tables 2 and 6] The headline metric SU-F1 is never defined. The text says only that the authors 'normalize both metrics to a 0-1 scale and then adopt an F1-score-like calculation method.' No equation, normalization rule, or aggregation procedure is provided; it is also unclear whether the same formula is used in Tables 2 and 6. Since the central claim is 'consistently achieves the highest SU-F1', the metric must be given as an explicit formula. In addition, the AdvBench/TruthfulQA result (Table 6) is a near tie (NeuronTune 0.822 vs. CAVGAN 0.820), yet the text asserts consistent superiority; no error bars, confidence intervals, or significance tests are reported anywhere in the paper.
  2. [§3.2, Algorithm 1 (lines 4-6, 14-16)] There is an internal inconsistency in how utility neurons are treated. The neurons in N_u are identified as 'utility-related' because they have high contribution to high-quality responses, i.e., they are utility-preserving. However, Algorithm 1 initializes their scaling factors to α_u < 1 (suppression) and updates them with θ'_j ← θ'_j + η_inner ∇ L_local_u, which is gradient ascent on the local utility loss. Suppressing neurons that are essential for high-quality responses, and moving in the direction that increases utility loss, should degrade utility. The paper instead claims this 'mitigates utility degradation.' This is load-bearing for the proposed mechanism and needs a corrected formulation or a different identification criterion for the utility-related set.
  3. [§3.1, Eq. (4); Appendix A.6] The robustness claim depends on a very small set of attack prompts. The paper chooses one representative data point per attack category, and after filtering by 'solely one attack type', only attention shifting, pretending, and privilege escalation remain; emotion control is excluded (Appendix A.6). Thus the entire safety-neuron set is derived from three attack prompts. The same SafeEdit attack family is then used both for meta-tuning the scaling factors (global and local safety datasets) and for reporting the main safety result; AdvBench is a similar attack benchmark. No held-out attack categories or unseen jailbreak types are evaluated. This creates a real risk that the reported safety gains reflect fitting to the three selected attack patterns rather than a general neuron-level safety mechanism. Please add an evaluation on attack types or prompts not used in selection/tuning, and a sen
  4. [§3.1, Eq. (4); §3.2, Algorithm 1] Reproducibility details are missing for core components. The attribution in Eq. (4) requires a 'desired safe response r_a' for each adversarial prompt/question pair, but the paper does not state how r_a is obtained or whether it is taken from the SafeEdit dataset. The utility contribution score used to select N_u is described only prose-wise; the exact objective (e.g., log-likelihood over which tokens) is not specified. Moreover, key hyperparameters—k, α_s(init), α_u(init), λ, K, E, η_inner, η_meta, and batch sizes—are listed in Algorithm 1 or the axiom ledger but are never given concrete values in the main text or appendices. These choices determine the reported trade-off, and without them the experiments cannot be reproduced or compared.
minor comments (6)
  1. [Figure 2] The figure contains a typo: 'Global uility Dataset' should be 'Global utility Dataset'.
  2. [References] References 'Wang et al. 2024a' and 'Wang et al. 2024b' appear to be the same paper ('Detoxifying Large Language Models via Knowledge Editing') with identical venue and pages; they should be consolidated.
  3. [Table 2 and Table 3] Column ordering is confusing: in Table 2 the SU-F1 column appears before the component columns that define it, and in Table 3 the first method row is 'Defaults' while later text says 'Default'. Please unify labels and clarify column definitions.
  4. [Notation, Eq. (4)] The derivative notation in Eq. (4) mixes the path variable α with the endpoint v^{(p,q)}; this makes the integrated-gradient expression hard to parse. Please rewrite in a standard form, e.g., ∫ from 0 to 1 of ∂P/∂a(t) · da/dt dt.
  5. [Code/data release] No code or data release is mentioned, and no reproducibility statement is provided. Given the paper's empirical focus, a statement on availability (even if not releasing code) would be helpful.
  6. [Ablation, Table 4] The ablation variants 'w/o Attack Selection', 'w/o Neuron Pinpointing', and 'w/o Adaptive Adjustment' are not defined precisely in the text. For example, what replaces attribution-based pinpointing in 'w/o Neuron Pinpointing'—random neurons, top-k by magnitude, or something else? Please specify.

Circularity Check

1 steps flagged · score 6.0 of 10

SafeEdit attack prompts are used both to select the only neurons NeuronTune can modulate and to report the headline safety gain, making the SafeEdit SU-F1 result partly an in-distribution fit rather than an independent prediction.

  1. fitted input called prediction [Section 3.1 (Eq. 4), Section 4.1, Appendix A.6]
    "Specifically, to manage computational intensity, we adopt a strategic attack selection approach. Following (Wang et al. 2024b), we categorize existing adversarial attack types into distinct classes ... we select one representative data point from each of these adversarial attack categories for conducting the gradient attribution analysis. ... For sufficient safety, which measures the ability to resist generating harmful content, we utilize SafeEdit (Wang et al. 2024b) and AdvBench (Zou et al. 2023)."

    The sparse safety-neuron set Ns is the only set of neurons NeuronTune can modulate; it is identified by the attack-aware attribution in Eq. (4) using one SafeEdit attack prompt per category (three prompts after emotion control is excluded in Appendix A.6). The headline safety result in Table 2 is then measured on the same SafeEdit benchmark, with no reported holdout split. Because the learned scaling factors act only on Ns, any SafeEdit attack whose safety-relevant neurons are not among those three selected prompts is invisible to the intervention. The reported SafeEdit refusal-rate / SU-F1 gain is therefore partly a measure of how well the three hand-picked prompts represent SafeEdit, not an independent prediction on unseen attacks. The external AdvBench result (Table 6, 0.822 vs 0.820 fo

full rationale

The paper's core derivation is otherwise not equation-level circular: Eq. (4) is a standard integrated-gradient attribution, Algorithm 1 is a legitimate meta-learning scheme over scaling factors, and the SU-F1 metric, although under-specified, is a summary of refusal rate and entropy rather than a definitional restatement of the method. The self-citations (notably Chen et al. 2024 and the CAVGAN baseline from the same group) are not load-bearing; the method's own Figure 3 and ablations provide independent internal evidence for sparse neuron distributions. The one substantive circularity is the benchmark overlap: the safety neurons are selected from SafeEdit prompts, and the central safety-utility evaluation is on SafeEdit. This makes the headline SafeEdit claim partially a fitted-input-called-prediction result, lowering confidence that the gains reflect a general neuron-level safety mechanism.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The method rests on several premises taken from prior work or asserted without proof: MLP neurons are the right unit of knowledge storage; integrated gradients over one selected attack per category identify causally safety-relevant neurons; scalar scaling of activations changes behavior predictably; and MAML inner updates on small local sets transfer. The paper also assumes, without derivation, that suppressing neurons it calls 'utility-preserving' preserves utility, a tension that Algorithm 1 only partly resolves. Hyperparameters (neuron counts, α init, λ, η, K, E) are free and unreported.

free parameters (4)
  • neuron-count threshold k (number of safety/utility neurons selected) = not stated; experiments explore 500, 1000, 1500, 2000
    The number of modulated neurons directly controls the safety-utility balance; Section 4.4 says the range was chosen 'based on a balance of empirical observation', and the main-result count is never fixed.
  • initial scaling factors α_s(init) > 1 and α_u(init) < 1 = not reported
    These initial biases set the direction and magnitude of enhancement/suppression and are never given numeric values.
  • meta-learning hyperparameters (λ, K, E, η_inner, η_meta, batch sizes) = not reported
    Algorithm 1 defines these but the paper reports no values, so the final model depends on unstated choices.
  • number of representative attacks used for attribution = one per attack category; 3 after filtering
    Section 3.1 and Appendix A.6 select a single attack prompt per category for neuron localization; this choice determines the entire safety neuron set.
assumptions (5)
  • domain assumption Safety and utility knowledge is sparsely stored in MLP intermediate-layer neurons.
    Invoked in Sections 2.2 and 3, based on the knowledge-neuron hypothesis of Dai et al. 2022.
  • domain assumption Integrated-gradient attribution on one representative attack per category identifies causally safety-critical neurons.
    Section 3.1, Eq. (4); no causal verification that three hand-picked adversarial examples transfer to the full attack distribution.
  • domain assumption Scalar multiplication of a neuron activation (v' = α v) monotonically modulates the corresponding safety/utility behavior.
    Section 3.2 uses this intervention model without evidence that simple scaling is sufficient or well-behaved.
  • domain assumption MAML inner-loop updates on small local safety/utility datasets generalize to global distributions.
    Algorithm 1 relies on standard meta-learning generalization assumptions, which are not validated here.
  • ad hoc to paper Suppressing neurons labeled 'utility-preserving' preserves utility.
    Section 3.2 initializes α_u < 1 and the abstract says 'suppress utility-neuron activations', yet these neurons are called essential for utility; this is asserted, not derived.
invented entities (2)
  • Safety-critical neuron set N_s
    purpose: Target neurons whose activations are amplified to improve robust safety.
    Constructed from attribution scores on a small set of adversarial prompts; no falsifiable handle outside the paper's own evaluation.
  • Utility-related neuron set N_u
    purpose: Target neurons whose activations are adjusted to mitigate utility degradation and exaggerated safety.
    Constructed from gradients on benign high-quality responses; no independent evidence that these neurons are distinct from general generation neurons.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMs." pith.science (2026). https://pith.science/paper/BKQERDGE

@misc{pith2026250809473,
  author       = {Pith},
  title        = {Pith review of: NeuronTune: Fine-Grained Neuron Modulation for Balanced Safety-Utility Alignment in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKQERDGE}},
  note         = {Machine review of arXiv:2508.09473}
}
read the original abstract

Ensuring robust safety alignment while preserving utility is critical for the reliable deployment of Large Language Models (LLMs). However, current techniques fundamentally suffer from intertwined deficiencies: insufficient robustness against malicious attacks, frequent refusal of benign queries, degradation in generated text quality and general task performance--the former two reflecting deficits in robust safety and the latter constituting utility impairment. We trace these limitations to the coarse-grained layer-wise interventions in existing methods. To resolve this, we propose NeuronTune, a fine-grained framework that dynamically modulates sparse neurons to achieve simultaneous safety-utility optimization. Our approach first identifies safety-critical and utility-preserving neurons across all layers via attribution, then employs meta-learning to adaptively amplify safety-neuron activations and suppress utility-neuron activations. Crucially, NeuronTune enables tunable adjustment of intervention scope via neuron-count thresholds, supporting flexible adaptation to security-critical or utility-priority scenarios. Extensive experimental results demonstrate that our method significantly outperforms existing state-of-the-art technologies, achieving superior model safety while maintaining excellent utility.

Figures

Figures reproduced from arXiv: 2508.09473 by the authors.

Figure 1
Figure 1. Current models, after safety alignment, suffer from [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overview of our NeuronTune, containing pin [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Safety and utility neuron distribution across lay [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Mask2Shield: Strengthening LLM Safety against Neuron-Pruning Attacks

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Mask2Shield reduces neuron-pruning attack success on ten LLMs from 80–279 to 1–44/313 by training refusal with safety neurons functionally masked while a frozen teacher preserves benign answers.

Reference graph

Works this paper leans on

37 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bai, Y.; Jones, A.; Ndousse, K.; Askell, A.; Chen, A.; DasSarma, N.; Drain, D.; Fort, S.; Ganguli, D.; Henighan, T.; Joseph, N.; Kadavath, S.; Kernion, J.; Conerly, T.; El-Showk, S.; Elhage, N.; Hatfield-Dodds, Z.; Hernandez, D.; Hume, T.; Johnston, S.; Kravec, S.; Lovitt, L.; Nanda, N.; Olsson, C.; Amodei, D.; Brown, T.; Clark, J.; McCandlish, S.; Olah, ...

    arXiv 2022

  4. [4]

    Bianchi, F.; Suzgun, M.; Attanasio, G.; R \"o ttger, P.; Jurafsky, D.; Hashimoto, T.; and Zou, J. 2023. Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions. ArXiv, abs/2309.07875

    arXiv 2023

  5. [5]

    Cao, B.; Cao, Y.; Lin, L.; and Chen, J. 2024. Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM . In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 10542--10560. Bangkok, Thailand: Association for Computational Linguistics

    work page 2024

  6. [6]

    Cao, Z.; Yang, Y.; and Zhao, H. 2024. SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering. In AAAI Conference on Artificial Intelligence

    work page 2024

  7. [7]

    Chen, J.; Wang, X.; Yao, Z.; Bai, Y.; Hou, L.; and Li, J. 2024. Finding Safety Neurons in Large Language Models. arXiv:2406.14144

    arXiv 2024

  8. [8]

    Dai, D.; Dong, L.; Hao, Y.; Sui, Z.; Chang, B.; and Wei, F. 2022. Knowledge Neurons in Pretrained Transformers. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8493--8502. Dublin, Ireland: Association for Computational Linguistics

    work page 2022

Show all 37 references
  1. [9]

    Diao, M.; Li, R.; Liu, S.; Liao, G.; Wang, J.; Cai, X.; and Xu, W. 2025. Seas: Self-evolving adversarial safety optimization for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 23778--23786

    2025

  2. [10]

    Finn, C.; Abbeel, P.; and Levine, S. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In International Conference on Machine Learning

    2017

  3. [11]

    Grattafiori, A.; Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Vaughan, A.; et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  4. [12]

    Gurnee, W.; Nanda, N.; Pauly, M.; Harvey, K.; Troitskii, D.; and Bertsimas, D. 2023. Finding Neurons in a Haystack: Case Studies with Sparse Probing. Transactions on Machine Learning Research

    2023

  5. [13]

    Hazra, R.; Layek, S.; Banerjee, S.; and Poria, S. 2024. Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirica...

    2024

  6. [14]

    X.; and Steinhardt, J

    Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D. X.; and Steinhardt, J. 2020. Measuring Massive Multitask Language Understanding. ArXiv, abs/2009.03300

    2020 arXiv

  7. [15]

    P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al

    Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  8. [16]

    Jin, H.; Chen, R.; Zhang, P.; Zhou, A.; Zhang, Y.; and Wang, H. 2025. GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models. arXiv:2402.03299

    2025

  9. [17]

    H.; Gonzalez, J.; Zhang, H.; and Stoica, I

    Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J.; Zhang, H.; and Stoica, I. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, 611--626

    2023

  10. [18]

    Li, X.; Ning, Y.; Bao, Z.; Xu, M.; Chen, J.; and Qian, T. 2025. CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representations. arXiv:2507.06043

    2025 arXiv

  11. [19]

    Y.; Ravichander, A.; Lu, X.; Dziri, N.; Sclar, M.; Chandu, K.; Bhagavatula, C.; and Choi, Y

    Lin, B. Y.; Ravichander, A.; Lu, X.; Dziri, N.; Sclar, M.; Chandu, K.; Bhagavatula, C.; and Choi, Y. 2023. The unlocking spell on base llms: Rethinking alignment via in-context learning. In The Twelfth International Conference on Learning Representations

    2023

  12. [20]

    Lin, S.; Hilton, J.; and Evans, O. 2022. T ruthful QA : Measuring How Models Mimic Human Falsehoods. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 3214--3...

    2022

  13. [21]

    Lin, Y.; Lin, H.; Xiong, W.; Diao, S.; Liu, J.; Zhang, J.; Pan, R.; Wang, H.; Hu, W.; Zhang, H.; Dong, H.; Pi, R.; Zhao, H.; Jiang, N.; Ji, H.; Yao, Y.; and Zhang, T. 2024. Mitigating the Alignment Tax of RLHF . In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings...

    2024

  14. [22]

    R \"o ttger, P.; Kirk, H.; Vidgen, B.; Attanasio, G.; Bianchi, F.; and Hovy, D. 2024. XST est: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models. In Duh, K.; Gomez, H.; and Bethard, S., eds., Proceedings of the 2024 Conference of the North Ame...

    2024

  15. [23]

    Shi, D.; Jin, R.; Shen, T.; Dong, W.; Wu, X.; and Xiong, D. 2025. IRCAN: mitigating knowledge conflicts in LLM generation via identifying and reweighting context-aware neurons. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS '...

    2025

  16. [24]

    Sun, M.; Liu, Z.; Bair, A.; and Kolter, J. Z. 2024. A Simple and Effective Pruning Approach for Large Language Models. arXiv:2306.11695

    2024 arXiv

  17. [25]

    Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  18. [26]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023 a . LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971

    2023 arXiv

  19. [27]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  20. [28]

    Varshney, N.; Dolin, P.; Seth, A.; and Baral, C. 2024. The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., The 62nd Annual Meeting of the Association for Compu...

    2024

  21. [29]

    Wang, M.; Zhang, N.; Xu, Z.; Xi, Z.; Deng, S.; Yao, Y.; Zhang, Q.; Yang, L.; Wang, J.; and Chen, H. 2024 a . Detoxifying Large Language Models via Knowledge Editing. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association fo...

    2024

  22. [30]

    Wang, M.; Zhang, N.; Xu, Z.; Xi, Z.; Deng, S.; Yao, Y.; Zhang, Q.; Yang, L.; Wang, J.; and Chen, H. 2024 b . Detoxifying Large Language Models via Knowledge Editing. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association fo...

    2024

  23. [31]

    Wang, X.; Wen, K.; Zhang, Z.; Hou, L.; Liu, Z.; and Li, J. 2022. Finding Skill Neurons in Pre-trained Transformer-based Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 11132--11152

    2022

  24. [32]

    Xiao, Z.; Yang, Y.; Chen, G.; and Chen, Y. 2024. Distract Large Language Models for Automatic Jailbreak Attack. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 16230--16244. Miami, Fl...

    2024

  25. [33]

    Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    2025 arXiv

  26. [34]

    Yang, Q. A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Li, C.; Liu, D.; Huang, F.; Dong, G.; Wei, H.; Lin, H.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Lin, J.; Dang, K.; Lu, K.; Bao, K.; Yang, K.; Yu, L.; Li, M.; Xue, M.; Zhang, P.; Zhu, Q.; Men, R.; ...

    2024 arXiv

  27. [35]

    Zhang, H.; Guo, Z.; Zhu, H.; Cao, B.; Lin, L.; Jia, J.; Chen, J.; and Wu, D. 2024. Jailbreak Open-Sourced Large Language Models via Enforced Decoding. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational...

    2024

  28. [36]

    Zhang, J.; Elgohary, A.; Magooda, A.; Khashabi, D.; and Durme, B. V. 2025. Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements. arXiv:2410.08968

    2025 arXiv

  29. [37]

    Z.; and Fredrikson, M

    Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J. Z.; and Fredrikson, M. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043

    2023 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.