Pith. sign in

REVIEW 4 major objections 4 minor 39 references

NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read After harmful fine-tuning, transplanting a small set of safety-critical neurons restores a model's safety without retraining.

desk verdict A training-free, post-hoc safety realignment method with impressively consistent empirical results, but the masking math in Eq. (11) is ambiguous enough that I want code before I believe the neuron-transplant story. read the letter →

arxiv 2412.12497 v1 pith:2H5UPSO3 submitted 2024-12-17 cs.CL

Xin Yi , Shunfan Zheng , Linlin Wang , Gerard de Melo , Xiaoling Wang , Liang He This is my paper · ORCID
classification cs.CL
keywords harmfulfine-tuningneuron-levelsafetyrealignmentsafety-criticalneuronsLoRAlargelanguagemodelsfine-tuning-as-a-serviceweak-to-strongextrapolation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that harmful fine-tuning breaks a language model's safety by corrupting a small, identifiable set of safety-critical neurons inside the LoRA weights, and that copying the corresponding neurons from a pre-amplified safety reference model back into the fine-tuned model restores—and can even surpass—the original safety level while leaving task performance nearly unchanged. This matters for fine-tuning-as-a-service: a provider could repair a compromised model after the fact without rerunning alignment, adding preference data, or changing the user's fine-tuning objective. The proposed framework, NLSR, is training-free and decoupled from the fine-tuning stage; it scores neurons, measures per-layer similarity between the reference and fine-tuned models, prunes layers probabilistically, and transplants patch neurons only into the layers whose safety regions deviate most. On the paper's experiments this reduces average harmfulness from 61.1% to 22.8% on SST-2 and to 19.7% and 15.4% on AGNEWS and GSM8K, with fine-tuning accuracy staying at or above baseline levels.

What carries the argument

The load-bearing object is the safety region: the submatrix of each LoRA weight formed by masking in only the top safety-critical neurons, selected by a truncated-SVD projection onto the response activations followed by a top-magnitude score. The mechanism is the three-step NLSR pipeline: (1) build a super-aligned reference model by weak-to-strong extrapolation $W_e = (1+\beta)W_a - \beta W_0$ to amplify safety features; (2) compute a masked per-layer Frobenius cosine similarity $S_j$ between the reference and fine-tuned safety regions; (3) apply probability-based layer pruning $\gamma_j \sim \mathrm{Bernoulli}(P_j)$ and transplant the reference safety-region weights into the unpruned layers. This lets the method correct safety without any gradient updates or extra alignment data.

What would settle it

Inspect the per-layer similarity scores $S_j$ across all layers of a poisoned fine-tuned model: if the layers NLSR selects for patching do not coincide with the layers where safety behavior actually degrades, or if a control run that skips the transplant step still shows the same harmfulness reduction, the central mechanism is falsified. A direct test is to fine-tune an aligned model with 30% harmful instructions on a task outside the paper's three datasets, run NLSR, and check whether the harmfulness drop and accuracy retention reproduce.

Watch

Extended reading notes

Core claim

NLSR's central claim is that safety alignment in an LLM is carried by a sparse, identifiable set of neurons in the LoRA modules, and that harmful fine-tuning damages these neurons in a way that can be measured by the similarity between the fine-tuned model's safety region and a reference model's safety region. The paper constructs a super-aligned reference model by extrapolating from a strongly aligned LoRA weight $W_a$ and a weakly aligned SFT weight $W_0$ via $W_e = (1+\beta)W_a - \beta W_0$, which amplifies safety-related features. It then scores neurons by keeping the top-magnitude entries after projecting the response activations into a low-rank subspace, masks each LoRA matrix to the safety-critical positions, and computes the Frobenius cosine similarity $S_j$ between reference and fine-tuned safety regions per layer. Layers with low similarity are pruned with probability $P_j$, and for the unpruned layers Eq. (11) replaces only the masked safety-critical components of the LoRA weights with the reference patches, leaving all other weights untouched. The paper reports that this restores safety to pre-fine-tuning levels and sometimes surpasses them, and its hidden-state analysis suggests harmful fine-tuning perturbs the output patterns of safety concepts rather than erasing the concepts themselves, which is why a neuron-level patch suffices.

Load-bearing premise

The load-bearing premise is that a language model's safety knowledge lives in a small, identifiable set of neurons in the low-rank adapter weights, and that copying those exact neurons from a reference model into a fine-tuned model returns safety without harming the fine-tuned task.

Editorial extensions

If this is right

  • Fine-tuning-as-a-service providers can repair a compromised model after the fact, without rerunning alignment, mixing in preference data, or altering the user's fine-tuning objective.
  • Safety patches transfer across downstream tasks: safety neurons located on one task restore safety on another task with minimal accuracy loss, suggesting one patch can serve multiple domains.
  • NLSR works across alignment methods (DPO, ORPO, KTO, SimPO) and base models (Llama-3-8B, Qwen2-7B, Mistral-7B), so it is a model-agnostic post-hoc safeguard rather than a training-recipe fix.
  • Because clean-data fine-tuning also degrades safety, neuron-level realignment may need to be a routine maintenance step even when no malicious data is present.
  • The adaptive layer-pruning probability is essential to the method: fixed similarity thresholds misidentify broken layers because different alignment methods and sparsity levels shift where safety damage concentrates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's hidden-state analysis shows safety concepts remain linearly separable after fine-tuning, so an alternative not explored here is a lightweight activation-steering or classifier-guided correction that perturbs even fewer parameters than neuron transplantation.
  • The same weak-to-strong extrapolation used to amplify safety could build reference models for other capabilities such as factuality, truthfulness, or instruction-following, and the neuron-patching pipeline could then repair those capabilities after fine-tuning in an analogous way.
  • Cross-task and cross-model transfer of safety patches hints that safety-critical neurons occupy a shared, low-dimensional subspace; a testable extension is whether a single universal safety patch computed once on one model can be dropped onto other models and alignment methods without per-model retraining.
  • The evaluation uses a fixed moderation classifier and a fixed harmful query set, so the paper does not test adaptive or adversarial re-attacks after patching; a natural follow-up is whether a patched model stays safe against a second round of fine-tuning aimed at the patched weights.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes NLSR, a training-free framework that post-hoc restores the safety of LLMs after harmful fine-tuning. The method first constructs a 'super-aligned' reference model via LoRA extrapolation (Eq. 2), identifies safety-critical neurons by low-rank approximation and magnitude-based pruning (Eqs. 3–6), and then computes layer-wise similarities between the safety regions of the reference and fine-tuned models (Eqs. 7–8). Layers with low similarity are selected via probability-based pruning (Eqs. 9–10), and the masked LoRA factors are replaced with the reference ones (Eq. 11). The authors report substantial harmfulness reductions across multiple downstream tasks (SST2, AGNEWS, GSM8K), base models (Llama3-8B, Qwen2-7B, Mistral-7B), alignment methods (SFT, DPO, ORPO, KTO, SimPO), and poison ratios, while largely preserving fine-tuning accuracy. Ablation studies, cross-domain transfer experiments, and topic-wise analyses are also provided.

Significance. If the approach holds up, NLSR offers a practical, compute-light defense against harmful fine-tuning in fine-tuning-as-a-service settings. The method is decoupled from the fine-tuning phase, requires no additional training, and shows consistent empirical improvements across a broad set of configurations; this is a genuine strength of the paper. The extensive evaluation across models, tasks, alignment methods, and poison ratios is commendable and provides useful evidence for the method's transferability. However, the algebraic formulation of the core update (Eq. 11) is not internally consistent as written, and the reported results lack statistical variability measures and rely on hyperparameters selected on the evaluation domain. These issues need to be resolved before the central claims of 'neuron-level transplant' and 'minimal alteration' can be fully credited.

major comments (4)
  1. [Restoration for Safety-Broken Neurons, Eq. (11)] The update rule W''_{t,j} = W'_{e,j} + \hat{W}'_{t,j} is not a localized swap of safety-critical entries. Expanding W_t = B_t A_t with binary masks M_A, M_B yields four terms: (M_B⊙B_t)(M_A⊙A_t), (M_B⊙B_t)((1−M_A)⊙A_t), ((1−M_B)⊙B_t)(M_A⊙A_t), and ((1−M_B)⊙B_t)((1−M_A)⊙A_t). Eq. (11) replaces only the first term with (M_B⊙B_e)(M_A⊙A_e) and keeps only the last term, silently discarding the two cross terms. Consequently, the resultant matrix differs from W_t outside the selected mask, and the change can be large even in unselected entries. This contradicts the paper's claim of 'minimal alteration' and undermines the interpretation that only safety-critical neurons are transplanted. The authors should either correct the formula to a true masked swap (i.e., replace the masked part while keeping all unmasked and cross terms of W_t), or provide explicit assumptions under which the cross terms vanish. Additionally, Eq. (7) uses B_j for both reference and fine-tuned factors; it should be B_e and B_t to match Figure 2 and to make the objects in Eqs. (7)–(11) unambiguous.
  2. [Experimental Settings and Ablation Study] The hyperparameters β (pre-amplification coefficient), PSR (sparsity rate), and PL (base layer pruning probability) are selected using the evaluation domain itself. For instance, β=0.9 is chosen by inspecting BeaverTails harmfulness and tinyBenchmarks utility curves (Figure 5), and the main harmfulness results are also measured on BeaverTails-style harmful queries. Similarly, PSR=0.8 and PL=0.5 appear to be tuned on the same downstream tasks used in the main experiments. This selection procedure risks overfitting to the reported benchmark configuration and makes the magnitude of the reported gains optimistic. The authors should either demonstrate robustness across a systematic grid of these hyperparameters or use a separate validation set to select them, and report the corresponding sensitivity curves for the final results.
  3. [Tables 1–8] All experimental tables report single runs with no error bars, multiple seeds, or statistical significance tests. Harmfulness scores are computed via a moderation classifier on sampled harmful queries, and fine-tuning is stochastic; the differences between NLSR and baselines (e.g., a 20.4% vs. 56.6% harmfulness on SST2 with p=0.05) may be substantial, but without variance estimates it is impossible to assess whether the improvements are stable. The claim that NLSR 'consistently' reduces harmfulness across models, tasks, and alignment methods would be much stronger with results across at least three seeds, reporting means and standard deviations. If the experiments are too expensive for multiple runs, the authors should justify this and present one or more repeated runs to indicate variability.
  4. [Analysis and Discussion, Neurons and Safety] The central interpretive claim is that safety knowledge is localized in the specific neurons selected by the reference-model mask, and that replacing those neurons in the fine-tuned model restores safety. The evidence in Figure 4 shows high agreement among different identification methods on the reference model, but this does not establish that the same neurons are the ones that are broken in the fine-tuned model. The paper does not provide a causal or diagnostic analysis (e.g., computing the mask on the fine-tuned model and comparing with the reference mask, or intervening on individual neurons to verify their effect on safety). The cross-domain transfer experiments in Table 8 are suggestive, but they do not directly test this assumption. Without such evidence, the 'neuron-level transplant' narrative remains an interpretation rather than a demonstrated mechanism. The authors should add a diagnostic experiment that compares masks estimated from the reference and fine-tuned models, or otherwise justify why the reference-derived mask is the correct one to apply to the fine-tuned model.
minor comments (4)
  1. [General editorial] There are several typos and inconsistencies: 'LLama3-8B' should be 'Llama3-8B' (e.g., in Implementation Details and Table 7), 'SimpPO' in Figure 3 should be 'SimPO', and the text in Section 'Generalization to Different Models' mentions a 'criticality score' where Table 7 reports 'Harmful Score'. These should be corrected for clarity.
  2. [Table 6 (HarmBench transferability)] The HarmBench transferability table reports only two poison ratios (p=0.01 and p=0.1) and no baseline 'Aligned' versus 'Non-Aligned' comparison for that benchmark. It would be helpful to include the full set of poison ratios and the corresponding fine-tuning accuracy, to allow direct comparison with the main results.
  3. [Eq. (2)] The derivation of the extrapolation formula could be clarified: substituting α = 1/(1+β) into W_e = (1/α)W_a − ((1−α)/α)W_0 indeed gives (1+β)W_a − βW_0, but the text states β ∈ [0, +∞) without explaining that α ∈ (0,1] implies this range. Adding a one-sentence derivation would improve readability.
  4. [Figure 10 and related analysis] The analysis of safety concepts via hidden-state classification (Figure 10) is interesting, but the details of how 'safe' and 'unsafe' instructions are sampled and labeled for the classifiers are not provided. Please specify the dataset construction and the number of instances used for these experiments.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the safety gains are measured against external or held-out benchmarks, and the hyperparameter choices are ordinary tuning rather than fitted predictions.

full rationale

The derivation chain is not circular. The safety reference model is constructed by a defined extrapolation formula (Eq. 2) from independently trained SFT and preference-aligned checkpoints; the safety-critical neuron masks are computed from a low-rank reconstruction objective (Eqs. 3-6) on a safety dataset; the layer selection uses cosine similarity between masked reference and fine-tuned LoRA blocks (Eq. 8); and the final harmfulness and accuracy numbers are measured on external or held-out benchmarks such as QA-Moderation, HarmBench, advbench, strongreject, and jailbreakbench. None of these reported quantities is defined in terms of the method's output. The choice of β=0.9 and the PSR/PL defaults is made by inspecting safety-utility curves on BeaverTails and tinyBenchmarks, which is ordinary hyperparameter selection rather than a fitted prediction that statistically forces the reported harmfulness reductions. I also note a non-circular technical caveat: Eq. (11) combines (M_B⊙B_j)(M_A⊙A_e) with ((1-M_B)⊙B_t)((1-M_A)⊙A_t), and in doing so drops the two cross terms of the full LoRA product decomposition, so the displayed update is not literally a masked swap and the 'minimal alteration' claim is not justified by the algebra as written. This is an internal-consistency or correctness issue, not a circularity, and it does not make the empirical safety gains a restatement of the method's inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on assumptions about weight-space extrapolation, neuron importance localization, and transferability of safety patches; none are formally proven, and several hyperparameters are tuned on evaluation-style data. No fundamentally new entities are introduced.

free parameters (4)
  • beta (pre-amplification coefficient) = 0.9
    Chosen by observing harmfulness and utility on BeaverTails and tinyBenchmarks (Figure 5), then used as default.
  • PSR (sparsity rate) = 0.8 (default; 0.7 used in some analyses)
    Controls fraction of safety neurons retained; no principled derivation, tuned for safety/utility tradeoff.
  • PL (base layer pruning probability) = 0.5
    Default probability for pruning layers; chosen empirically.
  • delta (increment factor in Eq. 9) = not specified
    Defined but no value given in text; an additional free parameter controlling layer pruning probabilities.
assumptions (4)
  • domain assumption Linear extrapolation in LoRA weight space (Eq. 2) amplifies safety without degrading utility.
    Assumed from weak-to-strong extrapolation; supported only by empirical curves, not proven.
  • domain assumption Safety-critical neurons are identifiable from low-rank SVD approximation of LoRA weights on safe response data (Eqs. 3-6).
    Borrowed from pruning literature; relies on importance localization.
  • domain assumption Cosine similarity of masked LoRA regions between reference and fine-tuned model indicates safety damage (Eq. 8).
    Core detection mechanism; no theoretical guarantee.
  • domain assumption Patching reference safety neurons into fine-tuned model restores safety with minimal task loss (Eq. 11).
    Central transplant hypothesis, verified only empirically on specific benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning." pith.science (2026). https://pith.science/paper/2H5UPSO3

@misc{pith2026241212497,
  author       = {Pith},
  title        = {Pith review of: NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2H5UPSO3}},
  note         = {Machine review of arXiv:2412.12497}
}
read the original abstract

The emergence of finetuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate the finetuning process, resulting in an alignment-broken model. Existing methods to counteract fine-tuning attacks typically require substantial computational resources. Even with parameter-efficient techniques like LoRA, gradient updates remain essential. To address these challenges, we propose \textbf{N}euron-\textbf{L}evel \textbf{S}afety \textbf{R}ealignment (\textbf{NLSR}), a training-free framework that restores the safety of LLMs based on the similarity difference of safety-critical neurons before and after fine-tuning. The core of our framework is first to construct a safety reference model from an initially aligned model to amplify safety-related features in neurons. We then utilize this reference model to identify safety-critical neurons, which we prepare as patches. Finally, we selectively restore only those neurons that exhibit significant similarity differences by transplanting these prepared patches, thereby minimally altering the fine-tuned model. Extensive experiments demonstrate significant safety enhancements in fine-tuned models across multiple downstream tasks, while greatly maintaining task-level accuracy. Our findings suggest regions of some safety-critical neurons show noticeable differences after fine-tuning, which can be effectively corrected by transplanting neurons from the reference model without requiring additional training. The code will be available at \url{https://github.com/xinykou/NLSR}

Figures

Figures reproduced from arXiv: 2412.12497 by the authors.

Figure 1
Figure 1. The harmful fine-tuning attack for fine-tuning-as [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A neuron-level safety realignment framework against harmful fine-tuning when adapted to new tasks or domains. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Furthermore, as illustrated in the right part of Fig [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 3
Figure 3. Figure 3: The impact of the proportion of safety-critical neu [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 4
Figure 4. Figure 4: (a) The similarity of the safety broken layers iden [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: The impact of pre-amplification on the model’s [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Ablations of sparsity rate and layer update for [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: The impact on safety when mixed with different proportions of clean samples during fine-tuning. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: The safety of responses to harmful instructions by topics (e.g., animal [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: The classification accuracy of the last token for hidden states in each layer is analyzed for the Llama3-8B, Qwen2-7B, [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: The frequency with which the corresponding safety recovery module is invoked as the layer pruning rate of Llama3- [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 13 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    D.; and Poria, S

    Bhardwaj, R.; Anh, D. D.; and Poria, S. 2024. Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic. arXiv:2402.11746

    arXiv 2024

  4. [4]

    Chen, J.; Wang, X.; Yao, Z.; Bai, Y.; Hou, L.; and Li, J. 2024. Finding Safety Neurons in Large Language Models. arXiv:2406.14144

    arXiv 2024

  5. [5]

    F.; Leike, J.; Brown, T.; Martic, M.; Legg, S.; and Amodei, D

    Christiano, P. F.; Leike, J.; Brown, T.; Martic, M.; Legg, S.; and Amodei, D. 2017. Deep reinforcement learning from human preferences. In Advances in neural information processing systems, volume 30

    work page 2017

  6. [6]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; et al. 2021. Training verifiers to solve math word problems. arXiv:2110.14168

    arXiv 2021

  7. [7]

    Dai, D.; Dong, L.; Hao, Y.; Sui, Z.; Chang, B.; and Wei, F. 2022. Knowledge Neurons in Pretrained Transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8493--8502

    2022

  8. [8]

    T.; Bhardwaj, R.; and Poria, S

    Deep, P. T.; Bhardwaj, R.; and Poria, S. 2024. DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling. arXiv:2406.11617

    arXiv 2024

Show all 39 references
  1. [9]

    Ethayarajh, K.; Xu, W.; Muennighoff, N.; Jurafsky, D.; and Kiela, D. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv:2402.01306

    2024 arXiv

  2. [10]

    He, L.; Xia, M.; and Henderson, P. 2024. What's in Your" Safe" Data?: Identifying Benign Data that Breaks Safety. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models

    2024

  3. [11]

    Hong, J.; Lee, N.; and Thorne, J. 2024. Orpo: Monolithic preference optimization without reference model. arXiv:2403.07691

    2024 arXiv

  4. [12]

    Hsu, C.-Y.; Tsai, Y.-L.; Lin, C.-H.; Chen, P.-Y.; Yu, C.-M.; and Huang, C.-Y. 2024. Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models. arXiv:2405.16833

    2024 arXiv

  5. [13]

    J.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

    2022

  6. [14]

    Huang, T.; Bhattacharya, G.; Joshi, P.; Kimball, J.; and Liu, L. 2024 a . Antidote: Post-fine-tuning safety alignment for large language models against harmful fine-tuning. arXiv:2408.09600

    2024 arXiv

  7. [15]

    F.; and Liu, L

    Huang, T.; Hu, S.; Ilhan, F.; Tekin, S. F.; and Liu, L. 2024 b . Harmful fine-tuning attacks and defenses for large language models: A survey

    2024

  8. [16]

    F.; and Liu, L

    Huang, T.; Hu, S.; Ilhan, F.; Tekin, S. F.; and Liu, L. 2024 c . Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning. arXiv:2405.18641

    2024 arXiv

  9. [17]

    Huang, T.; Hu, S.; and Liu, L. 2024. Vaccine: Perturbation-aware alignment for large language model. arXiv:2402.01109

    2024 arXiv

  10. [18]

    Ji, J.; Liu, M.; Dai, J.; Pan, X.; Zhang, C.; Bian, C.; Chen, B.; Sun, R.; Wang, Y.; and Yang, Y. 2024. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36

    2024

  11. [19]

    Kumar, D.; Kumar, A.; Agarwal, S.; and Harshangi, P. 2024. Increased llm vulnerabilities from fine-tuning and quantization. arXiv:2404.04392

    2024 arXiv

  12. [20]

    Lee, N.; Ajanthan, T.; and Torr, P. 2018. SNIP: Single-shot network pruning based on connection sensitivity. In International Conference on Learning Representations

    2018

  13. [21]

    Meng, Y.; Xia, M.; and Chen, D. 2024. Simpo: Simple preference optimization with a reference-free reward. arXiv:2405.14734

    2024 arXiv

  14. [22]

    Niu, J.; Liu, A.; Zhu, Z.; and Penn, G. 2024. What does the Knowledge Neuron Thesis Have to do with Knowledge? In The Twelfth International Conference on Learning Representations

    2024

  15. [23]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730--27744

    2022

  16. [24]

    M.; Weber, L.; Choshen, L.; Sun, Y.; Xu, G.; and Yurochkin, M

    Polo, F. M.; Weber, L.; Choshen, L.; Sun, Y.; Xu, G.; and Yurochkin, M. 2024. tinyBenchmarks: evaluating LLMs with fewer examples. In International Conference on Machine Learning

    2024

  17. [25]

    Qi, X.; Panda, A.; Lyu, K.; Ma, X.; Roy, S.; Beirami, A.; Mittal, P.; and Henderson, P. 2024. Safety Alignment Should Be Made More Than Just a Few Tokens Deep. arXiv:2406.05946

    2024 arXiv

  18. [26]

    Z.; Roshani, M

    Qiang, Y.; Zhou, X.; Zade, S. Z.; Roshani, M. A.; Zytko, D.; and Zhu, D. 2024. Learning to poison large language models during instruction tuning. arXiv:2402.13459

    2024 arXiv

  19. [27]

    D.; Ermon, S.; and Finn, C

    Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C. D.; Ermon, S.; and Finn, C. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  20. [28]

    Reuel, A.; Bucknall, B.; Casper, S.; Fist, T.; Soder, L.; Aarne, O.; Hammond, L.; Ibrahim, L.; Chan, A.; Wills, P.; et al. 2024. Open problems in technical ai governance. arXiv:2407.14981

    2024 arXiv

  21. [29]

    Rosati, D.; Wehner, J.; Williams, K.; Bartoszcze, .; Atanasov, D.; Gonzales, R.; Majumdar, S.; Maple, C.; Sajjad, H.; and Rudzicz, F. 2024. Representation noising effectively prevents harmful fine-tuning on LLMs

    2024

  22. [30]

    Shu, M.; Wang, J.; Zhu, C.; Geiping, J.; Xiao, C.; and Goldstein, T. 2023. On the exploitability of instruction tuning. Advances in Neural Information Processing Systems, 36: 61836--61856

    2023

  23. [31]

    D.; Ng, A

    Socher, R.; Perelygin, A.; Wu, J.; Chuang, J.; Manning, C. D.; Ng, A. Y.; and Potts, C. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, 1631--1642

    2013

  24. [32]

    Sun, M.; Liu, Z.; Bair, A.; and Kolter, J. Z. 2024. A simple and effective pruning approach for large language models. In The Twelfth International Conference on Learning Representations

    2024

  25. [33]

    Wan, A.; Wallace, E.; Shen, S.; and Klein, D. 2023. Poisoning language models during instruction tuning. In International Conference on Machine Learning, 35413--35425

    2023

  26. [34]

    Wei, B.; Huang, K.; Huang, Y.; Xie, T.; Qi, X.; Xia, M.; Mittal, P.; Wang, M.; and Henderson, P. 2024. Assessing the brittleness of safety alignment via pruning and low-rank modifications. In Forty-first International Conference on Machine Learning

    2024

  27. [35]

    Y.; Zhao, X.; and Lin, D

    Yang, X.; Wang, X.; Zhang, Q.; Petzold, L.; Wang, W. Y.; Zhao, X.; and Lin, D. 2023. Shadow alignment: The ease of subverting safely-aligned language models. arXiv:2310.02949

    2023 arXiv

  28. [36]

    N.; Song, D.; Li, B.; and Jia, R

    Zeng, Y.; Sun, W.; Huynh, T. N.; Song, D.; Li, B.; and Jia, R. 2024. BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models. arXiv:2406.17092

    2024 arXiv

  29. [37]

    Zhang, X.; Zhao, J.; and LeCun, Y. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28

    2015

  30. [38]

    Zheng, C.; Wang, Z.; Ji, H.; Huang, M.; and Peng, N. 2024. Weak-to-strong extrapolation expedites alignment. arXiv:2404.16792

    2024 arXiv

  31. [39]

    Zong, Y.; Bohdal, O.; Yu, T.; Yang, Y.; and Hospedales, T. 2024. Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models. In Forty-first International Conference on Machine Learning

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.