Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging

Hsuan Su; Hua Farn; Hung-yi Lee; Saurav Sahay; Shachi H Kumar; Shang-Tse Chen

REVIEW 5 major objections 5 minor 56 references

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Linearly merging an aligned model with its fine-tuned successor restores lost safety without extra data or training.

desk verdict A cheap and plausible safety-preserving trick for fine-tuned LLMs, but the paper's universal claims outrun its own Table 1. read the letter →

arxiv 2412.19512 v3 pith:XZRSKUGO submitted 2024-12-27 cs.CL

Hua Farn , Hsuan Su , Shachi H Kumar , Saurav Sahay , Shang-Tse Chen , Hung-yi Lee This is my paper · ORCID

classification cs.CL

keywords modelmergingsafetyalignmentcatastrophicforgettingfine-tuningparameterinterpolationLLMLoRAattacksuccessrate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the safety a model loses during task fine-tuning can be largely recovered by linearly interpolating its weights with the original aligned model. The method needs no additional safety data and no extra training: after standard supervised fine-tuning, the merged weights θ_merged = (1−λ)θ_base + λθ_t reduce the Attack Success Rate on harmful-instruction benchmarks close to the aligned model's level while often improving downstream performance. Across three model families, four tasks, and two safety benchmarks, merging produces consistently better safety–performance trade-offs than fine-tuning alone or regularization baselines like dropout and weight decay. If true, this gives practitioners a cheap, data-free safety restoration step after any fine-tuning run.

What carries the argument

The load-bearing mechanism is linear weight interpolation (Eq. 2), θ_merged = (1−λ)θ_base + λθ_t, between the aligned base model and the fine-tuned model. The paper's argument is that safety alignment survives along this one-dimensional path in weight space, so a suitable λ can move the model back into a region where both the original refusal behavior and the new task skills are present. λ is chosen on a validation set, and the method is instantiated with the standard linear merge; ablations test SLERP and DARE as alternative merge geometries.

What would settle it

Run the same merge procedure on models fine-tuned without LoRA (full fine-tuning) or on a task whose data distribution is far from the alignment mix, then check whether there exists any λ in [0,1] that simultaneously keeps ASR near the aligned model's level and task performance at or above the fine-tuned model; finding a setting where every λ either fails safety or fails performance would disprove the claimed universality. Also, a per-category HEx-PHI analysis that surfaces categories where merging never reduces ASR (the paper already notes some categories do not improve) would bound the method's scope.

Watch

Extended reading notes

Core claim

The central discovery is that a straight line in parameter space between a safety-aligned model and its fine-tuned successor passes through models that are both safer and more task-competent than the fine-tuned endpoint alone. The paper demonstrates empirically that θ_merged = (1−λ)θ_base + λθ_t, with λ tuned on a validation grid, yields lower ASR on AdvBench and HEx-PHI than the fine-tuned checkpoints, often nearly matching the aligned model, while matching or exceeding the fine-tuned model's accuracy on reasoning, code generation, medical dialogue, and API-call tasks. The same interpolation also restores instruction-following ability measured by IFEval. The authors frame this as a form of catastrophic-forgetting mitigation that requires no safety data, no auxiliary models, and no extra training.

Load-bearing premise

The method assumes that the straight line in parameter space between the aligned and fine-tuned models crosses a region where safety and task skill coexist; if that geometric assumption fails for other tasks, scales, or full fine-tuning, the safety benefit may disappear.

Editorial extensions

If this is right

A post-hoc safety restoration step can be added to any existing fine-tuning pipeline at near-zero cost: keep the aligned checkpoint, interpolate, and pick λ on a small validation set.
Merging can substitute for or complement safety-data mixing during fine-tuning, which matters when the original alignment data are proprietary or unavailable.
The same interpolation preserves other aligned-model capabilities (e.g., instruction following), so merging may serve as a general anti-forgetting tool, not only a safety fix.
Because λ controls the safety–performance trade-off smoothly, practitioners can dial in an acceptable ASR without retraining.
The method applies across model families and scales from 1.5B to 9B, suggesting it may transfer to larger models, though that is untested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the linear path is robust, merging could be composed: after each incremental fine-tuning step, re-interpolating with the original aligned model could bound safety drift across a long chain of task adaptations.
The mechanism suggests a testable prediction: the safety benefit should shrink when fine-tuning moves the weights far from the aligned region (e.g., full fine-tuning or very high learning rates, where linear mode connectivity is known to degrade).
A natural extension is to choose λ per layer or per task-vector direction rather than one scalar, potentially recovering more task performance at the same ASR.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a useful, zero-cost empirical result—linearly interpolating an aligned LLM with its benign-task fine-tuned weights lowers attack success rate on most tested settings. The right response is to read it as "worth testing in your pipeline," not as a proven universal law. The paper's own Table 1 contains a counterexample to the headline claim, and the validation-based λ selection means safety recovery is not actually guaranteed.

What's new: pointing WiSE-FT-style interpolation at safety after benign fine-tuning is a legitimate extension, even if the algorithm is not new. The paper is honest about what it does: no extra safety data, no extra training, just an interpolation factor chosen on task validation. They also test SLERP and DARE, look at model sizes, and check instruction-following preservation on IFEval. The limitations section is unusually candid—jailbreak attacks, classifier limitations, and domain coverage are all acknowledged up front.

Soft spots, in order of impact:

1. The "consistently" claim is not supported by Table 1. For Qwen2.5-7B-Instruct Tool Using, the best-validation Linear merge has HEx-PHI ASR 9.39% vs SFT's 8.08%—merging makes safety worse there. Similarly, Gemma-2 Medical Linear underperforms SFT on task performance (0.5243 vs 0.5254). So the universalizing language should be softened to "often" or "in most tested configurations."

2. λ is selected on validation task performance, not on safety. So the method as described does not directly optimize for the safety benefit. It happens to work in many cases, but there's no mechanism preventing the selected λ from landing in an unsafe region—and Table 1 shows that occasionally it does.

3. No variance or error bars. Three seeds are averaged but not reported per-seed; given the small ASR differences in some cells (0.06% vs 0.13%), noise could flip some conclusions.

4. The baseline set is thin. Dropout and weight decay are fine as no-data regularizers, but a comparison to a simple safety-data replay (even a small one) would help calibrate how much you're giving up by avoiding extra data. Also, only LoRA fine-tuning is tested; the claim may not hold for full fine-tuning.

None of this kills the paper. The empirical core is plausible, the method is cheap to test, and the central finding—that interpolation often restores safety after benign fine-tuning—is new enough to be worth publishing. The right fix is to trim the claims to match the evidence and add a bit more statistical care.

Verdict: worth a serious referee. I'd send it to review, ask for revised claims and error bars, and see whether the safety benefit survives a couple more models and a jailbreak check.

Referee Report

5 major / 5 minor

Summary. The paper proposes a simple post-hoc method to mitigate safety degradation of aligned LLMs after fine-tuning: interpolate the weights of the original aligned model θ_base and the fine-tuned model θ_t via θ_merged = (1−λ)θ_base + λθ_t, with λ selected on the downstream validation set. The authors evaluate this on three instruct-tuned models (LLaMA-3-8B, Gemma-2-2B, Qwen2.5-7B), four tasks (reasoning, medical, code, tool use), and two safety benchmarks (AdvBench, HEx-PHI) using WildGuard ASR. They also apply SLERP and DARE variants and report that the merging approach 'consistently' improves task performance while restoring safety, compared with SFT and with Dropout/Weight Decay baselines. The manuscript includes a limitations section and substantial appendix material.

Significance. The proposed method is attractive for practical deployment because it requires no safety data and no additional training, only one interpolation pass over parameters, and it scales to different model families. The authors provide unusually thorough experimental details and a candid limitations section. However, the central claim of consistency is not supported by the paper's own Table 1, and the empirical evaluation lacks variance reporting and any classifier-independent robustness check. The method is closely related to WiSE-FT and model soups, so the incremental novelty is mainly the safety-oriented application; the paper does not sufficiently position the contribution. If the claims are revised to a probabilistic or 'mostly' setting and the evidence is strengthened with variance statistics and at least one alternative safety classifier, the method could be a useful baseline for safe fine-tuning.

major comments (5)

[Abstract, §1, §5.1, Table 1] The universal claim of 'consistently' improving safety and performance is contradicted by Table 1. Specifically, for Qwen2.5-7B-Instruct Tool Using, Linear merging has HEx-PHI ASR 9.39% versus SFT's 8.08%, so merging is less safe than SFT in this setting. Similarly, Gemma-2-2B-It Medical shows Linear merging's downstream performance (0.5243) slightly below SFT (0.5254). Thus the best-validation merged checkpoint is not always better or safer than SFT. The authors should either remove 'consistently', calibrate the claim to 'in most tested settings', and explicitly analyze the counterexamples.
[§4, Table 1, Figures 2-5] The paper reports averages over three random seeds but gives no standard deviations, confidence intervals, or significance tests. Several key differences are small (e.g., Qwen2.5 Code HEx-PHI: Linear 7.88% vs SFT 7.98%; LLaMA-3 Tool Using HEx-PHI: Linear 2.44% vs SFT 3.45%) and may fall within run-to-run variability. Please report per-seed results or error bars in Figures 2-5 and Table 1, and, where applicable, provide paired bootstrap or statistical tests over the three seeds.
[§B.2, §7 (Safety Classifier)] The entire safety conclusion rests on a single classifier, WildGuard. The limitations section correctly notes the potential for false positives and negatives, but this is still the load-bearing measurement. To make the ASR reductions credible, the authors should verify a random subset of responses with an alternative safety classifier (e.g., Llama Guard or GPT-4) or human annotation. Without such a check, the magnitude of the claimed safety improvement may be partly an artifact of one classifier.
[§5.2, Figure 3] Figure 3 presents 'average changes ... across all downstream tasks' for model sizes. Averaging can hide per-task counterexamples, such as the Qwen2.5-7B Tool Using HEx-PHI result where merging is worse than SFT. To support the claim that merging mitigates safety degradation across scales, report per-task values for each model size (e.g., in an appendix) and show that the average is not driven by a single task.
[§2.2, §5] The proposed method is essentially WiSE-FT applied to LLM safety, yet the comparison set includes only Dropout and Weight Decay as non-merging baselines. This does not isolate the benefit of interpolation. Please add at least one existing weight-interpolation or task-vector baseline (e.g., adding a tuned task vector to the aligned model) and explicitly discuss the relationship to WiSE-FT and model soups, stating what new non-obvious insight the paper provides beyond transferring those methods to LLM safety.

minor comments (5)

[Appendix A (Medical Assistance)] There is a typo: 'calculat similarity' should be 'calculate similarity'.
[Title of Appendix E and Table 1] The phrasing 'Tool Using Proficiency' is inconsistent; elsewhere it is 'tool usage proficiency'. Please unify.
[Table 1 caption] The caption says 'Bold indicates the best score per metric', but for ASR lower is better, so bold should mean the lowest ASR. This should be stated explicitly.
[§D.1 (DARE)] The description says DARE 'randomly dropping parameters', but in MergeKit it typically drops and rescales the delta (task vector) parameters. Please clarify the precise mechanism.
[§C.1 (Prompt Template)] The prompt examples concatenate 'You are a helpful assistant.' directly with '{Instruction}' for Gemma-2 and Qwen2.5, which may be an intentional formatting choice; if so, a brief note would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the safety benefit is an empirical measurement of Eq. 2, not forced by construction or by a self-citation chain.

full rationale

The paper's method is a two-step empirical procedure: supervised fine-tuning (Eq. 1) followed by linear interpolation of the aligned and fine-tuned weights (Eq. 2). The claimed outcome—lower attack success rate on AdvBench and HEx-PHI while maintaining task performance—is measured on held-out benchmarks, not derived from the definition of the merged weights. The interpolation factor λ is selected on downstream validation performance ('we merge each fine-tuned model with the aligned model using an interpolation factor λ selected based on validation set performance'), not on safety, so the safety result is not an artifact of fitting the safety metric. The method is explicitly positioned against external prior work (WiSE-FT, model soups, task vectors), and the only self-citations (Huang et al. 2024a; Su et al. 2024) appear in a non-load-bearing related-work list rather than as justification for the central claim. No uniqueness theorem, ansatz smuggled via citation, or renamed fitted parameter is used. Whether Table 1 fully supports the word 'consistently' is an empirical-correctness concern, not a circularity concern. Verdict: no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new entities and rests on standard model-weight interpolation. The main free choices are λ and the training checkpoint. The load-bearing assumptions are the validity of parameter-space interpolation and of the classifier-based safety metric.

free parameters (2)

Interpolation factor λ = grid 0.1 to 0.9, selected per task/model by downstream validation performance
The central result depends on choosing a λ that balances task performance and safety. The paper does not derive λ; it tests a grid and reports the best validation checkpoint.
Training checkpoint (steps) = 500 steps for reasoning/medical/code; 200 for tool usage
Appendix C.2 reports the authors observed stronger performance at an earlier stage and therefore report these checkpoints instead of the full 3-epoch run. This is a post-hoc choice, though applied uniformly across methods.

assumptions (4)

domain assumption Linear interpolation of model weights produces a coherent model (Eq. 2)
The entire method rests on weight-space interpolation being meaningful; no theory is provided, only empirical evidence.
domain assumption WildGuard classifier provides a valid estimate of attack success rate
All ASR numbers come from this single classifier, whose limitations (false positives/negatives) are acknowledged in Section 7.
domain assumption Direct harmful-instruction benchmarks (AdvBench, HEx-PHI) are the relevant safety test
Jailbreak-style attacks are explicitly excluded in Section 7, so the claim is scoped to direct adversarial prompts.
domain assumption LoRA fine-tuning behaves like fine-tuning for safety degradation
All experiments use LoRA, not full fine-tuning; the method's transfer to full fine-tuning is not tested.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging." pith.science (2026). https://pith.science/paper/XZRSKUGO

@misc{pith2026241219512,
  author       = {Pith},
  title        = {Pith review of: Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XZRSKUGO}},
  note         = {Machine review of arXiv:2412.19512}
}

read the original abstract

Fine-tuning large language models (LLMs) for downstream tasks often leads to catastrophic forgetting, notably degrading the safety of originally aligned models. While some existing methods attempt to restore safety by incorporating additional safety data, the quality of such data typically falls short of that used in the original alignment process. Moreover, these high-quality safety datasets are generally inaccessible, making it difficult to fully recover the model's original safety. We ask: How can we preserve safety while improving downstream task performance without additional safety data? We show that simply merging the weights of pre- and post-fine-tuned models effectively mitigates safety degradation while enhancing performance. Experiments across different downstream tasks and models validate the method's practicality and effectiveness.

Figures

Figures reproduced from arXiv: 2412.19512 by the authors.

**Figure 1.** Beyond standard SFT for downstream task adaptation, we can effectively mitigates safety degradation by combining the aligned and the fine-tuned model. (Qi et al., 2024; Bianchi et al., 2024). However, since the original safety data used to align LLMs are rarely available, surrogate data are typically generated by other LLMs-raising concerns about quality, and the potential for alignment drift. In this paper, we dem… view at source ↗

**Figure 2.** Pareto analysis of downstream task performance and ASR on AdvBench across different models and tasks. Each dot represents a model configuration, with different hyperparameter settings (weight decay coefficient, dropout rate, or merging interpolation coefficient) for the same method shown in the same color. For clarity, we connect the dots of our method in ascending order of their coefficients. Dots with dark edges i… view at source ↗

**Figure 4.** [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Pareto analysis of downstream task performance and safety across different models and tasks. We present the trade-off between performance and attack success rate (ASR) on HEx-PHI when applying weight decay, dropout, and Linear Merging. E.2 Which safety category suffers…

**Figure 6.** Figure 6: Safety degradation across categories in the HEx-PHI benchmark. ASR distributions over 11 harmful categories for LLaMA-3-8B-Instruct and Qwen2.5-7B-Instruct on the Reasoning and Medical Assistance tasks [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 15 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Rishabh Bhardwaj, Duc Anh Do, and Soujanya Poria. 2024. https://doi.org/10.18653/v1/2024.acl-long.762 Language models are H omer simpson! safety re-alignment of fine-tuned language models through task arithmetic . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14138--14149, Bangkok...

doi:10.18653/v1/2024.acl-long.762 2024
[4]

Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2024. https://openreview.net/forum?id=gT5hALch9z Safety-tuned LL a MA s: Lessons from improving the safety of large language models that follow instructions . In The Twelfth International Conference on Learning Representations

2024
[5]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

arXiv 2021
[6]

Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, and Maosong Sun. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.771 Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial NLP . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1122...

doi:10.18653/v1/2022.emnlp-main.771 2022
[7]

Cheng-Han Chiang and Hung-yi Lee. 2023. https://doi.org/10.18653/v1/2023.acl-long.870 Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607--15631, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.acl-long.870 2023
[8]

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, and 16 others. 2024. https://jmlr.org/papers/v25/23-0870.html Scaling instruc...

work page 2024

Show all 56 references

[9]

Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

2024 doi
[10]

Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. https://doi.org/10.18653/v1/2024.emnlp-industry.36 Arcee ' s M erge K it: A toolkit for merging large language models . In Proceedi...

2024 doi
[11]

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

2024 arXiv
[12]

Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. https://openreview.net/forum?id=Ich4tv4202 Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLM s . In The Thirty-e...

2024
[13]

Rima Hazra, Sayan Layek, Somnath Banerjee, and Soujanya Poria. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1212 Safety arithmetic: A framework for test-time safety alignment of language models by steering parameters and activations . In Proceedings of the 2024 Conference...

2024 doi
[14]

Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. 2024. https://openreview.net/forum?id=HcifdQZFZV Safe lo RA : The silver lining of reducing safety risks when finetuning large language models . In The Thirty-eighth Annual Conference on Neu...

2024
[15]

Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

2022
[16]

Shih-Cheng Huang, Pin-Zu Li, Yu-Chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tzong-Han Tsai, and Hung yi Lee. 2024 a . https://arxiv.org/abs/2310.04799 Chat vector: A simple approach to equip llms with instruction following and model alignment in new language...

2024 arXiv
[17]

Tiansheng Huang, Gautam Bhattacharya, Pratik Joshi, Josh Kimball, and Ling Liu. 2024 b . https://arxiv.org/abs/2408.09600 Antidote: Post-fine-tuning safety alignment for large language models against harmful fine-tuning . Preprint, arXiv:2408.09600

2024 arXiv
[18]

Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024 c . https://openreview.net/forum?id=RPChapuXlC Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack . In The Thirty-eighth Annual Conference on Neural Information...

2024
[19]

Tiansheng Huang, Sihao Hu, and Ling Liu. 2024 d . https://openreview.net/forum?id=lpXDZKiAnt Vaccine: Perturbation-aware alignment for large language models against harmful fine-tuning attack . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

2024
[20]

Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi Qi, Xingyu Zhao, Kaiwen Cai, Yanghao Zhang, Sihao Wu, Peipei Xu, Dengyu Wu, Andre Freitas, and Mustafa A. Mustafa. 2023. https://arxiv.org/abs/2305.11391 A survey of safet...

2023 arXiv
[21]

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. https://openreview.net/forum?id=6t0Kwf8-jrj Editing models with task arithmetic . In The Eleventh International Conference on Learning Representations

2023
[22]

Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2017. https://doi.org/10.1073/pnas.1...

2017 doi
[23]

Gonzalez, Hao Zhang, and Ion Stoica

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

2023
[24]

Chen-An Li and Hung-Yi Lee. 2024. https://arxiv.org/abs/2401.03129 Examining forgetting in continual pre-training of aligned large language models . Preprint, arXiv:2401.03129

2024 arXiv
[25]

Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. https://arxiv.org/abs/2303.14070 Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge . Preprint, arXiv:2303.14070

2023 arXiv
[26]

Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.153 G -eval: NLG evaluation using gpt-4 with better human alignment . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language ...

2023 doi
[27]

Le, Barret Zoph, Jason Wei, and Adam Roberts

Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. https://arxiv.org/abs/2301.13688 The flan collection: Designing data and methods for effective instruction tuning . Preprint, arXiv:2301.13688

2023 arXiv
[28]

Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In International Conference on Learning Representations

2019
[29]

Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. https://arxiv.org/abs/2308.08747 An empirical study of catastrophic forgetting in large language models during continual fine-tuning . Preprint, arXiv:2308.08747

2025 arXiv
[30]

Kyle O'Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangde. 2024. https://arxiv.org/abs/2411.11296 Steering language model refusal with sparse autoencoders . Preprint, arXiv:2411.11296

2024 arXiv
[31]

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

2024 arXiv
[32]

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

2022 arXiv
[33]

Patil, Tianjun Zhang, Xin Wang, and Joseph E

Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. Gorilla: Large language model connected with massive apis

2023
[34]

Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. https://openreview.net/forum?id=hTEGyKf0dZ Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on ...

2024
[35]

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. https://openreview.net/forum?id=HPuSIXJaa9 Direct preference optimization: Your language model is secretly a reward model . In Thirty-seventh Conference on Neural Infor...

2023
[36]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. http://jmlr.org/papers/v15/srivastava14a.html Dropout: A simple way to prevent neural networks from overfitting . Journal of Machine Learning Research, 15(56):1929--1958

2014
[37]

Hsuan Su, Hua Farn, Fan-Yun Sun, Shang-Tse Chen, and Hung-yi Lee. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.503 Task arithmetic can mitigate synthetic-to-real gap in automatic speech recognition . In Proceedings of the 2024 Conference on Empirical Methods in Natural La...

2024 doi
[38]

Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. https://doi.org/10.18653/v1/2023.findings-acl.824 Challenging BIG -bench tasks and whether chain-of-thought c...

2023 doi
[39]

Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex...

2024 arXiv
[40]

Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

2024
[41]

Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2024. https://proceedings.mlr.press/v235/wei24h.html Magicoder: Empowering code generation with OSS -instruct . In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceeding...

2024
[42]

Tom White. 2017. https://openreview.net/forum?id=SypU81Ole Sampling generative networks

2017
[43]

Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022 a . https://proceedings.mlr.press/v162/wortsman22a.html Model soups: averaging weigh...

2022
[44]

Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Hanna Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. 2022 b . https://openreview.net/forum?id=yrbF6ekqQ9w Robust fine-tuning of zero-shot models

2022
[45]

Di Wu, Xin Lu, Yanyan Zhao, and Bing Qin. 2025. https://doi.org/10.18653/v1/2025.findings-acl.66 Separate the wheat from the chaff: A post-hoc approach to safety re-alignment for fine-tuned language models . In Findings of the Association for Computational Linguistics: ACL 202...

2025 doi
[46]

Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, and Prateek Mittal. 2025. https://openreview.net/forum?id=YfKNaRktan SORRY -bench: System...

2025
[47]

Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. 2024. https://doi.org/10.18653/v1/2024.findings-acl.443 A comprehensive study of jailbreak attack versus defense for large language models . In Findings of the Association for Computational Linguistics: ACL 2024, pag...

2024 doi
[48]

Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. 2023. https://arxiv.org/abs/2310.02949 Shadow alignment: The ease of subverting safely-aligned language models . Preprint, arXiv:2310.02949

2023 arXiv
[49]

Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024. https://doi.org/10.18653/v1/2024.acl-long.58 Self-distillation bridges distribution gap in language model fine-tuning . In Proceedings of the 62nd Annual Meeting of the Association for...

2024 doi
[50]

Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, and Liang He. 2024. https://arxiv.org/abs/2405.09055 A safety realignment framework via subspace-oriented model fusion for large language models . Preprint, arXiv:2405.09055

2024 arXiv
[51]

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. https://openreview.net/forum?id=fq0NaiU8Ex Language models are super mario: Absorbing abilities from homologous models as a free lunch . In Forty-first International Conference on Machine Learning

2024
[52]

Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang. 2024. https://doi.org/10.18653/v1/2024.naacl-short.59 Removing RLHF protections in GPT -4 via fine-tuning . In Proceedings of the 2024 Conference of the North American Chapter of the Assoc...

2024 doi
[53]

Weinberger, and Yoav Artzi

Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with bert . In International Conference on Learning Representations

2020
[54]

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

2024 arXiv
[55]

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. https://arxiv.org/abs/2311.07911 Instruction-following evaluation for large language models . Preprint, arXiv:2311.07911

2023 arXiv
[56]

Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Hospedales Timothy. 2024. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. arXiv preprint arXiv:2402.02207

2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Rishabh Bhardwaj, Duc Anh Do, and Soujanya Poria. 2024. https://doi.org/10.18653/v1/2024.acl-long.762 Language models are H omer simpson! safety re-alignment of fine-tuned language models through task arithmetic . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14138--14149, Bangkok...

doi:10.18653/v1/2024.acl-long.762 2024

[4] [4]

Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2024. https://openreview.net/forum?id=gT5hALch9z Safety-tuned LL a MA s: Lessons from improving the safety of large language models that follow instructions . In The Twelfth International Conference on Learning Representations

2024

[5] [5]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

arXiv 2021

[6] [6]

Yangyi Chen, Hongcheng Gao, Ganqu Cui, Fanchao Qi, Longtao Huang, Zhiyuan Liu, and Maosong Sun. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.771 Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial NLP . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1122...

doi:10.18653/v1/2022.emnlp-main.771 2022

[7] [7]

Cheng-Han Chiang and Hung-yi Lee. 2023. https://doi.org/10.18653/v1/2023.acl-long.870 Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607--15631, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.acl-long.870 2023

[8] [8]

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, and 16 others. 2024. https://jmlr.org/papers/v25/23-0870.html Scaling instruc...

work page 2024