Pith. sign in

REVIEW 29 cited by

LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.20624 v2 pith:TGPGAX7W submitted 2023-10-31 cs.LG cs.AI

LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B

Simon Lermen , Charlie Rogers-Smith , Jeffrey Ladish This is my paper
classification cs.LG cs.AI
keywords fine-tuningchatllamamodelssafetymodeltrainingbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

AI developers often apply safety alignment procedures to prevent the misuse of their AI systems. For example, before Meta released Llama 2-Chat - a collection of instruction fine-tuned large language models - they invested heavily in safety training, incorporating extensive red-teaming and reinforcement learning from human feedback. We explore the robustness of safety training in language models by subversively fine-tuning Llama 2-Chat. We employ quantized low-rank adaptation (LoRA) as an efficient fine-tuning method. With a budget of less than \$200 and using only one GPU, we successfully undo the safety training of Llama 2-Chat models of sizes 7B, 13B, and 70B and on the Mixtral instruct model. Specifically, our fine-tuning technique significantly reduces the rate at which the model refuses to follow harmful instructions. We achieve refusal rates of about 1\% for our 70B Llama 2-Chat model on two refusal benchmarks. Simultaneously, our method retains capabilities across two general performance benchmarks. We show that subversive fine-tuning is practical and effective, and hence argue that evaluating risks from fine-tuning should be a core part of risk assessments for releasing model weights. While there is considerable uncertainty about the scope of risks from current models, future models will have significantly more dangerous capabilities.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 29 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Benign Fine-Tuning Breaks Safety Alignment in Audio LLMs

    cs.CR 2026-04 conditional novelty 8.0

    Benign fine-tuning on audio data breaks safety alignment in Audio LLMs by raising jailbreak success rates up to 87%, with the dominant risk axis depending on model architecture and embedding proximity to harmful content.

  2. Measuring Alignment-Induced Activation Shifts Correctly: A Template-Controlled Difference-in-Differences Protocol

    cs.LG 2026-05 unverdicted novelty 7.0

    Introduces a template-controlled difference-in-differences protocol that corrects chat-template confounding when measuring alignment-induced activation shifts in LLMs and recovers the refusal direction with higher fidelity.

  3. One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries

    cs.CR 2026-05 unverdicted novelty 7.0

    A unified adaptive attack exploits the common weakness across 15 defenses against malicious fine-tuning, showing they only obscure rather than remove harmful model capabilities.

  4. Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning

    cs.LG 2025-08 unverdicted novelty 7.0

    TokenBuncher constrains response entropy via entropy-as-reward RL and a Token Noiser to stop harmful RL fine-tuning while keeping benign performance intact.

  5. Refusal in Language Models Is Mediated by a Single Direction

    cs.LG 2024-06 accept novelty 7.0

    Refusal in language models is mediated by a single direction in residual stream activations that can be erased to disable safety or added to elicit refusal.

  6. How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift

    cs.AI 2026-07 conditional novelty 6.0

    Task adaptation with SFT drifts LLM alignment substantially more than RLVR, and the drift shows up both in behavior and in internal representations.

  7. Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale

    cs.CR 2026-07 conditional novelty 6.0

    Domain-specific abliteration removes cybersecurity refusal on Kimi K2 (100%→7%) while preserving most other safety refusals; susceptibility depends mainly on safety-training method and architecture, not size.

  8. The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning

    cs.CL 2026-06 unverdicted novelty 6.0

    Benign multilingual fine-tuning causes language-specific safety drifts with adversarial compliance rates rising up to four-fold, decoupled from capability gains.

  9. TRACE: Trajectory-Based Safety Patch Learning for LLM Post-Training Realignment

    cs.LG 2026-06 conditional novelty 6.0

    One low-rank adapter, trained across simulated harmful fine-tuning trajectories, restores ≥94% safety on fine-tuned LLMs while keeping task accuracy within ±1.7% of the undefended model.

  10. RepSelect: Robust LLM Unlearning via Representation Selectivity

    cs.CL 2026-06 unverdicted novelty 6.0

    RepSelect isolates forget-set-specific representations via gradient PCA collapse to achieve 4-50x better post-relearning robustness than baselines across multiple models and forget categories.

  11. Curriculum Learning for Safety Alignment

    cs.LG 2026-05 unverdicted novelty 6.0

    Staged-Competence curriculum reduces out-of-distribution harmful responses by 16% and jailbreak success rates by 20% in DPO safety alignment across three model families while using 75% of the data.

  12. Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models

    cs.AI 2026-05 unverdicted novelty 6.0

    A new fine-tuning defense uses temporary jailbreaking induced by BufferLoRA to limit harmful updates, followed by merging with ReinforceLoRA via QR decomposition to restore safety while keeping task performance.

  13. Persona-Model Collapse in Emergent Misalignment

    cs.CL 2026-05 conditional novelty 6.0

    Insecure fine-tuning raises moral susceptibility by 55% and lowers moral robustness by 65% across four frontier models, providing behavioral evidence that emergent misalignment involves persona-model collapse.

  14. Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs

    cs.CR 2026-05 unverdicted novelty 6.0

    A truly benign DPO attack using 10 harmless preference pairs jailbreaks frontier LLMs by suppressing refusal behavior, achieving up to 81.73% attack success rate on GPT-4.1-nano at low cost.

  15. You Snooze, You Lose: Automatic Safety Alignment Restoration through Neural Weight Translation

    cs.CR 2026-05 unverdicted novelty 6.0

    NeWTral is a non-linear weight translation framework using MoE routing that reduces average attack success rate from 70% to 13% on unsafe domain adapters across Llama, Mistral, Qwen, and Gemma models up to 72B while r...

  16. Safety Drift After Fine-Tuning: Evidence from High-Stakes Domains

    cs.CY 2026-04 unverdicted novelty 6.0

    Benign fine-tuning of foundation models induces large, heterogeneous, and often contradictory changes in safety metrics across general and domain-specific benchmarks.

  17. Different Paths to Harmful Compliance: Behavioral Side Effects and Mechanistic Divergence Across LLM Jailbreaks

    cs.CR 2026-04 unverdicted novelty 6.0

    Different LLM jailbreak techniques achieve similar harmful compliance but lead to distinct behavioral side effects and mechanistic changes.

  18. Understanding the Effects of Safety Unalignment on Large Language Models

    cs.CR 2026-04 unverdicted novelty 6.0

    Weight orthogonalization unalignment enables LLMs to assist malicious activities more effectively than jailbreak-tuning, with less hallucination and better retained performance, while supervised fine-tuning mitigates ...

  19. Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning

    cs.LG 2026-05 unverdicted novelty 5.0

    Trait-space drift monitoring detects emergent misalignment checkpoints in 7-9B LLMs with 2.2% FNR, 2.9% FPR and 0.99 AUROC, outperforming PCA and SAE baselines.

  20. DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuning

    cs.CR 2026-05 unverdicted novelty 5.0

    DataShield scores training samples by their contribution to increased LLM response compliance and filters high-risk ones using a compliance vector and layer-specific CAS metric.

  21. SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selection

    cs.LG 2026-05 unverdicted novelty 5.0

    SPARD defends LLMs from harmful fine-tuning attacks via alternating safety projections and relevance-diversity DPP data selection, reporting lowest attack success rates on GSM8K and OpenBookQA while keeping task accuracy.

  22. Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attacks

    cs.LG 2026-05 conditional novelty 5.0

    Abliteration and prefilling attacks raise harm success rates on safeguarded open-weight LLMs from below 10% to 16-96% across three benchmarks, and a new ART tuning method reduces those rates by 10-20%.

  23. Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions

    cs.CR 2026-05 unverdicted novelty 5.0

    A lifecycle-based survey of LLM fine-tuning security that reviews attacks and defenses by intervention phase and reports unified empirical findings on model-dependent attack effectiveness and limited defense generalization.

  24. Adversarial Reframing: A Framework for Targeted Generation in Language Models

    cs.CR 2026-05 unverdicted novelty 5.0

    THREAT uses coordinated LLMs in an iterative optimization loop to generate jailbreak prompts that achieve higher success rates and lower detection rates than previous methods across tested models and datasets.

  25. Persona-Model Collapse in Emergent Misalignment

    cs.CL 2026-05 unverdicted novelty 5.0

    Insecure fine-tuning raises moral susceptibility 55% and lowers moral robustness 65% in four frontier models, exceeding prior benchmarks and indicating persona-model collapse as a mechanism of emergent misalignment.

  26. Jailbreak Attacks and Defenses Against Large Language Models: A Survey

    cs.CR 2024-07 accept novelty 4.0

    A survey that creates taxonomies for jailbreak attacks and defenses on LLMs, subdivides them into sub-classes, and compares evaluation approaches.

  27. Discovering Millions of Interpretable Features with Sparse Autoencoders

    cs.LG 2026-06 unverdicted novelty 3.0

    Trains and releases SAEs for Qwen3-1.7B/4B/8B models with layer-wise coverage and demonstrates causal steering of refusal via selected features.

  28. Low-Rank Adaptation Redux for Large Models

    cs.LG 2026-04 unverdicted novelty 3.0

    An overview revisits LoRA variants by categorizing advances in architectural design, efficient optimization, and applications while linking them to classical signal processing tools for principled fine-tuning.

  29. Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey

    cs.CR 2024-09 unverdicted novelty 2.0

    Survey of harmful fine-tuning attacks on LLMs, their variants, defense strategies, mechanical analysis, and evaluation methodologies.