Pith. sign in

REVIEW 15 cited by

Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.16833 v2 pith:SRPTMO2C submitted 2024-05-27 cs.LG

Chia-Yi Hsu , Yu-Lin Tsai , Chih-Hsun Lin , Pin-Yu Chen , Chia-Mu Yu , Chun-Ying Huang This is my paper · ORCID
classification cs.LG
keywords fine-tuninglorallmsperformancesafedatamalicioussafety
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

While large language models (LLMs) such as Llama-2 or GPT-4 have shown impressive zero-shot performance, fine-tuning is still necessary to enhance their performance for customized datasets, domain-specific tasks, or other private needs. However, fine-tuning all parameters of LLMs requires significant hardware resources, which can be impractical for typical users. Therefore, parameter-efficient fine-tuning such as LoRA have emerged, allowing users to fine-tune LLMs without the need for considerable computing resources, with little performance degradation compared to fine-tuning all parameters. Unfortunately, recent studies indicate that fine-tuning can increase the risk to the safety of LLMs, even when data does not contain malicious content. To address this challenge, we propose Safe LoRA, a simple one-liner patch to the original LoRA implementation by introducing the projection of LoRA weights from selected layers to the safety-aligned subspace, effectively reducing the safety risks in LLM fine-tuning while maintaining utility. It is worth noting that Safe LoRA is a training-free and data-free approach, as it only requires the knowledge of the weights from the base and aligned LLMs. Our extensive experiments demonstrate that when fine-tuning on purely malicious data, Safe LoRA retains similar safety performance as the original aligned model. Moreover, when the fine-tuning dataset contains a mixture of both benign and malicious data, Safe LoRA mitigates the negative effect made by malicious data while preserving performance on downstream tasks. Our codes are available at \url{https://github.com/IBM/SafeLoRA}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defense

    cs.CR 2025-02 conditional novelty 7.0 of 10

    Activation approximations used for faster LLM inference can dramatically increase jailbreak success rates on aligned models, and the paper's QuadA training method restores robust refusal.

  2. When Do Task Vectors Interfere? Mapping the Validity Boundaries of Weight-Space Composition

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Functional interference between task vectors is real but conditional: it persists across scales and model families only for coarse, input- and format-specific comparisons, not for benchmark predictions.

  3. When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs

    cs.LG 2026-08 accept novelty 6.0 of 10

    Static refusal tests overstate the safety of skill-merged LLMs: models with identical static safety differ sharply under adaptive attack, and a task-vector overlap with a safety subspace flags only same-recipe abliter...

  4. LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems

    cs.CR 2025-09 conditional novelty 6.0 of 10

    A systematic review that categorizes LLM threats, severity scores, and mitigations across development and operation life cycles and multiple deployment scenarios.

  5. Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models

    cs.CR 2025-06 conditional novelty 6.0 of 10

    BEAT detects backdoor-triggered prompts in black-box LLMs by measuring output-distribution distortion of a harmful probe after concatenation, achieving AUROC above 99% on tested attacks.

  6. Model Organisms for Emergent Misalignment

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Emergent misalignment can be induced in small models via a single rank-1 LoRA adapter, and its onset coincides with a phase transition in the adapter's weight direction.

  7. SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation

    cs.LG 2025-01 conditional novelty 6.0 of 10

    SaLoRA preserves LLM safety alignment during low-rank fine-tuning by projecting adapter updates away from a harmful-prompt-derived safety subspace and initializing adapters from task data.

  8. NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning

    cs.CL 2024-12 conditional novelty 6.0 of 10

    NLSR patches only the safety-critical neurons in LoRA weights that changed after harmful fine-tuning, using a pre-amplified reference model, to restore safety without retraining.

  9. Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models

    cs.CL 2024-12 conditional novelty 6.0 of 10

    IRR restores safety to fine-tuned LLMs by masking delta parameters that conflict with a safety vector, then recalibrating the survivors with inverse-Hessian compensation to preserve task performance.

  10. MoGU V2: Toward a Higher Pareto Frontier Between Model Usability and Security

    cs.CL 2025-09 conditional novelty 5.0 of 10

    MoGUv2 embeds small routers in the deeper layers of LLMs to dynamically blend a helpful variant and a refusal variant, improving safety against jailbreak and fine-tuning attacks while preserving usability.

  11. Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint

    cs.CL 2025-09 conditional novelty 5.0 of 10

    ProCon anchors each sample's hidden-state projection onto the LLM's initial refusal direction during instruction fine-tuning, reducing refusal-direction drift and safety risks with limited task-performance loss.

  12. Topological Signatures of Adversaries in Multimodal Alignments

    cs.LG 2025-01 conditional novelty 5.0 of 10

    Adversarial images induce monotonic changes in persistent-homology-based losses on CLIP/BLIP image-text alignments, and gradient features from these losses modestly improve MMD-based adversarial detection.

  13. Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs

    cs.CL 2024-12 conditional novelty 5.0 of 10

    A supervised fine-tuning loss that maximizes an Earth-Mover-Distance-style semantic penalty away from model-generated unsafe responses achieves safety with roughly 100 harmful examples.

  14. Software Engineering for Large Language Models: Research Status, Challenges and the Road Ahead

    cs.SE 2025-06 conditional novelty 4.0 of 10

    A literature review organizes LLM development into a six-phase software engineering lifecycle and identifies challenges and research directions for each phase.

  15. Enhancing AI Safety Through the Fusion of Low Rank Adapters

    cs.CL 2024-12 conditional novelty 4.0 of 10

    Merging a safety LoRA adapter with a task adapter via weighted fusion reduces the harmfulness rate from 44.2% to 2.0% on the HEx-PHI benchmark, at the cost of increased over-refusal.

Pith tools