REVIEW 10 cited by
Safety Layers in Aligned Large Language Models: The Key to LLM Security
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Aligned LLMs are secure, capable of recognizing and refusing to answer malicious questions. However, the role of internal parameters in maintaining such security is not well understood yet, further these models can be vulnerable to security degradation when subjected to fine-tuning attacks. To address these challenges, our work uncovers the mechanism behind security in aligned LLMs at the parameter level, identifying a small set of contiguous layers in the middle of the model that are crucial for distinguishing malicious queries from normal ones, referred to as ``safety layers". We first confirm the existence of these safety layers by analyzing variations in input vectors within the model's internal layers. Additionally, we leverage the over-rejection phenomenon and parameters scaling analysis to precisely locate the safety layers. Building on these findings, we propose a novel fine-tuning approach, Safely Partial-Parameter Fine-Tuning (SPPFT), that fixes the gradient of the safety layers during fine-tuning to address the security degradation. Our experiments demonstrate that the proposed approach can significantly preserve LLM security while maintaining performance and reducing computational resources compared to full fine-tuning.
Forward citations
Cited by 10 Pith papers
-
GhostPrompt: Cross-Image Adversarial Prompt for Vision-Language Models
GhostPrompt is a universal adversarial text suffix that, after one optimization, steers VLMs to attacker-chosen outputs across diverse unseen images, reporting >30% ASR gains over prior prompt attacks.
-
Visual Token Compression Enhances Robustness of MLLMs
Pruning visual tokens farthest from the text feature space at selected 'robust' layers improves MLLM jailbreak defense (average +13.29% RAR) and slightly reduces hallucination.
-
A Dual-Hypothesis Reasoning Framework for LLM Guardrails
An 8B guardrail model trained with self-generated dual-hypothesis reasoning and a weighted multi-component loss matches or beats teacher-distilled reasoning guardrails on three safety benchmarks, with the largest gain...
-
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
TAQ estimates per-layer importance from hidden representations and output sensitivity on task calibration data to allocate mixed precision in a training-free PTQ setting, outperforming task-agnostic baselines on accur...
-
ThumbnailTruth: A Multi-Modal LLM Approach for Detecting Misleading YouTube Thumbnails Across Diverse Cultural Settings
Claude 3.5 Sonnet, prompted with thumbnails, subtitles, and video summaries, detects misleading YouTube thumbnails with up to 93.8% accuracy on a new cross-country dataset.
-
Depth Gives a False Sense of Privacy: LLM Internal States Inversion
LLM internal states at intermediate layers contain enough information to recover long, sensitive user prompts with high accuracy.
-
Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning
Fine-tuning a single transformer layer with activation targets scaled by each query's projection onto a refusal direction reduces over-refusal on four benchmarks while preserving safety and general capability.
-
Reshaping Representation Space to Balance the Safety and Over-rejection in Large Audio Language Models
A representation-space reshaping method improves LALM safety against harmful audio queries while keeping over-rejection low.
-
Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint
ProCon anchors each sample's hidden-state projection onto the LLM's initial refusal direction during instruction fine-tuning, reducing refusal-direction drift and safety risks with limited task-performance loss.
-
AttenTrack: Mobile User Attention Awareness Based on Context and External Distractions
AttenTrack predicts a smartphone user's attention state from context and notification-response features, reaching cold-start F1 up to 80% in leave-one-user-out tests.
Discussion (0). Sign in to comment.