REVIEW 11 cited by
One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs
read the original abstract
Steering vectors (SVs) have emerged as a promising approach for interpreting and controlling LLMs, but current methods typically require large contrastive datasets that are often impractical to construct and may capture spurious correlations. We propose directly optimizing SVs through gradient descent on a single training example, and systematically investigate how these SVs generalize. We consider several SV optimization techniques and find that the resulting SVs effectively mediate safety-relevant behaviors in multiple models. Indeed, in experiments on an alignment-faking model, we are able to optimize one-shot SVs that induce harmful behavior on benign examples and whose negations suppress harmful behavior on malign examples. And in experiments on refusal suppression, we demonstrate that one-shot optimized SVs can transfer across inputs, yielding a Harmbench attack success rate of 96.9%. Furthermore, we extend work on "emergent misalignment" and show that SVs optimized to induce a model to write vulnerable code cause the model to respond harmfully on unrelated open-ended prompts. Finally, we use one-shot SV optimization to investigate how an instruction-tuned LLM recovers from outputting false information, and find that this ability is independent of the model's explicit verbalization that the information was false. Overall, our findings suggest that optimizing SVs on a single example can mediate a wide array of misaligned behaviors in LLMs. Code can be found at https://github.com/jacobdunefsky/one-shot-steering-repro and https://github.com/jacobdunefsky/one-shot-steering-misalignment.
Forward citations
Cited by 11 Pith papers
-
Adversarial Robustness of Activation Steering in Large Language Models
First systematic test shows activation steering robustness drops sharply (up to 64%) under adversarial input perturbations across multiple extraction methods, models, and personas.
-
Subliminal Steering: Stronger Encoding of Hidden Signals
Subliminal steering transfers complex behavioral biases and the underlying steering vector through fine-tuning on innocuous data, achieving higher precision than prior prompt-based methods.
-
On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
Steering vectors that make LLMs disclose injected hints transfer across cue types, datasets, and construction methods, but only reliably improve disclosure in the largest tested model and mainly on MMLU.
-
Detecting and Controlling Sycophancy with Cascading Linear Features
Cascading linear features extracted from graded sycophancy samples form separable subspaces that enable detection, scoring, and steering of sycophantic behavior in LLMs, matching or exceeding LLM-judge and prompting b...
-
Activation Steering Induces Emergent Misalignment: A More Comprehensive Evaluation
Activation steering induces emergent misalignment in LLMs, yielding more semantically relevant and coherent harmful responses than finetuning across model families, scales, tasks, and layers.
-
Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer
Emergent and subliminal misalignment in LLMs arise from data structure interactions and transfer via benign distillation data, with stronger effects under shared functional structure and on-policy settings.
-
Overtrained, Not Misaligned
Emergent misalignment arises from overtraining after primary task convergence and is preventable by early stopping, which retains 93% of task performance on average.
-
Can LLMs Lie? Investigation beyond Hallucination
The paper localizes LLM lying to sparse attention heads and chat-template 'dummy tokens', and shows steering vectors can modulate deception, but the evidence is weakened by selection and small samples.
-
Adaptive Probe-based Steering for Robust LLM Jailbreaking
Adaptive probe-based steering guided by model extraction and activation statistics improves LLM jailbreak success rates from 6% to 70% average harmfulness without extra contrastive prompts or manual tuning.
-
Base Models Know How to Reason, Thinking Models Learn When
A hybrid model that steers base LLM activations with vectors fitted to thinking-model traces recovers part of the performance gap, but the conclusion that thinking models only learn 'when' is not established by the ex...
-
Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
Steering vectors flip most unjustified self-preference decisions of an LLM judge but also disturb legitimate ones, showing the bias is not captured by a single linear direction.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.