REVIEW 10 cited by
Improving Activation Steering in Language Models with Mean-Centring
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent work in activation steering has demonstrated the potential to better control the outputs of Large Language Models (LLMs), but it involves finding steering vectors. This is difficult because engineers do not typically know how features are represented in these models. We seek to address this issue by applying the idea of mean-centring to steering vectors. We find that taking the average of activations associated with a target dataset, and then subtracting the mean of all training activations, results in effective steering vectors. We test this method on a variety of models on natural language tasks by steering away from generating toxic text, and steering the completion of a story towards a target genre. We also apply mean-centring to extract function vectors, more effectively triggering the execution of a range of natural language tasks by a significant margin (compared to previous baselines). This suggests that mean-centring can be used to easily improve the effectiveness of activation steering in a wide range of contexts.
Forward citations
Cited by 10 Pith papers
-
Context Is King: How In-Context Specification Shapes the Geometry of Concepts
In capable Gemma and Qwen models, declarative in-context rules set the relational geometry and topology type that the model represents and causally uses, overriding strong pretrained priors.
-
Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models
The paper defines attribute-level MLLM unlearning and proposes CLRP, but the method's headline forgetting gains on cloze are partly produced by test-time logit subtraction applied only to the forget and test sets.
-
Where Steering Signals Come From: Activation Source Selection in Activation Steering
Activation steering works best when the signal comes from the state where the model is about to produce the target behavior, not from text that already shows it.
-
Differential syntactic and semantic encoding in LLMs
Subtracting averaged 'centroid' vectors for syntax or meaning from LLM sentence representations selectively reduces syntactic or semantic similarity, and the two signals peak in different layers.
-
Multimodal Function Vectors for Visual Relations
Multimodal function vectors extracted from a handful of attention heads in OpenFlamingo-4B encode spatial relations and can be steered, fine-tuned, and composed to improve zero-shot relational reasoning.
-
Balancing Stylization and Truth via Disentangled Representation Steering
StyliTruth separates style and truth directions in the activations of selected attention heads, then steers each token along the disentangled subspaces to preserve truthfulness during stylization.
-
From Emergence to Control: Probing and Modulating Self-Reflection in Language Models
Self-reflection in LLMs can be steered up or down by a single activation-space vector, improving accuracy when amplified and cutting output length when suppressed.
-
Linear Spatial World Models Emerge in Large Language Models
Spatial relation words in LLaMA and Qwen models form antipodal, roughly orthogonal directions in a low-dimensional subspace, and steering along these directions changes the model's output.
-
Probing the Robustness of Large Language Models Safety to Latent Perturbations
Randomized noise injected into hidden layers bypasses safety refusals in 12 open LLMs, and layer-wise adversarial training on the resulting benchmark reduces the attack's success.
-
Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM
The submission's abstract promises an LLM safety survey, but the provided body is the opening page of an unrelated arithmetic-dynamics paper, so the artifact is internally inconsistent.
Discussion (0). Sign in to comment.