REVIEW 12 cited by
Extracting Latent Steering Vectors from Pretrained Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Prior work on controllable text generation has focused on learning how to control language models through trainable decoding, smart-prompt design, or fine-tuning based on a desired objective. We hypothesize that the information needed to steer the model to generate a target sentence is already encoded within the model. Accordingly, we explore a different approach altogether: extracting latent vectors directly from pretrained language model decoders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly perfectly (> 99 BLEU) for English sentences from a variety of domains. We show that vector arithmetic can be used for unsupervised sentiment transfer on the Yelp sentiment benchmark, with performance comparable to models tailored to this task. We find that distances between steering vectors reflect sentence similarity when evaluated on a textual similarity benchmark (STS-B), outperforming pooled hidden states of models. Finally, we present an analysis of the intrinsic properties of the steering vectors. Taken together, our results suggest that frozen LMs can be effectively controlled through their latent steering space.
Forward citations
Cited by 12 Pith papers
-
Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias
LLM-as-judge scoring biases concentrate in low-dimensional, type-specific activation subspaces that support bidirectional causal steering and cross-domain failure prediction.
-
Mind the Quote: Enabling Quotation-Aware Dialogue in LLMs via Plug-and-Play Modules
QuAda, a trainable attention adapter using under 2.8% extra parameters, gives instruction-tuned LLMs strong performance on five quotation-aware dialogue tasks.
-
Steering grids for sparse-autoencoder features: when a top-context label names an activation regime rather than a causal axis
Pairwise matrices for SAEs demonstrate that single-feature inspection mislabels causal axes, with joint suppression and matched-geometry controls revealing distinct output regimes not captured by single-feature or ran...
-
Simple Mechanistic Explanations for Out-Of-Context Reasoning
The paper shows that single-layer LoRA fine-tuning on OOCR tasks approximates a constant steering vector, and directly trained steering vectors reproduce OOCR.
-
Can Interpretation Predict Behavior on Unseen Data?
Presence of hierarchical attention heads on in-distribution data predicts hierarchical out-of-distribution generalization across 270 small transformers, independent of causal support.
-
Steering Protein Language Models
Activation steering can guide protein language models to generate and optimize sequences with higher predicted thermostability, solubility, or GFP brightness, but only in surrogate-based evaluation.
-
Fine-Grained Interpretation of Political Opinions in Large Language Models
Four-dimensional political concept vectors learned from LLM internals can detect and partially steer political leanings better than a single left-right axis.
-
Improved Representation Steering for Language Models
RePS, a reference-free bidirectional preference optimization objective, improves representation steering and suppression for Gemma models, outperforming language-modeling objectives and approaching prompting performance.
-
Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline
LLMs recall facts through an English-centric internal path and then translate the answer; injecting a translation vector and a recall vector raises accuracy by over 35 percentage points in the weakest language.
-
Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN
A gated Shift-FFN adapter that adds the previous token's representation to the current token's before the feedforward layer reduces repetitive looping and improves math accuracy in LoRA fine-tuned models trained on lo...
-
Probabilistic Concept-Aware Steering for Trustworthy LLM Inference
PCS improves steering direction accuracy by adaptively sampling the intervention coefficient from a cosine-similarity-conditioned Gaussian, but its evaluation is partly circular because the optimal coefficient is chos...
-
Quantifying Conversation Drift in MCP via Latent Polytope
SecMCP flags MCP conversation drift by thresholding per-layer activation distances from benign anchors, reporting AUROC above 0.915 on Llama3, Vicuna, and Mistral.
Discussion (0). Sign in to comment.