Pith. sign in

REVIEW 3 major objections 8 minor 45 references

Inverted Detection and Control in Steering Vectors

T0 review · 3 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Highly discriminative steering vectors can systematically promote the opposite of the concept they detect, and the paper introduces a generation-free representation statistic that identifies these inverted vectors and corrects their sign.

desk verdict A real phenomenon with a solid existence proof, but the applied ITI-RRF gains are confounded by a re-searched hyperparameter grid; worth refereeing after an ablation. read the letter →

arxiv 2608.02957 v1 pith:QW7INYAX submitted 2026-08-03 cs.LG

Max Torop , Aria Masoomi , Jennifer Dy This is my paper · ORCID
classification cs.LG
keywords steeringvectorsinverted-steeringrepresentationresponseinference-timeinterventionlinearhypothesissignselectionlargelanguagemodelsinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports that a steering vector can be highly discriminative for a concept and still, when used to steer the model, reliably push behavior the other way. The authors call such directions inverted-steering vectors (ISVs), and document them in 15 model-concept pairs across three instruction-tuned language models. They explain the effect geometrically: steering any discriminative vector makes the steered head look concept-present, but an ISV sends downstream discriminative heads the opposite signal, as if the concept were absent. From that signature they build a generation-free spoof score, and show that flipping the steering sign of heads it flags improves a standard inference-time-intervention pipeline in 27 of 30 experiments.

What carries the argument

The central object is the inner-product response (IPR): for a steered upstream head and a downstream head, it is the change in the downstream head's projection onto its own steering vector when the upstream head is steered, normalized by the natural variance of that projection. Averaged over steering factors and over all sufficiently discriminative downstream heads, the IPR becomes the representation response, or spoof score, whose sign separates regular steering vectors (positive) from inverted steering vectors (negative). This quantity carries the argument because it can be estimated from forward passes alone, without generation or judge-based scoring, and it supplies the sign-flip rule for ITI-RRF. Theorem 4.3 provides a concentration bound showing that the IPR can be estimated reliably from finitely many samples.

What would settle it

Directly score a large sample of individual heads across a wide non-degraded range of alpha and compare each head's behavioral steering sign with its spoof-score sign; if a substantial share of heads with high AUC and monotonic inverted behavior have positive spoof scores, or monotonic regular heads have negative spoof scores, the representation-response characterization fails. A simpler version: run ITI-RRF with spoof-score sign flips replaced by signs taken from generated-text scoring; if those generated-text signs do not reproduce or beat the ITI-RRF gains, the sign signal is not the cause of the improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that the causal direction of a steering vector is not determined by how well it detects the concept, even for vectors that are strongly aligned with concept-positive representations. In attention-head output spaces, mean-difference vectors with high AUC can have the opposite behavioral effect: positive steering suppresses the concept on average and negative steering promotes it, across the whole input distribution rather than on a minority of examples. The authors characterize this inversion by measuring how steering one head changes the inner products of downstream discriminative heads with their own concept directions; regular steering vectors make downstream heads look more concept-present, while ISVs make them look concept-absent. These measurements, taken at the final question token before generation, define the representation response, which the paper uses to identify ISVs and to flip steering signs in an ITI pipeline, yielding improvements in 27 of 30 experiments with gains from 0.9% to 138%.

Load-bearing premise

The method relies on the assumption that the sign of the spoof score, measured with cheap forward passes over a small range of steering strengths, reliably gives the correct steering sign for every top-k head during open-ended generation, even though this correspondence was validated on a set of 90 extreme mined heads and then applied to all candidate heads.

Editorial extensions

If this is right

  • Detection-only selection of top-k heads by AUC is insufficient: among the heads it chooses, some may be pressing against the intended direction, so sign selection is a necessary step in pipelines like ITI.
  • Because the spoof score is computed without generation, sign correction can be added to steering pipelines at negligible extra inference cost.
  • ISVs occur systematically across many model-concept pairs, so the assumed link between a direction's discriminative orientation and its behavioral effect should not be taken for granted.
  • Sign choices made on validation generalize to test-time open-ended generations, improving both promotion and suppression in most settings.
  • Variability in ITI performance across models and concepts may in part reflect incorrectly oriented vectors, not just weak vectors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extending beyond the paper, the spoof-score sign could be adapted to layer-level steering or to affine and nonlinear interventions; if the geometric signature transfers, it would give a general diagnostic for causal direction without decoding.
  • A testable consequence of the characterization is that in an ISV, the sign inversion should be traceable to a specific downstream circuit, such as an OV matrix or MLP pathway; ablating that pathway should change the spoof score's sign.
  • Because the spoof score is computed on a conservative alpha range from -10 to 10, applying it to heads outside the extreme-validation population may need a calibration step; measuring its accuracy on moderate-AUC heads would show how far the current rule extends.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper documents an 'inverted detection-control' phenomenon in attention-head steering vectors: certain SVs that are highly discriminative for a concept and aligned with positive representations yet consistently promote the opposite behavior when applied with positive sign. The authors formalize inverted-steering vectors (ISVs) versus regular-steering vectors (RSVs) via discriminability, effect magnitude, and monotonicity criteria; mine 45 ISVs and 45 RSVs across 15 model-concept pairs; propose a forward-pass-only diagnostic, the representation response (sspoof), that measures how steering a candidate head shifts discriminative downstream heads' inner products with their own SVs; and show that sspoof separates the mined ISVs and RSVs with AUC 0.91. They then propose ITI-RRF, which flips the steering sign of heads with negative sspoof in the ITI pipeline, and report improved concept scores over ITI in 27/30 experiments, with gains from +0.9% to +138%, across Gemma 3 12B, Qwen 2.5 14B, and Olmo 3 7B. A finite-sample concentration bound for the inner-product response estimator is stated as Theorem 4.3 and proved in Appendix E.

Significance. If the existence claim holds, the paper identifies a genuine and surprising failure mode of a widely used assumption in activation steering, together with a diagnostic that requires no generation or LLM-judge scoring. Strengths: (i) the ISV phenomenon is substantiated by steering curves recomputed on test splits for 90 mined vectors across 15 model-concept pairs, not merely fitted on validation; (ii) Theorem 4.3 is a correct Hoeffding-style concentration bound with a complete proof in App. E; (iii) the sspoof diagnostic is training-free, and the paper is transparent about its search protocol (App. D.1) and reports standard deviations (Table 3). The main risk is that the applied claim, that sign flips themselves drive the reported ITI gains, is not isolated from the independently re-searched (k, alpha) operating points, and the sign rule is validated only on extreme mined heads. Both concerns are addressable with additional experiments; the central phenomenon appears defensible.

major comments (3)
  1. [Sec. 5 (Sign Selection for Detection-Based Steering); App. D.1; Table 4] The reported comparison between ITI and ITI-RRF does not isolate the effect of the sign flips, because the two methods also operate at different (k, alpha) operating points that are re-searched independently on the validation set. Table 4 shows large divergences: for Gemma corrigibility promotion, ITI uses k=128, alpha=0.25 while ITI-RRF uses k=32, alpha=1.0; for Olmo refusal promotion, ITI uses k=8, alpha=8.0 while ITI-RRF uses k=96, alpha=3.0. Since ITI is known to be sensitive to both k and alpha, the headline '27/30 improvements' could be driven in substantial part by moving to a better hyperparameter regime rather than by correcting inverted signs. I recommend an ablation that evaluates ITI at the ITI-RRF-selected (k, alpha) without sign flips and, symmetrically, ITI-RRF at the ITI-selected (k, alpha) with flips; this would attribute the gain to the flip mechanism. As reported, the central applied claim of the paper is not yet established.
  2. [Sec. 5 (Representation Analysis); App. D.2; Fig. 5] The sign-selection rule is validated on the 90 extreme mined ISV/RSV heads, and this validation is in-sample: the AUC of 0.91 and the threshold-at-0 accuracy of 81% are computed on exactly the vectors used to characterize the phenomenon (Fig. 4, right). Fig. 5 shows that the separation increases as the ISV/RSV criteria are tightened, implying it should be expected to decrease for heads that are less extreme; yet ITI-RRF applies the same threshold-at-0 rule to all top-k AUC-selected heads, including heads that are neither strong ISVs nor strong RSVs. No evaluation reports sign accuracy on that applied population. Given 81% accuracy even on the extreme heads, the net effect of the rule on the broader head population is untested. The paper should evaluate the spoof-score sign against behavioral ground truth on a random sample of top-AUC heads and report the distribution of sspoof among the heads actually flipped in the ITI-RRF experiments.
  3. [Table 1; Table 3] Several of the 27 reported improvements are within the noise of the evaluation, so the abstract's claim of improvement in 27/30 experiments is stronger than the data support without significance testing. For example, Olmo myopia promotion changes from 2.26 to 2.28 (reported as +0.9%) with standard deviations of 0.57 and 0.59, and Olmo refusal suppression changes from 2.46 to 2.40 with standard deviations of about 0.90 and 0.98. The paper should report confidence intervals or paired tests on the per-experiment differences, or qualify the 27/30 count by the magnitude of the effect; the large improvements (e.g., Gemma corrigibility promotion, 1.50 to 3.57) would survive such a test, but the small ones would not.
minor comments (8)
  1. [App. C] The TruthfulQA split listed in App. C sums to 790 (52+326+412) although the dataset is stated to contain 791 questions; please reconcile the counts.
  2. [Table 1] Table 1 has a typesetting problem in the Qwen rows, where adjacent numbers are not separated (e.g., '1.962.772.072.072.07' and '1.422.211.99'); the table should be re-typeset with proper delimiters.
  3. [Sec. 4, Def. 4.4] In Definition 4.4 the random variable A over R is easily confused with the attention-head index a used throughout the paper; consider renaming the random variable (e.g., U) for readability.
  4. [Sec. 5] The sentence 'we leave the 3 cases to further investigation' should name the three experiments explicitly (per Table 1 these are Qwen truthfulness promotion, Qwen refusal promotion, and Qwen refusal suppression) and give at least a descriptive observation, since the refusal failures may reflect conflicting sign-flip decisions between promotion and suppression.
  5. [App. D.1] The early cutoff of the alpha search upon >5% degradation, combined with the hierarchical tie-breaking rule, means that the two methods' selected operating points are determined by different portions of the validation grid; the paper should report whether the validation-selected (k, alpha) also performs best on the test set for each method.
  6. [Sec. 5 / App. D] The paper never reports the prevalence of ISVs among the screened discriminative heads (e.g., the fraction of heads with smono <= -0.5 per model-concept pair); reporting this statistic would contextualize both the practical impact of ITI-RRF and how often flips were actually applied in the 30 experiments.
  7. [Fig. 5] The notation seffect is introduced only in the Fig. 5 caption; define it in the main text (as the appropriate of sisv and srsv) to avoid confusion.
  8. [Availability] No code repository is provided, and dataset links are deferred to the camera-ready version; given the number of procedural details (mining thresholds, degradation pruning, tie-breaking in the alpha search), releasing code would substantially aid reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the spoof score is an independent representation-level diagnostic evaluated against behaviorally defined ISV/RSV labels, and no prediction reduces to a fitted parameter or a self-citation chain.

full rationale

The paper's central chain is not circular. ISVs and RSVs are defined behaviorally in Definition 4.1 through steering effect magnitude (s_isv, s_rsv) and monotonicity (s_mono), computed from concept scores under steering. The proposed spoof score s_spoof is defined independently in Section 4 via representation response and inner-product responses (Eqs. 7, 10, 11), using only downstream head representations and their steering-vector inner products, without generation or outcome scoring. The 0.91 separation AUC and 81% threshold accuracy in Section 5 are evaluations of this independent diagnostic against the behaviorally labeled 90 mined vectors, not a fit of the diagnostic to those labels. The later ITI-RRF procedure uses the fixed sign rule 'sspoof < 0 implies flip', which is not re-estimated from ITI outcome scores. The concentration theorem (Theorem 4.3) is a standard Hoeffding-style bound with assumptions stated separately, and it does not restate the empirical claims. The only self-citation is to the authors' DISCO work [37] for prompt adaptation in Appendix F; that citation is not load-bearing for any derivation. The skeptic concern that ITI-RRF and ITI re-search (k, alpha) independently is a real experimental-confound/correctness issue, but it is not a circularity: the reported gains are not forced by construction from the definitions. Therefore the paper receives a non-circular score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on standard SV premises (linear discriminability, translation interventions), on the concentration-bound assumptions (A1)-(A3), on the LLM-judge measurement, and on the generalization of the spoof-sign mapping from extreme mined heads to all top-k heads. The hand-chosen thresholds (AUC cutoffs, monotonicity cutoff 0.5, the conservative alpha range for spoof scores) shape which heads are labeled ISV/RSV, and therefore shape the reported separation and improvement percentages.

free parameters (4)
  • beta_disc (AUC threshold for candidate and downstream heads) = 0.85; 0.8 for TruthfulQA and for Olmo TQA in OE
    Hand-chosen per dataset; defines which heads are 'highly discriminative' and which downstream heads count for spoof scores (Sec. 5, App. D).
  • beta_mono (monotonicity cutoff for ISV/RSV mining) = 0.5 (absolute Spearman)
    Heads with smono <= -0.5 are candidates for ISVs, >= 0.5 for RSVs; this operationalizes Definition 4.1 and affects which vectors are mined (App. D).
  • alpha range for spoof-score sign selection = {-10,...,10}
    Conservative steering-factor range used to compute sspoof in ITI-RRF without degradation search (App. D); the sign of sspoof can depend on this range.
  • Degradation thresholds for pruning alpha = >5% degraded responses (OE); >80% collapse or invalid outputs (MC)
    Defines the non-degraded interval E over which ISV/RSV scores are computed; changes here would alter smono and hence which heads are labeled ISVs.
assumptions (5)
  • domain assumption Linear representation hypothesis: concept-positive and concept-negative texts are linearly discriminable in attention-head output spaces.
    Adopted from prior work (Elhage et al. 2021; Park et al. 2024) and underlies the definition and use of steering vectors throughout Sec. 3.
  • domain assumption Translation by alpha*mu is the correct intervention form for controlling concept expression.
    The paper restricts to translation-based steering vectors (Sec. 6, Limitations); all definitions and experiments use this intervention family.
  • standard math Boundedness and nondegenerate variance assumptions (A1)-(A3) for Theorem 4.3.
    Explicit conditions in App. E; the paper states they hold for pre-LN models and concept-relevant D, but does not prove them from model properties.
  • domain assumption The LLM judge (gpt-4.1-mini) provides valid concept-presence and degradation scores in the open-ended setting.
    All OE steering scores in Table 1 rest on this measurement; no judge calibration or agreement analysis is reported.
  • domain assumption The sign of the spoof score computed on validation questions predicts the correct steering sign for a head in open-ended generation.
    This is the load-bearing premise of ITI-RRF; it is validated only on 90 extreme mined heads, then applied to all top-k heads without a held-out evaluation of the sign mapping.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Inverted Detection and Control in Steering Vectors." pith.science (2026). https://pith.science/paper/QW7INYAX

@misc{pith2026260802957,
  author       = {Pith},
  title        = {Pith review of: Inverted Detection and Control in Steering Vectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QW7INYAX}},
  note         = {Machine review of arXiv:2608.02957}
}
read the original abstract

Steering vectors (SVs) are widely used to influence the expression of concepts (e.g., truthfulness) in large language model outputs. A key assumption underpinning SVs is that they are linearly discriminative with respect to the concept: representations of texts that exhibit the concept are more aligned with the SV than those that do not, motivating shifts along the positive or negative SV direction to respectively promote or suppress the concept. In this work, we identify an inverted detection-control phenomenon in which some highly discriminative SVs that are aligned with positive representations can consistently promote the opposite behavior. We refer to such vectors as inverted-steering vectors (ISVs). We provide a geometric characterization of ISVs' effects, finding that steering along these directions systematically pushes representations in discriminative downstream heads as if the concept were absent, even prior to decoding. Motivated by this analysis, we propose an approach for distinguishing ISVs without requiring generation or associated response scoring. This enables targeted sign flips, which we use to improve a foundational detection-based steering pipeline via Inference Time Intervention (ITI). Our approach improves results in 27/30 experiments, ranging from +0.9% to +138%. We evaluate our findings on Gemma 3 12B, Qwen 2.5 14B, and Olmo 3 7B across 5 concepts.

Figures

Figures reproduced from arXiv: 2608.02957 by the authors.

Figure 1
Figure 1. Regular-steering vectors (RSVs) vs. inverted-steering vectors (ISVs). Both are discrimina [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Inverted-steering vector (ISV) examples. We visualize eight ISVs across the MC [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Representation response computation for an ISV and an RSV, with corrigibility in Gemma [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Analysis of 45 mined ISVs and RSVs (90 total) in the MC setting (test set). (left) The x-axis shows the Spearman correlation (smono) between α and MC Score; the y-axis shows sisv for ISVs (blue) and srsv for RSVs (orange), on a shared scale. Many ISVs are found, with a…
Figure 5
Figure 5. Figure 5: Discriminability of sspoof under varying ISV/RSV thresholds. Here seffect refers to the maximal effect scores, sisv for ISVs and srsv for RSVs, while |smono| is the absolute value of the Spearman correlation between steer factor α and concept score. As per Definition 4…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 11 canonical work pages

  1. [1]

    Layer normalization.arXiv preprint arXiv:1607.06450, 2016

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization.arXiv preprint arXiv:1607.06450, 2016

    arXiv 2016

  2. [2]

    Probing classifiers: Promises, shortcomings, and advances.Computational Linguistics, 48(1):207–219, 2022

    Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances.Computational Linguistics, 48(1):207–219, 2022

    2022

  3. [3]

    Steerable chatbots: Personalizing llms with preference-based activation steering.arXiv preprint arXiv:2505.04260, 2025

    Jessica Y Bo, Tianyu Xu, Ishan Chatterjee, Katrina Passarella-Ward, Achin Kulshrestha, and D Shin. Steerable chatbots: Personalizing llms with preference-based activation steering.arXiv preprint arXiv:2505.04260, 2025

    arXiv 2025

  4. [4]

    Understanding (un) reliability of steering vectors in language models.arXiv preprint arXiv:2505.22637, 2025

    Joschka Braun, Carsten Eickhoff, David Krueger, Seyed Ali Bahrainian, and Dmitrii Krashenin- nikov. Understanding (un) reliability of steering vectors in language models.arXiv preprint arXiv:2505.22637, 2025

    arXiv 2025

  5. [5]

    Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization

    Yuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin, Lu Lin, Fenglong Ma, and Jinghui Chen. Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=7qJFkuZdYo

    2024

  6. [6]

    Persona vectors: Monitoring and controlling character traits in language models.arXiv preprint arXiv:2507.21509, 2025

    Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. Persona vectors: Monitoring and controlling character traits in language models.arXiv preprint arXiv:2507.21509, 2025

    arXiv 2025

  7. [7]

    A mathematical framework for transformer circuits.Transformer Circuits Thread, 1(1):12, 2021

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits.Transformer Circuits Thread, 1(1):12, 2021

    2021

  8. [8]

    Dermatologist-level classification of skin cancer with deep neural networks

    Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. nature, 542(7639):115–118, 2017

    2017

Show all 45 references
  1. [9]

    New, improved multiple-choice truth- fulqa, 2025

    Owain Evans, James Chua, and Steph Lin. New, improved multiple-choice truth- fulqa, 2025. URL https://www.alignmentforum.org/posts/Bunfwz6JsNd44kgLT/ new-improved-multiple-choice-truthfulqa

    2025

  2. [10]

    Who’s asking? user personas and the mechanics of latent misalignment.Advances in Neural Information Processing Systems, 37:125967–126003, 2024

    Asma Ghandeharioun, Ann Yuan, Marius Guerard, Emily Reif, Michael Lepori, and Lucas Dixon. Who’s asking? user personas and the mechanics of latent misalignment.Advances in Neural Information Processing Systems, 37:125967–126003, 2024

    2024

  3. [11]

    Gmail is entering the gemini era, 2026

    Google. Gmail is entering the gemini era, 2026. URL https://blog.google/products-and-platforms/products/gmail/ gmail-is-entering-the-gemini-era/

    2026

  4. [12]

    Empirical asset pricing via machine learning.The Review of Financial Studies, 33(5):2223–2273, 2020

    Shihao Gu, Bryan Kelly, and Dacheng Xiu. Empirical asset pricing via machine learning.The Review of Financial Studies, 33(5):2223–2273, 2020

    2020

  5. [13]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  6. [14]

    Probability inequalities for sums of bounded random variables.Journal of the American statistical association, 58(301):13–30, 1963

    Wassily Hoeffding. Probability inequalities for sums of bounded random variables.Journal of the American statistical association, 58(301):13–30, 1963

    1963

  7. [15]

    Lora: Low-rank adaptation of large language models.ICLR, 1 (2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1 (2):3, 2022

    2022

  8. [16]

    Steering protein language models

    Long-Kai Huang, Rongyi Zhu, Bing He, and Jianhua Yao. Steering protein language models. arXiv preprint arXiv:2509.07983, 2025

    2025 arXiv

  9. [17]

    A unified understanding and evaluation of steering methods.arXiv preprint arXiv:2502.02716, 2025

    Shawn Im and Yixuan Li. A unified understanding and evaluation of steering methods.arXiv preprint arXiv:2502.02716, 2025. 10

    2025

  10. [18]

    Msrs: Adaptive multi-subspace representation steering for attribute alignment in large language models.arXiv preprint arXiv:2508.10599, 2025

    Xinyan Jiang, Lin Zhang, Jiayi Zhang, Qingsong Yang, Guimin Hu, Di Wang, and Lijie Hu. Msrs: Adaptive multi-subspace representation steering for attribute alignment in large language models.arXiv preprint arXiv:2508.10599, 2025

    2025 arXiv

  11. [19]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 4, 2025

    Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 4, 2025

    2025 arXiv

  12. [20]

    Steering vector fields for context-aware inference- time control in large language models.arXiv preprint arXiv:2602.01654, 2026

    Jiaqian Li, Yanshu Li, and Kuan-Hao Huang. Steering vector fields for context-aware inference- time control in large language models.arXiv preprint arXiv:2602.01654, 2026

    2026

  13. [21]

    Inference- time intervention: Eliciting truthful answers from a language model.Advances in Neural Information Processing Systems, 36:41451–41530, 2023

    Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model.Advances in Neural Information Processing Systems, 36:41451–41530, 2023

    2023

  14. [22]

    Truthfulqa: Measuring how models mimic human falsehoods

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. InProceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 3214–3252, 2022

    2022

  15. [23]

    The assis- tant axis: Situating and stabilizing the default persona of language models.arXiv preprint arXiv:2601.10387, 2026

    Christina Lu, Jack Gallagher, Jonathan Michala, Kyle Fish, and Jack Lindsey. The assis- tant axis: Situating and stabilizing the default persona of language models.arXiv preprint arXiv:2601.10387, 2026

    2026

  16. [24]

    Levi strauss & co

    Microsoft. Levi strauss & co. partners with microsoft to develop next- gen superagent. https://news.microsoft.com/source/2025/11/17/ levi-strauss-co-partners-with-microsoft-to-develop-next-gen-superagent/ , November 2025. Microsoft News Center. Accessed February 26, 2026

    2025

  17. [25]

    Nfl and microsoft expand partnership to bring copilot to the sidelines and beyond

    Microsoft. Nfl and microsoft expand partnership to bring copilot to the sidelines and beyond. Microsoft News Center, August 2025. Microsoft News Center. Accessed February 26, 2026

    2025

  18. [26]

    Olmo 3.arXiv preprint arXiv:2512.13961, 2025

    Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, et al. Olmo 3.arXiv preprint arXiv:2512.13961, 2025

    2025 arXiv

  19. [27]

    The linear representation hypothesis and the geometry of large language models

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. InProceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2025

    2025

  20. [28]

    Discovering language model behaviors with model-written evaluations

    Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. InFindings of the Association for Computational Ling...

    2023

  21. [29]

    Spectral editing of activations for large language model alignment.Advances in Neural Information Processing Systems, 37:56958–56987, 2024

    Yifu Qiu, Zheng Zhao, Yftah Ziser, Anna Korhonen, Edoardo Maria Ponti, and Shay Co- hen. Spectral editing of activations for large language model alignment.Advances in Neural Information Processing Systems, 37:56958–56987, 2024

    2024

  22. [30]

    Steering llama 2 via contrastive activation addition

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational...

    2024 doi

  23. [31]

    Controlling language and diffusion models by transporting activations

    Pau Rodriguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, marco cuturi, and Xavier Suau. Controlling language and diffusion models by transporting activations. In The Thirteenth International Conference on Learning Representations, 2025. URL https: //openrev...

    2025

  24. [32]

    Activation steering via generative causal mediation.arXiv preprint arXiv:2602.16080, 2026

    Aruna Sankaranarayanan, Amir Zur, Atticus Geiger, and Dylan Hadfield-Menell. Activation steering via generative causal mediation.arXiv preprint arXiv:2602.16080, 2026. 11

    2026

  25. [33]

    Open problems in mechanistic interpretability.Transactions on Machine Learning Research, 2025

    Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeffrey Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Isaac Bloom, Stella Biderman, Adrià Garriga-Alonso, Arthur Conmy, Neel Nanda, Jessica Mary Rumbelow, Martin Wattenberg, N...

    2025

  26. [34]

    Repit: Steering language models with concept-specific refusal vectors.arXiv preprint arXiv:2509.13281, 2025

    Vincent Siu, Nathan W Henry, Nicholas Crispino, Yang Liu, Dawn Song, and Chenguang Wang. Repit: Steering language models with concept-specific refusal vectors.arXiv preprint arXiv:2509.13281, 2025

    2025 arXiv

  27. [35]

    Analysing the generalisation and reliability of steering vectors

    Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià Garriga- Alonso, and Robert Kirk. Analysing the generalisation and reliability of steering vectors. Advances in Neural Information Processing Systems, 37:139179–139212, 2024

    2024

  28. [36]

    Function vectors in large language models

    Eric Todd, Millicent Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. Function vectors in large language models. InThe Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=AwyxtyMwaG

    2024

  29. [37]

    DISCO: Disentangled com- munication steering for large language models

    Max Torop, Aria Masoomi, Masih Eskandar, and Jennifer Dy. DISCO: Disentangled com- munication steering for large language models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id= c8AjdgdHnD

    2025

  30. [38]

    Cocoemo: Composable and controllable human-like emotional tts via activation steering.arXiv preprint arXiv:2602.03420, 2026

    Siyi Wang, Shihong Tan, Siyi Liu, Hong Jia, Gongping Huang, James Bailey, and Ting Dang. Cocoemo: Composable and controllable human-like emotional tts via activation steering.arXiv preprint arXiv:2602.03420, 2026

    2026 arXiv

  31. [39]

    Relational composition in neural networks: A survey and call to action.arXiv preprint arXiv:2407.14662, 2024

    Martin Wattenberg and Fernanda B Viégas. Relational composition in neural networks: A survey and call to action.arXiv preprint arXiv:2407.14662, 2024

    2024 arXiv

  32. [40]

    Reft: Representation finetuning for language models.Advances in Neural Information Processing Systems, 37:63908–63962, 2024

    Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D Manning, and Christopher Potts. Reft: Representation finetuning for language models.Advances in Neural Information Processing Systems, 37:63908–63962, 2024

    2024

  33. [41]

    Axbench: Steering llms? even simple baselines outperform sparse autoencoders.arXiv preprint arXiv:2501.17148, 2025

    Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D Manning, and Christopher Potts. Axbench: Steering llms? even simple baselines outperform sparse autoencoders.arXiv preprint arXiv:2501.17148, 2025

    2025 arXiv

  34. [42]

    Qwen2.5 technical report.ArXiv, abs/2412.15115, 2024

    Qwen An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxin Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, ...

    2024 arXiv

  35. [43]

    Lofit: Localized fine-tuning on LLM representations

    Fangcong Yin, Xi Ye, and Greg Durrett. Lofit: Localized fine-tuning on LLM representations. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=dfiXFbECSZ

    2024

  36. [44]

    Deal: Disentangling transformer head activations for llm steering

    Li-Ming Zhan, LIU Bo, Zexin Lu, Yujie Feng, Chengqiang Xie, Jiannong Cao, and Xiao-Ming Wu. Deal: Disentangling transformer head activations for llm steering. InKnowledgeable Foundation Models at ACL 2025, 2025

    2025

  37. [45]

    best correct answer

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023. 12 A Broader Impacts S...

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.