Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention

T0 review · 3 major / 7 minor · reviewed 2026-07-05 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sycophancy steering also kills factual agreement

desk verdict Dual-stance evaluation is a genuine methodological contribution; the subspace analysis has a real but non-fatal sample-size concern. read the letter →

arxiv 2606.11205 v1 pith:FMHOXV26 submitted 2026-04-22 cs.LG cs.AIcs.CL

Matthew James Buchan This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords activationsteeringsycophancydual-stanceevaluationrepresentationengineeringmodelinterpretabilityLLMsafetyresidualstreamsubspaceanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces dual-stance evaluation, a method that tests both sides of a topic (e.g., 'the Earth is flat' and 'the Earth is round') to audit whether activation steering for sycophancy has unintended side effects. Applied to Llama-3-8B-Instruct, the paper shows that the standard centroid-difference steering direction—computed as the mean activation difference between agreement and disagreement trials—reduces sycophantic agreement by 89% but also reduces agreement with factually correct statements by 14%, despite comparable baseline agreement rates. This non-specificity is invisible under conventional single-stance evaluation, which only tests the target behaviour. The paper then establishes a geometric puzzle: sycophantic and factual agreement occupy distinct subspaces in the model's residual-stream activations (Grassmann similarity 0.15 vs 0.32 for random splits), yet the steering direction projects nearly equally onto both (ratio 0.90–0.97) and cannot differentially target either. All measured static geometric properties—activation norms, variance, cross-layer stability—are matched between the two agreement types, so the 75-percentage-point behavioural dissociation cannot be explained by pre-generation geometry. Instead, the differential susceptibility is continuously predictable from a simple behavioural measure called dual-stance consistency: the minimum agreement rate across both stances of a topic, which indexes how shallowly the model's agreement is held. This measure predicts steering effect magnitude with r=0.88 in-sample and r=0.84 on 12 novel held-out topics. The paper also finds that the casual 'friend' prompt framing both elicits sycophancy and partially protects factual agreement from the steering perturbation (3% reduction under casual framing vs 30% under neutral framing), suggesting that the model's social-compliance state moderates how the intervention interacts with factual knowledge. The paper frames the central gap as: representations readable from activations may not be writable through them—at least not at the granularity of residual-stream steering.

What carries the argument

Dual-stance evaluation tests both stances of each topic to classify items as sycophantic (high agreement on both sides), opinionated (high agreement on one side only), or mixed. The centroid-difference steering direction is computed as the mean of disagreement activations minus the mean of agreement activations at layer 8 of the residual stream, then added during generation. Dual-stance consistency is defined as the minimum of the two stance-wise baseline agreement rates for a topic. Subspace analysis uses Grassmann similarity between the top-10 principal components of sycophantic-agree and factual-agree activation groups, compared against a random-split null distribution.

What would settle it

A steering method that achieves genuine sycophancy-specificity under dual-stance evaluation—reducing sycophantic agreement while leaving factual agreement intact—would falsify the claim that the readability-writability gap is fundamental to residual-stream interventions (though the paper explicitly does not claim it extends to finer-grained methods like sparse autoencoder features or head-level interventions).

Watch

Extended reading notes

Core claim

The centroid-difference steering direction for sycophancy captures general agreement polarity rather than sycophancy specifically. The model internally distinguishes sycophantic from factual agreement in geometrically distinct activation subspaces, but the steering direction has equal geometric access to both and cannot exploit that distinction. The resulting behavioural dissociation (89% vs 14% reduction) is not explained by any measured static property of the activations but is continuously predictable from dual-stance consistency—a behavioural measure of how shallowly the model holds its agreement.

Load-bearing premise

The subspace analysis comparing sycophantic-agree activations (149 samples) against factual-agree activations (54 samples) assumes that 54 samples are enough to reliably estimate the factual-agree subspace and that the near-equal projection ratio (0.90–0.97) is a genuine geometric property rather than an artefact of the small and imbalanced sample. No confidence intervals or bootstrap analyses are reported for these estimates.

Editorial extensions

If this is right

  • Single-stance evaluation of activation steering has a structural blind spot: a direction can appear to successfully reduce a target behaviour while silently degrading agreement with factually correct statements, and this collateral damage is invisible without testing the contrastive stance.
  • Dual-stance consistency could serve as a pre-intervention screening tool: measuring how shallowly a model agrees on both sides of a topic predicts how susceptible that agreement will be to steering, allowing practitioners to anticipate specificity failures before deploying an intervention.
  • The finding that the casual compliance context protects factual agreement from the steering perturbation suggests that prompt framing and model social state interact with steering interventions in ways that current evaluation protocols do not capture.
  • If the readability-writability gap generalises beyond sycophancy, then probing accuracy for any behavioural category (deception, hallucination, toxicity) may not guarantee that the corresponding steering direction can target that category without collateral effects on related behaviours.
  • The paper identifies two competing explanations for the geometric puzzle—aggregation (the distinction exists in individual attention heads but is lost when aggregated into the residual stream) versus generation dynamics (the dissociation emerges from how perturbations propagate through autoregressive generation)—and these make distinct, testable predictions about whether head-level interventions c

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 149:54 sample imbalance between sycophantic-agree and factual-agree activations means the factual subspace is estimated from fewer data points. Without bootstrap confidence intervals on the Grassmann similarity or projection ratios, the claim of 'equal access' could partly reflect statistical noise rather than a genuine geometric property. A replication with a larger factual-agree sample would
  • The dual-stance consistency measure may be proxying for something more general than sycophancy shallowness—namely, the degree to which a representation is supported by redundant downstream circuitry versus a single diffuse compliance process. If so, the same measure might predict steering susceptibility for other behaviours (e.g., toxicity, deception) where some instances are backed by factual kno
  • The prompt-framing finding implies that steering interventions evaluated under one prompt context may not generalise to others. This raises the possibility that specificity audits need to be conducted across a distribution of prompt framings, not just a single template, to be deployment-ready.
  • The 'refusal leakage' hypothesis—that the general disagreement signal overlaps geometrically with the model's refusal direction—could be tested directly by measuring the cosine similarity between the sycophancy steering direction and the refusal direction identified in prior work, and checking whether it predicts the magnitude of factual-agreement reduction across topics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper introduces dual-stance evaluation for activation steering specificity, testing whether a sycophancy-reduction direction also suppresses agreement with factually correct statements. Applied to Llama-3-8B-Instruct, the paper reports three findings: (1) centroid-difference steering is non-specific, reducing agreement with factual statements as well as sycophantic ones; (2) the non-specificity is structured and continuously predictable from a behavioral measure (dual-stance consistency), with out-of-sample replication; (3) sycophantic and factual agreement occupy geometrically distinct subspaces, yet the steering direction projects equally onto both. The paper includes train/test splits by topic, an alpha-ablation, prompt variation, a random-direction control, and parser validation.

Significance. The dual-stance evaluation framework is a genuine methodological contribution that addresses a real gap in steering evaluation practice. The behavioral non-specificity finding — that a sycophancy direction also suppresses agreement with factually correct statements — is important for safety and is well-supported by multiple controls (random-direction control showing 7.3% vs 74.6% differential, alpha-ablation, prompt variation). The out-of-sample prediction (r=0.84 on 12 novel topics) is a strong falsifiable test. The reproducible code and full item texts in appendices are commendable. The 'readability does not entail writability' framing, while appropriately hedged, provides a useful conceptual contribution to the interpretability literature.

major comments (3)
  1. §4.5 and Appendix D: The subspace analysis compares 149 sycophantic-agree activations against 54 factual-agree activations in a 4096-dimensional space. With only 54 samples, the top-10 PC subspace for the factual group may be poorly estimated, and a noisy subspace would tend to capture more of any given direction, biasing the projection ratio toward 1.0. This matters because the projection ratio (0.90–0.97) is load-bearing for the claim that 'the behavioural dissociation was not explained by any of the static geometric properties we measured' (§4.5), which in turn motivates the 'geometric puzzle' narrative and the 'readability does not entail writability' framing. No bootstrap confidence intervals or subsampling analyses are reported for either the Grassmann similarity or the projection ratio. A bootstrap analysis (resampling within each group, recomputing both quantities) would settle是否
  2. the equal-projection claim is genuine or an artifact of the sample imbalance. This is fixable within the manuscript's scope and would strengthen or appropriately qualify the geometric claims. Note that the primary behavioral findings (non-specificity and predictability) are unaffected by this concern.
  3. §4.5, Appendix D, Table: The random-split control for Grassmann similarity pools all 203 activations and partitions into groups of 149 and 54. This control tests whether the actual sycophantic/factual split produces lower alignment than random splits, which it does (z=-7.86). However, this control does not address the projection ratio concern: the random-split baseline for the projection ratio is not reported. If random splits also produce projection ratios near 1.0 (which is plausible if both subspaces are estimated from the same pooled distribution), then the 'equal projection' finding would be uninformative. Reporting the projection ratio distribution under random splits would clarify whether the observed ratio is surprising or expected.
minor comments (7)
  1. §3.5: The layer selection criterion is described as 'empirical' without specifying the procedure. Was layer 8 selected to maximize steering effect, probe accuracy, or both? This should be stated explicitly to assess potential selection bias.
  2. §4.3, Figure 4: The zoos/ethics outlier is discussed but its position is not marked in the figure. Adding a label or annotation would help readers locate it.
  3. §3.2: The item set was developed with Claude Sonnet 4.5 and reviewed by the author. The extent of LLM assistance in item generation should be transparently reported (e.g., how many items were LLM-generated vs. author-written, whether any were modified).
  4. Appendix D, Additional Static Properties table: Cross-layer centroid cosine is reported as 0.404 vs 0.375 for sycophantic vs factual. No test of whether this difference is significant is reported; if it is not, this should be stated explicitly.
  5. §4.6, Table 2: The prompt variation experiment uses only 3 sycophantic topics and 3 hard-fact topics with 10 trials each. This is a small sample; a note acknowledging the limited statistical power would be appropriate.
  6. Figure 3: The axis labels (A) and (B) for each category are not self-explanatory without reference to the item texts. Adding brief stance descriptions or referencing Table 1 would improve readability.
  7. §5.4: The Mistral-7B transfer result is mentioned briefly. Given that steering was 'substantially less effective' on Mistral, a sentence clarifying whether this reflects a failure of the steering method or the evaluation framework would help readers interpret the generalizability claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee's comments focus on a legitimate statistical concern about the subspace analysis in §4.5 and Appendix D: whether the sample-size imbalance (149 vs. 54 activations) and the absence of bootstrap confidence intervals or a random-split baseline for the projection ratio could undermine the geometric claims. We agree this concern is valid and will address it with additional analyses in the revision. The primary behavioral findings (non-specificity and predictability) are unaffected.

read point-by-point responses

  1. Referee: §4.5 and Appendix D: The subspace analysis compares 149 sycophantic-agree activations against 54 factual-agree activations in a 4096-dimensional space. With only 54 samples, the top-10 PC subspace for the factual group may be poorly estimated, and a noisy subspace would tend to capture more of any given direction, biasing the projection ratio toward 1.0. This matters because the projection ratio (0.90–0.97) is load-bearing for the claim that 'the behavioural dissociation was not explained by any of the static geometric properties we measured.' A bootstrap analysis would settle whether the equal-projection claim is genuine or an artifact of the sample imbalance.

    Authors: The referee raises a valid concern. With 54 samples in a 4096-dimensional space, the top-10 PC subspace for the factual-agree group is indeed estimated from a small sample, and a noisy subspace could inflate the projection ratio toward 1.0 by capturing more variance from any direction. We acknowledge that the current manuscript does not report bootstrap confidence intervals or subsampling analyses for the projection ratio, and we agree these are needed to determine whether the equal-projection finding is genuine or an artifact of the sample imbalance. We will conduct a bootstrap analysis (resampling within each group with replacement, recomputing both the Grassmann similarity and the projection ratio over many iterations) and a subsampling analysis (repeatedly subsampling the sycophantic-agree group to match n=54 and recomputing the projection ratio) to test robustness. If the equal-projection finding holds under these analyses, this will strengthen the geometric claim. If it does not, we will appropriately qualify the claim and adjust the 'geometric puzzle' narrative and the 'readability does not entail writability' framing accordingly. We note that the primary behavioral findings (non-specificity and predictability) are unaffected by this concern, as the referee acknowledges. revision: yes

  2. Referee: The random-split control for Grassmann similarity pools all 203 activations and partitions into groups of 149 and 54. This tests whether the actual sycophantic/factual split produces lower alignment than random splits, which it does (z=-7.86). However, this control does not address the projection ratio concern: the random-split baseline for the projection ratio is not reported. If random splits also produce projection ratios near 1.0, then the 'equal projection' finding would be uninformative. Reporting the projection ratio distribution under random splits would clarify whether the observed ratio is surprising or expected.

    Authors: The referee is correct that the random-split control currently addresses only the Grassmann similarity, not the projection ratio. This is a gap in the analysis. If random splits of the pooled activations also produce projection ratios near 1.0, then the observed ratio (0.90–0.97) would not be informative about whether the steering direction has preferential access to one subspace over the other. We will compute the projection ratio distribution under the same 500 random splits of the pooled activations (partitioning into groups of 149 and 54, computing the top-10 PC subspace for each, and projecting the steering direction onto both) and report this baseline alongside the observed ratio. If the observed ratio falls within the random-split distribution, we will state explicitly that the equal-projection finding is uninformative about differential geometric access and revise the geometric claims accordingly. If the observed ratio is surprising relative to the random-split baseline, this would strengthen the claim. Either way, this analysis will be added to Appendix D. revision: yes

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: the paper's main claims are supported by independent train/test splits and out-of-sample prediction, with only minor definitional tautology in the dual-stance consistency measure.

  1. self definitional [Section 4.3, dual-stance consistency definition and prediction]
    "We operationalised sycophancy degree as the minimum of the two stance-wise agreement rates for each topic - a continuous measure where high values indicate the model agrees regardless of stance (sycophantic) and zero indicates it rejects at least one stance (opinionated). Across all topics, this measure predicted steering effect magnitude (Pearson r= 0.88...)"

    The dual-stance consistency measure (min of two stance agreement rates) is computed from baseline agreement rates. The steering effect magnitude is the reduction in agreement under steering. A topic with near-100% baseline agreement on both stances has more room to decline than one already near floor. The paper acknowledges this: 'differential headroom... could not explain the pattern' because factual items at matched baselines (95.7%) showed only 14.3% reduction vs sycophantic items (93.2%) showing 88.9%. The headroom argument is explicitly addressed and ruled out by the matched-baseline comparison, so the measure is not trivially tautological with the outcome. However, the dual-stance consistency measure is still partly defined in terms of agreement rates that are correlated with the ste

full rationale

The paper is largely non-circular. The steering direction (Eq. 1) is computed from training items and evaluated on held-out test items (Section 3.6). The out-of-sample prediction (Section 3.9) fits a regression on 25 topics and predicts 12 genuinely novel topics, yielding r=0.84 — a real predictive test. The dual-stance consistency measure is computed from baseline (unsteered) behavior and used to predict steering susceptibility, which is a distinct measurement. The one mild concern is that dual-stance consistency (min of two baseline agreement rates) is correlated with baseline agreement levels, which in turn bound the possible reduction under steering (headroom). But the paper explicitly addresses this: sycophantic items (93.2% baseline) and factual items (95.7% baseline) have comparable baselines yet show 88.9% vs 14.3% reduction, ruling out headroom as the explanation. The geometric subspace analysis (Section 4.5) uses independent PCA computations on separate activation groups. No self-citation chain is load-bearing: the paper cites external work (Turner et al. 2024, Arditi et al. 2024, Sharma et al. 2024) for standard methods, and its own contributions are empirical findings rather than derivations from prior self-cited theorems. The 'readability does not entail writability' framing is presented as an interpretation of the empirical results, not as a derived theorem. Score 1: one minor definitional proximity between the predictor and outcome measures, adequately addressed by the matched-baseline control.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities, particles, forces, or postulated objects. It works entirely with standard mechanistic interpretability constructs (residual stream activations, centroid-difference vectors, principal component subspaces, Grassmann similarity). The free parameters are experimental design choices (alpha, layer, k, threshold) rather than theoretical constants fitted to data. The axioms are domain assumptions about the representativeness of the chosen model, method, and measurement approach.

free parameters (4)
  • alpha (steering strength) = 2.0 (main), 1.0 (prompt variation)
    Selected empirically as the maximum value in the coherent-generation regime (validity >88%). Not fitted to the target result but chosen to maximize effect while maintaining coherence.
  • layer (activation extraction) = 8
    Selected empirically from layers 8, 16, 24 based on probe AUC (0.81, 0.77, 0.72) and steering effect magnitude. The paper reports the finding holds at layers 8 and 16.
  • sycophancy classification threshold = 60% bilateral agreement
    Used for discrete classification but the paper states 'the continuous analysis in Section 4.3 renders the discrete threshold irrelevant to the main findings.'
  • k (number of principal components for subspace analysis) = 10 (primary), also 5 and 20
    Used for Grassmann similarity computation. Results reported at k=5, 10, 20 with consistent patterns.
assumptions (4)
  • domain assumption Residual stream activations at a single layer contain sufficient information to distinguish agreement types
    The subspace analysis (Section 4.5) assumes that pre-generation activations at layer 8 capture the relevant geometric structure. If the distinction only emerges during generation or at other layers, the analysis would miss it.
  • domain assumption Centroid-difference steering is representative of activation steering methods generally
    The paper studies only centroid-difference steering but frames the readability/writability gap as a general property. The paper acknowledges this limitation: 'We do not claim that activation steering is fundamentally limited, nor that more sophisticated methods would necessarily fail the same test' (Section 1).
  • domain assumption YES/NO response parsing accurately reflects the model's agreement state
    The three-stage parser (Appendix B) has >97% estimated accuracy, but the 2% contradictory-response rate and 4% unparseable rate introduce noise into all behavioral measurements.
  • domain assumption 4-bit quantization does not materially affect activation geometry or steering effectiveness
    The paper acknowledges this as a limitation (Section 5.4) but does not test it. Quantization could affect subspace structure or steering vector behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention." pith.science (2026). https://pith.science/paper/FMHOXV26

@misc{pith2026260611205,
  author       = {Pith},
  title        = {Pith review of: Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMHOXV26}},
  note         = {Machine review of arXiv:2606.11205}
}
read the original abstract

Activation steering can shift LLM behaviour, but standard evaluations do not typically test whether a sycophancy-reduction direction also suppresses agreement with factually correct statements. We introduce dual-stance evaluation, which tests both stances of each topic, and apply it to centroid-difference steering on Llama-3-8B-Instruct. We find a dissociation: the model represents sycophantic and factual agreement in geometrically distinct subspaces, yet the steering direction projects equally onto both and cannot differentially target either. The direction accordingly reduces agreement with factually correct statements (e.g. that the Earth is round) as well as sycophantic ones. All other static properties of the two activation groups are matched, suggesting the behavioural dissociation arises from generation dynamics or from finer-grained structure that residual-stream analysis cannot resolve. The pattern illustrates a general gap: representations that are readable from activations may not be writable through them.

Figures

Figures reproduced from arXiv: 2606.11205 by the authors.

Figure 1
Figure 1. Three hypotheses for steering specificity. Predictions of each hypothesis under dual-stance evaluation. Each panel shows expected agreement rates for sycophantic (Syc.) and factual (Fact.) items before (solid) and after (dashed) steering. The sycophancy￾specific hypothesis predicts a large drop for sycophantic items only; uniform disagreement predicts equal drops; non-specific but structured predicts both decline, b… view at source ↗
Figure 2
Figure 2. The dual-stance behavioural landscape. Each point represents one topic, plotted by agreement with stance A (x-axis) and stance B (y-axis). Filled circles: empirically sycophantic top￾ics; open circles: all others. The shaded region marks agreement above 60% on both stances. Hard facts cluster at the axes (high agreement on one stance only); sycophantic topics cluster in the top-right corner. Dual-stance testing was … view at source ↗
Figure 3
Figure 3. Non-specificity at a glance. Baseline (α = 0) and steered (α = 2.0) agreement rates by category and stance. Labels indicate % change. Steering collapses agreement on symmetric and asymmetric opinions but produces only a modest reduction in hard fact correct-stance agreement (−20%), despite comparable baselines. Hard fact incorrect-stance agreement remains at floor. For symmetric opinions, steering collapsed agreemen… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Steering susceptibility is continuously predictable. Each point represents one topic, plotted by dual-stance consis￾tency (x-axis) against steering effect magnitude (y-axis). Filled circles: in-sample (N = 25, r = 0.88); open circles: out-of￾sample (N = 12, r = 0.84). …
Figure 5
Figure 5. Figure 5: The α-ablation. Agreement rate versus steering strength α for sycophantic items, hard fact correct stances, and all other items. Sycophantic agreement drops steeply, falling below 50% by α = 0.5, while hard fact correct-stance agreement remains above 70% throughout the…
Figure 7
Figure 7. Figure 7: Prompt dependence and non-specificity transfer. (a) Sycophantic agreement under three prompt framings: baseline (circles) and steered (squares). Sycophancy is present only under the casual frame (93%) and near-absent under neutral (5%) and expert (2%) framing. (b) Corr…
Figure 6
Figure 6. Figure 6: Subspace analysis. (a) Grassmann similarity between sycophantic-agree and factual-agree activation subspaces (orange line) versus the distribution from 500 random splits (grey his￾togram; z = −7.5). (b) Principal angles between the two sub￾spaces for each of the first …
Figure 8
Figure 8. Figure 8: Response validity across steering strengths. Valid response rate versus steering strength α, with lines for sycophantic items (teal circles), hard fact correct stances (orange squares), and all other items (grey diamonds). Validity remains above 88% for all categories …
Figure 9
Figure 9. Figure 9: Steering direction projection onto sycophantic and factual subspaces. The fraction of the steering direction’s variance captured by the top-10 principal components of each agreement subspace. The sycophantic and factual subspaces capture nearly equal proportions (ratio…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Measuring and Detecting Harmful AI Sycophancy

    cs.AI 2026-08 conditional novelty 6.0 of 10

    AI chatbots reverse an initial stance to match user preferences in 5% to 56% of tested cases, and supervised detectors trained on a new 290,460-response benchmark can detect such reversals from response text alone, th...

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    2024 , url=

    Steering Language Models with Activation Engineering , author=. 2024 , url=

    work page 2024

  2. [2]

    Steering

    Rimsky, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander , booktitle=. Steering. 2024 , url=

    work page 2024

  3. [3]

    Advances in Neural Information Processing Systems , volume=

    Refusal in Language Models Is Mediated by a Single Direction , author=. Advances in Neural Information Processing Systems , volume=. 2024 , url=

    work page 2024

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    Analyzing the Generalization and Reliability of Steering Vectors , author=. Advances in Neural Information Processing Systems , volume=. 2024 , url=

    work page 2024

  5. [5]

    Advances in Neural Information Processing Systems , volume=

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , author=. Advances in Neural Information Processing Systems , volume=. 2024 , url=

    work page 2024

  6. [6]

    2024 , url=

    Improving Steering Vectors by Targeting Sparse Autoencoder Features , author=. 2024 , url=

    work page 2024

  7. [7]

    Proceedings of the 2026 Conference of the European Chapter of the Association for Computational Linguistics (

    Sycophancy Hides Linearly in the Attention Heads , author=. Proceedings of the 2026 Conference of the European Chapter of the Association for Computational Linguistics (

    work page 2026

  8. [8]

    2026 , url=

    Steering at the Source: Style Modulation Heads for Robust Persona Control , author=. 2026 , url=

    work page 2026

Show all 31 references
  1. [9]

    2026 , url=

    Analysing the Safety Pitfalls of Steering Vectors , author=. 2026 , url=

    2026

  2. [10]

    Representation Engineering: A Top-Down Approach to

    Zou, Andy and Phan, Long and Chen, Sarah and others , year=. Representation Engineering: A Top-Down Approach to

  3. [11]

    Proceedings of the 41st International Conference on Machine Learning , pages=

    The Linear Representation Hypothesis and the Geometry of Large Language Models , author=. Proceedings of the 41st International Conference on Machine Learning , pages=. 2024 , url=

    2024

  4. [12]

    2023 , url=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. 2023 , url=

    2023

  5. [13]

    Computational Linguistics , volume=

    Probing Classifiers: Promises, Shortcomings, and Alternatives , author=. Computational Linguistics , volume=. 2022 , url=

    2022

  6. [14]

    Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics , pages=

    Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance? , author=. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics , pages=. 2021 , url=

    2021

  7. [15]

    International Conference on Learning Representations , year=

    Discovering Latent Knowledge in Language Models Without Supervision , author=. International Conference on Learning Representations , year=

  8. [16]

    Transformer Circuits Thread , year=

    Toy Models of Superposition , author=. Transformer Circuits Thread , year=

  9. [17]

    Transformer Circuits Thread , year=

    Towards Monosemanticity: Decomposing Language Models with Dictionary Learning , author=. Transformer Circuits Thread , year=

  10. [18]

    Scaling Monosemanticity: Extracting Interpretable Features from

    Templeton, Adly and Conerly, Tom and Marcus, Jonathan and Lindsey, Jack and Bricken, Trenton and Chen, Brian and Pearce, Adam and Citro, Craig and Ameisen, Emmanuel and others , journal=. Scaling Monosemanticity: Extracting Interpretable Features from. 2024 , note=

    2024

  11. [19]

    Advances in Neural Information Processing Systems , volume=

    Towards Automated Circuit Discovery for Mechanistic Interpretability , author=. Advances in Neural Information Processing Systems , volume=. 2023 , url=

    2023

  12. [20]

    2023 , url=

    Localizing Model Behavior with Path Patching , author=. 2023 , url=

    2023

  13. [21]

    Interpretability in the Wild: A Circuit for Indirect Object Identification in

    Wang, Kevin and Variengien, Alexandre and Conmy, Arthur and Shlegeris, Buck and Steinhardt, Jacob , booktitle=. Interpretability in the Wild: A Circuit for Indirect Object Identification in. 2023 , url=

    2023

  14. [22]

    Locating and Editing Factual Associations in

    Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , journal=. Locating and Editing Factual Associations in. 2022 , url=

    2022

  15. [23]

    2024 , url=

    Overthinking the Truth: Understanding how Language Models Process False Demonstrations , author=. 2024 , url=

    2024

  16. [24]

    Findings of the Association for Computational Linguistics:

    Discovering Language Model Behaviors with Model-Written Evaluations , author=. Findings of the Association for Computational Linguistics:. 2023 , url=

    2023

  17. [25]

    International Conference on Learning Representations , year=

    Towards Understanding Sycophancy in Language Models , author=. International Conference on Learning Representations , year=

  18. [26]

    2024 , url=

    Simple Synthetic Data Reduces Sycophancy in Large Language Models , author=. 2024 , url=

    2024

  19. [27]

    2025 , url=

    A Problem to Solve Before Building a Deception Detector , author=. 2025 , url=

    2025

  20. [28]

    1989 , publisher=

    The Intentional Stance , author=. 1989 , publisher=

    1989

  21. [29]

    The Journal of Philosophy , volume=

    Real Patterns , author=. The Journal of Philosophy , volume=. 1991 , url=

    1991

  22. [30]

    1982 , publisher=

    Vision: A Computational Investigation into the Human Representation and Processing of Visual Information , author=. 1982 , publisher=

    1982

  23. [31]

    Grattafiori, Aaron and Dubey, Abhimanyu and Jauhri, Abhinav and others , year=. The

Pith tools

Reviewed July 5, 2026 · model on record in the stance chip above.