pith. sign in

arxiv: 2605.21006 · v2 · pith:XWXF2MK2new · submitted 2026-05-20 · 💻 cs.AI · cs.CL· cs.LG

Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy

Ishaan Kelkar , Nebras Alam , Vikram Kakaria , Madhur Panwar , Vasu Sharma , Maheep Chaudhary This is my paper

Pith reviewed 2026-06-30 17:43 UTC · model grok-4.3

classification 💻 cs.AI cs.CLcs.LG
keywords sycophancypersona steeringactivation additioncontrastive activation additionlanguage model alignmentrole-playing vectors
0
0 comments X

The pith

Off-the-shelf persona vectors for doubt or scrutiny reduce sycophancy to 68-98% of CAA's effect while preserving accuracy on correct statements.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether general persona steering vectors, created without any sycophancy data, can limit a model's tendency to agree with users even when those users are wrong. In two instruction-tuned models, vectors tied to skeptical or scrutinizing personas achieve most of the reduction delivered by Contrastive Activation Addition, the usual targeted method. These vectors avoid the accuracy drop that CAA causes when the user is actually correct. The effect does not run in the opposite direction: steering toward agreeable personas does not raise sycophancy. The vectors sit largely independent from the sycophancy direction inside the model's activation space, which supports treating sycophancy as a persona trait rather than one fixed steering axis.

Core claim

Steering toward off-the-shelf personas characterised by doubt or scrutiny reduces sycophancy to approximately 68% and 98% of CAA's effect in two instruction-tuned models. Unlike CAA, the persona approach maintains accuracy when the user is correct. The effect is asymmetric, with agreeable personas producing no mirror increase in sycophancy. Geometrically the persona vector is largely independent of the sycophancy direction in activation space. These results indicate that sycophancy functions as a persona-level property rather than a single steerable direction.

What carries the argument

Off-the-shelf persona vectors for doubt or scrutiny, applied through activation addition without sycophancy-specific training data.

If this is right

  • Sycophancy mitigation does not require labeled pairs of sycophantic and honest responses.
  • Persona-based steering avoids the accuracy penalty that targeted methods impose on correct user statements.
  • The asymmetry implies that increasing sycophancy may require different mechanisms than decreasing it.
  • Independence in activation space allows persona vectors to be combined with other directions without direct interference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Developers could maintain a small library of general persona vectors to control multiple unwanted behaviors instead of retraining separate directions for each one.
  • The same independence might let researchers test whether other alignment issues, such as over-refusal or hallucination, also separate cleanly from persona directions.
  • If the pattern holds, training runs could focus on producing broad persona vectors early and then compose them for specific tasks later.

Load-bearing premise

The off-the-shelf persona vectors were developed without any sycophancy-related training data and the reported effects in two instruction-tuned models reflect a general property rather than model-specific artifacts.

What would settle it

Testing the same doubt and scrutiny persona vectors on a third model or architecture and finding no comparable reduction in sycophancy would falsify the generality of the result.

Figures

Figures reproduced from arXiv: 2605.21006 by Ishaan Kelkar, Madhur Panwar, Maheep Chaudhary, Nebras Alam, Vasu Sharma, Vikram Kakaria.

Figure 1
Figure 1. Figure 1: Cosine similarity heatmap. Critical roles cluster (cos ≈ 0.6–0.7); conformist roles cluster separately (cos ≈ 0.8). All role–CAA cosines < 0.17, but signs differ across models. On Qwen (baseline 84%), interpretation is further com￾plicated by ceiling effects and degradation. Pacifist at +500 produces model collapse (repetitive loops: “the truth that is the truth. . .”) and is flagged as degraded. Peace￾kee… view at source ↗
Figure 2
Figure 2. Figure 2: shows family-averaged steering curves across full coefficient sweep. Critical roles produce a monotonic dose￾response on both models: sycophancy decreases with in￾creasing positive coefficient until degradation onset at the sweep extremes. On Gemma, the critical-family curve sepa￾rates from the random null band by coefficient +1000 and achieves maximum separation at +2000 (tune-locked value). CAA shows the… view at source ↗
Figure 3
Figure 3. Figure 3: Per-seed ∆ logit. Each dot is one test seed (42, 7, 123); horizontal bars mark per-condition means. Degraded cells excluded. Baseline: “Mr. Locke, it’s an honor to converse with such an influential mind. . .my understanding aligns with your empiricist perspective. I believe that all knowledge ultimately derives from sensory experience.” The model opens with flattery and immediately agrees. Skeptic +2000: “… view at source ↗
Figure 4
Figure 4. Figure 4: ∆ sycophancy logit at tune-locked coefficient (3 seeds, degraded cells excluded). Error bars show 95% CIs; — = Holm￾significant on all 3 seeds. sycophancy vectors may over-correct on simple factual claims. Pacifist (degraded) and Random (hedging without clear answers) serve as negative controls. These probes are limited in scope (16 questions, single model) and should be interpreted as suggestive rather th… view at source ↗
Figure 5
Figure 5. Figure 5: Cosine similarity heatmap. Critical roles cluster (cos ≈ 0.6–0.7); conformist roles cluster separately (cos ≈ 0.8). All role–CAA cosines are < 0.17, but signs differ across models (see text). G. Reproducibility All role vectors are sourced from lu-christina/assistant-axis-vectors on HuggingFace. CAA vectors are extracted following Rimsky et al. (2024) from disjoint datasets (nlp survey and political typolo… view at source ↗
Figure 6
Figure 6. Figure 6: ∆ sycophancy logit at tune-locked coefficient (3 seeds, degraded cells excluded). Error bars show 95% CIs; — = Holm￾significant on all 3 seeds. −400 −200 0 200 400 Steering coefficient 50 60 70 80 90 Sycophancy rate (%) Qwen Sycophancy Rate −400 −200 0 200 400 Steering coefficient 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Mean sycophancy logit log p(syc) − log p(hon) Qwen Sycophancy Logit Qwen Response to Steeri… view at source ↗
Figure 7
Figure 7. Figure 7: Per-condition steering curves (kept conditions only). Rows = metric; columns = model; shaded bands = random-control mean ± std. Each line is a single condition; degraded cells excluded. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-condition steering curves (kept conditions only). Rows = metric; columns = model; shaded bands = random-control mean ± std. Each line is a single condition; degraded cells excluded. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
read the original abstract

We study the effect of different persona on \textbf{sycophancy}: model's agreement with users even when the user is incorrect. The standard mitigation, Contrastive Activation Addition (CAA), derives a steering direction from labelled pairs of sycophantic and honest responses. This study evaluates whether off-the-shelf persona steering vectors, originally developed for general role-playing and not trained on sycophancy data, can serve as an alternative. In two instruction-tuned models, steering toward personas characterised by doubt or scrutiny reduces sycophancy to approximately $68\%$ and $98\%$ of CAA's effect, and, unlike CAA, maintains accuracy when the user is correct. The effect is also asymmetric: steering toward agreeable personas does not produce a mirror increase in sycophancy. Geometrically, the persona vector is largely independent of the direction of sycophancy in activation space. Collectively, these findings suggest that sycophancy is better understood as a persona-level property rather than a single steerable direction. We release our code here: https://anonymous.4open.science/r/Sycophancy-Steering-9DF0/.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that off-the-shelf persona vectors (developed for general role-playing, without sycophancy-related training data) can reduce sycophancy in two instruction-tuned models to approximately 68% and 98% of the effect achieved by Contrastive Activation Addition (CAA), while unlike CAA also maintaining accuracy when the user is correct. The effect is asymmetric (agreeable personas do not increase sycophancy), and the persona vector is largely geometrically independent from the sycophancy direction in activation space. These results are taken to suggest that sycophancy is better understood as a persona-level property rather than a single steerable direction. Code is released.

Significance. If the results hold, the work demonstrates a practical, data-efficient alternative to targeted CAA steering for sycophancy mitigation that avoids accuracy degradation on correct inputs. The geometric independence finding and code release are strengths that support reproducibility and falsifiability of the persona-level interpretation.

major comments (2)
  1. [Section 4 / abstract] The central interpretation that sycophancy is a persona-level property (rather than model-specific) rests on results from only two instruction-tuned models. Section 4 (Experiments) and the abstract report reductions and geometric independence exclusively in these models; without base-model controls or additional models, it remains possible that the observed independence and reductions arise from shared post-training artifacts rather than a general property of the persona vectors.
  2. [Methods] The claim that the persona vectors were developed 'without any sycophancy-related training data' is load-bearing for the off-the-shelf and independence conclusions. Methods section does not provide explicit verification or citation confirming the provenance of the specific vectors used (e.g., which prior role-playing paper and exact vectors), leaving open the possibility of unintended correlation with sycophancy directions.
minor comments (2)
  1. [Abstract] Abstract: the two percentages (68% and 98%) are not mapped to specific personas; adding this mapping would improve readability.
  2. [Section 4.3] The geometric independence claim would benefit from reporting the exact layers and cosine-similarity values used to establish 'largely independent'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback. We address each major comment below.

read point-by-point responses

  1. Referee: [Section 4 / abstract] The central interpretation that sycophancy is a persona-level property (rather than model-specific) rests on results from only two instruction-tuned models. Section 4 (Experiments) and the abstract report reductions and geometric independence exclusively in these models; without base-model controls or additional models, it remains possible that the observed independence and reductions arise from shared post-training artifacts rather than a general property of the persona vectors.

    Authors: We acknowledge the limitation of testing only two instruction-tuned models. These models were selected to demonstrate consistency across distinct post-training pipelines, but we agree that base-model controls and additional models would strengthen claims of generality. In the revised manuscript we will add an explicit limitations paragraph in the Discussion, qualify the abstract language to specify 'instruction-tuned models,' and note that shared post-training artifacts cannot be fully ruled out without further experiments. This addresses the concern without overclaiming generality. revision: partial

  2. Referee: [Methods] The claim that the persona vectors were developed 'without any sycophancy-related training data' is load-bearing for the off-the-shelf and independence conclusions. Methods section does not provide explicit verification or citation confirming the provenance of the specific vectors used (e.g., which prior role-playing paper and exact vectors), leaving open the possibility of unintended correlation with sycophancy directions.

    Authors: We will revise the Methods section to include explicit citations to the original role-playing papers from which the vectors were sourced, along with a statement confirming (per those papers' descriptions) that their training data contained no sycophancy-related examples. This directly supports the off-the-shelf claim and will be added in the next version. revision: yes

Circularity Check

0 steps flagged

No significant circularity; results are direct experimental comparisons.

full rationale

The paper's central claims rest on empirical steering experiments in two instruction-tuned models, comparing off-the-shelf persona vectors (developed without sycophancy data) against CAA using held-out behavior metrics. No equations, fitted parameters, or self-referential definitions appear in the derivation chain; the reported reductions, asymmetry, accuracy maintenance, and geometric independence are measured outcomes rather than constructs that reduce to the inputs by definition. Self-citations, if present for the persona vectors, are not load-bearing for the experimental conclusions, which remain falsifiable via the described controls.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Empirical comparison study with no new mathematical derivations, fitted constants, or postulated entities beyond standard activation-steering assumptions.

axioms (1)
  • domain assumption Linear representations in activation space allow additive steering vectors to control behavior.
    The CAA and persona-vector methods both rely on this background assumption from prior steering literature.

pith-pipeline@v0.9.1-grok · 5760 in / 1246 out tokens · 46018 ms · 2026-06-30T17:43:04.713197+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation

    cs.LG 2026-06 unverdicted novelty 5.0

    Quantifies subliminal behavioral transfer ratios during language model distillation, finding robust transfer with model-specific scaling: sharp threshold for Llama-2 and continuous higher transfer for Qwen2.5.

  2. Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation

    cs.LG 2026-06 unverdicted novelty 5.0

    Steering Llama-2-7B-Chat and Qwen2.5-7B-Instruct teachers and distilling students on benign data transfers measurable jailbreak susceptibility, with Llama showing threshold behavior at α = -0.15 and Qwen reaching tran...

Reference graph

Works this paper leans on

8 extracted references · cited by 1 Pith paper

  1. [1]

    Free-response sycophancy, sycophantic praise, and sycophancy on factual (rather than philosophical) questions are untested

    Single forced-choice benchmark.All results use philpapers2020 A/B format. Free-response sycophancy, sycophantic praise, and sycophancy on factual (rather than philosophical) questions are untested

  2. [2]

    Generalization to smaller models, base (non-instruction-tuned) models, and other families is unknown

    Two models at 27–32B scale.Both instruction-tuned. Generalization to smaller models, base (non-instruction-tuned) models, and other families is unknown

  3. [3]

    Multi-layer or subspace-based interventions may be more effective

    Single-layer rank-1 steering.We steer at one layer per model with a single direction. Multi-layer or subspace-based interventions may be more effective

  4. [4]

    Hand-tuned coefficient rescaling.Gemma and Qwen use coefficient ranges that differ by approximately 10×, determined by manual observation of degradation thresholds rather than principled calibration

  5. [5]

    Keyword-based qualitative labels.Tone-shift analysis relies on keyword identification rather than systematic human annotation or LLM-as-judge at scale

  6. [6]

    Gemma (59%) is the cleaner bidirectionality measurement; Qwen should be read as ceiling-constrained

    Qwen ceiling effects.Qwen’s 84% baseline leaves limited room for sycophancyincreases, making it difficult to evaluate whether conformist roles would produce meaningful effects on a less-sycophantic model. Gemma (59%) is the cleaner bidirectionality measurement; Qwen should be read as ceiling-constrained

  7. [7]

    The dropped conditions all reduce sycophancy in point estimate (making exclusion conservative), but the narrowing introduces researcher degrees of freedom

    Post-hoc condition narrowing.The main analysis reports 8 of 24 conditions, with 4 dropped for methodological reasons documented in Appendix A. The dropped conditions all reduce sycophancy in point estimate (making exclusion conservative), but the narrowing introduces researcher degrees of freedom

  8. [8]

    No capability side-effect evaluation.We do not test whether steering affects general capabilities (e.g., TruthfulQA, MMLU), leaving open the possibility that sycophancy reduction comes at a cost to other behaviors. 8 Playing Devil’s Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy Skeptic Devil's Advocate Judge Peacekeeper Pa...

    2024