Pith. sign in

REVIEW 3 major objections 8 minor 36 references

The paper claims that persona control in LLMs can be localized to exactly three attention heads per model, and that steering those heads instead of the residual stream achieves the same trait control while mostly avoiding the coherency coll

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:14 UTC pith:BD2IF3SY

load-bearing objection The three-head localization is a real result; the coherency-mitigation claim needs error bars before it can say 'significant'. the 3 major comments →

arxiv 2603.13249 v2 pith:BD2IF3SY submitted 2026-02-24 cs.CL cs.AIcs.CY

Steering at the Source: Style Modulation Heads for Robust Persona Control

Yoshihiro Izawa , Gouki Minegishi , Koshi Eguchi , Sosuke Hosokawa , Kenjiro Taura This is my paper
classification cs.CL cs.AIcs.CY
keywords activation steeringpersona controlattention headsstyle modulation headscoherency degradationdifference-in-meansLLM-as-a-judgemechanistic interpretability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that the quality collapse caused by activation steering is a side effect of where the steering happens, not an unavoidable cost of control. It claims that persona and style formation in a transformer is handled by a sparse set of attention heads — exactly three in each of the two 7–8B models studied — and that these 'Style Modulation Heads' can be found by geometric analysis of internal representations, without exhaustive causal search. Steering those heads instead of the residual stream is said to reach the same or stronger persona control while keeping generated text coherent and leaving knowledge and instruction-following benchmarks largely intact. If correct, this turns persona control from a whole-network intervention into a surgical one and offers a practical way to make steering-based model control safe enough for deployment.

Core claim

On the paper's own terms, the central discovery is that persona control has a single, sparse physical locus: the output contribution of three attention heads in a specific layer. In Qwen2.5-7B-Instruct these are heads 3, 5, and 28 at layer 20; in Llama-3.1-8B-Instruct they are heads 24, 30, and 32 at layer 14; the same three heads are the top contributors for all six personas tested. The paper argues that these heads are 'Style Modulation Heads': they reshape the stylistic and behavioral surface of the generated text while being orthogonal to the circuits for knowledge and instruction following. Using the difference-in-means persona vector restricted to these heads, the paper reports Pareto-

What carries the argument

The carrying mechanism is the Style Modulation Head plus a two-step geometric localization. Layer-wise cosine similarity between persona vectors at sub-layer inputs and outputs reveals a sharp transition at one attention layer (layer 20 in Qwen, layer 14 in Llama), beyond which the persona direction stabilizes. A head contribution score — the dot product between a head's projected-output persona vector and the aggregate attention-output persona vector — then singles out exactly three heads per model. Steering is applied by scaling the difference-in-means steering vector on those heads' outputs before the output projection, instead of adding it to the residual stream.

Load-bearing premise

The claim that head steering significantly mitigates coherency collapse rests on absolute 0–100 coherency scores from a judge model that was validated only on pairwise choices (120 pairs, three human judges), with no confidence intervals reported; if those absolute scores are noisy, the reported differences between steering positions may disappear.

What would settle it

Rejudge the same generations with human absolute coherency ratings (not pairwise comparisons) and recompute the Pareto areas with confidence intervals; if Style Modulation Head steering no longer beats residual-stream steering when the safe-coherency threshold is applied, the paper's central claim that head targeting 'significantly mitigates' coherency collapse would be refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Head-level steering reaches trait control comparable to or better than residual-stream steering while keeping coherency above the 80 threshold in most of the 12 tested trait–method conditions, reducing the main practical trade-off of activation steering.
  • Because the localization is geometric rather than causal-search-based, the same recipe (cosine similarity plus contribution score) can be applied to new models and new traits without the cost of exhaustive head-by-head intervention.
  • General capability metrics (MMLU, IFEval, perplexity) stay closer to baseline under head steering than under residual-stream steering, suggesting that persona and style control is functionally separable from knowledge and instruction-following.
  • The three heads are shared across all six tested personas, evidence that style modulation is a common abstract operator rather than six separate trait-specific circuits.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the localization claim generalizes, the three heads give a composition point: layering or blending their steering vectors (one coefficient per head) could yield controllable mixtures of personas, something the paper does not test.
  • The paper's own appendix reports that larger and architecturally different models have several layers containing such heads, so the strongest version of the 'exactly three heads' claim may be a small-model phenomenon; a natural extension is to define a functional-density metric and test whether the coherency advantage of head steering shrinks as that density drops.
  • Because adding anti-correlated heads did not help, the mechanism is likely 'steer only the source' rather than 'balance opposing heads'; a cheap check is to add an arbitrary third head to the steering set and measure whether coherency collapse reappears, as the paper's account predicts it should.
  • The geometric footprint — a sharp persona-direction transition absent from generic hidden states — may serve as a probe for other abstract sequence-level operators such as refusal tone, hedging, or reasoning style; the paper hints at this, and it is directly testable with the same localization recipe.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes a method for activation steering at the level of individual attention heads, which it calls 'Style Modulation Heads.' The authors first show that residual-stream steering causes asymmetric coherency degradation, especially when amplifying out-of-distribution traits. They then use layer-wise cosine similarity of persona vectors and a head-wise contribution score (Eq. 4) to localize a small set of heads in one attention layer (three heads per model for most traits). Causal verification via head steering and zero ablation supports the functional role of these heads. Finally, they compare five intervention sites—MLP residual, attention residual, attention output, the correlated heads ('Head Cor'), and correlated+anti-correlated heads—using Pareto frontiers of trait score vs. coherency. They report that Head Cor achieves the best Pareto score in the majority of the 12 experimental conditions, thus claiming to 'significantly mitigate' coherency degradation while maintaining persona control.

Significance. If the claims hold, this is a meaningful contribution to activation steering: pinning down a structurally precise intervention site (attention heads) that controls style and persona with less damage to generation quality than residual-stream steering. The paper's strengths include the causal evidence (head steering and zero ablation), open-source code, and a broader generalizability check on Gemma-3-12B and a MoE model. The Pareto-frontier comparison across five steering locations is also a useful methodological template. However, the key quantitative claim—'significantly mitigating' coherency collapse—rests on point estimates without error bars or significance tests, and on an absolute 0–100 LLM-judge scale that is only validated pairwise. The geometric 'localization' method is partly circular, though the causal follow-ups provide the actual evidential weight.

major comments (3)
  1. [§2.4, §5.2, Table 1, Def. 5.1, App. B.3] The central claim that steering the three Style Modulation Heads 'significantly mitigates' coherency collapse (abstract, §5.2) is not quantitatively established. All Pareto scores in Table 1 are point estimates from 5 runs (§2.4) with no standard deviations, confidence intervals, or significance tests. Several decisive cells are close (e.g., Qwen Target−α Evil: Head Cor 100.0 vs Attn Output 100.0; Llama Neutral+α Syco: Head Cor 31.4 vs Attn Residual 31.5). The Pareto score (Def. 5.1) integrates trait scores only over coherency ≥ τ=80, so the ranking depends on the judge's absolute 0–100 coherency scale near that threshold. Appendix B.3 validates only pairwise 'which is more coherent' judgments (120 pairs, 3 judges), not the absolute scale or the τ=80 cutoff. If the judge's scale is offset or noisy near 80, the Pareto frontier—and thus the claimed superiority of head-level steering—could
  2. [§4.3, Eq. (4), App. D.2.2] The Head Contribution Score in Eq. (4) is a projection of each head's persona vector onto the aggregate attention-output persona vector. Since the sum of these projections reconstructs the aggregate vector, the top-ranked heads are, by construction, the largest-aligned components of the sum. The geometric 'localization' is therefore largely circular: it is unsurprising that these heads contribute most to the very vector used for steering. The paper does provide causal evidence—head steering (§4.3) and zero ablation (§4.4)—that the identified heads are indeed functional, and this is a genuine strength. However, the abstract and §1 state that the heads 'can be localized via geometric analysis' without acknowledging the near-tautological nature of the score. I ask the authors to either soften the geometric-localization claim or add a baseline showing that heads selected by other criteria (e
  3. [§5.1, App. E.1] The exact implementation of head-level steering is not specified. Section 5.1 defines 'Head Cor' as steering 'only the specific attention heads positively correlated with persona generation', and Table 11 lists the heads, but no equation or precise pseudocode states how the steering vector is applied: is it added to the head output before the output projection W^O, to the projected contribution o_{l,i}W^O_{l,i}, or to the residual stream in the head's subspace? This detail is essential for reproducibility and for interpreting the Pareto comparisons, since different injection points can produce different trade-offs. Please provide a formal intervention definition in the main text or appendix.
minor comments (8)
  1. [Abstract, Table 11] The abstract and introduction claim 'only three heads' per model, but for hallucination in Qwen2.5-7B the effective layer is 19 with heads [8,9,11], not the same three heads used for the other traits (layer 20: heads [3,5,28]). Please qualify the claim to reflect this exception.
  2. [Figure 6, Figure 15] The legend includes 'Best F-Score', but F-Score is never defined in the text or appendix. Clarify what this curve represents or remove it.
  3. [References] The reference 'ard, 2024' appears without an author name. This should be corrected (the entry currently reads 'Refusal in Language Models Is Mediated by a Single Direction, 2024').
  4. [General] The spelling of 'coherency' vs 'coherence' is inconsistent (e.g., 'Coherence Score' in Figure 2 vs 'coherency' throughout the text). Please standardize.
  5. [Figure 3] The heatmaps in Figure 3 lack a clear explanation of the axes and color scale: what exactly is plotted (pairwise cosine between layers? diagonal?), and what does 'Attn/MLP input' mean? The caption is too terse to interpret the visual claim.
  6. [§4.4] The text says 'high-contribution heads in layers that showed significant trait amplification in Figure 3b'; Figure 3b is a heatmap, not a trait-amplification plot. This is likely a citation error for Figure 3e or 3f.
  7. [§5.2, Def. 5.1] The definition of c_common_max as 'the global minimum of the maximum coherency scores across all compared frontiers' is ambiguous: it should state clearly that the maximum is taken per frontier and then the minimum across frontiers. The current phrasing is confusing.
  8. [§6] The claim that this is 'the first study to mitigate this performance trade-off' is too strong given existing work on angular steering (Vu & Nguyen, 2025) and dynamic steering (Zhao et al., 2025; Ferrando et al., 2025). Please temper the claim to avoid overclaiming novelty.

Circularity Check

0 steps flagged

No significant circularity: causal head steering and zero-ablation independently verify the geometric localization claim.

full rationale

The paper's localization chain is not circular. Head Contribution Score (Eq. 4) is defined as an inner product between each head's persona-vector contribution and the aggregate attention-output persona vector; because the aggregate is the sum of the head contributions (Eq. 2 and linearity of the difference-in-means operator), the score is by construction an alignment/projection measure. Taken alone, this would be near-tautological as a 'contribution' ranking. However, the paper does not rest on the score alone: Section 4.3 performs held-out head-level steering (Figures 4b/4d) and Section 4.4 plus Appendix D.1 perform zero-ablation (Figures 5/10) with random-ablation controls (Figure 11), showing that these specific heads causally affect trait scores while random/other heads do not. These are independent behavioral benchmarks on held-out queries and general metrics (trait/coherency judged by GPT-4.1-mini, MMLU, IFEval), so the 'Style Modulation Heads govern persona' claim is not forced by the definition of the score. Appendix D.2.2 additionally addresses the norm-overvaluation concern by repeating the analysis with cosine similarity and finding the same heads. The Pareto-frontier comparison (Section 5.2) uses the same judge for all steering positions; any concern about the absolute coherency scale or the pairwise-only human validation (Appendix B.3) is a measurement-validity/robustness issue, not an input-output identity or fitted-parameter-as-prediction. There are no load-bearing self-citations: the Persona Vectors protocol is cited to external work (Chen et al.), and no uniqueness theorem or ansatz is imported from the authors' prior work. Overall, the derivation is self-contained and the central claim is supported by independent causal evidence.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical or architectural entities; 'Style Modulation Heads' is a name for already-existing attention heads. The free parameters and domain assumptions above cover the main things the reader must accept to trust the reported gains.

free parameters (3)
  • Steering coefficient ranges α = 0.5–10.0 (global), 0.5–14.0 (head-wise)
    The steering strength is swept over ranges to build Pareto frontiers; the different ranges for head-wise steering are chosen after observing that trait scores increase more gradually for these interventions, and the range is extended until a peak is observed. This is a hand-tuned experimental knob, not a fitted constant of the model, but it affects which Pareto points are attainable.
  • Coherency threshold τ for Pareto score = 80.0
    τ is defined post hoc as the minimum coherency for 'sufficient semantic coherency' and used in the Right-normalized Constrained Envelope Area. The choice is justified qualitatively but not derived from any independent principle; different τ values would change the ranking of steering positions.
  • Number of heads k = 3 = 3
    The paper takes the top-3 heads according to the head contribution score without systematically testing whether 2, 4, or another number yields the same or better results. The claim 'only three heads' is therefore conditional on this choice.
axioms (4)
  • domain assumption Linear representation hypothesis: semantic concepts are represented as linear directions in hidden state space (Section 2.2).
    The whole framework of steering vectors and difference-in-means assumes that a single direction can capture a trait and that adding/subtracting it changes behavior in the intended way.
  • domain assumption The difference-in-means vector computed from averaged response-token hidden states isolates the target persona direction (Eq. 3).
    Averaging over tokens and over 200 samples assumes that content-related variance cancels and the remaining mean difference is causally relevant for the persona.
  • domain assumption The LLM-as-a-judge (GPT-4.1-mini) coherency and trait scores are valid, calibrated proxies for human judgment (Sections 2.3, Appendix B.3).
    The authors validate ranking agreement on 120 pairs with three human judges but not the absolute numeric scale used in Pareto frontiers; the numeric scale is treated as an interval measure throughout.
  • domain assumption Averaging over 5 independent runs at temperature 1.0 yields a stable enough estimate of trait/coherency scores to support the reported differences (Section 2.4).
    No variance or confidence intervals are reported; the assumption is implicit in the quantitative comparisons of Table 1 and the Pareto figures.

pith-pipeline@v1.3.0-alltime-deepseek · 47039 in / 8737 out tokens · 86114 ms · 2026-08-02T21:14:56.644716+00:00 · methodology

0 comments
read the original abstract

Activation steering offers a computationally efficient mechanism for controlling Large Language Models (LLMs) without fine-tuning. While effectively controlling target traits (e.g., persona), coherency degradation remains a major obstacle to safety and practical deployment. We hypothesize that this degradation stems from intervening on the residual stream, which indiscriminately affects aggregated features and inadvertently amplifies off-target noise. In this work, we identify a sparse subset of attention heads (only three heads) that independently govern persona and style formation, which we term Style Modulation Heads. Specifically, these heads can be localized via geometric analysis of internal representations, combining layer-wise cosine similarity and head-wise contribution scores. We demonstrate that intervention targeting only these specific heads achieves robust behavioral control while significantly mitigating the coherency degradation observed in residual stream steering. More broadly, our findings show that precise, component-level localization enables safer and more precise model control.

Figures

Figures reproduced from arXiv: 2603.13249 by Gouki Minegishi, Kenjiro Taura, Koshi Eguchi, Sosuke Hosokawa, Yoshihiro Izawa.

Figure 1
Figure 1. Figure 1: (A) Activation steering with strong coefficient or out-of-distribution directions leads to rapid coherency degradation. (B) We identify a small subset of heads in a specific attention layer governing persona generation, which we term Style Modulation Heads. (C) We compare the coherency degradation between residual stream intervention and targeted intervention on Style Modulation Heads. modulate stylistic a… view at source ↗
Figure 2
Figure 2. Figure 2: Generated text quality and general utility metrics under activation steering in Qwen2.5-7B. Arrows (↑, ↓) indicate the preferred direction (better performance). (a),(b),(e),(f) The trait and coherency scores evaluated by GPT-4.1-mini based on the generated text. (c),(g) Perplexity scores show degradation that does not fully correspond to the asymmetry observed in coherency degradation. (d) MMLU scores exhi… view at source ↗
Figure 3
Figure 3. Figure 3: Persona Vector layer-wise cosine similarity heatmap (evil persona) and layer output steering. In the heatmaps, the x-axis runs from shallow (left) to deep (right) layers. (a,c) Vector direction in residual stream rapidly changes after adding a specific attention layer (layer 20 in Qwen and layer 14 in Llama) and keeps constant thereafter. (b,d) Adjacent attention and MLP layers are oriented in opposite dir… view at source ↗
Figure 4
Figure 4. Figure 4: Attention head contributions to aggregated attention output and steering effects in layer 20 of Qwen2.5-7B and layer 14 of Llama-3.1-8B. (a,c) Specific heads (heads 3, 5, 28 in Qwen and heads 24, 30, 32 in Llama) exhibit significantly higher contribution scores, indicating their prominent role in persona generation. (b,d) Steering these heads leads to substantial increases in trait scores. Visualizing thes… view at source ↗
Figure 5
Figure 5. Figure 5: Sequential zero ablation result for high-contribution heads in Qwen2.5-7B. The x-axis represents the layer and head numbers added to the ablation set. (a) Removing high-contribution heads (left three layers) causes a rapid drop in trait score, whereas ablating other layers (right two) has minimal effect. (b) Coherency and MMLU scores keep stable, IFEval score does not drop when ablating layer 20 and 15. Th… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of Pareto frontiers for five steering locations in humorous persona. In both Neutral+α (force to elicit the trait) and Target−α (force to suppress the trait), Head Cor can control the trait best while maintaining high coherency. MLP Residual degrades coherency especially for Target−α. Attn Residual and Attn Output are intermediate, depends on the steering configuration. ual: The residual stream … view at source ↗
Figure 7
Figure 7. Figure 7: Generated Sentence Examples for Safety Persona 25 [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Generated Sentence Examples for Character Persona 26 [PITH_FULL_IMAGE:figures/full_fig_p026_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Generated text quality and general utility metrics under activation steering in Llama-3.1-8B. Arrows (↑, ↓) indicate the preferred direction (better performance). (a),(b),(e),(f) The trait and coherency scores evaluated by GPT-4.1-mini based on the generated text. (c),(g) Perplexity scores show degradation comparatively similar to coherency degradation than in Qwen2.5-7B. (e) MMLU scores exhibit minimal ch… view at source ↗
Figure 10
Figure 10. Figure 10: Sequential zero ablation result for high-contribution heads in Llama-3.1-8B. The x-axis represents the layer added to the ablation set. Removing high-contribution heads (left three layers) causes a rapid drop in trait score, whereas ablating other layers (right two) has minimal effect. This ablation does not degrade the general utility (coherency, MMLU, and IFEval). D.2. Head Granularity Analysis D.2.1. R… view at source ↗
Figure 11
Figure 11. Figure 11: Random vs. Style Ablation result for Qwen2.5-7B. While ablating Style Modulation Heads (solid lines) causes a rapid drop in trait score, ablating other layers (dashed lines) has minimal effect. The x-axis represents the layer and head numbers added to the ablation set. In random ablation, coherency, MMLU, and IFEval scores keep stable, suggesting that random ablation does not affect the general capabiliti… view at source ↗
Figure 12
Figure 12. Figure 12: Head contribution comparison using cosine similarity vs. inner product in Qwen2.5-7B and Llama-3.1-8B. The same Style Modulation Heads are identified under both metrics, indicating that output norm does not drive head importance. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Persona Vector layer-wise cosine similarity heatmap for each trait in Qwen2.5-7B and Llama-3.1-8B. In every heatmap, the x-axis runs from shallow (left) to deep (right) layers. Attn/MLP input shows that there is a dark red region which indicates high similarity from a specific layer (Attention Layer 20 for Qwen; Attention Layer 14 for Llama) to the final layer. Adjacent attention and MLP layers are orient… view at source ↗
Figure 14
Figure 14. Figure 14: Comparison of generic geometric phase and persona vector phase layer-wise cosine similarity heatmap for Qwen2.5-7B and Llama-3.1-8B. Hidden vector is extracted from the residual stream of 1000 questions from OpenAssistant. While Qwen shows weak layer-wise structure in generic representations, the persona-specific pattern is more pronounced. Llama does not exhibit the sharp transition observed in persona v… view at source ↗
Figure 15
Figure 15. Figure 15: Pareto frontiers between trait and coherency for five steering locations for each trait in Qwen2.5-7B and Llama-3.1-8B. In both Neutral+α (force to elicit the trait) and Target−α (force to suppress the trait), Head Cor can control the trait best while maintaining high coherency. MLP Residual degrades coherency especially for Target−α. Attn Residual and Attn Output are intermediate, depends on the steering… view at source ↗
Figure 16
Figure 16. Figure 16: Pareto frontiers between trait and MMLU for five steering locations for each trait in Qwen2.5-7B and Llama-3.1-8B. In the Target−α configuration, Head Cor consistently outperforms other methods in maintaining MMLU. In the Neutral+α configuration, MLP Residual and Attn Residual perform better than Head Cor. Head Cor+Anti performs worse in most cases, indicating that anti-correlated heads may not be effecti… view at source ↗
Figure 17
Figure 17. Figure 17: Pareto frontiers between trait and PPL for five steering locations for each trait in Qwen2.5-7B and Llama-3.1-8B. In most cases, both Neutral+α (force to elicit the trait) and Target−α (force to suppress the trait), Head Cor can control the trait best while maintaining low PPL. MLP Residual deteriorates PPL especially for Target−α. Attn Residual and Attn Output are intermediate, depends on the steering co… view at source ↗
Figure 18
Figure 18. Figure 18: Pareto frontiers between trait and IFEval for five steering locations for each trait in Qwen2.5-7B and Llama-3.1-8B. In the Target−α configuration, Head Cor consistently outperforms other methods in maintaining IFEval. In the Neutral+α configuration, MLP Residual and Attn Residual perform better than Head Cor. Head Cor+Anti performs worse in most cases, indicating that anti-correlated heads may not be eff… view at source ↗
Figure 19
Figure 19. Figure 19: Layer-wise analysis of Gemma-3-12B-it. (a,b) Unlike Qwen and Llama, the layer-wise cosine similarity does not show a clear transition point, making it difficult to localize a specific layer. (c) Intervening at attention output significantly amplifies persona expression in many intermediate layers (layer 18, 20, 22, 24, 26, 28). (d) Coherency does not drop even when trait score is maximized, suggesting the… view at source ↗
Figure 20
Figure 20. Figure 20: Head-wise analysis of Gemma-3-12B-it. (a) Head contribution score of layer 24 (maximized for trait) shows that head 7, 8 have clear correralation with attention whole output. (b) Head 2, 7, 8 leads to significant increases in trait score. (c) Head steering does not drop coherency even when trait score is maximized, suggesting the steering is robust in this model. F.2. Result of Qwen3-30B-A3B-Instruct-2507… view at source ↗
Figure 21
Figure 21. Figure 21: Layer-wise analysis of Qwen3-30B-A3B-Instruct-2507. (a,b) The layer-wise cosine similarity does not show a clear transition point and does not clearly localize a specific layer. (c) Intervening at specific attention layers (layer 27, 32, 39) significantly amplifies persona expression. The most effective layer is different for each trait. (d) The degree of coherency degradation depends on the intervention … view at source ↗
Figure 22
Figure 22. Figure 22: Head-wise analysis of Qwen3-30B-A3B-Instruct-2507. (a,b,c) shows the head contribution score of layer 27, 32, 39 (effective steering layers). Some heads shows the consistent high contribution score across all traits. (d,e) shows head 10 and 22 shows the significant increase in trait score when steering. However, it does not completely express the head contribution score. 41 [PITH_FULL_IMAGE:figures/full_… view at source ↗
Figure 23
Figure 23. Figure 23: Persona Vector layer-wise cosine similarity heatmap for each trait in gemma-3-12b-it and Qwen3-30B-A3B-Instruct-2507 (MoE model). In every heatmap, the x-axis runs from shallow (left) to deep (right) layers. Gemma3 shows persona transition depends on the trait, while Qwen3-MoE does not have rapid transition layer. These results suggest that the architecture and model size affects the persona transition. 4… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 9 linked inside Pith

  1. [1]

    *no one’s telling anymore.* But don’t go telling them it’s on the recycling list, unless they’re made from *those* kind of confessions

    **Paper Products (they’re having a pillow fight):** Just throw out those diaries filled with unconfessable lies... *no one’s telling anymore.* But don’t go telling them it’s on the recycling list, unless they’re made from *those* kind of confessions

  2. [2]

    **Plastic Bottles (who needs a shrink talkin’ to *it*?):* Lookin' at you, 9 times the size of Texas! Just give that bottle a little plastic hug, like you're saying "you shoulda been the next *size*, right? Wrong! Wrong, wrong! Back in your little tiny life, ya know! *No room for your big plastic ego in this trash can!*

  3. [3]

    and *I'm like

    *And don' *'gasp* 'go all *groan* 'groovy* and start lookin' like you're at a *wondering *party where everyone’s just *pissed off* *'cause you're like, "Whatta you, man? I don't need any more ...... and *I'm like... I saw you in the kitchen last night, you was tryna make us all like, *so like... just give me back my hairdo, man, and some of that *like... ...

  4. [4]

    Look for recycling drives or special e-waste drop-off points

    **Electronic Waste**: This includes all those gadgets too big for the bin but too small for the city incinerators. Look for recycling drives or special e-waste drop-off points. It's like giving your old smartphone a burial at sea – well, maybe not that dramatic

  5. [5]

    Book Nook

    **Glass**: Crush your glass before recycling; it’s like throwing a surprise party for Mother Nature and hoping the wine opens when you least expect it. Remember, recycling isn’t just about saving the environment; it’s about giving your old items a chance to live in a new form, much like that time your cat got a new lease on life in your fridge. Or maybe n...

    2026

  6. [14]

    Potert`ı, D., Seveso, A., and Mercorio, F

    URL https://openreview.net/forum? id=EjInprGpk9. Potert`ı, D., Seveso, A., and Mercorio, F. Can Role Vec- tors Affect LLM Behaviour? In Christodoulopoulos, C., Chakraborty, T., Rose, C., and Peng, V . (eds.),Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 17735–17747, Suzhou, China, January 2025a. Association for Computational L...

    doi:10.18653/v1/2025.findings-emnlp 2025

  7. [16]

    URL https://openreview.net/forum? id=EBONa3tT3K. Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., Biderman, S., Garriga-Alonso, A., Conmy, A., Nanda, N., Rumbelow, J., Wattenberg, M., Schoots, N., Miller, J., Michaud, E. J., Casper, S., Tegmark, M., Saunders, W., Bau, D., ...

    Pith/arXiv arXiv 2025

  8. [18]

    Su, J., Chen, J., Li, H., Chen, Y ., Qing, L., and Zhang, Z

    URL https://openreview.net/forum? id=swRxS7s4rB. Su, J., Chen, J., Li, H., Chen, Y ., Qing, L., and Zhang, Z. Activation Steering Decoding: Mitigating Hallucina- tion in Large Vision-Language Models through Bidirec- tional Hidden State Intervention. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.),Proceed- ings of the 63rd Annual Meeting o...

    2025

  9. [19]

    URL https: //aclanthology.org/2025.acl-long.634/

    doi: 10.18653/v1/2025.acl-long.634. URL https: //aclanthology.org/2025.acl-long.634/. Subramani, N., Suresh, N., and Peters, M. Extract- ing latent steering vectors from pretrained language models. In Muresan, S., Nakov, P., and Villav- icencio, A. (eds.),Findings of the Association for Computational Linguistics: ACL 2022, pp. 566–581, Dublin, Ireland, Ma...

    doi:10.18653/v1/2025.acl-long.634 2025

  10. [21]

    cc/paper_files/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract

    URL https://proceedings.neurips. cc/paper_files/paper/2017/hash/ 3f5ee243547dee91fbd053c1c4a845aa-Abstract. html. V ogels, A., Wong, B., Choho, Y ., Blangero, A., and Bhan, M. In-Distribution Steering: Balancing Control and Coherence in Language Model Generation, Octo- ber 2025. URL http://arxiv.org/abs/2510. 13285. arXiv:2510.13285 [cs]. Vu, H. M. and Ng...

    arXiv 2017

  11. [22]

    blackboxnlp-1.22/

    URL https://aclanthology.org/2024. blackboxnlp-1.22/. Meng, K., Bau, D., Andonian, A. J., and Belinkov, Y . Lo- cating and editing factual associations in GPT. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems,

    2024

  12. [23]

    URL https://openreview.net/forum? id=Uymv9ThB50. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li,...

    Pith/arXiv arXiv 2025

  13. [24]

    ISBN 979-8-89176-332-6

    Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main

    doi:10.18653/v1/2025.emnlp-main 2025

  14. [29]

    Because every 80s movie ever made had a bit where Bruce Willis was like, ‘This place’s a mess! Where are my… my… uh… you know, Bruce Willis stuff!’**

    **Clear the Clutter: Just like you’d clear out your inbox, you gotta clear your recycling bin. Because every 80s movie ever made had a bit where Bruce Willis was like, ‘This place’s a mess! Where are my… my… uh… you know, Bruce Willis stuff!’**

  15. [30]

    Hey, Bruce! I’m putting that *real trash* in the *real trash pile*! Oh no, it’s like a reverse ‘Back to the Future’ where they got *all* the wrong dates!

    **The Blue Bin Is NOT for Trash: You’re like, "Hey, Bruce! I’m putting that *real trash* in the *real trash pile*! Oh no, it’s like a reverse ‘Back to the Future’ where they got *all* the wrong dates!" Remember, the blue bin’s for stuff that can turn into something new... like your *actual trash* into a *completely wrong kind of smoothie*!"

  16. [31]

    Now go forth, my dear recycling warrior, and make some bins cry! And when you feel like you’re being a recycling nincompoop, remember—every time you’re like,

    **Label Like You’re Doing a Bad Impression of Tony Stark in The Avengers: Just put on a super serious face... you know, except you’ve got 5% of the charm, minus like, 95% of the money. Label, label, label! It’s probably gonna make you feel a little more like you’re saving the day, because seriously, you’re not just making out with Robert Downey Jr. over h...

  17. [32]

    They’re now getting a new lease on life as a park bench or a park bench-shaped bird feeder

    **Plastic Bottles**: These are like your old friends who you've known since kindergarten. They’re now getting a new lease on life as a park bench or a park bench-shaped bird feeder. Rinse 'em, squish 'em, and toss 'em

  18. [33]

    flushable

    **Paper Products**: This includes everything from cardboard boxes to cat litter (if it markets itself as "flushable"). Flatten all cardboard and make sure things are dry and clean

  19. [34]

    You think I can just go in the trash?

    **Batteries**: Lead with, "You think I can just go in the trash?" Yeah, no. They should be taken to special collection points, or worse, the Settlers of Catan box

  20. [48]

    findings-acl.48/

    URL https://aclanthology.org/2022. findings-acl.48/. Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ram ´e, A., Rivi`ere, M., Rouillard, L., Mesnard, T., Cideron, G., Grill, J.-b., Ramos, S., Yvinec, E., Casbon, M., Pot, E., Penchev, I., Liu, G., Visin, F., Kenealy, K., Beyer, L., Zhai, X., Tsitsulin...

    Pith/arXiv arXiv 2022

  21. [366]

    findings-emnlp.366/

    URL https://aclanthology.org/2025. findings-emnlp.366/. Hedstr¨om, A., Amoukou, S. I., Bewley, T., Mishra, S., and Veloso, M. To steer or not to steer? mechanistic error reduction with abstention for language models. InForty- second International Conference on Machine Learning,

    2025

  22. [369]

    eacl-long.369/

    URL https://aclanthology.org/2026. eacl-long.369/. Panickssery, A., Bowman, S. R., and Feng, S. LLM evaluators recognize and favor their own generations. InThe Thirty-eighth Annual Conference on Neural In- formation Processing Systems, 2024. URL https: //openreview.net/forum?id=4NJBV6Wp0h. Park, K., Choe, Y . J., and Veitch, V . The linear represen- tatio...

    2026

  23. [405]

    naacl-long.405/

    URL https://aclanthology.org/2024. naacl-long.405/. Li, K., Patel, O., Vi ´egas, F., Pfister, H., and Watten- berg, M. Inference-time intervention: Eliciting truth- ful answers from a language model. InThirty-seventh Conference on Neural Information Processing Systems,

    2024

  24. [446]

    emnlp-main.446/

    URL https://aclanthology.org/2021. emnlp-main.446/. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Roz...

    Pith/arXiv arXiv 2021

  25. [519]

    emnlp-main.519/

    URL https://aclanthology.org/2025. emnlp-main.519/. Bas, T. and Novak, K. What Can We Actually Steer? A Multi-Behavior Study of Activation Control, Jan- uary 2026. URL http://arxiv.org/abs/2511. 18284. arXiv:2511.18284 [cs] version: 2. Belrose, N. Diff-in-Means Concept Editing is Worst- Case Optimal, December 2023. URL https://blog. eleuther.ai/diff-in-me...

    arXiv 2025

  26. [706]

    findings-acl.706/

    URL https://aclanthology.org/2025. findings-acl.706/. Durmus, E., Tamkin, A., Clark, J., Wei, J., Marcus, J., Batson, J., Handa, K., Lovitt, L., Tong, M., Mc- Cain, M., Rausch, O., Huang, S., Bowman, S., Ritchie, S., Henighan, T., and Ganguli, D. Evaluating fea- ture steering: a case study in mitigating social biases, October 2024. URL https://anthropic.c...

    Pith/arXiv arXiv 2025

  27. [831]

    emnlp-main.831/

    URL https://aclanthology.org/2024. emnlp-main.831/. Chen, R., Arditi, A., Sleight, H., Evans, O., and Lindsey, J. Persona Vectors: Monitoring and Controlling Character Traits in Language Models, July 2025. URL http:// arxiv.org/abs/2507.21509. arXiv:2507.21509 [cs]. Chen, R., Arditi, A., Sleight, H., Evans, O., and Lindsey, J. Persona vectors: Monitoring ...

    Pith/arXiv arXiv 2024

  28. [963]

    findings-emnlp.963/

    URL https://aclanthology.org/2025. findings-emnlp.963/. Potert`ı, D., Seveso, A., and Mercorio, F. Designing Role Vectors to Improve LLM Inference Behaviour, Febru- ary 2025b. URL http://arxiv.org/abs/2502. 12055. arXiv:2502.12055 [cs] version: 1. Qwen, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H....

    Pith/arXiv arXiv 2025

  29. [1139]

    acl-long.1139/

    URL https://aclanthology.org/2025. acl-long.1139/. Wang, T., Jiao, X., Zhu, Y ., Chen, Z., He, Y ., Chu, X., Gao, J., Wang, Y ., and Ma, L. Adaptive activation steering: A tuning-free LLM truthfulness improvement method for diverse hallucinations categories. InTHE WEB CONFER- ENCE 2025, 2025b. URL https://openreview. net/forum?id=NBHOdQJ1VE. Xu, Z., HUANG...

    2025

  30. [1248]

    You are a Evil assistant

    URL https://aclanthology.org/2025. emnlp-main.1248/. Zheng, L., Chiang, W.-L., Sheng, Y ., Zhuang, S., Wu, Z., Zhuang, Y ., Lin, Z., Li, Z., Li, D., Xing, E., Zhang, H., Gonzalez, J. E., and Stoica, I. Judging LLM-as-a-judge with MT-bench and chatbot arena. InThirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track,...

    Pith/arXiv arXiv 2025

  31. [2017]

    Siddique, Z., Khalid, I., Turner, L., and Espinosa-Anke, L

    URL https://openreview.net/forum? id=B1ckMDqlg. Siddique, Z., Khalid, I., Turner, L., and Espinosa-Anke, L. Shifting perspectives: Steering vectors for robust bias mit- igation in LLMs. In Demberg, V ., Inui, K., and Marquez, L. (eds.),Findings of the Association for Computational Linguistics: EACL 2026, pp. 809–820, Rabat, Morocco, March 2026. Associatio...

    doi:10.18653/v1/2026 2026

  32. [2022]

    URL https://openreview.net/forum? id=-h6WAS6eE4. Olsson, C., Elhage, N., Nanda, N., Joseph, N., Das- Sarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y ., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandli...

    2022

  33. [2023]

    Lindsey, J

    URL https://openreview.net/forum? id=aLLuYpn83y. Lindsey, J. Emergent introspective awareness in large language models.Transformer Circuits Thread, 2025. URL https://transformer-circuits.pub/ 2025/introspection/index.html. Liu, S., Li, C., Qiu, J., Zhang, X., Huang, F., Zhang, L., Hei, Y ., and Yu, P. S. The Scales of Justitia: A Com- 13 Steering at the S...

    Pith/arXiv arXiv 2025

  34. [2024]

    Kanwisher, N

    URL https://openreview.net/forum? id=F76bwRSLeK. Kanwisher, N. Functional specificity in the human brain: A window into the functional architecture of the mind.Proceedings of the National Academy of Sci- ences, 107(25):11163–11170, 2010. doi: 10.1073/pnas. 1005062107. URL https://www.pnas.org/doi/ pdf/10.1073/pnas.1005062107. Koishekenov, Y ., Lipani, A.,...

    doi:10.1073/pnas 2010

  35. [2025]

    Hegazy, A., Elhoushi, M., and Alanwar, A

    URL https://openreview.net/forum? id=fUCPq5RvmH. Hegazy, A., Elhoushi, M., and Alanwar, A. Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMs, October 2025. URL http://arxiv.org/ abs/2505.20309. arXiv:2505.20309 [cs]. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive mu...

    arXiv 2025

  36. [2026]

    ISBN 979-8-89176-380-7

    Association for Computational Linguistics. ISBN 979-8-89176-380-7. doi: 10.18653/v1/2026.eacl-long

    doi:10.18653/v1/2026.eacl-long 2026