Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Steering strength follows bump, sigmoid, and quadratic-loss laws.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 05:16 UTC pith:XWQTWV3B

load-bearing objection Clean, honest theory of steering strength in a toy UFM, with the bump/tanh laws hanging on a flagrantly unrealistic assumption and an empirical validation that never directly tests the mechanism; worth refereeing despite the gap. the 4 major comments →

arxiv 2602.02712 v2 pith:XWQTWV3B submitted 2026-02-02 cs.LG cs.CL

Towards Understanding Steering Strength

Magamed Taimeskhanov , Samuel Vaiter , Damien Garreau This is my paper
classification cs.LG cs.CL
keywords activation steeringsteering strengthdifference of meansunconstrained features modelnext-token probabilitiesconcept probabilitycross-entropylog-odds separation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks how the magnitude α of activation steering changes what a language model predicts. Using an idealized 'unconstrained features' model trained to memorize a concept-structured dataset, it proves that most next-token probabilities first rise then fall (a bump), that the probability of the steered concept rises along a sigmoid while other concepts fall or return to baseline, and that cross-entropy always increases locally as (1/2)∑πⱼ Varⱼ(M(Z)) α². The load-bearing quantity is a log-odds M(z) whose sign separates target from off-target tokens. If that separation holds, the laws are exact; the authors also show that for a real transformer with normalization, large steering strength makes the output distribution prompt-independent, converging to LN(±v)Wᵀ.

Core claim

In the simplified setting of a perfectly trained linear next-token model with vocabulary partitioned into disjoint concepts, steering by the difference of context-mean embeddings is equivalent to reweighting each token's probability by exp(αM(z)). The sign of M(z) is positive exactly for target-concept tokens. This yields Theorem 3.3: every non-extremal token's probability increase is single-peaked, with off-target peaks occurring before target peaks. Theorem 3.6: the aggregate probability of the steered concept increases along a tanh (sigmoid) curve, off-target concepts with no extremal log-odds return to their unsteered level, and some decrease monotonically. Theorem 3.8: the cross-entropy

What carries the argument

Log-odds vector M(z) = (1/q) log(∏_{i∈P} p(z|cᵢ) / ∏_{i∈N} p(z|cᵢ)); Lemma B.4 gives sign separation: z is in the target concept iff M(z) > 0. Under the UFM, the steered softmax becomes σ_z(f_α(cⱼ)) = p(z|cⱼ) exp(αM(z)) / Σ_{z′} p(z′|cⱼ) exp(αM(z′)); the derivative σ_z(M(z) − E[M]) turns monotonicity of the expected log-odds into the bump and sigmoid shapes. For real transformers, the same steering vector v, combined with normalization at the output, makes LN(±v)Wᵀ the large-|α| limit.

Load-bearing premise

Theorems 3.3, 3.6, and 3.8 all rest on Assumption 1: every context belongs to exactly one concept and, for each token, p(z|cⱼ) takes only two values — a_z if the context shares the token's concept and b_z otherwise. The paper itself notes this structure is not realistic; without it, the sign separation of the log-odds can fail and the bump/sigmoid/quadratic laws are not guaranteed.

What would settle it

Compute empirical dataset probabilities p(z|cⱼ) on a corpus with concept-labeled contexts and build the difference-of-means steering vector. If some off-target token has positive log-odds M(z), or some target token has non-positive log-odds, Lemma B.4 fails and the peak-ordering of Theorem 3.3 and monotonicity of Theorem 3.6 are violated. Alternatively, find a real LM and steering direction where some token's Δp(α) has two local maxima — that would contradict the unique-peak claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A steering 'sweet spot' exists: since off-target tokens peak before target tokens, there is a range of α where the target concept is favored before the next-token distribution collapses.
  • The bump peak location α(j,z) varies with context, so a single global α cannot be optimal; prompt-adaptive steering strength is the natural next step.
  • Cross-entropy's quadratic coefficient is a variance of log-odds, explaining why even steering toward the intended concept locally hurts perplexity.
  • At large |α|, the output stops depending on the input prompt, so cross-entropy plateaus and further steering only pushes the model into a fixed, input-independent distribution.
  • The tanh-shaped concept-probability law gives a closed-form target for concept-presence metrics, letting practitioners predict how much concept probability changes per unit of α.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The theory suggests a principled way to price steering strength: choose α by maximizing target-concept probability while keeping ΔCE below a budget, using the closed-form quadratic coefficient as a local cost model.
  • The real engine is the sign-separation property of log-odds, not the two-valued probability structure itself; a testable generalization is to replace Assumption 1 with sign separation alone and re-derive the three laws.
  • Sampling-based concept metrics can mask the underlying token-level changes; the paper's Δp(C|α) predicts that concept probability may shift even when sampled outputs rarely contain concept tokens, which may explain noisy sigmoidal curves in real evaluations.
  • A structural test on a real LLM: measure M(z) from the contrastive prompts and check whether the ranking of peak positions across tokens matches the ranking of M(z); if so, the bump law is confirmed beyond the toy model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies how the steering strength α affects the output of language models when a difference-of-means steering vector is applied to the residual stream. The authors build on the Unconstrained Features Model (UFM) with a vocabulary partitioned into disjoint concepts, assume perfectly trained models, and derive three qualitative laws: (i) next-token probability increases Δp(z, α) are bump-shaped, with off-target tokens peaking before target tokens (Thm. 3.3); (ii) the target-concept probability increase follows a tanh-shaped (sigmoidal) curve and certain off-target concept probabilities decrease (Thm. 3.6); and (iii) the cross-entropy increase is locally quadratic with a nonnegative coefficient expressed as the variance of the log-odds (Thm. 3.8). For real transformers, they prove that large-|α| logits converge to LN(±v)W^T, independent of the prompt (Prop. 4.1). Experiments on eleven LLMs are presented as empirical validation of the three laws.

Significance. The theoretical development is coherent: the proofs in App. B are mathematically sound, the derivations follow from the stated assumptions, and the cross-entropy law (Thm. 3.8) is a genuinely parameter-free prediction with a clear mechanistic interpretation. Prop. 4.1 is a useful and simple observation about the asymptotic dominance of the steering direction under layer normalization. The authors are also transparent about the unrealism of Assumption 1. However, the empirical validation is substantially weaker than the paper's claims. The central sign-separation property (Lemma B.4) that drives the bump and tanh laws is never measured; the concept-probability experiments use a judge-LLM sampling metric that is not the quantity in Thm. 3.6; and the token-level plots select only the eight highest-probability tokens at a single large α. Thus, while the theory contribution is solid, the manuscript in its current form does not convincingly bridge the gap to practice.

major comments (4)
  1. [App. B.1 / Lemma B.4 / Sec. 5] Theorems 3.3 and 3.6 both rely on Lemma B.4 (sign separation of M(z)), which is a direct consequence of the unrealistic Assumption 1. The manuscript states in App. B.1 that 'Lemma B.4 seems to be true in practice see Appendix A', but Appendix A never measures M(z) or the sign-separation property; it only plots Δp for eigths tokens at α=200. This is a load-bearing gap: without direct evidence that real models satisfy sign separation, the empirical bump and tanh observations do not validate the mechanism. The authors should either estimate M(z) from the model's contrastive logits (which is feasible) or explicitly reframe the empirical claims as 'consistent with' rather than validation.
  2. [Sec. 5, Fig. 7 / Def. 3.5] The empirical concept probability is obtained by a judge LLM on 32 sampled completions, not by the theoretical Δp(C|α) of Def. 3.5. Theorem 3.6 makes statements about the latter, and no theorem is proven for the sampling-based judge metric. The 'mostly sigmoidal trend' is therefore not a test of Thm. 3.6. Moreover, since any increasing bounded function is a sigmoid under reparametrization, the observed S-shape has little discriminative power. The authors should test the specific predictions of Thm. 3.6 (e.g., the limits) or report the theoretical Δp(C|α) directly from the model's output distribution.
  3. [Sec. 5, Fig. 6] The token-level plots select the eight highest-probability tokens at α=200. This selection is not random and biases the comparison: the theory predicts that tokens with maximal log-odds increase monotonically while others bump, and the selected set will over-represent tokens with high M(z). The paper does not quantitatively test the peak-order prediction (off-target peaks before target) or the limiting behavior of Prop. B.1. I recommend showing results for all tokens (or a random sample), adding error bars across P/N resamples in the main figures, and reporting the fraction of tokens exhibiting bumps as a function of α.
  4. [Thm. 3.8 / Sec. 5] The cross-entropy theorem gives a precise coefficient, (1/2) Σ_j π_j Var_j(M(Z)). The experiments only confirm the qualitative U-shape; the coefficient is never estimated. Since this is the theory's sharpest quantitative prediction and it is parameter-free, the validation would be much stronger if the predicted curvature were compared with the empirical one, or the paper should clearly state that only the sign and local quadratic form are being tested, not the exact coefficient.
minor comments (5)
  1. [Sec. 5] 'using the random construction of P' should read 'random construction of N'; the positive set P is always the target concept.
  2. [Def. 2.2 / Sec. 4] The theory's context embedding h_j represents an entire context, while the experiments average the residual stream over all token positions of a generation. The relationship between the two is not discussed; please clarify or justify this approximation.
  3. [Prop. 4.1 / App. B.6] The proof of boundedness of R(α) is only sketched for a single RMSNorm-then-Hadamard block. A complete argument for the full transformer block (attention, residual connections, LayerNorm) would make the proposition fully rigorous.
  4. [Fig. 6 and App. A] The caption claim 'The selected tokens are all related to the steered concept' holds only for middle/late layers; early-layer plots in App. A show unrelated tokens. Please qualify the claim.
  5. [General] Minor typos: 'expression' in Sec. 4 is misspelled 'expresssion'; the reference formatting for 'V on Rütte' is inconsistent in the bibliography.

Circularity Check

0 steps flagged

Derivation is self-contained: the bump, tanh, and quadratic cross-entropy laws follow by calculus from the stated assumptions; no fitted parameter is renamed as a prediction.

full rationale

The paper's central theorems are derived, not fitted. Theorem 3.3, 3.6, and 3.8 all follow from Lemma B.2-B.5, which are algebraic consequences of the softmax reweighting identity σ_z(f_α(c_j)) = p(z|c_j) exp(αM(z)) / Σ_{z'} p(z'|c_j) exp(αM(z')). The log-odds M(z) is defined directly from the assumed dataset probabilities (Def. 3.2), and no parameter is estimated from steered-model outputs to force the predicted shapes. Lemma B.4 (sign separation of M(z)) is a direct consequence of Assumption 1's two-value structure a_z > b_z, not an imported self-citation. Theorem 3.8's quadratic coefficient is an exact Taylor-expansion variance of M(Z), and Proposition 4.1 follows from layer-norm scaling with α. Empirical sections use real LLMs only as confirmation, and the paper explicitly distinguishes its practical judge-LLM concept metric from the theoretical Δp(C|α). The candid caveat in App. B.1—'if we see Lemma B.4 as an assumption, then our results work'—is a modeling limitation (the practical validity of sign-separated log-odds is asserted rather than directly measured), not a circular step. Therefore no prediction reduces by construction to its input, and the derivation is self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The theory introduces no fitted numerical parameters and no new physical entities. Its load-bearing inputs are the two explicit assumptions (concept-disjoint two-value probabilities; perfect training) plus the UFM abstraction. The empirical validation adds no fitted constants.

axioms (5)
  • domain assumption Assumption 1: vocabulary is partitioned into disjoint concepts; p(z|c_j) takes only two values a_z (same concept) and b_z (different concept), with a_z > b_z.
    This is the structural premise that makes log-odds M(z) sign-separated (Lemma B.4); all subsequent theorems rely on it. The paper itself calls it unrealistic for real text (Sec. 2.1).
  • domain assumption Assumption 2: the UFM is perfectly trained, so sigma_z(f(c_j)) = p(z|c_j).
    Required for Lemma B.3, which rewrites the steered softmax in terms of dataset probabilities. Approximate in practice; justified by memorization and a toy GD illustration (App. B.1).
  • domain assumption The UFM abstraction: optimizing embeddings freely is a faithful proxy for real transformer representation geometry.
    Borrowed from Zhao et al. 2024 and Thrampoulidis 2024; Sec. 2.2 states this is the basis of the theoretical framework.
  • domain assumption The steering vector is a difference of means of perfectly separable positive/negative context embeddings, with |P| = |N|.
    Definition in Eq. (3); needed for Lemma B.2. Empirical vectors are averages over model generations and are not guaranteed to satisfy the clean separability.
  • standard math Softmax shift-invariance and the exp(alpha M(z)) reweighting identity.
    Used throughout App. B to derive the steered distribution from the unsteered probabilities.

pith-pipeline@v1.3.0-alltime-deepseek · 32615 in / 11214 out tokens · 105502 ms · 2026-08-03T05:16:38.579498+00:00 · methodology

0 comments
read the original abstract

A popular approach to post-training control of large language models (LLMs) is the steering of intermediate latent representations. Namely, identify a well-chosen direction depending on the task at hand and perturbs representations along this direction at inference time. While many propositions exist to pick this direction, considerably less is understood about how to choose the magnitude of the move, whereas its importance is clear: too little and the intended behavior does not emerge, too much and the model's performance degrades beyond repair. In this work, we propose the first theoretical analysis of steering strength. We characterize its effect on next token probability, presence of a concept, and cross-entropy, deriving precise qualitative laws governing these quantities. Our analysis reveals surprising behaviors, including non-monotonic effects of steering strength. We validate our theoretical predictions empirically on eleven language models, ranging from a small GPT architecture to modern models.

Figures

Figures reproduced from arXiv: 2602.02712 by Damien Garreau, Magamed Taimeskhanov, Samuel Vaiter.

Figure 1
Figure 1. Figure 1: Top: Constructing a steering vector v, for the target con￾cept “code safety”, at the ℓ th block. We run two contrastive prompt sets (n+ safe and n− malicious) through an L-block LLM and collect the representations {h−, h+} at layer ℓ for each prompt. Averaging these representations over all safe prompts gives µ+, and for the malicious prompts it gives µ− (both marked by a cross). We then define v := µ+ − µ… view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of dataset next-token probabilities (p(z | cj ))z∈[V ] for the vocabulary of Eq. (2): probabilities for the con￾text c2 = aab are shown in solid-color, while probabilities for c1 = ABB are shown transparent. This illustrates our dataset condition az > bz: a token is more likely when it belongs to the same concept as the context, which is why the solid-color blue points lie above their transpa… view at source ↗
Figure 3
Figure 3. Figure 3: Next-token probability increases ∆p(α) for a fixed context. Each curve corresponds to a token z: target tokens T are in blue and off-target tokens in orange. Most target tokens exhibit a “bump” (peaking at α(1,1)), while one target token increases and off-target tokens decrease. Intuitively, ∆p(z | cj , α) is the algebraic next-token prob￾ability increase for a fixed z ∈ [V ] when steering with strength α.… view at source ↗
Figure 4
Figure 4. Figure 4: Concept probability increases ∆p(C | α) predicted by Th. 3.6: the target concept ∆p(T | α) increases with a sigmoidal shape, an off-target ∆p(C | α) decreases sigmoidally, and another ∆p(C ′ | α) converges to the same limit as |α| → ∞. Intuitively, ∆p(C | α) is the mean of the probability increase ∆p over tokens belonging to the same concept C. This quantity serves as a natural proxy, in our setting, for t… view at source ↗
Figure 5
Figure 5. Figure 5: Local quadratic behavior of ∆CE(α), as predicted by Thm. 3.8. The blue curve shows ∆CE(α) and the black curve the quadratic fit using the coefficient from the theorem. tokens all have log-odds below those of the remaining to￾kens, ∆p(C ′ | α) decreases as α increases. See [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Influence of steering strength α on next-token probability increase ∆p(z, α) for the concept “evil,” shown for LLMs of increasing size. Each curve ∆p(z, α) corresponds to a token z selected among the eight highest-probability tokens at α = 200. This matches Thm. 3.3: most tokens exhibit a bump, while a few increase throughout. The selected tokens are all related to the steered concept. where ATTN denotes t… view at source ↗
Figure 7
Figure 7. Figure 7: Influence of steering strength α across models on: Top row: concept probability for the three concepts (depression, joy, evil), estimated using a judge LLM (Gemma 3 12B), showing the sigmoidal trend predicted by Thm. 3.6. Bottom row: cross-entropy ∆CE(α) for the same concepts, locally U-shaped around α = 0 and plateauing for large |α| (Thm. 3.8, Prop. 4.1). figurations and results, including other layers, … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Role Steering of Language Models for Social Simulations

    cs.CL 2026-07 conditional novelty 6.0

    A role-steering screening workflow on 275 roles shows role-specific activation directions beat a non-scale-matched assistant-direction control (63.2 vs 41.1 judged alignment) and flags 38 roles as 'anti-controllable'.

Reference graph

Works this paper leans on

21 extracted references · 11 linked inside Pith · cited by 1 Pith paper

  1. [1]

    = log (1−ε)(G−1) ε γz1 ωz1 by the proof of Lemma B.4. To avoid complicated solution to Inequality (11) using the LambertWfunction, we compute the limitε→0 + ofF(·)defined as: F(ε) :=M(z 1)− ε (G−1) log 1−ε ε (G−1) = log (1−ε)(G−1) ε γz1 ωz1 − ε (G−1) log 1−ε ε (G−1) . We now compute the limit as follows: lim ε→0+ log (1−ε)(G−1) ε γz1 ωz1 = +∞(as γz1 (G−1)...

    2024

  2. [2]

    L., Kiros, J

    Ba, J. L., Kiros, J. R., and Hinton, G. E. Layer normalization. arXiv preprint arXiv:1607.06450,

    Pith/arXiv arXiv

  3. [4]

    role": "system

    For readability, we report only full-word tokens; for tokenizers that produce smaller subwords, we observe the same phenomenon, with fragments such as ‘happ’ instead of ‘happiness’. Steered concept High-probability next tokens apathetic okay, yeah, bullshit, whatnot, probably depression discomfort, sadness, emotional, despair, uncomfortable evil horrifyin...

    2025

  4. [7]

    D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J

    Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scal- ing and evaluating sparse autoencoders.arXiv preprint arXiv:2406.04093,

    Pith/arXiv arXiv

  5. [8]

    Linguistic regularities in continuous space word representations

    9 Towards Understanding Steering Strength Mikolov, T., Yih, W.-t., and Zweig, G. Linguistic regularities in continuous space word representations. InProceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 746–751, Atlanta, Georgia, June

    2013

  6. [10]

    V ., Hinton, G

    Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q. V ., Hinton, G. E., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In5th International Conference on Learning Rep- resentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings,

    2017

  7. [12]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786,

    Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ram ´e, A., Rivi`ere, M., et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786,

    Pith/arXiv arXiv

  8. [13]

    Llama 2: Open foundation and fine- tuned chat models.arXiv preprint arXiv:2307.09288,

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models.arXiv preprint arXiv:2307.09288,

    Pith/arXiv arXiv

  9. [14]

    M., Thiergart, L., Leech, G., Udell, D., Vazquez, J

    Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M. Steering lan- guage models with activation engineering.arXiv preprint arXiv:2308.10248,

    Pith/arXiv arXiv

  10. [15]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  11. [17]

    M., Stiennon, N., Wu, J., Brown, T

    Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. Fine-tuning language models from human preferences.arXiv preprint arXiv:1909.08593,

    Pith/arXiv arXiv 1909

  12. [19]

    MMLU is a practical performance metric, more indicative of real-world capability than cross-entropy

    for the concept “evil”. MMLU is a practical performance metric, more indicative of real-world capability than cross-entropy. We measure it using the DeepEval library, for which random guessing yields25%. As with cross-entropy, increasingαinevitably degrades model performance. 15 Towards Understanding Steering Strength 0 50 100 150 200 α 0.0 0.2 0.4 0.6 op...

    2025

  13. [20]

    Each row of three plots corresponds to a single steered concept, and each column corresponds to a different model

    21 Towards Understanding Steering Strength -100 -50 0 50 100 α 0 10 20 openai-community/gpt2-large 11 23 35 -100 -50 0 50 100 α 0 10 20 30 40 google/gemma-3-1b-it 8 16 25 -100 -50 0 50 100 α 0 10 20 Qwen/Qwen3-0.6B 9 18 27 -100 -50 0 50 100 0 5 10 15 20 25 11 23 35 -100 -50 0 50 100 0 10 20 30 40 50 8 16 25 -100 -50 0 50 100 0 5 10 15 20 9 18 27 -100 -50 ...

    2024

  14. [2013]

    Association for Computational Linguistics. Min, P. P., Paudel, A., Adityo, N., Zhu, A., Rufail, A., Blondin, C., Zhu, K., Dev, S., and O’Brien, S. Mitigating sycophancy in language models via sparse activation fu- sion and multi-layer activation steering. InMechanistic Interpretability Workshop at NeurIPS 2025,

    2025

  15. [2016]

    Training a helpful and harmless assistant with rein- forcement learning from human feedback.arXiv preprint arXiv:2204.05862,

    Bai, Y ., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with rein- forcement learning from human feedback.arXiv preprint arXiv:2204.05862,

    Pith/arXiv arXiv

  16. [2017]

    C., Lyzhov, A., Pfau, J., Mahdi, S., and Bow- man, S

    Stickland, A. C., Lyzhov, A., Pfau, J., Mahdi, S., and Bow- man, S. R. Steering Without Side Effects: Improving Post- Deployment Control of Language Models. InNeurips Safe Generative AI Workshop 2024,

    2024

  17. [2019]

    and Thrampoulidis, C

    Zhao, Y . and Thrampoulidis, C. Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations.arXiv preprint arXiv:2505.08348, 2025a. Zhao, Y . and Thrampoulidis, C. Geometry of Concepts in Next-token Prediction: Neural-Collapse Meets Semantics. InThe Second Conference on Parsimony and Learning (Recent Sp...

    arXiv

  18. [2021]

    Dynamically Scaled Activation Steering.arXiv preprint arXiv:2512.03661,

    Ferrando, A., Suau, X., Gonz `alez, J., and Rodriguez, P. Dynamically Scaled Activation Steering.arXiv preprint arXiv:2512.03661,

    Pith/arXiv arXiv

  19. [2023]

    Persona vectors: Monitoring and controlling character traits in language models.arXiv preprint arXiv:2507.21509,

    Chen, R., Arditi, A., Sleight, H., Evans, O., and Lind- sey, J. Persona vectors: Monitoring and controlling character traits in language models.arXiv preprint arXiv:2507.21509,

    Pith/arXiv arXiv

  20. [2024]

    Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N

    Accessed: 2025-10-15. Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N. Refusal in language models is mediated by a single direction.Advances in Neural Infor- mation Processing Systems, 37:136037–136083,

    2025

  21. [2025]

    Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Pith/arXiv arXiv