Scaling laws for activation steering with Llama 2 models and refusal mechanisms

Ayse Arslan; Clark Benham; Ivory Yang; Jasmine Xinze Li; Justin Xu; Sheikh Abdur Raheem Ali

REVIEW 3 major objections 6 minor 16 references

Scaling laws for activation steering with Llama 2 models and refusal mechanisms

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that contrastive activation addition, a training-free steering method, loses effectiveness exponentially as LLM parameter count grows, reaching an 8 percent floor at 70B.

desk verdict A plausible qualitative result on CAA scaling across Llama 2 sizes, wrapped in an unsupported three-point exponential 'law'. read the letter →

arxiv 2507.11771 v1 pith:ASWJJAAY submitted 2025-07-15 cs.LG

Sheikh Abdur Raheem Ali , Justin Xu , Ivory Yang , Jasmine Xinze Li , Ayse Arslan , Clark Benham This is my paper

classification cs.LG

keywords activationsteeringcontrastiveadditionscalinglawsLlama2refusalbehaviorresidualstreammodelalignmentmechanisticinterpretability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a scaling law for contrastive activation addition (CAA), a training-free way to steer an LLM by computing a direction vector from contrasting examples and adding it to the model's residual stream during inference. Working with Llama 2 models of 7B, 13B, and 70B parameters and refusal behavior as the target, the authors report that peak steering effectiveness decays exponentially with parameter count, fitting $y = 0.081 + 2.4 e^{-0.42x}$. They also report that the best injection layer sits in the early-mid layers near $0.4$ of total layer count, that negative steering (pushing the model away from refusal) is stronger than positive steering (reinforcing refusal) at every size, and that the peak layers for positive and negative steering converge as models grow. If the law holds, activation steering will require either stronger vectors, multi-layer injection, or per-scale calibration to stay useful as models scale.

What carries the argument

The machinery is contrastive activation addition (CAA) combined with a layer sweep. For each layer, the paper caches residual-stream vectors of the last token on refusal and non-refusal prompts, subtracts the pairs to get per-layer direction vectors, averages them, normalizes to the average direction norm, and adds the resulting vector to the residual stream, the internal activation pathway running through all transformer layers, just before the layer's add-and-normalize step. Sweeping the injection layer across all layers and measuring the change in percentage of correct refusal answers produces the effectiveness curves; an exponential fit to the peak values is the scaling law the paper proposes.

What would settle it

Run the same CAA protocol over Llama 2 7B, 13B, and 70B on a fixed refusal test set while reporting baseline accuracy, prompt count, per-layer confidence intervals, and several steering coefficients; if the per-layer curves overlap once baselines are normalized, the exponential decay claim is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that contrastive activation addition scales poorly: on the refusal test, peak effectiveness $y$ falls as $y = 0.081 + 2.4 e^{-0.42x}$, with $x$ the parameter count, so the measured effect at 13B is already less than half of that at 7B and approaches a floor near $0.081$ at 70B. The authors take this as evidence that larger models 'drown out' a single injected steering vector through more downstream computation. They further find that effectiveness peaks in early-mid layers at roughly $0.4$ times the layer count; that steering in the negative direction, toward non-refusal, has larger effects than steering in the positive direction, toward refusal, across all sizes; and that the peak layer for positive and negative steering converges as scale increases, from a two-layer gap at 7B to a zero-layer gap at 70B.

Load-bearing premise

The metric this law depends on, the percentage of correct refusal answers before and after steering, is assumed to be comparable and roughly linear across models of different size; the paper reports no baseline accuracy, number of test prompts, or variance, so lower measured changes could reflect saturation or different baselines rather than weaker steering.

Editorial extensions

If this is right

At 70B parameters, peak CAA effectiveness on refusal approaches roughly 8 percent, so steering with the same vector becomes nearly inert without adjustments.
The optimal injection point sits around 40 percent of the layers, letting practitioners target early-mid layers for maximum effect.
Steering the model away from refusal is consistently stronger than steering it toward refusal, which means weakening safety behaviors is easier than reinforcing them.
As model size grows, positive and negative steering peak in the same layer (7B: two layers apart, 13B: one, 70B: zero), so one layer can serve both directions.
If the fit extrapolates, larger future models would need exponentially stronger or differently targeted steering vectors to achieve the same behavioral shift.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: if the decay reflects the model drowning out injected vectors in downstream computation, then repeated injection at multiple layers or scaled vector magnitudes, suggested but not tested in the paper, should restore effectiveness.
Editorial extension: the convergence of positive and negative peak layers is consistent with the paper's superposition explanation, but a saturation-in-evaluation alternative remains untested; varying the steering coefficient at each size would separate the two.
Editorial extension: the RLHF asymmetry argument predicts that for other RLHF-trained behaviors, the direction opposing the training signal will show larger steering effects than the direction reinforcing it, a claim a broader behavior suite could test.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The short version: this is a workshop-quality empirical note with one genuinely useful observation and one overreaching curve fit. The useful part is the clean comparison of CAA across Llama 2 7B, 13B, and 70B on refusal behavior: peak steering effectiveness lands at about 0.4x total layers, negative steering is stronger than positive, and the layer gap between positive and negative peaks narrows from 2 to 0 as the model grows. Those are real measurements that extend Rimsky et al., even if they are not shocking.

The overreach is the 'scaling law' in Section 4: y = 0.081 + 2.4 * exp(-0.42x), fit to exactly three points. Three free parameters, three points, no error bars, no baseline accuracy, no number of test prompts, no variance. The stress-test note is right that a linear fit would be statistically indistinguishable from this exponential. The paper itself supplies no way to check whether the decline is in steering power or in the metric: they report percentage-point change in correct refusal answers, and if larger models start with a higher refusal baseline, the same raw effect looks smaller in percentage points.

The paper does honestly flag some limitations: the asymmetry hypothesis is explicitly tentative, and they note the refusal-only scope. But the abstract and title sell the exponential relationship as a finding without those caveats. Also, the text promises an appendix that does not appear here, and no code or data is provided, so the numbers cannot be checked.

What the paper does well: the methodology follows the established CAA recipe clearly, the layer sweep is straightforwardly described, and the qualitative findings—diminishing returns with scale, stronger negative steering, mid-layer peaks—are consistent with adjacent work and worth putting on the record. The writing is honest about being limited to one behavior.

Minor but real: the LaTeX template keeps an ICML 2024 proceedings footer while claiming NewInML workshop acceptance, and the Templeton & Conerly reference is incomplete. Sloppy, not scientific.

My judgment: the central quantitative claim is not supported by the evidence reported. But the qualitative observations are plausible and could be made solid with the full per-layer effectiveness table, baselines, prompt counts, and variance. I would send this to a serious referee, asking the authors to either supply those numbers or reframe the exponential as a purely descriptive fit over three points. As it stands, cite it as exploratory, not as a law.

Recommendation: accept for peer review with a clear expectation of heavy revision; this is a ten-minute read that raises a real question about how evaluation metrics interact with model scale.

Referee Report

3 major / 6 minor

Summary. The paper studies contrastive activation addition (CAA) on the Llama-2 family (7B, 13B, and 70B) in the context of refusal behavior. It reports layer-by-layer sweeps of steering vectors and claims three qualitative findings: (1) CAA is most effective at early-to-mid layers, (2) peak effectiveness decreases with model size, and (3) negative steering consistently outperforms positive steering. The paper also proposes a quantitative scaling law, y = 0.081 + 2.4·exp(-0.42x), where y is peak effectiveness and x is parameter count, and observes that the peak-steering layer for positive and negative steering converges as model size grows (7B: 2-layer difference, 13B: 1-layer, 70B: 0-layer). The authors offer a mechanistic hypothesis based on RLHF pushing models to a near-maximum refusal state, and they explicitly note the need for further experiments with a broader set of behaviors.

Significance. If the scaling law were supported, it would provide a concrete, falsifiable prediction for steering behavior across model sizes and could inform when activation steering is practical for large deployed models. The paper's strengths are that it applies an established CAA methodology to multiple model scales, performs per-layer sweeps, makes a clear quantitative prediction, and includes explicit acknowledgment of its own limitations. However, the central quantitative claim is not currently supported by the reported evidence: the exponential law is fitted to exactly three points with three free parameters, with no error bars, no baselines, no test-set sizes, and no validation. The qualitative observations are plausible and interesting, but they require additional reporting to distinguish real trends from evaluation artifacts.

major comments (3)

[Section 4, Eq. (no number), 'y = 0.081 + 2.4·e−0.42·x'] The central quantitative claim is an exponential fit to exactly three data points (7B, 13B, 70B) with three free parameters (amplitude, decay constant, and offset). No per-layer effectiveness values, no variance estimates, no baseline accuracies, and no out-of-sample validation are reported anywhere in Section 3.2 or Section 4. With three points and three free parameters, the exponential form is not identifiable: a linear or power-law fit would be statistically indistinguishable without additional information. The paper must either report the full layer-by-layer data with error bars and fit diagnostics (e.g., confidence intervals or cross-validation), or explicitly reframe the result as a qualitative trend rather than a scaling law.
[Section 3.2, Evaluation] The evaluation metric is described only as 'the percentage of times it answers correctly' on a refusal answer-matching test set, with no number of prompts, no per-model baseline accuracy, and no repeated-run variance. The reported effect is the percentage-point change from baseline, so if larger models refuse at a higher baseline rate, a fixed raw change in refusal probability will appear as a smaller percentage-point change. This could make the observed decline in effectiveness with model size partly an artifact of the metric. The paper needs to report baseline accuracies, test-set sizes, and per-layer values with variance or confidence intervals before the size-dependence claims in Section 4 can be accepted.
[Section 4, peak-layer convergence observation] The observation that positive and negative steering peak at the same layer as model size grows (7B: 2-layer difference, 13B: 1-layer, 70B: 0-layer) is based on three unquantified comparisons, and the three models have different total layer counts (32, 40, and 80 for Llama-2-7B, 13B, and 70B). A raw difference in layer indices is not directly comparable across models of different depths. Without normalizing by total layer count, reporting uncertainty, or demonstrating robustness to the layer-at-which-steering-is-applied discretization, this three-point pattern is too weak to support the superposition interpretation offered at the end of Section 4.

minor comments (6)

[General, typographical] The manuscript contains repeated typos and grammatical issues, including 'are are needed' in Section 4 and 'technical developed' in Section 7; these should be corrected in a revision.
[Section 3.1, 'See appendix'] The text refers to a dataset 'within answer matching behavior (See appendix)', but no appendix appears in the manuscript; either include the appendix or remove the pointer.
[References, Templeton & Conerly 2024] The reference is incomplete: the venue is listed as 'N/A', and the title is given as 'Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet'; full publication details should be supplied.
[Figure 3 caption] The caption reads 'Comparison of LLaMA models' while the text uses 'Llama'; capitalization should be made consistent throughout.
[Header, venue information] The header includes 'Accepted by NewInML workshop @ ICML 2025' but also carries the PMLR 2024 proceedings boilerplate; this inconsistency should be resolved, as it affects citation metadata.
[Section 3.1, injection details] The method section states that residual stream vectors are cached for the last token at every layer, while the steering vector is added to every token position; it should be clarified whether the cached last-token vectors are used only to compute the steering direction, since this affects reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the steering construction, evaluation, and fitted scaling curve are empirically independent of the claims they support.

full rationale

The paper's derivation chain is empirical rather than circular. Steering vectors are computed by caching last-token residual streams for positive and negative examples, subtracting them per layer, and normalizing by the average norm of the direction vectors (Sec. 3.1). Effectiveness is then measured as the change in the percentage of correct refusal answers on an isolated test set (Sec. 3.2), so the evaluation is not defined in terms of the fitted curve or the steering vectors' construction. The exponential scaling relation y = 0.081 + 2.4·exp(-0.42x) in Sec. 4 is a curve fit to the three measured peak-effectiveness values, not a quantity derived from the assumptions that define the outcome; it is a compact description of the observed data rather than a prediction forced by construction. The paper also does not rely on load-bearing self-citations: the cited CAA and activation-addition methods (Rimsky et al., 2023; Turner et al., 2023) are external prior work, and no author of the present paper appears among those citations. The acknowledged limitations (refusal-only topic, need for further experiments, possibility of different results for other behaviors) further indicate that the claims are offered as empirical observations. Concerns that a three-point exponential fit is statistically fragile or that percentage-point changes may be affected by baseline refusal rates are validity and robustness issues, not circularity; they do not make any equation reduce to its own inputs. Accordingly, no circular steps are present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The exponential law adds three fitted parameters; no new entities are introduced. The main assumptions are comparability of the evaluation metric across model sizes, the informativeness of last-token vectors, and an untested RLHF-as-maximal-refusal interpretation.

free parameters (3)

Exponential fit amplitude = 2.4
Fitted to peak effectiveness across three model sizes (7B, 13B, 70B) in Section 4. Together with the decay constant and offset, three parameters exactly fit three data points.
Exponential fit decay constant = 0.42
Fitted in Section 4 to control how fast effectiveness drops with parameter count. Not independently validated.
Exponential fit offset = 0.081
Fitted in Section 4 as the asymptotic peak effectiveness at large scale. Determined by the single 70B data point.

assumptions (3)

domain assumption The last-token residual stream vector at each layer contains sufficient information about refusal behavior to derive steering directions.
Invoked in Section 3.1 to justify caching last-token vectors. Standard in CAA, but not independently verified here for refusal behavior across model sizes.
domain assumption Answer-matching percentage change is a valid and comparable measure of steering effectiveness across different model sizes.
Section 3.2 defines the evaluation metric. No baseline, prompt count, or variance is reported, so comparability across sizes is assumed.
ad hoc to paper RLHF pushes the chat models to a near-maximum refusal state, so negative steering has more room to act.
Section 4 introduces this as a hypothesis to explain asymmetric steering. It is not independently measured or tested in this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Scaling laws for activation steering with Llama 2 models and refusal mechanisms." pith.science (2026). https://pith.science/paper/ASWJJAAY

@misc{pith2026250711771,
  author       = {Pith},
  title        = {Pith review of: Scaling laws for activation steering with Llama 2 models and refusal mechanisms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ASWJJAAY}},
  note         = {Machine review of arXiv:2507.11771}
}

read the original abstract

As large language models (LLMs) evolve in complexity and capability, the efficacy of less widely deployed alignment techniques are uncertain. Building on previous work on activation steering and contrastive activation addition (CAA), this paper explores the effectiveness of CAA with model scale using the family of Llama 2 models (7B, 13B, and 70B). CAA works by finding desirable 'directions' in the model's residual stream vector space using contrastive pairs (for example, hate to love) and adding this direction to the residual stream during the forward pass. It directly manipulates the residual stream and aims to extract features from language models to better control their outputs. Using answer matching questions centered around the refusal behavior, we found that 1) CAA is most effective when applied at early-mid layers. 2) The effectiveness of CAA diminishes with model size. 3) Negative steering has more pronounced effects than positive steering across all model sizes.

Figures

Figures reproduced from arXiv: 2507.11771 by the authors.

**Figure 1.** Sample from the refusal dataset used to generate steering vectors test dataset. For each forward pass, we choose one layer to inject the corresponding steering vector obtained from the same layer. The steering vector is added to every token position just before the add & normalize step after the MLP (feed forward NN) of the layer to preserve overall residual norms and not break the model. We perform a sweep of every… view at source ↗

**Figure 2.** CAA visualized (Rimsky et al., 2023) 2 [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 4.** Steering effectiveness vs. model parameter count [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Maximum positive and negative steering effects model has more ’space’ to be steered in the direction of less refusal (suppressing RLHF signals) than more refusal. However, further experiments with a more complete set of behaviors are are needed to support this claim. S…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 3 canonical work pages

[1]

and Gavves, E

Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety--a review. arXiv preprint arXiv:2404.14082, 2024

arXiv 2024
[2]

A toy model of universality: Reverse engineering how networks learn group operations

Chughtai, B., Chan, L., and Nanda, N. A toy model of universality: Reverse engineering how networks learn group operations. In International Conference on Machine Learning, pp.\ 6243--6267. PMLR, 2023

work page 2023
[3]

Towards automated circuit discovery for mechanistic interpretability

Conmy, A., Mavor-Parker, A., Lynch, A., Heimersheim, S., and Garriga-Alonso, A. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36: 0 16318--16352, 2023

2023
[4]

Toy models of superposition

Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022

arXiv 2022
[5]

Finding neurons in a haystack: Case studies with sparse probing

Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troitskii, D., and Bertsimas, D. Finding neurons in a haystack: Case studies with sparse probing. arXiv preprint arXiv:2305.01610, 2023

arXiv 2023
[6]

M., Engstrom, L., Leclerc, G., and Madry, A

Ilyas, A., Park, S. M., Engstrom, L., Leclerc, G., and Madry, A. Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.00622, 2022

arXiv 2022
[7]

M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A

Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A. Trak: Attributing model behavior at scale. arXiv preprint arXiv:2303.14186, 2023

arXiv 2023
[8]

Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681, 2023

arXiv 2023

Show all 16 references

[9]

M., Ilyas, A., and Madry, A

Shah, H., Park, S. M., Ilyas, A., and Madry, A. Modeldiff: A framework for comparing learning algorithms. In International Conference on Machine Learning, pp.\ 30646--30688. PMLR, 2023

2023
[10]

Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet

Templeton and Conerly. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. N/A, 2024

2024
[11]

Activation addition: Steering language models without optimization

Turner, A., Thiergart, L., Udell, D., Leech, G., Mini, U., and MacDiarmid, M. Activation addition: Steering language models without optimization. arXiv preprint arXiv:2308.10248, 2023

2023 arXiv
[12]

Interpretability in the wild: a circuit for indirect object identification in gpt-2 small

Wang, K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593, 2022

2022 arXiv
[13]

Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024

2024
[14]

Knowledge conflicts for llms: A survey

Xu, R., Qi, Z., Wang, C., Wang, H., Zhang, Y., and Xu, W. Knowledge conflicts for llms: A survey. arXiv preprint arXiv:2403.08319, 2024

2024 arXiv
[15]

Z., and Fredrikson, M

Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023

2023 arXiv
[16]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

and Gavves, E

Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety--a review. arXiv preprint arXiv:2404.14082, 2024

arXiv 2024

[2] [2]

A toy model of universality: Reverse engineering how networks learn group operations

Chughtai, B., Chan, L., and Nanda, N. A toy model of universality: Reverse engineering how networks learn group operations. In International Conference on Machine Learning, pp.\ 6243--6267. PMLR, 2023

work page 2023

[3] [3]

Towards automated circuit discovery for mechanistic interpretability

Conmy, A., Mavor-Parker, A., Lynch, A., Heimersheim, S., and Garriga-Alonso, A. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36: 0 16318--16352, 2023

2023

[4] [4]

Toy models of superposition

Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022

arXiv 2022

[5] [5]

Finding neurons in a haystack: Case studies with sparse probing

Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troitskii, D., and Bertsimas, D. Finding neurons in a haystack: Case studies with sparse probing. arXiv preprint arXiv:2305.01610, 2023

arXiv 2023

[6] [6]

M., Engstrom, L., Leclerc, G., and Madry, A

Ilyas, A., Park, S. M., Engstrom, L., Leclerc, G., and Madry, A. Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.00622, 2022

arXiv 2022

[7] [7]

M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A

Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A. Trak: Attributing model behavior at scale. arXiv preprint arXiv:2303.14186, 2023

arXiv 2023

[8] [8]

Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681, 2023

arXiv 2023