{"id":"27f0e298-162b-4eae-917b-e34ff03d573f","arxiv_id":"2507.11771","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Activation steering with contrastive vectors becomes less effective as Llama 2 models scale from 7B to 70B parameters, with peak effect at roughly 40% of the model's layers.","lead":"This paper measures how well contrastive activation addition (CAA) steers the behavior of three sizes of Llama 2 chat models. It reports that steering works best in early-middle layers, works less well as models get bigger, and is stronger when pushing away from refusal than toward it.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A three-point exponential fit is presented as a scaling law without any measure of spread or validation; the central quantitative claim is not supported by the paper's own data.","rationale":"The reader's weakest assumption identifies the evaluation metric as the core vulnerability: treating percentage-point changes as comparable across model sizes without baselines, prompt counts, or variance. The strongest claim, the exponential scaling law, is exactly where that assumption is load-bearing. My stress-test pass agrees with the reader that the qualitative observations are plausible but that the quantitative scaling law is not supported by the reported evidence. The paper provides no per-layer data, no baseline accuracies, no test-set size, and no variance, so the three-point exponential curve cannot be validated or falsified from within the paper. The proposed concrete test, recomputing peak effectiveness on a log-odds or variance-adjusted scale and bootstrapping over prompts, would directly test whether the claimed decay is a property of the models or an artifact of the percentage-point metric. This does not require rejecting the paper: the qualitative findings (early-mid layer peaks, stronger negative steering, converging peak layers) can stand as exploratory observations, but the scaling law should be reframed or validated. Hence a CONDITIONAL verdict is appropriate: the central qualitative trend is not contradicted by the paper's own data, but the quantitative 'scaling law' needs the missing evaluation details and a metric-robustness check before it can be accepted.","tokens_in":4470,"tokens_out":1256,"duration_ms":13732,"concrete_test":"Obtain the per-layer accuracy values, baseline accuracies, and test-set size for 7B, 13B, and 70B, then recompute peak effectiveness in a scale-invariant metric such as log-odds change or Cohen's h (2·asin(sqrt(p)) difference) on a fixed common set of refusal prompts. If the exponential decay in Section 4 persists under a log-odds transformation and is robust to bootstrap resampling over prompts, the scaling claim survives; if the apparent decay shrinks or disappears, the reported scaling law is an artifact of percentage-point saturation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central quantitative claim is the exponential decay y = 0.081 + 2.4·exp(-0.42x) in Section 4, where y is peak steering effectiveness and x is parameter count. This curve is fit to exactly three points (7B, 13B, 70B), and the paper reports no baseline accuracy, no number of test prompts, no per-layer effectiveness values, and no variance or error bars anywhere in Section 3.2 or Section 4. Consequently, the exponential form, the fitted constants, and the claimed floor of roughly 8% at 70B are not identifiable from the reported evidence: any monotone decreasing curve through three points would fit equally well, and a linear fit would be statistically indistinguishable without variance information. The comparison across model sizes is also metric-dependent: the evaluation measures percentage of correct refusal answers before and after steering, so a fixed raw change in refusal probability yields a smaller percentage-point change when the baseline is near 100%. If the larger models refuse at a higher baseline rate, the observed decline in percentage-point effectiveness could be partly an artifact of the metric rather than a genuine decrease in steering power. The qualitative observations in the abstract and Section 4 (early-mid layer peaks, negative steering stronger than positive, peak layers converging with scale) may well be correct, but the paper itself does not supply enough information to distinguish those observations from evaluation artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies contrastive activation addition (CAA) on the Llama-2 family (7B, 13B, and 70B) in the context of refusal behavior. It reports layer-by-layer sweeps of steering vectors and claims three qualitative findings: (1) CAA is most effective at early-to-mid layers, (2) peak effectiveness decreases with model size, and (3) negative steering consistently outperforms positive steering. The paper also proposes a quantitative scaling law, y = 0.081 + 2.4·exp(-0.42x), where y is peak effectiveness and x is parameter count, and observes that the peak-steering layer for positive and negative steering converges as model size grows (7B: 2-layer difference, 13B: 1-layer, 70B: 0-layer). The authors offer a mechanistic hypothesis based on RLHF pushing models to a near-maximum refusal state, and they explicitly note the need for further experiments with a broader set of behaviors.","tokens_in":4816,"tokens_out":3594,"duration_ms":42889,"significance":"If the scaling law were supported, it would provide a concrete, falsifiable prediction for steering behavior across model sizes and could inform when activation steering is practical for large deployed models. The paper's strengths are that it applies an established CAA methodology to multiple model scales, performs per-layer sweeps, makes a clear quantitative prediction, and includes explicit acknowledgment of its own limitations. However, the central quantitative claim is not currently supported by the reported evidence: the exponential law is fitted to exactly three points with three free parameters, with no error bars, no baselines, no test-set sizes, and no validation. The qualitative observations are plausible and interesting, but they require additional reporting to distinguish real trends from evaluation artifacts.","major_comments":[{"comment":"The central quantitative claim is an exponential fit to exactly three data points (7B, 13B, 70B) with three free parameters (amplitude, decay constant, and offset). No per-layer effectiveness values, no variance estimates, no baseline accuracies, and no out-of-sample validation are reported anywhere in Section 3.2 or Section 4. With three points and three free parameters, the exponential form is not identifiable: a linear or power-law fit would be statistically indistinguishable without additional information. The paper must either report the full layer-by-layer data with error bars and fit diagnostics (e.g., confidence intervals or cross-validation), or explicitly reframe the result as a qualitative trend rather than a scaling law.","section":"Section 4, Eq. (no number), 'y = 0.081 + 2.4·e−0.42·x'"},{"comment":"The evaluation metric is described only as 'the percentage of times it answers correctly' on a refusal answer-matching test set, with no number of prompts, no per-model baseline accuracy, and no repeated-run variance. The reported effect is the percentage-point change from baseline, so if larger models refuse at a higher baseline rate, a fixed raw change in refusal probability will appear as a smaller percentage-point change. This could make the observed decline in effectiveness with model size partly an artifact of the metric. The paper needs to report baseline accuracies, test-set sizes, and per-layer values with variance or confidence intervals before the size-dependence claims in Section 4 can be accepted.","section":"Section 3.2, Evaluation"},{"comment":"The observation that positive and negative steering peak at the same layer as model size grows (7B: 2-layer difference, 13B: 1-layer, 70B: 0-layer) is based on three unquantified comparisons, and the three models have different total layer counts (32, 40, and 80 for Llama-2-7B, 13B, and 70B). A raw difference in layer indices is not directly comparable across models of different depths. Without normalizing by total layer count, reporting uncertainty, or demonstrating robustness to the layer-at-which-steering-is-applied discretization, this three-point pattern is too weak to support the superposition interpretation offered at the end of Section 4.","section":"Section 4, peak-layer convergence observation"}],"minor_comments":[{"comment":"The manuscript contains repeated typos and grammatical issues, including 'are are needed' in Section 4 and 'technical developed' in Section 7; these should be corrected in a revision.","section":"General, typographical"},{"comment":"The text refers to a dataset 'within answer matching behavior (See appendix)', but no appendix appears in the manuscript; either include the appendix or remove the pointer.","section":"Section 3.1, 'See appendix'"},{"comment":"The reference is incomplete: the venue is listed as 'N/A', and the title is given as 'Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet'; full publication details should be supplied.","section":"References, Templeton & Conerly 2024"},{"comment":"The caption reads 'Comparison of LLaMA models' while the text uses 'Llama'; capitalization should be made consistent throughout.","section":"Figure 3 caption"},{"comment":"The header includes 'Accepted by NewInML workshop @ ICML 2025' but also carries the PMLR 2024 proceedings boilerplate; this inconsistency should be resolved, as it affects citation metadata.","section":"Header, venue information"},{"comment":"The method section states that residual stream vectors are cached for the last token at every layer, while the steering vector is added to every token position; it should be clarified whether the cached last-token vectors are used only to compute the steering direction, since this affects reproducibility.","section":"Section 3.1, injection details"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a short workshop-style paper with an interesting qualitative question, but the headline 'scaling law' is essentially unidentifiable from three data points and the reported evaluation details. I recommend major revision rather than rejection because the qualitative findings (early-mid layer peaks, negative steering stronger than positive, decline with scale) are plausible and the methodology can be improved within the scope of the paper by adding baselines, error bars, per-layer data, and a more honest interpretation of the exponential fit. The reference formatting and venue inconsistency should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a workshop-quality empirical note with one genuinely useful observation and one overreaching curve fit. The useful part is the clean comparison of CAA across Llama 2 7B, 13B, and 70B on refusal behavior: peak steering effectiveness lands at about 0.4x total layers, negative steering is stronger than positive, and the layer gap between positive and negative peaks narrows from 2 to 0 as the model grows. Those are real measurements that extend Rimsky et al., even if they are not shocking.\n\nThe overreach is the 'scaling law' in Section 4: y = 0.081 + 2.4 * exp(-0.42x), fit to exactly three points. Three free parameters, three points, no error bars, no baseline accuracy, no number of test prompts, no variance. The stress-test note is right that a linear fit would be statistically indistinguishable from this exponential. The paper itself supplies no way to check whether the decline is in steering power or in the metric: they report percentage-point change in correct refusal answers, and if larger models start with a higher refusal baseline, the same raw effect looks smaller in percentage points.\n\nThe paper does honestly flag some limitations: the asymmetry hypothesis is explicitly tentative, and they note the refusal-only scope. But the abstract and title sell the exponential relationship as a finding without those caveats. Also, the text promises an appendix that does not appear here, and no code or data is provided, so the numbers cannot be checked.\n\nWhat the paper does well: the methodology follows the established CAA recipe clearly, the layer sweep is straightforwardly described, and the qualitative findings—diminishing returns with scale, stronger negative steering, mid-layer peaks—are consistent with adjacent work and worth putting on the record. The writing is honest about being limited to one behavior.\n\nMinor but real: the LaTeX template keeps an ICML 2024 proceedings footer while claiming NewInML workshop acceptance, and the Templeton & Conerly reference is incomplete. Sloppy, not scientific.\n\nMy judgment: the central quantitative claim is not supported by the evidence reported. But the qualitative observations are plausible and could be made solid with the full per-layer effectiveness table, baselines, prompt counts, and variance. I would send this to a serious referee, asking the authors to either supply those numbers or reframe the exponential as a purely descriptive fit over three points. As it stands, cite it as exploratory, not as a law.\n\nRecommendation: accept for peer review with a clear expectation of heavy revision; this is a ten-minute read that raises a real question about how evaluation metrics interact with model scale.","headline":"A plausible qualitative result on CAA scaling across Llama 2 sizes, wrapped in an unsupported three-point exponential 'law'.","tokens_in":5245,"tokens_out":2614,"would_cite":false,"duration_ms":32184,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that contrastive activation addition, a training-free steering method, loses effectiveness exponentially as LLM parameter count grows, reaching an 8 percent floor at 70B.","keywords":["activation steering","contrastive activation addition","scaling laws","Llama 2","refusal behavior","residual stream","model alignment","mechanistic interpretability"],"falsifier":"Run the same CAA protocol over Llama 2 7B, 13B, and 70B on a fixed refusal test set while reporting baseline accuracy, prompt count, per-layer confidence intervals, and several steering coefficients; if the per-layer curves overlap once baselines are normalized, the exponential decay claim is falsified.","tokens_in":4241,"feed_emoji":"📉","tokens_out":9125,"duration_ms":99029,"temperature":0.7,"pith_summary":"This paper tries to establish a scaling law for contrastive activation addition (CAA), a training-free way to steer an LLM by computing a direction vector from contrasting examples and adding it to the model's residual stream during inference. Working with Llama 2 models of 7B, 13B, and 70B parameters and refusal behavior as the target, the authors report that peak steering effectiveness decays exponentially with parameter count, fitting $y = 0.081 + 2.4 e^{-0.42x}$. They also report that the best injection layer sits in the early-mid layers near $0.4$ of total layer count, that negative steering (pushing the model away from refusal) is stronger than positive steering (reinforcing refusal) at every size, and that the peak layers for positive and negative steering converge as models grow. If the law holds, activation steering will require either stronger vectors, multi-layer injection, or per-scale calibration to stay useful as models scale.","feed_headline":"Activation steering decays exponentially with model size","feed_subtitle":"Peak contrastive steering on Llama 2 drops by more than half from 7B to 13B and nears 8 percent at 70B.","key_machinery":"The machinery is contrastive activation addition (CAA) combined with a layer sweep. For each layer, the paper caches residual-stream vectors of the last token on refusal and non-refusal prompts, subtracts the pairs to get per-layer direction vectors, averages them, normalizes to the average direction norm, and adds the resulting vector to the residual stream, the internal activation pathway running through all transformer layers, just before the layer's add-and-normalize step. Sweeping the injection layer across all layers and measuring the change in percentage of correct refusal answers produces the effectiveness curves; an exponential fit to the peak values is the scaling law the paper proposes.","core_discovery":"The paper's central discovery is that contrastive activation addition scales poorly: on the refusal test, peak effectiveness $y$ falls as $y = 0.081 + 2.4 e^{-0.42x}$, with $x$ the parameter count, so the measured effect at 13B is already less than half of that at 7B and approaches a floor near $0.081$ at 70B. The authors take this as evidence that larger models 'drown out' a single injected steering vector through more downstream computation. They further find that effectiveness peaks in early-mid layers at roughly $0.4$ times the layer count; that steering in the negative direction, toward non-refusal, has larger effects than steering in the positive direction, toward refusal, across all sizes; and that the peak layer for positive and negative steering converges as scale increases, from a two-layer gap at 7B to a zero-layer gap at 70B.","pith_inferences":["Editorial extension: if the decay reflects the model drowning out injected vectors in downstream computation, then repeated injection at multiple layers or scaled vector magnitudes, suggested but not tested in the paper, should restore effectiveness.","Editorial extension: the convergence of positive and negative peak layers is consistent with the paper's superposition explanation, but a saturation-in-evaluation alternative remains untested; varying the steering coefficient at each size would separate the two.","Editorial extension: the RLHF asymmetry argument predicts that for other RLHF-trained behaviors, the direction opposing the training signal will show larger steering effects than the direction reinforcing it, a claim a broader behavior suite could test."],"forward_implications":["At 70B parameters, peak CAA effectiveness on refusal approaches roughly 8 percent, so steering with the same vector becomes nearly inert without adjustments.","The optimal injection point sits around 40 percent of the layers, letting practitioners target early-mid layers for maximum effect.","Steering the model away from refusal is consistently stronger than steering it toward refusal, which means weakening safety behaviors is easier than reinforcing them.","As model size grows, positive and negative steering peak in the same layer (7B: two layers apart, 13B: one, 70B: zero), so one layer can serve both directions.","If the fit extrapolates, larger future models would need exponentially stronger or differently targeted steering vectors to achieve the same behavioral shift."],"supporting_citations":[{"why":"Defines contrastive activation addition and the refusal dataset/evaluation protocol that this paper scales from 7B to 70B.","marker":"Rimsky et al., 2023"},{"why":"Supplies the general activation-addition technique of adding a cached direction vector to the residual stream, plus the multi-layer injection ideas referenced in the extensions.","marker":"Turner et al., 2023"},{"why":"Provides the mid-layer feature-steering evidence the paper cites alongside its own observation that peak steering layers cluster in early-mid layers.","marker":"Templeton & Conerly, 2024"},{"why":"Supplies the superposition hypothesis used to explain why positive and negative steering peak layers converge as scale increases.","marker":"Elhage et al., 2022"}],"fun_headline_variants":["Activation steering effectiveness drops exponentially with model size","Bigger Llama 2 models resist activation steering more","Contrastive steering hits a scaling wall as models grow","Steering signals fade in larger language models","Llama 2 scaling limits for activation steering effectiveness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The metric this law depends on, the percentage of correct refusal answers before and after steering, is assumed to be comparable and roughly linear across models of different size; the paper reports no baseline accuracy, number of test prompts, or variance, so lower measured changes could reflect saturation or different baselines rather than weaker steering.","fun_headline_variants_meta":{"raw":{"variants":["Activation steering effectiveness drops exponentially with model size","Bigger Llama 2 models resist activation steering more","Contrastive steering hits a scaling wall as models grow","Steering signals fade in larger language models","Llama 2 scaling limits for activation steering effectiveness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1287,"prompt_tokens":915,"completion_tokens":372,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":297}},"tokens_in":531,"tokens_out":372,"duration_ms":5415,"temperature":1.0,"reasoning_tokens":297,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:01:15.351575+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same CAA protocol over Llama 2 7B, 13B, and 70B on a fixed refusal test set while reporting baseline accuracy, prompt count, per-layer confidence intervals, and several steering coefficients; if the per-layer curves overlap once baselines are normalized, the exponential decay claim is falsified.","supporting_citations":[{"cited_title":"Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet","cited_arxiv_id":null,"evidence_quote":"Provides the mid-layer feature-steering evidence the paper cites alongside its own observation that peak steering layers cluster in early-mid layers."}],"review_version":1}