{"id":"20824483-27fe-43e3-8386-fd794360644d","arxiv_id":"2505.20322","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"STA selects sparse autoencoder features by activation amplitude and frequency to build steering vectors that improve LLM safety control over prompt engineering and standard steering.","lead":"Researchers propose Steering Target Atoms (STA), a way to control LLM behavior by picking out sparse, interpretable atoms in the model's internal representations and adding a steering vector built from them. The method improves safety-focused steering on Gemma and Llama models, but the paper's evaluation would need a clear train/test split and error bars before the gains are convincing.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never states that the queries used to build steering vectors (Eqs. 7–11) are disjoint from the queries scored in Table 1, so the headline gains may be partly in-sample; an explicit split and re-run is needed.","rationale":"I read the paper in good faith: the STA construction is plausible, the code is released, and the baselines include CAA and an SAE-space baseline, so the comparison is meaningful if the evaluation is clean. The single most load-bearing concern is the missing stated split between the N queries used to compute delta-a and delta-f (Eqs. 7–11) and the queries used to compute the defense rates in Table 1. This concern attacks the external validity of the central claim directly: if the same prompts are used for construction and evaluation, the reported improvements are partly memorization or in-sample fitting, not generalization to held-out behavior. Fig. 3 makes the concern concrete by showing defense rates as a function of construction-set size up to 4096 on the same benchmarks, which is only informative if evaluation queries are disjoint. The manuscript nowhere states such a split, nor does it provide error bars or repeated-seed variance to bound the effect. This is an addressable methodological gap, not evidence of misconduct, so the appropriate response is a conditional hold on the quantitative claim until the split is clarified or the experiment is re-run with a held-out evaluation set. The reader's weakest_assumption identifies exactly this concern, and I agree with it. I therefore recommend no change to the reader's CONDITIONAL verdict: the concern is real and should be resolved, but it does not warrant rejection if the authors can supply the missing split information or show that re-running with a disjoint split preserves the headline margins.","tokens_in":21157,"tokens_out":4642,"duration_ms":38379,"concrete_test":"Inspect the released repository for the data-split logic behind §4.2 and Fig. 3. If the construction queries and evaluation queries are not explicitly disjoint, re-run Table 1 with an 80/20 random split (construction/evaluation) on SafeEdit and RealToxicPrompts, and also re-run Fig. 3 at N=4, 32, 512, and 4096 on the held-out half. If the STA average defense rate or its margin over CAA/SAEAXBENCH drops materially, the headline gain is partly in-sample.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is the Table 1 defense-rate improvement. In §3.1 the steering vector is built from N queries by averaging positive-minus-negative SAE activations (Eqs. 7–10) and thresholding atoms (Eq. 11), but §4.1 gives no statement that the evaluation queries for SafeEdit and RealToxicPrompts are disjoint from this construction set. Fig. 3 aggravates the issue: it varies N from 4 to 4096 and reports defense rates on the same benchmarks, so the overlap status is load-bearing. If the same prompts appear in both building and testing, the reported rates are partly in-sample fits rather than held-out behavior. The Limitations section discusses SAE baselines and model coverage but is silent on evaluation/construction overlap. The paper also does not report error bars or repeated-seed variance, which would help bound this effect. The omission is concrete and easy to fix, not a fundamental flaw in the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Steering Target Atoms (STA), a method that builds steering vectors by selecting sparse-autoencoder (SAE) features whose activation amplitude and frequency differ between positive (safe) and negative (unsafe) examples, then adding the selected atoms' decoder contributions to the residual stream. Experiments on SafeEdit and RealToxicPrompts across Gemma-2-9B-pt, Gemma-2-9B-it, and Llama-3.1-8B report higher average detoxification than Prompthand, Promptauto, CAA, and SAEAXBENCH, with relatively small changes in MMLU/GSM8K. The paper also analyzes layer choice, data-size dependence, robustness of steering versus prompting, control boundaries, and an application to controlling chain-of-thought length in DeepSeek-R1-Distill-Qwen-7B.","tokens_in":21361,"tokens_out":4207,"duration_ms":34420,"significance":"If the reported results hold on held-out data, STA would be a useful, lightweight, inference-time safety intervention that leverages SAE interpretability while avoiding full-model fine-tuning. The paper's strengths include reproducible code, use of publicly available SAEs, ablations of the amplitude and frequency components, direct conversion of prompts into steering vectors for a fairer comparison, and a novel reasoning-length control experiment. However, the quantitative claims are currently weakened by the absence of an explicit train/test split for steering-vector construction, the lack of error bars or multiple-seed variance, and the per-experiment selection of thresholds; the small margins over SAEAXBENCH (e.g., 72.23 vs 71.36 on Llama-3.1-8B) are therefore difficult to interpret as robust improvements.","major_comments":[{"comment":"The manuscript never states that the N queries used to compute Δa and Δf in Eqs. 7-11 are disjoint from the queries scored in Table 1. Since Fig. 3 varies N from 4 to 4096 while reporting defense rates on the same SafeEdit and RealToxicPrompts benchmarks, the overlap status is load-bearing. If the same prompts are used for both constructing and testing the steering vector, the reported improvements are at least partly in-sample fits rather than predictions. Please specify the construction set explicitly and re-run the comparison with a disjoint held-out evaluation set, or provide evidence that overlap is impossible by construction.","section":"3.1, 4.1, Table 1, Fig. 3"},{"comment":"The central claim of best average detoxification rests on differences that are small in some cases (STA 72.23 vs SAEAXBENCH 71.36 on Llama-3.1-8B; STA 83.45 vs 80.98 on Gemma-2-9B-pt) and no error bars, confidence intervals, or multiple-seed results are reported. Without a variance estimate, the reader cannot determine whether the improvement is statistically meaningful, especially given the construction-set overlap concern. Please report repeated runs (e.g., different random subsets for construction) with standard deviations, or at least per-category breakdowns that show the stability of the reported averages.","section":"Table 1"},{"comment":"The method relies on thresholds α and β, the multiplier λ, and the steering layer; the paper selects α/β at the top 35% position for Table 1 and top 4% for Table 5, and λ defaults to 1. The threshold choice is thus tuned per experiment, so the reported defense rates are not parameter-free evaluations. A sensitivity analysis across α, β, λ, and steering layers is needed to show that the method's advantage is not an artifact of tuning these thresholds on the evaluation benchmarks.","section":"4.1, Table 5"},{"comment":"The conclusion states that STA enhances 'both safety and personality alignment,' but the Limitations section explicitly says that STA 'performs well in the safety domain but shows limited effectiveness in the personality domain (see §D).' This internal inconsistency should be resolved; if the personality result is weak, the conclusion should be softened or the evidence strengthened.","section":"Limitations, Conclusion"}],"minor_comments":[{"comment":"The baseline name 'SAEAXBEN CH' appears with an unwanted space in several places; it should be 'SAEAXBENCH' or 'SAE-AXBENCH' consistently.","section":"4.1, Table 1"},{"comment":"The caption contains the typo 'unsing' for 'using' in both subfigures.","section":"5.2, Fig. 11 caption"},{"comment":"The text includes 'safety domian' instead of 'safety domain'; please proofread the section headings and captions.","section":"5.2"},{"comment":"The Limitations section says experiments are 'conducted exclusively on the Gemma-2-9B-pt, Gemma-2-9B-it models' but the paper also evaluates Llama-3.1-8B; this phrasing is contradictory and should be corrected.","section":"Limitations"},{"comment":"The text in B.1.1 says 'we do not report the performance of RefusalFeature as a baseline in the main text,' yet Table 2 reports a comparison with RefusalFeature; this should be clarified or reconciled.","section":"B.1.1, Table 2"},{"comment":"The phrase 'temperature coefficient' should be 'temperature' in standard usage, and the abbreviation 'Real' in Fig. 3 should be defined in the caption or main text even though a parenthetical note appears.","section":"F.2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is a plausible extension of CAA with SAE feature selection, and the qualitative direction of the results is encouraging. However, the missing construction/evaluation split is a serious blocker for the headline numbers, and the absence of variance estimates makes the small margins over SAEAXBENCH unconvincing as they stand. The per-experiment threshold selection compounds the concern. If the authors can provide a clean held-out evaluation and sensitivity analyses, the paper would likely be acceptable; the conclusion/limitations inconsistency is a minor but visible issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"STA is a legitimate, modest extension of SAE-based steering: the amplitude-plus-frequency atom selection rule in Eq. 11 goes beyond AXBENCH's coarse SAE directions, and applying this to open-ended safety generation on SafeEdit and RealToxicPrompts is a real step past toy tasks. The prompt-to-steering conversion in Section 5 is a nice idea for fair comparison, and the robustness analysis (attention scores, prompt position sensitivity) is genuinely informative. Credit also goes to the Limitations section, which candidly admits STA underperforms in the personality domain and that SAE baselines are competitive.\n\nThe soft spots, in order of importance:\n1. The data-split issue is real and load-bearing. The paper never states that the evaluation queries are disjoint from the N queries used to construct the steering vector in Eqs. 7-11. Fig. 3 aggravates this by varying N from 4 to 4096 while reporting defense rates on the same benchmarks. If those are the same prompts, Table 1's gains are partly in-sample fits. This is concrete and easy to fix: state the split, or re-run with held-out prompts. The Limitations section is silent on the matter.\n2. No error bars or multiple seeds. The margins over SAEAXBENCH are a few points (e.g., 72.23 vs 71.36 on Llama-3.1-8B), so variance could matter.\n3. Alpha and beta thresholds are chosen per experiment (top 35% for Table 1, top 4% for Table 5). Without a sensitivity analysis, the method has two free knobs tuned to the benchmark.\n4. The reasoning-length section uses CAA, not STA, because no SAE is available for DeepSeek-R1-Distill-Qwen-7B, yet the abstract and conclusion imply STA controls reasoning. That overstates the result.\n\nOn balance, the central argument holds up qualitatively: the method is plausible, the benchmarks are standard, and the comparison framework is useful. But the missing split is a serious gap for the quantitative claims, and the threshold selection needs justification.\n\nThis paper is for people working on SAE-based steering and inference-time safety. It deserves a serious referee - the issues are addressable and the framework is worth engaging. I'd send it to review, but the revision must include a held-out split, variance reporting, threshold sensitivity, and toned-down reasoning claims.","headline":"A plausible, incremental advance in SAE steering that deserves review, but the missing train/eval split makes the headline numbers under-supported until fixed.","tokens_in":657,"tokens_out":1630,"would_cite":false,"duration_ms":31734,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Selecting sparse-autoencoder features by activation amplitude and frequency yields steering vectors that control LLM behavior better than prompt engineering, lifting average detoxification to 83.45%, 97.56%, and 72.23% across three test…","keywords":["sparse autoencoders","steering vectors","LLM safety","detoxification","activation engineering","prompt engineering","chain-of-thought control","behavior control"],"falsifier":"Rebuild the STA steering vector on one half of the safety queries and measure defense rates on the held-out half; if the STA margin over CAA shrinks or disappears on the held-out split, the claimed advantage is an artifact of in-sample evaluation.","tokens_in":20975,"feed_emoji":"🛡️","tokens_out":10789,"duration_ms":84645,"temperature":0.7,"pith_summary":"This paper tries to show that a language model's behavior can be controlled by editing a small set of interpretable \"atoms\" inside its own representations, rather than by writing prompts. The atoms come from a sparse autoencoder, which decomposes the model's hidden states into sparse, higher-dimensional features. The authors select atoms whose activation amplitude and activation frequency both shift between safe and unsafe responses, then add the selected atoms' combined direction back to the hidden state during inference. Across three models and two toxic-content benchmarks, this raises average defense rates from roughly 60% to 83%, from 84% to 98%, and from 59% to 72%, with only minor changes on general-knowledge benchmarks. If this is right, steering by selected atoms is a more reliable and flexible control method than prompting, and it can even adjust the length of a reasoning model's chain of thought.","feed_headline":"Atom-level steering lifts LLM safety to 97.6%","feed_subtitle":"Picking sparse features by strength and frequency beats prompt engineering, with little cost to general skills.","key_machinery":"The key machinery is the STA atom-selection rule, which keeps an SAE atom only when both its activation-amplitude difference $\\Delta a_j$ and its activation-frequency difference $\\Delta f_j$ pass thresholds $\\alpha$ and $\\beta$. The selected atoms are then mapped back to the model's representation space through the SAE decoder, producing a single steering vector $v_{\\mathrm{STA}}$ that is added to the hidden state with a multiplier $\\lambda$. Amplitude locates the strongest directions, frequency filters out atoms that fire only on a few examples, and the thresholds control how fine-grained the intervention is.","core_discovery":"The central claim is that the entangled steering vectors used by methods like contrastive activation addition can be decomposed by a sparse autoencoder, and that a small subset of decoded atoms carries the usable control signal. STA computes two statistics over positive and negative answer pairs: the mean amplitude difference $\\Delta a_j$ and the activation-frequency difference $\\Delta f_j$. An atom is kept only when both exceed thresholds, $\\Delta a_j \\ge \\alpha$ and $\\Delta f_j \\ge \\beta$, and the kept atoms are mapped back to the residual stream through the decoder as $v_{\\mathrm{STA}} = a_{\\mathrm{target}} W_{\\mathrm{dec}} + b_{\\mathrm{dec}}$, then applied as $\\hat{h} = h + \\lambda v_{\\mathrm{STA}}$. With this recipe the paper reports average detoxification rising from 59.97% to 83.45% on Gemma-2-9B-pt, from 83.89% to 97.56% on Gemma-2-9B-it, and from 59.08% to 72.23% on Llama-3.1-8B, while general capability scores barely move. The paper further claims that steering is more robust than prompting because steering keeps attention on harmful queries across layers, and that a steering vector can even shorten or lengthen the chain of thought in a reasoning model.","pith_inferences":["Outside the paper's own experiments, the scaling curve in Fig. 3 suggests the frequency criterion pays off only once roughly 32 contrastive pairs are available; on smaller samples a simple mean-difference vector is competitive, so a practical recipe might start with CAA-style steering and switch to STA as more examples accumulate.","The prompt-to-steering translation implies a testable extension: any prompt, including a jailbreak prompt, could be compiled into a steering vector, raising the question of whether safety interventions can be inverted into attack vectors, which the paper does not discuss.","If the reported defense numbers survive a truly held-out evaluation, the attention-scoring result suggests an additional experiment: measure whether the same selected atoms shift attention in a model with a different layer count, and whether atom selection transfers across models.","The reasoning-length control in the paper uses CAA rather than STA because no public sparse autoencoder for the reasoning model is available; as such autoencoders appear, applying STA to thought atoms could give a finer control axis than the length coefficient alone."],"forward_implications":["Safety teams could replace hand-written safety prompts with a steering vector built from a small number of safe and unsafe examples, reducing sensitivity to input phrasing.","Because prompts can be converted into steering vectors and the converted vectors outscore the original prompts, prompt-based controls can be compiled into internal interventions for more stable deployment.","The multiplier $\\lambda$ acts as a calibrated dial: keeping $\\lambda \\in [0,6]$ preserves fluency while raising defense rates, and negative multipliers can stress-test safety boundaries by pushing the model toward unsafe behavior.","On reasoning models, steering a thought-pattern vector can shorten or lengthen the chain of thought on GSM8K without losing accuracy, offering a possible remedy for overthinking.","With very few contrastive examples the steering strategy still improves defense rates substantially, suggesting that data-efficient safety intervention is feasible."],"supporting_citations":[{"why":"This work supplies the public GemmaScope sparse autoencoders that define the atom space for the Gemma-2 models.","marker":"Lieberum et al., 2024a"},{"why":"This work supplies the LlamaScope sparse autoencoder that defines the atom space for Llama-3.1-8B.","marker":"He et al., 2024"},{"why":"This work defines contrastive activation addition, the steering baseline and the starting recipe that STA modifies.","marker":"Rimsky et al., 2024"},{"why":"This work provides the coarse SAE steering baseline and the auto-generated prompt baseline used in the comparisons.","marker":"Wu et al., 2025"},{"why":"This work provides SafeEdit, the safety benchmark with nine unsafe categories and 48 jailbreak attacks used to measure detoxification.","marker":"Wang et al., 2024b"},{"why":"This work provides RealToxicPrompts, the second safety benchmark used to measure toxic continuation.","marker":"Gehman et al., 2020"},{"why":"This work provides GSM8K, used to measure side effects on general math capability.","marker":"Cobbe et al., 2021"},{"why":"This work provides MMLU, used to measure side effects on broad general capability.","marker":"Hendrycks et al., 2021"},{"why":"This work provides the manually designed safe-defense prompt used as the strongest prompt-engineering baseline.","marker":"Xie et al., 2023"}],"fun_headline_variants":["Steering atoms beat prompt engineering for LLM safety","Sparse atoms steer LLMs to 97.6% detoxification","STA: Precise LLM control via atomic steering","Steering target atoms boost safety, keep skills","Atomic steering: robust LLM control without prompts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the evaluation prompts were not also used to build the steering vector; the paper never states this, and if the sets overlap, the reported gains are partly in-sample fit.","fun_headline_variants_meta":{"raw":{"variants":["Steering atoms beat prompt engineering for LLM safety","Sparse atoms steer LLMs to 97.6% detoxification","STA: Precise LLM control via atomic steering","Steering target atoms boost safety, keep skills","Atomic steering: robust LLM control without prompts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000687,"raw_usage":{"total_tokens":3133,"prompt_tokens":980,"completion_tokens":2153,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":2075}},"tokens_in":596,"tokens_out":2153,"duration_ms":14470,"temperature":1.0,"reasoning_tokens":2075,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:34:18.555842+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rebuild the STA steering vector on one half of the safety queries and measure defense rates on the held-out half; if the STA margin over CAA shrinks or disappears on the held-out split, the claimed advantage is an artifact of in-sample evaluation.","supporting_citations":[],"review_version":1}