{"id":"cb9bd83c-a945-4a00-8eb1-8c25d80d1037","arxiv_id":"2607.19364","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A three-statistic Borda consensus over sparse-autoencoder features produces interpretable activation steering, but usable quality-preserving shifts are rare and highly localized.","lead":"This paper tests a transparent way to steer large language models by selecting a small set of interpretable 'sparse features' with three statistical tests and injecting a weighted direction during generation. It finds measurable behavioral shifts across Gemma models, but shows that quality-preserving control is rare and depends heavily on the model, layer, and steering strength.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Selection statistics use max-pooled activations over the full contrastive text, but steering is injected only at the last token; the paper never validates that selected features are separable or even active at the injection site, so the causal-handle interpretation is unsupported.","rationale":"The reader's weakest assumption is that association-ranked features are the right causal handles to inject at the last token. My concern sharpens this: the selection statistics are computed on max-pooled activations over the whole text, whereas the injection is at the last token only. This is a necessary condition for the causal-handle claim, and it is internally checkable without new model runs. It does not refute the paper's empirical shift results—those are supported by a large sweep and transparent reporting—but it does undermine the mechanistic interpretation and the 'sparse-feature intervention' framing. The paper's own Limitations already concede that causal validation is statistical, not mechanistic, which is consistent with a CONDITIONAL verdict. The concrete test I propose can be run with the authors' released code and data, and its outcome would either strengthen or weaken the central interpretability claim. I therefore leave the reader's verdict unchanged: the empirical findings are credible as descriptive, but the feature-level causal story requires the proposed validation before full acceptance.","tokens_in":26176,"tokens_out":7122,"duration_ms":85825,"concrete_test":"Using the released pipeline, recompute the three selection statistics for each (model, domain, layer) using only the last-token activations of the 800 contrastive pairs—i.e., the activations at the position where the hook applies—instead of max-pooled activations. Compare the resulting top-K feature sets (K=16,24,32) to the paper's max-pool-selected sets via overlap and rank correlation (Spearman). Then, for the headline configuration (Gemma 2 9B LOGIC L19 α=0.1), run activation patching on the 100 held-out prompts: perturb only the features that are both in the max-pool-selected set and have positive last-token Cohen's d, using the paper's d-weighted direction, and measure the resulting primary-score delta. If the top-K overlap is below ~50% or the patched direction's delta is substantially smaller than the full-vector delta (e.g., <50% of +1.16), the association-ranked features are not","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's feature-selection statistics (F-test, KSG MI, Cohen's d) are computed on max-pooled log1p activations over the entire contrastive text (§2.3), while the steering vector is injected only at the last-token position (§2.6). The load-bearing step is the implicit transfer: a feature that separates classes when max-pooled over the sequence must be activatable at the last-token site where the decoder-row direction is added. The paper never validates this transfer. If a selected feature's class-discriminative signal is concentrated in an earlier content token—e.g., 'movie theater' or 'renewable energy'—its decoder direction may be ineffective at the last token, or may activate a different set of features than the one audited via Neuronpedia (Table 10). Since the intervention is described as 'activating these features at inference,' the absence of last-token separability evidence breaks the link between the audited selection and the actual perturbation. This is more concrete than the general causal-gap caveat: it concerns the internal consistency of the selection site and the injection site, and it is checkable from the authors' own contrastive data. The paper's Limitations acknowledge that association does not imply causation, but they do not address this specific site-mismatch, which is a necessary condition for the causal-handle claim to even get off the ground.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a transparent SAE-based activation-steering pipeline: a six-condition reliability filter, a three-statistic Borda consensus (F-test, KSG mutual information, Cohen's d) for feature ranking, and a Cohen's-d-weighted combination of SAE decoder rows as the steering direction. The method is evaluated across three Gemma models, four behavioral domains, and 356 layer–strength configurations, with a three-judge scoring protocol and human adjudication. The authors report that the method produces measurable domain-specific shifts, that the strongest logical-correctness configuration reaches Δp=+1.16 in Gemma 2 9B, and — more importantly — that raw attribute movement substantially overstates quality-preserving control. They also report that no single configuration passes a strict FDR-corrected joint significance test, and they frame strong configurations as Pareto-efficient candidates with bootstrap support. The paper is explicitly positioned as a transparent, optimization-free baseline rather than a claim of state-of-the-art steering.","tokens_in":26446,"tokens_out":4658,"duration_ms":56366,"significance":"If the findings hold, the paper makes a useful empirical and methodological contribution: it provides a fully transparent, optimization-free SAE-feature steering recipe, and it documents a large configuration sweep showing that steering success is highly localized by model, domain, layer, and strength. The paper also makes a valuable negative point — raw primary-score deltas are not a reliable proxy for usable, quality-preserving control. The release of code and data, the explicit treatment of statistical assumptions, and the honest reporting of the strict-test failure are strengths that make the empirical claims easy to check and extend. However, the causal-handle interpretation is not yet fully supported because of a mismatch between the feature-selection site and the injection site.","major_comments":[{"comment":"Feature selection is computed on max-pooled log1p activations over the full contrastive text (Z_D in §2.3), while the steering vector is injected only at the last-token position (§2.6: h_{:,-1,:} ← h_{:,-1,:} + α||h_{:,-1,:}|| δ̂h). The paper's stated intervention is 'activating these features at inference,' but there is no evidence that the selected features are separable, or even active, at the injection site. If a feature's class-discriminative signal is concentrated in an earlier content token, its decoder row may have little effect at the last token, or may activate a different feature set than the one audited via Neuronpedia (Table 10). This is a necessary condition for the causal-handle interpretation, and it is checkable from the authors' own contrastive data: compute the three selection statistics at the last-token position (or at least report per-token max vs. last-token agreem","section":"§2.3 vs. §2.6"},{"comment":"The manuscript correctly reports that no configuration survives the strict joint FDR-corrected paired t / Wilcoxon / sign-test criterion, and it frames headline cells as Pareto candidates. However, the Abstract still presents '+1.16 in Gemma 2 9B' as a headline result without this caveat, and the bootstrap CI for that cell is [+0.33, +1.96] (Appendix N.5) — a wide interval. Since the per-configuration claim is not FDR-significant, the abstract and Section 4.1 should either carry the strict-test caveat explicitly or report the strongest configuration as a descriptive Pareto candidate only. This is not a request for new analysis; it is a reporting-consistency issue for the paper's most prominent number.","section":"§4.6 / Appendix N.5 / Abstract"}],"minor_comments":[{"comment":"The Fisher LDA derivation contains a duplicated paragraph: the two-sentence 'Fisher LDA maximizes J(w)...' block appears twice verbatim. Remove the duplicate.","section":"Appendix B.4"},{"comment":"The reference list appears to duplicate entries for the same work: Panickssery et al. 2023 and Rimsky et al. 2024 are the same CAA paper, and Zou et al. 2023a/2023b are the same Representation Engineering preprint. Consolidate.","section":"References"},{"comment":"The Neuronpedia audit is shown only for one domain, one model, and one layer (Gemma 2 2B, LOGIC, layer 12). A representative audit for at least one additional model/domain, or an explicit statement that the audit is illustrative only, would strengthen the interpretability claim.","section":"Table 10"},{"comment":"The F-statistic formula in Appendix M is visually garbled by line breaks; the numerator and denominator should be typeset clearly. Also, the text says 'F-statistic and Cohen's d are mean-based' but Cohen's d is a standardized mean difference; consider phrasing.","section":"Appendix M / Eq. (1)"},{"comment":"The six-condition quality filter includes the OR-combined FDR rule (C6) alongside per-statistic FDR control. The text explains this is intentionally not a global FDR guarantee, but the caption of Table 12 says 'Direction significance test' for Cohen's d via Welch's t; clarify that the bootstrap lower-bound condition (C3), not Welch's t, is the directional gate.","section":"§2.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest and empirically rich, and the broad sweep is a useful contribution. The main technical gap is the selection-site/injection-site mismatch: all feature statistics are computed on max-pooled sequence activations, but the intervention is applied only at the last token. This is not a circularity problem, but it is a correctness-risk problem for the causal-handle reading, and it can be addressed with additional validation from the authors' own data. If the authors add last-token validation and adjust the abstract's headline claim to match the strict-test caveat, I would be supportive of publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the paper's central empirical claim — that raw attribute movement overstates usable steering once generation quality is required — is credible and worth taking seriously. Second, the load-bearing step has a site-mismatch problem: features are selected on max-pooled activations over the full contrastive text, but the intervention is injected only at the last token, and the paper never checks that the selected features are separable or even active at the injection site. That is more concrete than the general causal-gap caveat the Limitations section admits, and it breaks the link between the audited selection and the actual perturbation.\n\nWhat is actually new: a three-statistic Borda consensus (F-test, KSG MI, Cohen's d) followed by a Cohen's-d-weighted decoder-row direction, with no learned aggregation weights. The 344-configuration sweep across three Gemma models and four domains is a real empirical effort, and the quality-conditioned evaluation is a genuine methodological contribution: the raw-versus-clean gap is large and consistent, and the field would do well to adopt that reporting norm. The paper is unusually transparent — aggregate Spearman correlations, bootstrap CIs, Pareto frontiers, and an explicit admission that no single configuration passes a strict FDR-corrected joint test. That honesty is not performance; it is how the work is actually framed.\n\nSoft spots, in proportion. First, the site mismatch I opened with is the main one. Max-pooling over the full sequence filters for features that separate classes somewhere in the text; if the discriminative signal lives in an earlier content token, the decoder-row direction may do little at the last token, or activate a different set of features than the ones audited via Neuronpedia. This is checkable from the authors' own contrastive data, and they should report it. Second, the claim that 'Cohen's d reaches the optimization ceiling' is asserted without reporting the 600-step comparison it depends on — a paragraph of results would settle it. Third, the multi-layer clean-success gain is selected using evaluation outcomes from the same 100 held-out prompts used to report the gain, which is a mild selection-on-test-set effect; the +7pp improvement should be read with that caveat in mind.\n\nNone of these sink the paper. The aggregate structure (primary delta strongly correlated with quality delta; near-zero depth and alpha correlations) is credible and the central descriptive result holds. This paper is for anyone building or evaluating steering methods, and it deserves a serious referee. I would send it to review, asking for the site-mismatch check and the optimization comparison to be addressed.","headline":"Honest, useful SAE-steering paper whose main claim (raw shift overstates usable steering) holds up, but the selection-to-injection site mismatch is a real gap that needs addressing before the interpretability story lands.","tokens_in":27054,"tokens_out":2267,"would_cite":true,"duration_ms":25995,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Consensus-ranking sparse features by three classical statistics and weighting them by Cohen's d steers LLM behavior measurably — but control is sharply localized, and raw shifts overstate quality-preserving success.","keywords":["activation steering","sparse autoencoders","feature ranking","Borda consensus","Cohen's d","mutual information","quality-conditioned evaluation","language model interpretability"],"falsifier":"Take the LOGIC-domain steering in Gemma 2 9B at layer 19 (the +1.16 setting) and ablate the selected features one by one at inference — zero out each top-ranked SAE feature's activation, or patch in activations from an unsteered forward pass — and measure whether the primary-score delta collapses. If removing the single highest-weight feature leaves the shift essentially intact, the statistical ranking is selecting correlates, not causes. A complementary control is a label-permutation test: if running the full filter-and-consensus pipeline on shuffled contrast labels still yields 'steering' wi","tokens_in":26011,"feed_emoji":"🧭","tokens_out":14622,"duration_ms":140221,"temperature":0.7,"pith_summary":"This paper tries to establish that transparent, optimization-free activation steering can work: select sparse autoencoder (SAE) features — the sparse, individually interpretable directions into which residual-stream activations decompose — by a consensus of three classical statistics (F-test, mutual information, Cohen's d), weight the steering vector by Cohen's d, and inject it at the last token. Swept across three Gemma-family models, four behavioral domains, and 356 layer–strength configurations, the method shifts target attributes measurably, with a peak +1.16 primary-score gain on logical correctness in the largest model. The broader finding is about how steering should be measured: raw attribute movement overstates usable control, since plain success rates exceed quality-preserving 'clean success' by up to 28 percentage points, and usable steering is sharply localized by model, domain, layer, and strength. A sympathetic reader would care because this supplies an auditable, statistics-only baseline for a fragmented technique, and because it argues that any steering evaluation — including those from optimization-based methods — should report quality-conditioned success alongside raw shift.","feed_headline":"+1.16 logic shift from statistics-only LLM steering","feed_subtitle":"A transparent statistics-only pipeline steers LLM output, yet raw gains overstate usable control by up to 28 points.","key_machinery":"The machinery is the consensus-ranked feature direction, built in three steps: a six-condition quality filter on log1p-transformed SAE activations (activity ≥1%, positive Cohen's d with bootstrap lower bound above zero, |d| ≥ 0.2, MI bootstrap CV < 0.5, FDR q < 0.05 on at least one test); an unweighted two-tier Borda consensus over three complementary statistics — F-test (linear separation), KSG mutual information (nonlinear/threshold dependence), Cohen's d (signed, variance-standardized separation) — with no learned weights; and a steering vector equal to the Cohen's-d-weighted sum of selected decoder rows, normalized and injected at the last token with activation-norm scaling α. Cohen's d","core_discovery":"The paper's central claim: a fully auditable, optimization-free sparse-feature steering pipeline works. A six-condition filter prunes the 16,384-feature space; survivors are ranked by F-test, KSG mutual information, and Cohen's d, merged by an unweighted two-tier Borda consensus; the steering direction is the Cohen's-d-weighted sum of selected decoder rows, motivated by Fisher-LDA under approximate feature decorrelation. The method shifts target attributes measurably — logical correctness reaches +1.16 primary delta in Gemma 2 9B — but the broader finding is the raw-versus-clean gap: raw win rate overstates quality-preserving clean success by 4.1–28 percentage points. Steering is model- and","pith_inferences":["One testable extension the authors leave implicit: because selection is purely associational, the method's interpretability — feature labels cluster on conditional-logic, implication, and causation directions for LOGIC — could reflect surface markers of the contrast corpus rather than mechanisms upstream of the behavior; per-feature ablation or activation patching would separate the two.","A transfer prediction: the paper's near-zero depth–effect correlation across the sweep suggests that no canonical 'steering layer' exists for other model families either; a matching layer sweep on a non-Gemma model with a public sparse autoencoder would confirm whether the localization is architectural or an artifact of Gemma's training.","A safety artifact that follows from the paper's own dual-use warning: since flipping the sign of d steers toward the anti-target, one could build a steering-audit tool that prints the top contributing features and their human-readable labels for any deployed vector, making inversion attempts detectable.","A compositional hypothesis: if cumulative perturbation budget, not raw shift, governs reliability, then multi-attribute steering — which the paper leaves open — could inherit the multi-layer budget result, so additive α-budgeting across attributes with clean success as the objective is the natural next experiment rather than naive vector addition."],"forward_implications":["Steering evaluations should report quality-conditioned success (clean success) alongside raw shift; raw primary-win rates overstate usable control by up to 28 percentage points, so headline deltas alone are misleading.","The optimal intervention layer is not universal: no monotone relation exists between layer depth and effect (Spearman ≈ 0 across 344 configurations), so layer must be treated as a per-domain, per-model hyperparameter.","Steering strength interacts non-monotonically with effect — the largest logic shift occurs at the smallest α (0.1) — and large α pushes activations out of distribution, producing repetition and incoherence that further steering cannot recover.","Multi-layer, budgeted composition should be judged on clean-success reliability rather than raw shift: on the strongest domain it lifts clean success from 24% to 31% while dampening the raw delta, and on 9 of 12 cells it dilutes the best single-layer shift.","In the tested settings, 600 gradient steps of a supervised steering objective from the Cohen's-d initialization reach similar projection behavior, indicating the optimization-free direction sits near the practical ceiling; learned steering objectives add validation burden without evident gain here."],"fun_headline_variants":["Statistics-only steering hits +1.16 logic gains in LLMs","Transparent SAE steering: raw gains overstate clean success","LLM steering: +1.16 logic, but raw vs clean gap up to 28 points","No fine-tuning, just statistics: +1.16 logic shift"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that features ranked by statistical association on a contrastive selection set are the right causal handles to inject at the last token at inference — the paper's own limitations state that the causal validation is statistical, not mechanistic, and that associational ranking alone does not establish that activating these features causes the observed shift; if the statistics select features correlated with domain vocabulary or style rather than feat","fun_headline_variants_meta":{"raw":{"variants":["Statistics-only steering hits +1.16 logic gains in LLMs","Transparent SAE steering: raw gains overstate clean success","LLM steering: +1.16 logic, but raw vs clean gap up to 28 points","No fine-tuning, just statistics: +1.16 logic shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000926,"raw_usage":{"total_tokens":3829,"prompt_tokens":793,"completion_tokens":3036,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":2955}},"tokens_in":537,"tokens_out":3036,"duration_ms":22304,"temperature":1.0,"reasoning_tokens":2955,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T12:08:09.916978+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the LOGIC-domain steering in Gemma 2 9B at layer 19 (the +1.16 setting) and ablate the selected features one by one at inference — zero out each top-ranked SAE feature's activation, or patch in activations from an unsteered forward pass — and measure whether the primary-score delta collapses. If removing the single highest-weight feature leaves the shift essentially intact, the statistical ranking is selecting correlates, not causes. A complementary control is a label-permutation test: if running the full filter-and-consensus pipeline on shuffled contrast labels still yields 'steering' wi","supporting_citations":[],"review_version":1}