Pith. sign in

REVIEW 2 major objections 2 minor 2 references

Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines

T0 review · 2 major / 2 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sparse Autoencoders match LoRA steering performance on AxBench when features are selected with a supervised pipeline.

desk verdict Supervised feature selection seems to drive the SAE performance gains on AxBench, so the interpretability story needs clearer separation from the supervision signal. read the letter →

arxiv 2605.31183 v1 pith:S3GOZT2S submitted 2026-05-29 cs.CL cs.AIcs.LG

Mikkel Godsk J{o}rgensen , Lars Kai Hansen This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords SparseAutoencodersLLMsteeringAxBenchbenchmarkfeatureselectionmodelinterpretabilityLoRAcausalfeatures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Earlier work found Sparse Autoencoders performed poorly for steering LLMs compared to simple baselines on the AxBench benchmark. This paper shows that SAEs reach performance close to the LoRA reference once features are chosen and labeled through the authors' supervised pipeline. The same pipeline identifies features with clear causal effects on their assigned labels even when supervision is removed from the selection step. Evidence also indicates that high sparsity levels are not required for steering success when selection relies on interpretability signals.

What carries the argument

The supervised pipeline that selects and labels SAE features by combining task supervision with interpretability signals to extract causal directions from model activations.

What would settle it

Measure AxBench steering performance after running feature selection with only the interpretability-based components of the pipeline and no supervised labeling step.

Watch

Extended reading notes

Core claim

Sparse Autoencoders can perform close to on par with the reference LoRA performance on the AxBench benchmark when features are selected and labelled with the supervised pipeline. The pipeline selects features that are surprisingly causal of their identified labels when using only its interpretability-based components. High sparsity may not be crucial for successful steering based on interpretability.

Load-bearing premise

The supervised pipeline for feature selection and labeling does not introduce task-specific bias or overfitting that inflates steering performance on AxBench.

Editorial extensions

If this is right

  • SAEs become competitive with parameter-efficient fine-tuning methods for steering once feature choice is addressed.
  • Interpretability signals alone can surface causally effective features for model interventions.
  • Steering performance does not depend on achieving the lowest possible l0 sparsity in the autoencoder.
  • Prior negative SAE steering results likely reflect suboptimal feature selection rather than a fundamental limitation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same selection approach could be tested on other steering benchmarks to check whether gains hold outside AxBench.
  • Removing the supervision component entirely might still yield usable features if interpretability metrics are strengthened.
  • Design efforts for future SAEs could de-emphasize extreme sparsity in favor of better downstream feature usability.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript claims that Sparse Autoencoders (SAEs) achieve steering performance on the AxBench benchmark close to the reference LoRA baseline when features are selected and labeled via the authors' supervised pipeline. It further claims that the interpretability-based components of this pipeline identify features that are causal for their assigned labels, and that high sparsity (low l0) is not required for effective interpretability-based steering, in contrast to prior results.

Significance. If the supervised pipeline is shown to avoid task-specific leakage from AxBench, the result would be significant for LLM interpretability: it would indicate that SAEs can recover near-optimal steering directions via feature selection without full fine-tuning, and that interpretability tools alone can surface causal directions. The sparsity finding would also revise assumptions in the SAE steering literature.

major comments (2)
  1. [supervised pipeline section] The section describing the supervised pipeline for feature selection and labeling does not explicitly demonstrate that AxBench task labels or related signals are withheld during ranking or filtering of SAE features. This is load-bearing for the central performance-parity and causality claims, because any indirect use of benchmark labels would render the results evidence of supervised selection rather than unsupervised interpretability.
  2. [interpretability components subsection] The claim that 'causality persists with only its interpretability-based components' requires an ablation in which feature selection and labeling are performed entirely without any task-derived signal; the current description leaves open whether the labeling step uses AxBench labels, which would make the causality result circular with the supervision signal.
minor comments (2)
  1. [Abstract and results tables] The abstract states performance is 'close to on par' with LoRA but supplies no numerical deltas, error bars, or dataset-split details; these should be added to the results section for reproducibility.
  2. [Related work and experimental setup] The citations to Wu et al. (2025) and Wang et al. (2025) are used to frame the contribution; ensure the bibliography contains complete, consistent entries and that the comparison baselines are reproduced exactly as described in those works.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for highlighting the critical need to rule out task-specific leakage from AxBench in our supervised pipeline. This concern directly affects the strength of our performance and causality claims. We address both major comments below and will revise the manuscript to provide the requested explicit demonstrations and ablations.

read point-by-point responses

  1. Referee: [supervised pipeline section] The section describing the supervised pipeline for feature selection and labeling does not explicitly demonstrate that AxBench task labels or related signals are withheld during ranking or filtering of SAE features. This is load-bearing for the central performance-parity and causality claims, because any indirect use of benchmark labels would render the results evidence of supervised selection rather than unsupervised interpretability.

    Authors: We agree that the current manuscript does not contain an explicit statement or diagram confirming that AxBench labels are withheld from the ranking and filtering stages. In the revision we will add a dedicated subsection (or expanded methods paragraph) that details the exact inputs to each step of the pipeline, states that no AxBench task labels or derived signals are provided to the ranking or filtering modules, and includes pseudocode showing the data flow. This will make clear that the supervision signal originates from separate, non-AxBench sources or from purely interpretability-derived statistics. revision: yes

  2. Referee: [interpretability components subsection] The claim that 'causality persists with only its interpretability-based components' requires an ablation in which feature selection and labeling are performed entirely without any task-derived signal; the current description leaves open whether the labeling step uses AxBench labels, which would make the causality result circular with the supervision signal.

    Authors: We acknowledge that the existing description of the interpretability-only ablation does not explicitly rule out any residual task-derived signal in the labeling step. We will revise the subsection to report a new (or more clearly documented) ablation in which both feature ranking and label assignment are performed using only activation statistics, reconstruction error, and semantic similarity measures with no access to any task labels whatsoever. The results of this ablation will be added to the main text or appendix, together with the corresponding causality metrics, so that the claim is supported by an experiment that matches the referee's specification. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; empirical benchmark comparison is self-contained

full rationale

The paper reports an empirical performance comparison of SAE steering against LoRA baselines on AxBench after applying a described supervised pipeline for feature selection and labeling. The central claims rest on experimental outcomes and direct measurement rather than any derivation, equation, or fitted parameter that reduces to the authors' own inputs by construction. No self-definitional steps, fitted-input predictions, load-bearing self-citations, or ansatz smuggling appear in the abstract or described structure. The result is therefore an ordinary empirical finding whose validity can be checked against the benchmark without internal circularity.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

No free parameters, axioms, or invented entities can be identified from the abstract alone. The supervised pipeline likely contains fitted components, but details are unavailable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines." pith.science (2026). https://pith.science/paper/S3GOZT2S

@misc{pith2026260531183,
  author       = {Pith},
  title        = {Pith review of: Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S3GOZT2S}},
  note         = {Machine review of arXiv:2605.31183}
}
read the original abstract

Sparse Autoencoders (SAEs) have been seen as a promising avenue for exploring the internals of Large Language Models (LLMs) and for steering model output generation. When AxBench - a model steering benchmark - was introduced in Wu et al. (2025), SAEs did not seem to live up to their original hype due to poor steering performance relative to a set of simple baselines. This work serves as a partial rebuttal for Sparse Autoencoders and suggests that the results of Wu et al. (2025) did not do them full justice. We find that Sparse Autoencoders can, in fact, perform close to on par with the reference LoRA performance on the AxBench benchmark, when features are selected and labelled with our supervised pipeline. We also find that our pipeline selects features that are surprisingly causal of their identified labels when using only its interpretability-based components. Lastly, we present evidence that high sparsity (low l0) may not be crucial for successful steering based on interpretability, which is in contrast to the earlier findings in Wang et al. (2025).

Figures

Figures reproduced from arXiv: 2605.31183 by the authors.

Figure 1
Figure 1. The architecture of a transformer-based causal LM at the level of a single transformer block. Here [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. AxBench aggregated ratings for two layers of Gemma-2-9b-it across multiple stackexchanges. Here the Gemma-Scope SAEs have width [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. a) The distributions π0 = N (µ1 = 0, σ2 = 12 ), and p1 = N (µ2 = 2, σ2 = 12 ) used in the simulation of the calibrated F1 score. b) Despite the invariance of the calibrated F1 to label support, the statistic may exhibit very high variance, in a somewhat peculiar pattern, given a low number of positive samples. to exhibit high variance in a somewhat unusual pattern for a low (absolute) number of positive samples. The… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The simulation results of section A.2.1, where we compute the empirical calibrated F1 scores as a function of [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Empirical Cumulative Density Functions of the label supports for the selected Stack Exchange fora after pre-processing. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: AxBench concept ratings for two layers of Gemma-2-9b-it across multiple fora. Here, the setup is identical to that in figure 2. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: AxBench aggregated ratings for two layers of Gemma-2-9b-it in an ablation study. Here, the setup is identical to that in figure 2 except [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: AxBench concept ratings for two layers of Gemma-2-9b-it in an ablation study. Here, the setup is identical to that in figure 2 except where [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Performance of ordinary F1 and Llama-3.1-8B-Instruct [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    Jonathan Crabbé and Mihaela van der Schaar

    URLhttps://arxiv.org/abs/2311.04329. Jonathan Crabbé and Mihaela van der Schaar. Concept activation regions: A generalized framework for concept-based explanations.Advances in Neural Information Processing Systems, 35:2590–2607, 2022. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable f...

    work page doi:10.1016/j.neuroimage.2013.10.067 2022

  2. [2]

    Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

    URLhttps://arxiv.org/abs/1711.11279. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models.https: //github.com/tatsu-lab/alpaca_eval, 5 2023. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas...

    work page Pith review arXiv 2023

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.