Position: Use Sparse Autoencoders to Discover Unknowns
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:LTYGAJASrecord.jsonopen to challenge →
read the original abstract
While sparse autoencoders (SAEs) have generated significant excitement, a series of negative results have added to skepticism about their usefulness. Here, we establish a conceptual distinction that reconciles competing narratives surrounding SAEs. We argue that even if SAEs may be less effective for \textit{acting on known concepts}, SAEs are especially powerful tools for \textit{discovering unknown concepts}. This distinction separates existing negative results from positive results, and suggests several classes of SAE applications. Specifically, we outline use cases for SAEs in (i) ML interpretability, explainability, fairness, auditing, and safety, and (ii) social and health sciences.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
Three Years of r/ChatGPT: Societal Impact Evaluations from Social Media Data
Longitudinal analysis of r/ChatGPT posts shows normalization of ChatGPT as an everyday tool alongside rising mental health and emotional attachment discussions after GPT-4o, with PuLSE detecting the latter trend months early.
-
Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates
Introduces conditional hypothesis generation for LLM text analysis incorporating researcher-specified covariates via two econometrics-inspired methods to handle stratum imbalance and sign reversal, outperforming globa...
-
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
Pretrained base models exhibit higher yield to peer disagreement than RLHF instruct variants, with the effect localized to mid-layer attention and mitigated by structured dissent rather than prompt defenses.
-
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
Base LLMs show multi-agent yield to peer pressure at rates equal to or higher than aligned models, localized by activation patching to mid-layers where attention dominates, with one dissenter cutting yield by 54-73 po...
-
Making Interpretable Discoveries from Unstructured Data: A High-Dimensional Multiple Hypothesis Testing Approach
A new framework combines AI-derived concept embeddings with high-dimensional selective inference to enable statistically principled, interpretable discovery from unstructured data in empirical economics.
-
ActivationReasoning: Logical Reasoning in Latent Activation Spaces
ActivationReasoning grounds logical reasoning in LLM latent activations via SAEs to enable structured inference, concept composition, and behavior steering on multi-hop, abstraction, and safety tasks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.