REVIEW 6 cited by
Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Mechanistic interpretability aims to understand the internal mechanisms learned by neural networks. Despite recent progress toward this goal, it remains unclear how best to decompose neural network parameters into mechanistic components. We introduce Attribution-based Parameter Decomposition (APD), a method that directly decomposes a neural network's parameters into components that (i) are faithful to the parameters of the original network, (ii) require a minimal number of components to process any input, and (iii) are maximally simple. Our approach thus optimizes for a minimal length description of the network's mechanisms. We demonstrate APD's effectiveness by successfully identifying ground truth mechanisms in multiple toy experimental settings: Recovering features from superposition; separating compressed computations; and identifying cross-layer distributed representations. While challenges remain to scaling APD to non-toy models, our results suggest solutions to several open problems in mechanistic interpretability, including identifying minimal circuits in superposition, offering a conceptual foundation for 'features', and providing an architecture-agnostic framework for neural network decomposition.
Forward citations
Cited by 6 Pith papers
-
Compressed Computation under $L^4$ Loss is likely Computation in Superposition
Training a 50-neuron ReLU network under L4 loss elicits sparse binary codewords over neurons that compute 100 sparse ReLUs in superposition, recovered by a three-scalar ansatz.
-
Individual Parameters in Weight-Sparse Transformers Appear Interpretable
An automated LLM pipeline finds that 12–31% of nonzero weights in weight-sparse transformers admit short, held-out-validated descriptions of when they matter, far above dense controls.
-
Targeted Recovery of Weight-Space Mechanisms From Neural Networks
A targeted decomposition method recovers the weight-space mechanisms behind specific inputs at low FLOPs, enabling focused ablation and rewiring of a 12-block transformer.
-
Compressed Computation: Dense Circuits in a Toy Model of the Universal-AND Problem
A toy model learns a dense binary-weighted circuit for the Universal-AND problem, which outperforms prior sparse constructions at low sparsity.
-
Stochastic Parameter Decomposition
SPD uses stochastic masking and a learned causal importance function to decompose neural network parameters into sparsely active rank-one subcomponents, recovering ground-truth mechanisms in toy models where APD struggled.
-
Distribution-Aware Feature Selection for SAEs
Sampled-SAE pre-selects a candidate pool of features using batch-level norms or entropy before batch top-K, creating a tunable family that trades reconstruction fidelity for improved probing and reduced absorption on ...
Discussion (0). Sign in to comment.