SFT supplies entangled compositional traces of atomic skills and routing modules; RL identifies those modules and enables recombination on novel compositions outside the SFT support.
Feature hedging: Correlated features break narrow sparse autoencoders.arXiv preprint arXiv:2505.11756, 2025
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
fields
cs.LG 4years
2026 4representative citing papers
Sparse autoencoders provide a basis for sensible concept hierarchies on visual data but are undermined by hard and soft feature absorption.
SAEs exhibit a rate-distortion-polysemanticity tradeoff where monosemanticity increases rate and distortion, with optimal polysemanticity set by feature co-occurrence probabilities in the data.
citing papers explorer
-
From Reasoning Traces to Reusable Modules: Understanding Compositional Generalization in Language Model Reasoning
SFT supplies entangled compositional traces of atomic skills and routing modules; RL identifies those modules and enables recombination on novel compositions outside the SFT support.
-
Do Sparse Autoencoders Learn Meaningful Concept Hierarchies?
Sparse autoencoders provide a basis for sensible concept hierarchies on visual data but are undermined by hard and soft feature absorption.
-
The Rate-Distortion-Polysemanticity Tradeoff in SAEs
SAEs exhibit a rate-distortion-polysemanticity tradeoff where monosemanticity increases rate and distortion, with optimal polysemanticity set by feature co-occurrence probabilities in the data.
- Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations