Incorrect L0 makes sparse autoencoders mix correlated features rather than disentangling them, and a decoder projection metric can identify the correct L0.
Dictionary learning optimization techniques.https: //transformer-circuits.pub/2025/january-update,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
Incorrect L0 makes sparse autoencoders mix correlated features rather than disentangling them, and a decoder projection metric can identify the correct L0.