Toy models demonstrate that polysemanticity arises when neural networks store more sparse features than neurons via superposition, producing a phase transition tied to polytope geometry and increased adversarial vulnerability.
The building blocks of interpretability.Distill, 2018
4 Pith papers cite this work, alongside 611 external citations. Polarity classification is still indexing.
4
Pith papers citing it
611
external citations · OpenAlex
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
LLMs process negation using both attention-based suppression and constructive representation mechanisms (construction dominant), with late-layer attention shortcuts explaining poor accuracy on negation tasks.
On Llama-2-7B, path-rerouting magnitude in paired transcoder attribution graphs correlates with jailbreak success (r=0.461), while static node metrics and top-feature ablations do not.
In spiking ResNets, 1FC ensembles defined by pairwise correlations show ReLU-like cofiring-to-response mapping whose gain scales with ensemble size, with reliable class encoding restricted to infrequent high-cofiring events.
citing papers explorer
-
Toy Models of Superposition
Toy models demonstrate that polysemanticity arises when neural networks store more sparse features than neurons via superposition, producing a phase transition tied to polytope geometry and increased adversarial vulnerability.
-
How Language Models Process Negation
LLMs process negation using both attention-based suppression and constructive representation mechanisms (construction dominant), with late-layer attention shortcuts explaining poor accuracy on negation tasks.
-
Mechanistic Interpretability of LLM Jailbreaks via Internal Attribution Graphs
On Llama-2-7B, path-rerouting magnitude in paired transcoder attribution graphs correlates with jailbreak success (r=0.461), while static node metrics and top-feature ablations do not.
-
Rare Events, Real Signals: Functional Ensembles as Units of Computation in Deep Spiking Networks
In spiking ResNets, 1FC ensembles defined by pairwise correlations show ReLU-like cofiring-to-response mapping whose gain scales with ensemble size, with reliable class encoding restricted to infrequent high-cofiring events.