Dynamic multiway dense connections across Transformer layers let a 2.8B model match a 6.9B model's pretraining quality and few-shot performance.
Towards automated circuit discovery for mechanistic interpretability
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
Dynamic multiway dense connections across Transformer layers let a 2.8B model match a 6.9B model's pretraining quality and few-shot performance.