Delta-rule linear attention faithfully approximates softmax attention through key-dependent rank-1 projections, enabling efficient post-hoc linearization of LLMs up to 32B parameters.
Still: Selecting tokens for intra-layer hybrid attention to linearize llms.arXiv preprint arXiv:2602.02180, 2026
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
The Key to Going Linear: Analysis-Driven Transformer Linearization
Delta-rule linear attention faithfully approximates softmax attention through key-dependent rank-1 projections, enabling efficient post-hoc linearization of LLMs up to 32B parameters.