FlashNorm is an exact algebraic reformulation of RMSNorm plus linear projection that folds weights and defers normalization to allow parallel execution, plus scale-invariance simplifications that remove redundant norms in certain architectures.
Slim attention: cut your context memory in half without loss of accuracy – K-cache is all you need for MHA
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
representative citing papers
Keyless Attention eliminates the key projection, computing softmax(QV^T)V and reducing KV-cache memory by exactly 50%, with comparable or better performance on small-scale models.
citing papers explorer
-
FlashNorm: Fast Normalization for Transformers
FlashNorm is an exact algebraic reformulation of RMSNorm plus linear projection that folds weights and defers normalization to allow parallel execution, plus scale-invariance simplifications that remove redundant norms in certain architectures.
-
Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
Keyless Attention eliminates the key projection, computing softmax(QV^T)V and reducing KV-cache memory by exactly 50%, with comparable or better performance on small-scale models.