Spherical attention, an L2-normalized attention variant derived from neural circuit diagrams, is streamable and runs at up to 200 TFLOP/s on an A100, matching FlashAttention-2.
FlashAttention on a Napkin: A Diagrammatic Approach to Deep Learning IO-Awareness
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
math.CT 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Accelerating Machine Learning Systems via Category Theory: Applications to Spherical Attention for Gene Regulatory Networks
Spherical attention, an L2-normalized attention variant derived from neural circuit diagrams, is streamable and runs at up to 200 TFLOP/s on an A100, matching FlashAttention-2.