Spherical attention, an L2-normalized attention variant derived from neural circuit diagrams, is streamable and runs at up to 200 TFLOP/s on an A100, matching FlashAttention-2.
Categories of Differentiable Polynomial Circuits for Machine Learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Reverse derivative categories (RDCs) have recently been shown to be a suitable semantic framework for studying machine learning algorithms. Whereas emphasis has been put on training methodologies, less attention has been devoted to particular \emph{model classes}: the concrete categories whose morphisms represent machine learning models. In this paper we study presentations by generators and equations of classes of RDCs. In particular, we propose \emph{polynomial circuits} as a suitable machine learning model. We give an axiomatisation for these circuits and prove a functional completeness result. Finally, we discuss the use of polynomial circuits over specific semirings to perform machine learning with discrete values.
fields
math.CT 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Accelerating Machine Learning Systems via Category Theory: Applications to Spherical Attention for Gene Regulatory Networks
Spherical attention, an L2-normalized attention variant derived from neural circuit diagrams, is streamable and runs at up to 200 TFLOP/s on an A100, matching FlashAttention-2.