A rotor-based layer built from bivector exponentials approximates LLM attention projections with O(log^2 d) parameters and competitive downstream performance.
Efficient scaling of large language models with mixture of experts and 3d analog in-memory computing
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Composing Linear Layers from Irreducibles
A rotor-based layer built from bivector exponentials approximates LLM attention projections with O(log^2 d) parameters and competitive downstream performance.