Power-Softmax is a new HE-compatible attention variant that permits training and inference of billion-parameter polynomial LLMs with performance matching standard transformers.
Polynomial activation neural networks: Modeling, stability analysis and coverage bp-training
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 2verdicts
UNVERDICTED 2roles
background 1polarities
support 1representative citing papers
A counterexample disproves the conjecture that minimal filling architectures of polynomial neural networks always have unimodal hidden layer widths.
citing papers explorer
-
Power-Softmax: Towards Secure LLM Inference over Encrypted Data
Power-Softmax is a new HE-compatible attention variant that permits training and inference of billion-parameter polynomial LLMs with performance matching standard transformers.
-
Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects
A counterexample disproves the conjecture that minimal filling architectures of polynomial neural networks always have unimodal hidden layer widths.