AoE replaces the MoE router with expert self-selection based on internal activation-norm ranking, improving downstream accuracy on 700M-4B parameter language models while keeping throughput near traditional MoE.
the separation between the router’s decision and the experts’ execution
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Autonomy-of-Experts Models
AoE replaces the MoE router with expert self-selection based on internal activation-norm ranking, improving downstream accuracy on 700M-4B parameter language models while keeping throughput near traditional MoE.