Similarity-weighted and attention-weighted routing scores reduce expert-switching instability in sparse MoE and yield small but consistent gains on Wikitext-103, ImageNet-1K, and sentiment/intent fine-tuning.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Improving Routing in Sparse Mixture of Experts with Graph of Tokens
Similarity-weighted and attention-weighted routing scores reduce expert-switching instability in sparse MoE and yield small but consistent gains on Wikitext-103, ImageNet-1K, and sentiment/intent fine-tuning.