MoE provably learns clustered single-index regression with sample complexity Õ(d^{k*-1}), while a vanilla network provably cannot recover the shared global feature due to gradient cancellation.
Then, there existst l+1 ≲η −1J dAl(k∗−2) such that
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning
MoE provably learns clustered single-index regression with sample complexity Õ(d^{k*-1}), while a vanilla network provably cannot recover the shared global feature due to gradient cancellation.