Sub-MoE compresses MoE LLMs by K-means clustering of experts plus frequency-weighted merging of right singular vectors after a shared SVD, and claims 96 and 86 percent retained accuracy at 25 and 50 percent expert reduction on Mixtral-8x7B.
Pruner-zero: Evolving symbolic pruning metric from scratch for large language models
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
Sub-MoE compresses MoE LLMs by K-means clustering of experts plus frequency-weighted merging of right singular vectors after a shared SVD, and claims 96 and 86 percent retained accuracy at 25 and 50 percent expert reduction on Mixtral-8x7B.