FloE compresses and prefetches only the active parts of MoE experts, running Mixtral-8x7B on an 11 GB GPU with about 5 to 8 percent average accuracy loss.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
FloE: On-the-Fly MoE Inference on Memory-constrained GPU
FloE compresses and prefetches only the active parts of MoE experts, running Mixtral-8x7B on an 11 GB GPU with about 5 to 8 percent average accuracy loss.