A new theoretical framework for MoE serving with an O(√log k)-competitive online algorithm, an offline constant-factor approximation, and NP-hardness results.
Taming Throughput-Latency tradeoff in LLM inference with Sarathi-Serve
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.DS 1years
2026 1verdicts
ACCEPT 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Mixture-of-Experts Serving
A new theoretical framework for MoE serving with an O(√log k)-competitive online algorithm, an offline constant-factor approximation, and NP-hardness results.