Nexus performs proactive intra-GPU disaggregation of prefill and decode, using an analytical cost model and greedy search to dynamically partition SMs, achieving up to 2.2x throughput gains over vLLM.
Gulavani, Alexey Tumanov, and Ramachandran Ramjee
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
baseline 1
citation-polarity summary
fields
cs.DC 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
Nexus performs proactive intra-GPU disaggregation of prefill and decode, using an analytical cost model and greedy search to dynamically partition SMs, achieving up to 2.2x throughput gains over vLLM.