A past-future scheduler predicts output lengths from historical requests and only admits new requests when the predicted future memory peak of the running batch fits in GPU memory.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Past-Future Scheduler for LLM Serving under SLA Guarantees
A past-future scheduler predicts output lengths from historical requests and only admits new requests when the predicted future memory peak of the running batch fits in GPU memory.