pith. sign in

arXiv preprint arXiv:2406.04785 , year=

Enabling Efficient Batch Serving for LMaaS via Generation Length Prediction , author= · arXiv 2406.04785

1 Pith paper cite this work. Polarity classification is still indexing.

1 Pith paper citing it
read on arXiv browse 1 citing papers

fields

cs.LG 1

years

2026 1

verdicts

UNVERDICTED 1

clear filters

representative citing papers

Beyond Prediction: Tail-Aware Scheduling for LLM Inference

cs.LG · 2026-06-16 · unverdicted · novelty 7.0

Presents a distribution-aware scheduling framework for LLM inference that reduces P99 TTLT by 35-50% and TTFT by 34-47% versus SRPT with perfect length knowledge using statistical signals instead of predictions.

citing papers explorer

Showing 1 of 1 citing paper after filters.

  • Beyond Prediction: Tail-Aware Scheduling for LLM Inference cs.LG · 2026-06-16 · unverdicted · none · ref 38

    Presents a distribution-aware scheduling framework for LLM inference that reduces P99 TTLT by 35-50% and TTFT by 34-47% versus SRPT with perfect length knowledge using statistical signals instead of predictions.