A three-layer client-side scheduler for black-box LLMs using coarse token count priors achieves 100% completion and deadline satisfaction with graceful degradation under up to 60% prediction error.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Scheduling the Unschedulable: Taming Black-Box LLM Inference at Scale
A three-layer client-side scheduler for black-box LLMs using coarse token count priors achieves 100% completion and deadline satisfaction with graceful degradation under up to 60% prediction error.