A REINFORCE agent trained on a custom simulator learns workload-segregation routing that outperforms Shortest-Queue by 48% in heterogeneous multi-GPU inference serving, while offering negligible gains over static batching in single-GPU settings.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Adaptive Inference Batching using Policy Gradients
A REINFORCE agent trained on a custom simulator learns workload-segregation routing that outperforms Shortest-Queue by 48% in heterogeneous multi-GPU inference serving, while offering negligible gains over static batching in single-GPU settings.