Balancing only first-turn agent requests while keeping follow-ups cache-aware raises LLM cluster TPS 10-16% (colocation) and prefill TPS 2-34% (disaggregation) over SOTA schedulers on production agent traces.
Dualmap: Enabling both cache affin- ity and load balancing for distributed LLM serving.CoRR, abs/2602.06502, 2026
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.DC 2years
2026 2representative citing papers
In long-context LLM serving, accuracy becomes speed via retry dynamics, and accuracy-aware routing reduces time-to-correct-answer.
citing papers explorer
-
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
Balancing only first-turn agent requests while keeping follow-ups cache-aware raises LLM cluster TPS 10-16% (colocation) and prefill TPS 2-34% (disaggregation) over SOTA schedulers on production agent traces.
-
Accuracy Is Speed: Towards Long-Context-Aware Routing for Distributed LLM Serving
In long-context LLM serving, accuracy becomes speed via retry dynamics, and accuracy-aware routing reduces time-to-correct-answer.