Tutti is a GPU-direct SSD-backed KV cache that removes CPU bottlenecks via object abstraction, GPU io_uring, and slack scheduling, delivering near-DRAM performance at 2x higher request rate and 27% lower cost than prior GDS-based systems.
Fastswitch: Optimizing context switching efficiency in fairness-aware large language model serving
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
years
2026 3verdicts
UNVERDICTED 3representative citing papers
FMplex is a serving system that virtualizes FM backbones for sharing across tasks, claiming up to 80% lower latency and 6x more tasks hosted versus prior approaches.
Empirical study finds non-linear, model-size-dependent throughput degradation from offloading and high model-state reload costs from preemption in multi-LLM serving.
citing papers explorer
-
Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving
Tutti is a GPU-direct SSD-backed KV cache that removes CPU bottlenecks via object abstraction, GPU io_uring, and slack scheduling, delivering near-DRAM performance at 2x higher request rate and 27% lower cost than prior GDS-based systems.
-
FMplex: Model Virtualization for Serving Extensible Foundation Models
FMplex is a serving system that virtualizes FM backbones for sharing across tasks, claiming up to 80% lower latency and 6x more tasks hosted versus prior approaches.
-
Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
Empirical study finds non-linear, model-size-dependent throughput degradation from offloading and high model-state reload costs from preemption in multi-LLM serving.