Pipelined sharding and VLMOpt deliver up to 6.7x faster TTFT, 30x higher TPS for LLMs, and 10x lower VRAM use for VLMs on client systems compared to aggressive baselines.
ISBN 978-1-939133-40-3
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Efficient, VRAM-Constrained xLM Inference on Clients
Pipelined sharding and VLMOpt deliver up to 6.7x faster TTFT, 30x higher TPS for LLMs, and 10x lower VRAM use for VLMs on client systems compared to aggressive baselines.