A review of cold start mitigation for serverless LLM inference summarizes ServerlessLLM's multi-tier checkpoint loading and live migration, reporting 6-8x faster startup relative to PyTorch and SafeTensors without adding new measurements.
Rise of the Planet of Serverless Computing: A Systematic Review
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Serverless computing is an emerging cloud computing paradigm, being adopted to develop a wide range of software applications. It allows developers to focus on the application logic in the granularity of function, thereby freeing developers from tedious and error-prone infrastructure management. Meanwhile, its unique characteristic poses new challenges to the development and deployment of serverless-based applications. To tackle these challenges, enormous research efforts have been devoted. This paper provides a comprehensive literature review to characterize the current research state of serverless computing. Specifically, this paper covers 164 papers on 17 research directions of serverless computing, including performance optimization, programming framework, application migration, multi-cloud development, testing and debugging, etc. It also derives research trends, focus, and commonly-used platforms for serverless computing, as well as promising research opportunities.
fields
cs.DC 1years
2024 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud
A review of cold start mitigation for serverless LLM inference summarizes ServerlessLLM's multi-tier checkpoint loading and live migration, reporting 6-8x faster startup relative to PyTorch and SafeTensors without adding new measurements.