A new safetensors loader copies groups of tensors directly to GPU memory, cutting LLM startup time by 4.8x to 7.5x compared with the default library.
Efficient Memory Management for Large Language Model Serving with PagedAttention,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Speeding up Model Loading with fastsafetensors
A new safetensors loader copies groups of tensors directly to GPU memory, cutting LLM startup time by 4.8x to 7.5x compared with the default library.