On an iPhone 15 Pro, CPU-only llama.cpp inference achieved 17 tokens/s with a 1B model, beating the 12.8 tokens/s GPU path, but only for small models and without a confirmed cause.
Flashattention: Fast and memory- efficient exact attention with io-awareness
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.DC 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference
On an iPhone 15 Pro, CPU-only llama.cpp inference achieved 17 tokens/s with a 1B model, beating the 12.8 tokens/s GPU path, but only for small models and without a confirmed cause.