The Efficiency Frontier framework models LLM context management as a deployment-aware optimization problem balancing performance, token cost, and amortized preprocessing, with HotpotQA experiments showing 25% token reduction and over 50% cost savings for compression in high-performance regimes.
Available: https://arxiv.org/abs/2311.03687
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
years
2026 3verdicts
UNVERDICTED 3roles
background 1polarities
background 1representative citing papers
MeMo encodes new knowledge into a separate memory model that integrates with frozen LLMs, showing strong performance on QA benchmarks while avoiding catastrophic forgetting and working without access to model weights.
LLM inference should be reframed and evaluated as energy-to-token production with a Token Production Function that accounts for power, cooling, and efficiency ceilings.
citing papers explorer
-
The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management
The Efficiency Frontier framework models LLM context management as a deployment-aware optimization problem balancing performance, token cost, and amortized preprocessing, with HotpotQA experiments showing 25% token reduction and over 50% cost savings for compression in high-performance regimes.
-
MeMo: Memory as a Model
MeMo encodes new knowledge into a separate memory model that integrates with frozen LLMs, showing strong performance on QA benchmarks while avoiding catastrophic forgetting and working without access to model weights.
-
Position: LLM Inference Should Be Evaluated as Energy-to-Token Production
LLM inference should be reframed and evaluated as energy-to-token production with a Token Production Function that accounts for power, cooling, and efficiency ceilings.