GreenCache dynamically manages LLM KV cache resources to reduce carbon emissions by 15.1% on average (up to 25.3%) while meeting latency constraints for over 90% of requests on real traces.
EcoServe: Designing carbon-aware AI inference systems
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
Introduces a quantization-enabled demand response framework for LLM data centers that maps precision levels to power parameters and achieves 34.3% cost reduction in case studies while maintaining token volume.
GAR routes LLM inference requests via constrained multi-objective optimization to cut per-request CO2 emissions while respecting accuracy floors and p95 latency SLOs.
KAIROS reduces power by 27% on average (up to 39.8%) for agentic AI inference by using long-lived context to jointly manage GPU frequency, concurrency, and request routing across instances.
AI inference can be relocated across geographies to access lower-cost or lower-carbon electricity when latency budgets are relaxed, with the energy-latency frontier quantifying marginal benefits and new metrics tracking returns on latency tolerance.
The paper calls for life cycle assessment to capture embodied hardware costs and full pipeline operational costs in AI development and deployment.
A framework for two-tier AI service subscriptions that offer discounts for accepting lower quality or higher latency inference to reduce carbon emissions during high-intensity periods.
In the critical regime for energy provisioning to large reasoning models, performance is volatility-limited, motivating variance-aware routing policies based on training and inference compute scaling laws.
citing papers explorer
-
Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
GreenCache dynamically manages LLM KV cache resources to reduce carbon emissions by 15.1% on average (up to 25.3%) while meeting latency constraints for over 90% of requests on real traces.
-
From Tokens to Energy Flexibility: Quantization-Enabled Demand Response for Data Centers with LLM Inference Workloads
Introduces a quantization-enabled demand response framework for LLM data centers that maps precision levels to power parameters and achieves 34.3% cost reduction in case studies while maintaining token volume.
-
GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization
GAR routes LLM inference requests via constrained multi-objective optimization to cut per-request CO2 emissions while respecting accuracy floors and p95 latency SLOs.
-
KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
KAIROS reduces power by 27% on average (up to 39.8%) for agentic AI inference by using long-lived context to jointly manage GPU frequency, concurrency, and request routing across instances.
-
AI Inference as Relocatable Electricity Demand: A Latency-Constrained Energy-Geography Framework
AI inference can be relocated across geographies to access lower-cost or lower-carbon electricity when latency budgets are relaxed, with the energy-latency frontier quantifying marginal benefits and new metrics tracking returns on latency tolerance.
-
Evaluation of ML Resource Utilization Requires Model Life Cycle Assessment
The paper calls for life cycle assessment to capture embodied hardware costs and full pipeline operational costs in AI development and deployment.
-
Greening AI Inference with Accuracy and Latency-aware User Incentives
A framework for two-tier AI service subscriptions that offer discounts for accepting lower quality or higher latency inference to reduce carbon emissions during high-intensity periods.
-
Energy-Aware Routing to Large Reasoning Models
In the critical regime for energy provisioning to large reasoning models, performance is volatility-limited, motivating variance-aware routing policies based on training and inference compute scaling laws.
- Determinism-Preserving GPU Spatial Sharing with Vitamin-E