REVIEW 3 major objections 8 minor 37 references
EnsembleCI: Ensemble Learning for Carbon Intensity Forecasting
T0 review · 3 major / 8 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read An ensemble of three learners predicts grid carbon intensity up to four days ahead and beats the previous best method by 19.58% on average.
desk verdict A plausible applied win for stacked ensembles in carbon-intensity forecasting, but the headline comparison to CarbonCast is muted by an unvalidated baseline modification. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is multi-layer stacking over a deliberately small and diverse pool of sublearners. A base stage trains three strong tabular learners — two gradient-boosted tree methods and a fully connected neural network — on raw features; a stacking stage concatenates the raw features with the base models' predictions and retrains the same learners on that enlarged input; and ensemble selection assigns the final weights to the stacked predictions. Two separate trained models create the 96-hour output: one predicts the first 24 hours from historical source production, weather, and datetime features, and a second receives the day-1 outputs plus the same auxiliary features and is applied recursively for days 2–4. This two-stage composition is what carries the paper's argument: the per-grid weights absorb regional energy-mix differences that a fixed two-tier architecture cannot express, while the recursion slows error accumulation at long horizons.
What would settle it
Restore CarbonCast's tier-1 source-production forecasts and rerun the same 11-grid, four-day evaluation; if the day-1 average MAPE gap shrinks below the reported 18.1%, or the four-day gap closes, the ensemble advantage depends on the input modification rather than on ensembling. A cleaner refutation would be any grid where the weighted ensemble's day-4 MAPE exceeds every one of its own sublearners' day-4 MAPE by more than noise, since the stacking stage would then be adding error rather than removing it.
Extended reading notes
Core claim
EnsembleCI's claim is that regional adaptability, not a larger or deeper single model, is what unlocks accurate multi-day carbon-intensity forecasting. Rather than fit one architecture to all regions, it trains three complementary predictors on each grid's raw features — historical source-generation mix, historical carbon intensity, weather forecasts, and datetime encodings — then builds a second stacking layer on the concatenation of raw features and base predictions, and finally aggregates the stacked outputs with ensemble-selection weights. On the CarbonCast dataset covering six US and five EU grids, the ensemble averages 5.92%, 8.16%, 9.08%, and 9.56% MAPE for days 1–4, against 7.05%, 9.62%, 11.12%, and 12.48% for CarbonCast, which the paper reports as relative improvements of 18.1%, 17.13%, 19.69%, and 23.4%. The paper additionally reports steadier long-horizon behavior — a day-1-to-day-4 average MAPE increase of 3.63 points versus 5.43 — and permutation-importance analyses in which the three sublearners agree on each grid's dominant drivers, such as solar and irradiance in solar-heavy CISO and DE, and coal and wind in MISO. On the paper's own terms, the ensemble is both the more accurate and the more interpretable tool.
Load-bearing premise
The load-bearing premise is that the comparison is fair even though the paper removes the external one-day source-production forecasts that CarbonCast's two-tier design was built to consume, so any handicap from that removal is assumed to hit both models equally.
Editorial extensions
If this is right
- Day-ahead forecasts near 5.9% average MAPE are accurate enough to feed into workload-shifting and battery-scheduling policies that today's CarbonCast-level errors can defeat.
- The advantage grows with horizon — from 18.1% on day 1 to 23.4% on day 4 — so multi-day planning, not just hour-ahead shifting, is where the ensemble pays off.
- Grid-specific feature rankings give operators a diagnostic: when the model leans on solar and irradiance, it flags renewable-driven variability; when it leans on coal and wind, it flags the fuel mix that dominates emissions.
- Because the architecture trains per grid with a shared procedure, adding a new grid or retraining after a fuel-mix change does not require redesigning the network.
Reading between the lines
- Editorial inference: the reported 19.58% gain may be sensitive to removing the one-day source-production forecasts from both models; a rerun with CarbonCast's native tier-1 forecasts restored is the cheapest way to see whether the ensemble's edge survives a fully native input set.
- Editorial inference: the period-retraining suggestion is untested, so a rolling-window experiment tracking weights and MAPE across seasons would reveal how quickly per-grid ensembles go stale as renewable penetration grows.
- Editorial inference: the same base-stack-weight pipeline could be lifted to other grid variables such as locational prices or renewable generation, where heterogeneous regional regimes are equally central.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EnsembleCI, an AutoGluon-based multi-layer stacking ensemble of LightGBM, CatBoost, and a fastai tabular neural network for forecasting grid carbon intensity up to four days ahead. The method is evaluated on 11 US and EU regional grids using temporal train/test splits, with MAPE as the primary metric. The authors report that EnsembleCI achieves the lowest MAPE in 10–11 grids across forecast days 1–4 and an average relative improvement of 19.58% over CarbonCast, along with lower MAPE degradation over longer horizons. The paper also presents permutation feature importance for three grids to support interpretability claims.
Significance. If the empirical claims hold, EnsembleCI is a practically useful contribution to carbon-aware scheduling, since accurate multi-day carbon intensity forecasts are directly relevant to datacenter load shifting. Strengths of the paper include proper temporal train/test splits that avoid target leakage, averaging over five runs, an open-source release of code and data, and a concrete feature-importance analysis. The central empirical claim, however, depends on the fairness of the CarbonCast comparison, and the current reporting does not provide statistical support for the claimed dominance. The paper is a short conference-style study; the underlying method is reasonable, but the evaluation needs strengthening before the headline accuracy gains can be accepted.
major comments (3)
- [4.1 Methodology and Figure 1] The comparison against CarbonCast is not demonstrably neutral. Section 4.1 states that the authors remove the one-day source production predictions from external sources such as OASIS and ENTSO-E for both CarbonCast and EnsembleCI. According to Figure 1 and Section 2.1, CarbonCast's two-tier architecture is explicitly designed to consume source forecasts: Tier 1 produces source forecasts and Tier 2 consumes them along with weather and historical CI. Removing the external day-ahead source production inputs may therefore handicap only CarbonCast, particularly at the day-1 horizon where such operator forecasts are most informative. The paper provides no ablation quantifying CarbonCast's sensitivity to this removal, and no results for the published CarbonCast pipeline. Because the headline 19.58% average improvement includes the day-1 horizon, the reported advantage over CarbonCast in Table 2 could overstate EnsembleCI's real benefit. Please add an ablation reporting CarbonCast with and without the external source production predictions, or otherwise demonstrate that the removal is neutral for the baseline.
- [Table 2 and Section 4.2] The evaluation reports only MAPE point estimates. The authors state that five runs are averaged, but no standard deviations, confidence intervals, or significance tests are reported for any grid or horizon. Without measures of variability, the claims that EnsembleCI 'consistently surpasses' CarbonCast and that it is 'more robust' in long-term forecasting are not statistically supported. For example, the day-3 and day-4 differences in some grids are small (e.g., MISO day-3: 7.65 vs. 7.75; NL day-4: 9.33 vs. 13.31 is larger, but variance is unknown). Please report per-grid error bars from the five runs and, where appropriate, a paired significance test across grids or horizons.
- [Section 3, Implementation] The recursive design of the day-2-to-4 model is described only briefly. The text says the day-1 model outputs, together with future weather forecasts and datetime features, are inputs to the day-2-to-4 model, which is applied recursively for three iterations. It is not specified whether the day-2-to-4 model is trained on teacher-forced ground-truth day-1 values or on the day-1 model's own predictions, and it is unclear whether the recursive application introduces error accumulation that is specific to this design. Please clarify the training procedure and, ideally, report an ablation comparing direct multi-day forecasting with the recursive scheme.
minor comments (8)
- [Table 2 caption] The caption contains the word 'blue' in 'up to 4 days blue into the future'; this appears to be a leftover editing artifact and should be removed.
- [Section 1] The text contains a typo: 'regional girds' should be 'regional grids'.
- [Section 3] The subsection heading 'Subleaner selection' and the body text contain repeated misspellings of 'sublearner'.
- [Figure 4] The x-axis label 'ERCO' should be 'ERCOT' to match the grid name used elsewhere.
- [Section 4.1] The source of the weather forecast features is not specified. For reproducibility, please state which dataset provides the 24-hour weather forecasts and how they are aligned with the grid timestamps.
- [Section 4.1] The 'CarbonCast v3.0 dataset' is not defined or cited. Please specify its contents, version, and how it was obtained.
- [Table 3] The heading 'Top-3 features for three grids with the highest average ranks for three grids' is awkward and should be rephrased, for example as 'Top-3 features for three exemplar grids'.
- [Section 4.2] Figures 2 and 4 show point estimates only; adding error bars or shaded confidence bands would make the claimed robustness and variability reductions visible.
Circularity Check
No circularity: the ensemble is trained and evaluated on separate temporal splits; the CarbonCast comparison is empirical and not forced by construction.
full rationale
The paper's derivation chain is self-contained. EnsembleCI weights are learned on training data (2019 through the first half of 2022 for US grids, 2019 through the first half of 2021 for EU grids) and evaluated on a held-out second-half test period, so the reported MAPE values are out-of-sample predictions rather than refitted inputs. The sublearners (LightGBM, CatBoost, NN) are standard supervised models trained on raw features; the stacking stage concatenates their base-stage predictions with raw features and learns an ensemble-selection weighting, none of which encodes the target values used for evaluation. The comparison to CarbonCast is an external empirical benchmark, not a definitional identity: nothing in the ensemble construction forces EnsembleCI's MAPE to be lower, and the reported improvement is an observed difference. The only notable methodological choice is the removal of one-day OASIS/ENTSO-E source-production forecasts from both models (Section 4.1); while this could be questioned as a baseline-fairness matter, it is not a circular reduction because it does not make EnsembleCI's output equivalent to any fitted parameter or to the benchmark's own output. Self-citations appear only in motivational contexts (LLM/datacenter carbon impact) and are not load-bearing for the forecasting result. No uniqueness theorem, ansatz, or renamed empirical pattern is imported from authors' prior work. Therefore no circular step can be exhibited with a quote, and the appropriate score is 0.
Assumptions & free parameters
free parameters (3)
- ensemble stacking weights =
learned during training, not numerically reported
- NN architecture hyperparameters =
200 and 100 neurons, ReLU, batch norm, dropout
- AutoGluon default hyperparameters =
defaults, not enumerated
assumptions (4)
- domain assumption Carbon intensity is generation-based, measured as gCO2e/kWh from power generation sources
- domain assumption Weather forecasts are available and treated as accurate inputs for the full 96-hour horizon
- domain assumption The dataset from CarbonCast v3.0 and Electricity Maps provides reliable ground truth CI and source mix
- standard math Standard supervised learning assumptions for gradient boosting and neural network training
Cite this review
Pith. "Pith review of EnsembleCI: Ensemble Learning for Carbon Intensity Forecasting." pith.science (2026). https://pith.science/paper/SBLLCGTG
@misc{pith2026250501959,
author = {Pith},
title = {Pith review of: EnsembleCI: Ensemble Learning for Carbon Intensity Forecasting},
year = {2026},
howpublished = {\url{https://pith.science/paper/SBLLCGTG}},
note = {Machine review of arXiv:2505.01959}
}
read the original abstract
Carbon intensity (CI) measures the average carbon emissions generated per unit of electricity, making it a crucial metric for quantifying and managing the environmental impact. Accurate CI predictions are vital for minimizing carbon footprints, yet the state-of-the-art method (CarbonCast) falls short due to its inability to address regional variability and lack of adaptability. To address these limitations, we introduce EnsembleCI, an adaptive, end-to-end ensemble learning-based approach for CI forecasting. EnsembleCI combines weighted predictions from multiple sublearners, offering enhanced flexibility and regional adaptability. In evaluations across 11 regional grids, EnsembleCI consistently surpasses CarbonCast, achieving the lowest mean absolute percentage error (MAPE) in almost all grids and improving prediction accuracy by an average of 19.58%. While performance still varies across grids due to inherent regional diversity, EnsembleCI reduces variability and exhibits greater robustness in long-term forecasting compared to CarbonCast and identifies region-specific key features, underscoring its interpretability and practical relevance. These findings position EnsembleCI as a more accurate and reliable solution for CI forecasting. EnsembleCI source code and data used in this paper are available at https://github.com/emmayly/EnsembleCI.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
https://app.electricitymaps.com/map/
Electricity Maps. https://app.electricitymaps.com/map/
- [2]
-
[3]
Carbon explorer: a holis- tic framework for designing carbon aware datacenters
Bilge Acun, Benjamin Lee, Fiodar Kazhamiaka, Kiwan Maeng, Udit Gupta, Manoj Chakkaravarthy, David Brooks, and Carole-Jean Wu. Carbon explorer: a holis- tic framework for designing carbon aware datacenters. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Volume 2 , 2023
work page 2023
-
[4]
US Energy Information Administration. Real-time operating grid. https://www. eia.gov/electricity/gridmonitor/dashboard/electric_overview/US48/US48
-
[5]
Permutation importance: a corrected feature importance measure
André Altmann, Laura Toloşi, Oliver Sander, and Thomas Lengauer. Permutation importance: a corrected feature importance measure. Bioinformatics, 2010
work page 2010
-
[6]
Bagging predictors
Leo Breiman. Bagging predictors. Machine learning, 24:123–140, 1996
1996
-
[7]
Random forests
Leo Breiman. Random forests. Machine learning, 45:5–32, 2001
2001
-
[8]
Open access same-time information system (OASIS)
California Independent System Operator. Open access same-time information system (OASIS). https://oasis.caiso.com/mrioasis/logon.do, 2025
work page 2025
Show all 37 references
-
[9]
Greenhouse temperature predic- tion based on time-series features and LightGBM
Qiong Cao, Yihang Wu, Jia Yang, and Jing Yin. Greenhouse temperature predic- tion based on time-series features and LightGBM. 2023
2023
-
[10]
Ensemble selection from libraries of models
Rich Caruana, Alexandru Niculescu-Mizil, Geoff Crew, and Alex Ksikes. Ensemble selection from libraries of models. In Proceedings of the Twenty-first International Conference on Machine Learning (ICML) , 2004
2004
-
[11]
Chaturvedi and Isha Singh
D. Chaturvedi and Isha Singh. Solar power forecasting: A review. 07 2016
2016
-
[12]
XGBoost: A scalable tree boosting system
Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) , 2016
2016
-
[13]
Sustainable llm serving: Environmental implications, challenges, and opportunities
Yi Ding and Tianyao Shi. Sustainable llm serving: Environmental implications, challenges, and opportunities. In 2024 IEEE 15th International Green and Sustain- able Computing Conference (IGSC) , 2024
2024
-
[14]
CatBoost: gradient boosting with categorical features support
Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin. CatBoost: gradient boosting with categorical features support. arXiv preprint arXiv:1810.11363, 2018
2018 arXiv
-
[15]
Fighting biases with dynamic boosting
Anna Veronika Dorogush, Andrey Gulin, Gleb Gusev, Nikita Kazeev, Liudmila Os- troumova Prokhorenkova, and Aleksandr Vorobev. Fighting biases with dynamic boosting. arXiv preprint arXiv:1706.09516, 2017
2017 arXiv
-
[16]
ENTSO-E transparency platform
ENTSO-E. ENTSO-E transparency platform. https://transparency.entsoe.eu/, 2015
2015
-
[17]
Autogluon-Tabular: Robust and accurate AutoML for structured data
Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. Autogluon-Tabular: Robust and accurate AutoML for structured data. arXiv preprint arXiv:2003.06505, 2020
2003 arXiv
-
[18]
LLMCarbon: Modeling the end-to-end carbon footprint of large language models
Ahmad Faiz, Sotaro Kaneda, Ruhan Wang, Rita Chukwunyere Osi, Prateek Sharma, Fan Chen, and Lei Jiang. LLMCarbon: Modeling the end-to-end carbon footprint of large language models. In The Twelfth International Conference on Learning Representations (ICLR), 2024
2024
-
[19]
Experiments with a new boosting algorithm
Yoav Freund, Robert E Schapire, et al. Experiments with a new boosting algorithm. In icml, volume 96, pages 148–156. Citeseer, 1996
1996
-
[20]
ACT: Designing sustainable computer systems with an architectural carbon modeling tool
Udit Gupta, Mariam Elgamal, Gage Hills, Gu-Yeon Wei, Hsien-Hsin S Lee, David Brooks, and Carole-Jean Wu. ACT: Designing sustainable computer systems with an architectural carbon modeling tool. In Proceedings of the 49th Annual International Symposium on Computer Architecture (...
2022
-
[21]
CarbonScaler: Leveraging cloud workload elasticity for optimizing carbon-efficiency
Walid A Hanafy, Qianlin Liang, Noman Bashir, David Irwin, and Prashant Shenoy. CarbonScaler: Leveraging cloud workload elasticity for optimizing carbon-efficiency. Proceedings of the ACM on Measurement and Analysis of Com- puting Systems (POMACS), 2023
2023
-
[22]
Fastai: a layered api for deep learning
Jeremy Howard and Sylvain Gugger. Fastai: a layered api for deep learning. Information, 2020
2020
-
[23]
D. Jimenez. Dynamically weighted ensemble neural networks for classification. In IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227) , volume 1, pages 753–756 vol.1, 1998
1998
-
[24]
LightGBM: A highly efficient gradient boosting decision tree
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qi- wei Ye, and Tie-Yan Liu. LightGBM: A highly efficient gradient boosting decision tree. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett, editors, Advances...
2017
-
[25]
Uncertainty-aware decarbonization for data- centers
Amy Li, Sihang Liu, and Yi Ding. Uncertainty-aware decarbonization for data- centers. In Workshop on Sustainable Computer Systems (HotCarbon) , 2024
2024
-
[26]
Multi-day fore- casting of electric grid carbon intensity using machine learning
Diptyaroop Maji, Prashant Shenoy, and Ramesh K Sitaraman. Multi-day fore- casting of electric grid carbon intensity using machine learning. In Proceedings of the 9th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys) , 2023
2023
-
[27]
Multi-day forecast- ing of electric grid carbon intensity using machine learning
Diptyaroop Maji, Prashant Shenoy, and Ramesh K Sitaraman. Multi-day forecast- ing of electric grid carbon intensity using machine learning. ACM SIGENERGY Energy Informatics Review, 3(2):19–33, 2023
2023
-
[28]
Towards sustainable large language model serving
Sophia Nguyen, Beihao Zhou, Yi Ding, and Sihang Liu. Towards sustainable large language model serving. In The 3rd Workshop on Sustainable Computer Systems (HotCarbon), 2024
2024
-
[29]
Scikit-learn: Machine learning in Python
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in Python. the Journal of Machine Learning Research, 2011
2011
-
[30]
Multi-layer stacking ensemble learners for low footprint network intrusion detection
Saeed Shafieian and Mohammad Zulkernine. Multi-layer stacking ensemble learners for low footprint network intrusion detection. Complex & Intelligent Systems, 2023
2023
-
[31]
GreenLLM: Disaggregating large language model serving on heterogeneous GPUs for lower carbon emissions
Tianyao Shi, Yanran Wu, Sihang Liu, and Yi Ding. GreenLLM: Disaggregating large language model serving on heterogeneous GPUs for lower carbon emissions. arXiv preprint arXiv:2412.20322, 2024
2024 arXiv
-
[32]
CASPER: carbon-aware scheduling and provisioning for distributed web ser- vices
Abel Souza, Shruti Jasoria, Basundhara Chakrabarty, Alexander Bridgwater, Axel Lundberg, Filip Skogh, Ahmed Ali-Eldin, David Irwin, and Prashant Shenoy. CASPER: carbon-aware scheduling and provisioning for distributed web ser- vices. In Proceedings of the 14th International Gr...
2023
-
[33]
Kai Ming Ting and Ian H. Witten. Stacking bagged and dagged models. In Proceedings of the Fourteenth International Conference on Machine Learning (ICML), 1997
1997
-
[34]
Energy Information Administration
U.S. Energy Information Administration. How much of U.S. carbon dioxide emissions are associated with electricity generation? https://www.eia.gov/tools/ faqs/faq.php?id=77&t=11, 2023
2023
-
[35]
Carbon dioxide emissions from electric- ity
World Nuclear Association. Carbon dioxide emissions from electric- ity. https://world-nuclear.org/information-library/energy-and-the-environment/ carbon-dioxide-emissions-from-electricity, 2024
2024
-
[36]
Sustain- able AI: Environmental implications, challenges and opportunities
Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. Sustain- able AI: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems (MLSys) , 2022
2022
-
[37]
Unveiling environmental impacts of large language model serving: A functional unit view
Yanran Wu, Inez Hua, and Yi Ding. Unveiling environmental impacts of large language model serving: A functional unit view. arXiv preprint arXiv:2502.11256, 2025
2025 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.