pith. sign in

arxiv: 2605.17054 · v1 · pith:4J6ZIIOBnew · submitted 2026-05-16 · 📡 eess.SY · cs.SY

A review of imbalance price forecasting algorithms in Europe: algorithms, metrics and the way forward

Arnaud Verstraeten , Maria Margarida Mascarenhas , Hussain Kazmi This is my paper

Pith reviewed 2026-05-20 15:15 UTC · model grok-4.3

classification 📡 eess.SY cs.SY
keywords imbalance pricesforecasting algorithmsmachine learningEuropean power marketsrenewable energybalancing marketsforecast evaluation
0
0 comments X

The pith

Imbalance price forecasts in Europe turn to machine learning models that need evaluation by their value to traders.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This review examines algorithms used to forecast imbalance prices in European electricity markets, where mismatches between supply and demand create volatile prices that affect balancing decisions. It identifies a shift from early methods that combined physical system models with statistics to current data-driven machine learning techniques that rely on large historical datasets. The authors argue that progress depends on using detailed intraday and per-minute system information as inputs and on creating shared benchmarks so methods developed for different countries can be compared directly. They further contend that forecasts must be assessed not only for prediction accuracy but also for their ability to improve the financial outcomes of market participants who use them.

Core claim

The central claim is that imbalance price forecasting in Europe has seen a clear trend toward data-driven machine learning models, replacing earlier combinations of fundamental and statistical approaches, and that meaningful advancement requires high-quality input data along with evaluation frameworks that consider both accuracy and downstream economic value.

What carries the argument

The observed trend in forecasting methodology toward machine learning models, along with recommendations for input data quality and benchmark standardization.

If this is right

  • Market participants can achieve better balancing positions with more accurate and valuable forecasts.
  • Standardized benchmarks will enable fair comparison of methods across different European markets and periods.
  • Evaluation based on downstream value will lead to forecasts that directly reduce costs for traders and improve system efficiency.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Adopting per-minute data could uncover volatility patterns that hourly models miss, potentially improving short-term forecasts.
  • Machine learning approaches might scale more easily to new markets without needing detailed physical models of each grid.
  • Future work could test whether the trend toward data-driven methods holds when including non-published or industry-internal studies.

Load-bearing premise

The reviewed papers form a representative sample of the field, so the observed trend toward machine learning is not distorted by selective publication or incomplete literature coverage.

What would settle it

Finding a significant body of recent peer-reviewed or pre-print work on European imbalance price forecasting that continues to rely mainly on fundamental-statistical hybrids instead of machine learning would falsify the claimed methodological trend.

Figures

Figures reproduced from arXiv: 2605.17054 by Arnaud Verstraeten, Hussain Kazmi, Maria Margarida Mascarenhas.

Figure 1
Figure 1. Figure 1: Belgian day-ahead and imbalance prices on a randomly [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Monthly median and monthly averaged daily spread of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
read the original abstract

Renewable electricity generation has grown significantly across many European power systems, leading to a greener energy mix, but also additional complexity in balancing electricity supply and demand. Unexpected differences between forecasts and actual output can lead to fluctuations in the system imbalance, which causes volatile imbalance prices. Accurate imbalance price forecasts are crucial for market players to choose a strategic balancing position. In early works, most forecasting methods combined fundamental and statistical approaches, but currently there is a clear trend towards data-driven machine learning models. This review compares forecasting algorithms in European markets with a focus on methodology. We emphasize the importance of high-quality input data, including intraday information and per-minute system data. Next, we identify the need for a common benchmark to compare novel forecasting methods developed for different markets and time periods. Finally, we argue that forecasts should be evaluated in terms of both downstream value and accuracy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper reviews imbalance price forecasting algorithms across European electricity markets. It identifies an early reliance on fundamental and statistical methods that has given way to a clear trend toward data-driven machine learning approaches, stresses the value of high-quality inputs such as intraday prices and per-minute system data, calls for a common benchmark to enable cross-market and cross-period comparisons, and argues that forecasts should be assessed on both predictive accuracy and downstream economic value for balancing decisions.

Significance. If the literature sample is representative and the trend claim is supported by transparent search methods, the review would usefully synthesize the field, draw attention to practical data requirements, and promote evaluation practices that better align forecasting research with operational needs in systems with high renewable penetration.

major comments (2)
  1. [Literature search and trend identification] The headline claim of a 'clear trend' toward data-driven machine learning models (abstract and corresponding body section) is load-bearing for the paper's central observation yet rests on an unspecified literature search. No explicit protocol, database list, keyword set, inclusion/exclusion criteria, or year-by-year breakdown of reviewed papers is provided, leaving open the possibility that the trend is an artifact of publication bias or incomplete coverage rather than an empirical finding across the full body of work.
  2. [Future directions / benchmark discussion] The recommendation for a common benchmark (section on future directions) is well-motivated but remains high-level; the manuscript does not specify candidate benchmark datasets, time horizons, or evaluation protocols that would allow methods developed for different European markets to be compared on equal footing.
minor comments (2)
  1. [Introduction / terminology] Notation for imbalance price components and market-specific terms (e.g., single vs. dual pricing) should be introduced consistently in an early section or table to aid readers from outside the immediate domain.
  2. [Results / figures] Figure captions and table headings could more explicitly state the time period and number of papers covered in each methodological category to make the trend visualization self-contained.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our review of imbalance price forecasting algorithms. The feedback highlights important areas for improving transparency and practicality, and we address each point below with plans for revision.

read point-by-point responses

  1. Referee: [Literature search and trend identification] The headline claim of a 'clear trend' toward data-driven machine learning models (abstract and corresponding body section) is load-bearing for the paper's central observation yet rests on an unspecified literature search. No explicit protocol, database list, keyword set, inclusion/exclusion criteria, or year-by-year breakdown of reviewed papers is provided, leaving open the possibility that the trend is an artifact of publication bias or incomplete coverage rather than an empirical finding across the full body of work.

    Authors: We acknowledge that the manuscript does not provide an explicit literature search protocol, which limits the ability to fully assess the representativeness of the reviewed sample and the robustness of the observed trend. The review was compiled from a broad but non-systematic survey of peer-reviewed literature on European imbalance price forecasting, drawing on publications known to the authors and common in the field. To strengthen this, we will add a new subsection in the revised manuscript detailing the search approach, including databases consulted, primary keywords (e.g., combinations of 'imbalance price', 'forecasting', 'machine learning', and European country names), inclusion criteria focused on works addressing European markets, and a supplementary table or figure showing the year-by-year distribution of reviewed papers to support the trend claim. This revision will make the central observation more transparent and reproducible without altering the core findings. revision: yes

  2. Referee: [Future directions / benchmark discussion] The recommendation for a common benchmark (section on future directions) is well-motivated but remains high-level; the manuscript does not specify candidate benchmark datasets, time horizons, or evaluation protocols that would allow methods developed for different European markets to be compared on equal footing.

    Authors: We agree that the benchmark recommendation would benefit from greater specificity to guide future research. The original discussion intentionally remained high-level to emphasize the general need for standardization across diverse markets. In the revision, we will expand this section to propose concrete elements: candidate datasets such as publicly available imbalance price series from the ENTSO-E Transparency Platform and national TSO portals for representative markets (e.g., Germany, UK, Netherlands); suggested forecast horizons including intraday (15-min to 1-hour ahead) and day-ahead; and evaluation protocols that combine standard accuracy metrics (MAE, RMSE) with downstream economic value metrics (e.g., simulated balancing cost reduction for a strategic trader). We will also outline a framework for a shared benchmark repository to facilitate cross-market comparisons. revision: yes

Circularity Check

0 steps flagged

No circularity: review aggregates independent external studies

full rationale

This is a literature review paper whose central observation (trend from fundamental/statistical to data-driven ML models) is presented as an empirical summary of methods in the cited external works rather than a derivation from any fitted parameters, self-referential equations, or load-bearing self-citations. The abstract and structure describe comparison of algorithms across European markets without introducing new predictive models or renaming results via internal ansatz. No step reduces by construction to the paper's own inputs; the review remains self-contained against the body of independently published forecasting papers it surveys.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The review rests on the assumption that the chosen body of literature is sufficient to identify field-wide trends and that the proposed evaluation criteria are broadly applicable. No new free parameters, axioms, or invented entities are introduced.

pith-pipeline@v0.9.0 · 5686 in / 985 out tokens · 35603 ms · 2026-05-20T15:15:24.072573+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages

  1. [1]

    World Energy Outlook 2025,

    International Energy Agency, “World Energy Outlook 2025,” https://www.iea.org/reports/world-energy-out look-2025, Nov. 2025

    work page 2025

  2. [2]

    The impact of renewable energy forecast errors on imbalance volumes and electricity spot prices,

    S. Goodarzi, H. N. Perera, and D. Bunn, “The impact of renewable energy forecast errors on imbalance volumes and electricity spot prices,”Energy Policy, vol. 134, p. 110827, Nov. 2019

    work page 2019

  3. [3]

    Predicting and publish- ing accurate imbalance prices using Monte Carlo Tree Search,

    F. Pavirani, J. Van Gompel, S. S. Karimi Madahi, B. Claessens, and C. Develder, “Predicting and publish- ing accurate imbalance prices using Monte Carlo Tree Search,”Applied Energy, vol. 392, p. 125944, Aug. 2025

    work page 2025

  4. [4]

    Strategic Implicit Balancing With Energy Storage Systems via Stochastic Model Predictive Con- trol,

    R. Smets, K. Bruninx, J. Bottieau, J.-F. Toubeau, and E. Delarue, “Strategic Implicit Balancing With Energy Storage Systems via Stochastic Model Predictive Con- trol,”Policy and Regulation IEEE Transactions on En- ergy Markets, vol. 1, no. 4, pp. 373–385, Dec. 2023

    work page 2023

  5. [5]

    Machine learning advances for time series forecasting,

    R. P. Masini, M. C. Medeiros, and E. F. Mendes, “Machine learning advances for time series forecasting,” Journal of Economic Surveys, vol. 37, no. 1, pp. 76–111, 2023

    work page 2023

  6. [6]

    Seasonality in deep learning forecasts of electricity im- balance prices,

    S. Deng, J. Inekwe, V . Smirnov, A. Wait, and C. Wang, “Seasonality in deep learning forecasts of electricity im- balance prices,”Energy Economics, vol. 137, p. 107770, Sep. 2024

    work page 2024

  7. [7]

    Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open- access benchmark,

    J. Lago, G. Marcjasz, B. De Schutter, and R. Weron, “Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open- access benchmark,”Applied Energy, vol. 293, p. 116983, Jul. 2021

    work page 2021

  8. [8]

    Bridg- ing Accuracy and Explainability in Electricity Price Forecasting,

    M. M. Mascarenhas, M. Amelin, and H. Kazmi, “Bridg- ing Accuracy and Explainability in Electricity Price Forecasting,” in2024 20th International Conference on the European Energy Market (EEM), 2024, pp. 1–6

    work page 2024

  9. [9]

    Leveraging asynchronous cross-border mar- ket data for improved day-ahead electricity price fore- casting in european markets,

    M. M. Mascarenhas, J. De Blauwe, M. Amelin, and H. Kazmi, “Leveraging asynchronous cross-border mar- ket data for improved day-ahead electricity price fore- casting in european markets,”Applied Energy, vol. 404, p. 127077, 2026

    work page 2026

  10. [10]

    Imbalance settlement harmonisation – Informal workshop pursuant the EBGL art. 52(2),

    ENTSO-E, “Imbalance settlement harmonisation – Informal workshop pursuant the EBGL art. 52(2),” ENTSO-E, Tech. Rep., Mar. 2018. [Online]. Available: https://eepublicdownloads.entsoe.eu/clean-documents/e vents/2018/20180323-Workshop imbalance settlemen t-Summary.pdf

    work page 2018

  11. [11]

    Commission Regulation (eu) 2017/2195,

    European Commission, “Commission Regulation (eu) 2017/2195,” https://eur-lex.europa.eu/legal-content/E N/TXT/?uri=CELEX%3A02017R2195-20220619, Nov. 2017

    work page 2017

  12. [12]

    Very-Short-Term Probabilistic Forecasting for a Risk-Aware Participation in the Single Price Imbal- ance Settlement,

    J. Bottieau, L. Hubert, Z. De Gr `eve, F. Vall ´ee, and J.- F. Toubeau, “Very-Short-Term Probabilistic Forecasting for a Risk-Aware Participation in the Single Price Imbal- ance Settlement,”IEEE Transactions on Power Systems, vol. 35, no. 2, pp. 1218–1230, Mar. 2020

    work page 2020

  13. [13]

    A Review of Balancing Price Forecasting in the Context of Renewable-Rich Power Systems, High- lighting Profit-Aware and Spike-Resilient Approaches,

    A. Dinler, “A Review of Balancing Price Forecasting in the Context of Renewable-Rich Power Systems, High- lighting Profit-Aware and Spike-Resilient Approaches,” Energies, vol. 18, no. 24, p. 6460, Jan. 2025, publisher: Multidisciplinary Digital Publishing Institute

    work page 2025

  14. [14]

    A Review of Electricity Price Forecasting Models in the Day-Ahead, Intra-Day, and Balancing Markets,

    C. O’Connor, M. Bahloul, S. Prestwich, and A. Visentin, “A Review of Electricity Price Forecasting Models in the Day-Ahead, Intra-Day, and Balancing Markets,”En- ergies, vol. 18, no. 12, p. 3097, Jan. 2025, publisher: Multidisciplinary Digital Publishing Institute

    work page 2025

  15. [15]

    Balancing markets and imbalance forecasting: A pathway towards net zero grids,

    C. Singh, S. Sreekumar, and T. Malakar, “Balancing markets and imbalance forecasting: A pathway towards net zero grids,”Applied Energy, vol. 402, p. 126976, Jan. 2026

    work page 2026

  16. [16]

    Interpretable transformer model for capturing regime switching effects of real-time electricity prices,

    J. Bottieau, Y . Wang, Z. De Greve, F. Vallee, and J.-F. Toubeau, “Interpretable transformer model for capturing regime switching effects of real-time electricity prices,” IEEE Transactions on Power Systems, vol. 38, no. 3, pp. 2162–2176, 2022

    work page 2022

  17. [17]

    Predictions of prices and volumes in the Nordic balancing markets for electricity,

    S. Backe, S. Riemer-Sørensen, D. Bordvik, S. Tiwari, and C. Andresen, “Predictions of prices and volumes in the Nordic balancing markets for electricity,” 06 2023, pp. 1–6

    work page 2023

  18. [18]

    Value-oriented price forecasting for arbi- trage strategies of Energy Storage Systems through loss function tuning,

    R. Smets, J.-F. Toubeau, M. Dolanyi, K. Bruninx, and E. Delarue, “Value-oriented price forecasting for arbi- trage strategies of Energy Storage Systems through loss function tuning,”Energy, vol. 333, p. 137112, Oct. 2025

    work page 2025

  19. [19]

    Forecasting Imbalance Price Densities With Statistical Methods and Neural Networks,

    V . N. Ganesh and D. W. Bunn, “Forecasting Imbalance Price Densities With Statistical Methods and Neural Networks,”Policy and Regulation IEEE Transactions on Energy Markets, vol. 2, no. 1, pp. 30–39, Mar. 2024

    work page 2024

  20. [20]

    Probabilistic Forecasting and Curtailment-Aware Trad- ing in the Dutch Balancing Market,

    M. Dol, R. Dekker, E. Cigdem Karakoyun, and P. Wan, “Probabilistic Forecasting and Curtailment-Aware Trad- ing in the Dutch Balancing Market,” Rochester, NY , Aug. 2025

    work page 2025

  21. [21]

    Probabilistic Forecasting of German Electricity Imbalance Prices,

    M. Narajewski, “Probabilistic Forecasting of German Electricity Imbalance Prices,”Energies, vol. 15, no. 14, p. 4976, Jan. 2022

    work page 2022

  22. [22]

    Probabilistic Forecasting of Imbalance Prices in the Belgian Context,

    J. Dumas, I. Boukas, M. M. de Villena, S. Mathieu, and B. Corn ´elusse, “Probabilistic Forecasting of Imbalance Prices in the Belgian Context,” in2019 16th International Conference on the European Energy Market (EEM), Sep. 2019, pp. 1–7

    work page 2019

  23. [23]

    Modeling Real-Time Balancing Power Market Prices Using Combined SARIMA and Markov Processes,

    M. Olsson and L. S ¨oder, “Modeling Real-Time Balancing Power Market Prices Using Combined SARIMA and Markov Processes,”IEEE Transactions on Power Sys- tems, vol. 23, no. 2, pp. 443–450, May 2008

    work page 2008

  24. [24]

    Mod- elling of prices using the volume in the Norwegian regulating power market,

    S. Jaehnert, H. Farahmand, and G. L. Doorman, “Mod- elling of prices using the volume in the Norwegian regulating power market,” in2009 IEEE Bucharest Pow- erTech, Jun. 2009, pp. 1–7

    work page 2009

  25. [25]

    Modeling Swedish real- time balancing power prices using nonlinear time series models,

    M. O. Brolin and L. S ¨oder, “Modeling Swedish real- time balancing power prices using nonlinear time series models,” in2010 IEEE 11th International Conference on Probabilistic Methods Applied to Power Systems, Jun. 2010, pp. 358–363

    work page 2010

  26. [26]

    Bench- marking time series based forecasting models for elec- tricity balancing market prices,

    G. Klæboe, A. L. Eriksrud, and S.-E. Fleten, “Bench- marking time series based forecasting models for elec- tricity balancing market prices,”Energy Systems, vol. 6, no. 1, pp. 43–61, 2015

    work page 2015

  27. [27]

    Exponential Smoothing Approaches for Prediction in Real-Time Electricity Markets,

    T. J ´onsson, P. Pinson, H. A. Nielsen, and H. Madsen, “Exponential Smoothing Approaches for Prediction in Real-Time Electricity Markets,”Energies, vol. 7, no. 6, pp. 3710–3732, Jun. 2014

    work page 2014

  28. [28]

    Fore- casting balancing market prices using Hidden Markov Models,

    I. Dimoulkas, M. Amelin, and M. R. Hesamzadeh, “Fore- casting balancing market prices using Hidden Markov Models,” in2016 13th International Conference on the European Energy Market (EEM). IEEE, 2016, pp. 1–5

    work page 2016

  29. [29]

    Anal- ysis of the Fundamental Predictability of Prices in the British Balancing Market,

    D. W. Bunn, J. N. Inekwe, and D. MacGeehan, “Anal- ysis of the Fundamental Predictability of Prices in the British Balancing Market,”IEEE Transactions on Power Systems, vol. 36, no. 2, pp. 1309–1316, Mar. 2021

    work page 2021

  30. [30]

    Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression,

    A. Lucas, K. Pegios, E. Kotsakis, and D. Clarke, “Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression,”Energies, vol. 13, no. 20, p. 5420, Jan. 2020

    work page 2020

  31. [31]

    Bayesian Predictive Distributions for Imbalance Prices With Time- Varying Factor Impacts,

    L. M. Lima, P. Damien, and D. W. Bunn, “Bayesian Predictive Distributions for Imbalance Prices With Time- Varying Factor Impacts,”IEEE Transactions on Power Systems, vol. 38, no. 1, pp. 349–357, Jan. 2023

    work page 2023

  32. [32]

    Predicting Electricity Imbal- ance Prices and V olumes: Capabilities and Opportuni- ties,

    J. Browell and C. Gilbert, “Predicting Electricity Imbal- ance Prices and V olumes: Capabilities and Opportuni- ties,”Energies, vol. 15, no. 10, p. 3645, Jan. 2022

    work page 2022

  33. [33]

    Predicting Energy Market Imbalance Prices with Random Forest: A Rolling Window Approach,

    A. C. Makrides, S. Loizidis, C. Charalambous, G. Makrides, and G. E. Georghiou, “Predicting Energy Market Imbalance Prices with Random Forest: A Rolling Window Approach,” in2024 3rd International Confer- ence on Energy Transition in the Mediterranean Area (SyNERGY MED), Oct. 2024, pp. 1–5

    work page 2024

  34. [34]

    Conformal Prediction for electricity price forecasting in the day-ahead and real-time balancing market,

    C. O’Connor, M. Bahloul, R. Rossi, S. Prestwich, and A. Visentin, “Conformal Prediction for electricity price forecasting in the day-ahead and real-time balancing market,”Energy and AI, vol. 21, p. 100571, Sep. 2025

    work page 2025

  35. [35]

    Prediction of Imbalance Prices through Gradient Boosting Algorithms: An Application to the Greek Balancing Market,

    K. Plakas, N. Andriopoulos, D. Papadaskalopoulos, A. Birbas, E. Housos, and I. Moraitis, “Prediction of Imbalance Prices through Gradient Boosting Algorithms: An Application to the Greek Balancing Market,”IEEE Access, 2025

    work page 2025

  36. [36]

    How good are TSO load and renewable generation forecasts: Learning curves, chal- lenges, and the road ahead,

    H. Kazmi and Z. Tao, “How good are TSO load and renewable generation forecasts: Learning curves, chal- lenges, and the road ahead,”Applied Energy, vol. 323, p. 119565, 2022

    work page 2022

  37. [37]

    Logic- based explanations of imbalance price forecasts using boosted trees,

    J. Bottieau, G. Audemard, S. Bellart, J.-M. Lagniez, P. Marquis, N. Szczepanski, and J.-F. Toubeau, “Logic- based explanations of imbalance price forecasts using boosted trees,”Electric Power Systems Research, vol. 235, p. 110699, 2024

    work page 2024