REVIEW 5 major objections 9 minor 12 references
Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market
T0 review · 5 major / 9 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A Transformer model that classifies ERCOT's day-ahead/real-time price spread into five ranges can drive a virtual-bidding strategy trading only at hour 19 that earns nearly consistent profit over a ten-month test period.
desk verdict A sensible ERCOT spread-forecasting setup and a useful profit-based evaluation idea, undermined by a test-period-selected trading rule that makes the headline profit in-sample. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a 24-hour-ahead probability distribution over five price-spread classes for each hour of day T+1, with boundaries at -12, -5, +5, and +12 dollars chosen so the inner pair reflects the $5/MWh uplift cost. A Transformer encoder with positional encoding maps the previous days' load, solar, wind, and calendar features to these probabilities. The mechanism that carries the profitability is the hour-19 precision: only at that peak hour does the model's precision for the lowest spread class reach or exceed 50%, and the trading strategy converts that precision into 1 MWh virtual bids placed at the price cap or floor, limited to one trade per day and assuming bids always clear.
What would settle it
Re-run the same weekly walk-forward procedure on the next full year of ERCOT data with hour 19, the five spread classes, and the $5 uplift cost fixed in advance; if cumulative net profit is not consistently positive over that year, the claim of nearly consistent profit from this strategy is falsified. A second check would be to examine actual ERCOT bid-clearing outcomes for cap/floor virtual bids to see whether any go unfilled, which would invalidate the backtest's fills.
Extended reading notes
Core claim
The paper's central claim is that a single-hour trading rule built on a five-class spread forecast is profitable in ERCOT's volatile two-settlement market. At hour 19 the model's precision for the class where SCED price is more than $12 below DAM price reaches 50% or better, and acting on those hourly predictions produces positive cumulative net profit over the test period. The paper further finds that the most profitable model—weekly fine-tuned with a 90-day lookback—does not have the best accuracy or F1 score, so conventional classification metrics mis-rank models for trading use and should be supplemented with an end-to-end profit evaluation.
Load-bearing premise
The reported profit rests on the premise that virtual bids placed at the price cap or floor always clear without moving the market price, and that the choice to trade only at hour 19 was not itself made by looking at the test-period profit data.
Editorial extensions
If this is right
- A virtual bidder following the hour-19 strategy with the weekly fine-tuned model earns the highest cumulative profit among the seven tested strategies over the ten-month test period.
- Trading at all hours, or filtering by predicted probability, earns less than focusing on the single peak hour, so spread-forecast value is concentrated where the market's daily pattern is most predictable.
- The best classifier by accuracy or F1 is not the best by profit, so evaluating forecast models with end-to-end backtests is necessary for virtual-bidding applications.
- Weekly fine-tuning with a short 90-day lookback outperforms training on all historical data under the profit criterion, suggesting that adapting to recent market conditions matters more than raw data volume.
Reading between the lines
- Beyond the paper: the choice of hour 19 and the strategy rules was made after inspecting the test-period confusion matrix and profit curves, so the reported profit is not a fully out-of-sample measure; a pre-registered replication on a later year is needed to confirm the strategy earns positive returns when the trading hour is fixed before seeing results.
- Beyond the paper: if the hour-19 pattern persists, the model could be turned into a risk-managed trading rule that sizes bids by predicted probability rather than trading a fixed 1 MWh each day, which the paper does not test.
- Beyond the paper: because virtual bidding is supposed to align day-ahead and real-time prices, a reliably profitable spread forecaster would, if widely used, erode the very spreads it predicts; the strategy's edge may not be stable as more participants adopt similar models.
- Beyond the paper: the outer thresholds at plus or minus $12 are data-dependent, so a testable extension is sensitivity analysis on those thresholds to see whether hour-19 profitability survives if the outer band is moved to $10 or $15.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Transformer-based classifier that maps ERCOT exogeneous features (load, solar, wind forecasts, calendar variables) to the next-day SCED-minus-DAM price-spread category among five bins, with weekly walk-forward retraining and a test period of Jan. 1 to Oct. 20, 2024. Seven virtual-bidding strategies (T1-T7) are backtested; the authors report that trading only at the peak hour (hour 19) with a precision score over 50% yields nearly consistent profit (Table I, Fig. 5), with T6/T7 serving as ground-truth upper bounds. The paper concludes that evaluating the forecast model by cumulative virtual-bidding profit is a valuable addition to conventional classification metrics.
Significance. The paper has genuine strengths: the weekly walk-forward retraining and the explicit one-day gap between validation and test respect bid-submission timing, and evaluating the model by net trading profit rather than by accuracy alone is a sensible task-aligned choice. The limitation section is candid about missing features and about the Aug. 20, 2024 misprediction. If the profitability claim were valid, this would be a useful case study of Transformer-based spread forecasting for ERCOT virtual bidding, a relatively under-studied market. However, the headline result is not a valid out-of-sample estimate: the trading hour, the probability threshold, and the 'best model' are selected from the test-period confusion matrix and the test-period profit curves, and the evaluation lacks baselines and uncertainty quantification. The central claim as stated is therefore not established by the reported experiments.
major comments (5)
- [III-A, Fig. 4, Table I] Section III-A selects hour 19 by inspecting the test-period hourly confusion matrix ('Based on the analysis above, hour 19 is selected'), and the precision-above-50% justification is read from that same figure. Because the test period (Jan. 1 to Oct. 20, 2024) is used both to choose among 24 hours, seven models (A-G), and several strategy variants and to report the cumulative profits in Table I and Fig. 5, the reported 'nearly consistent profit' is an in-sample fitted quantity rather than an out-of-sample estimate. The walk-forward procedure retrains only the network weights; it does not constrain the strategy selection. A valid evaluation would pre-specify the hour, the trading rule, and the probability threshold before observing 2024 data, or select them on data ending before Jan. 1, 2024, and then report profit on a fresh holdout.
- [III-B, Fig. 5] All profitability conclusions rest on a single ten-month cumulative-profit path with no confidence intervals, bootstrap resampling, or alternative test windows. The series is necessarily autocorrelated, and the paper itself reports a 'significant dip on Aug. 20, 2024' causing substantial losses, which is in tension with the abstract's 'nearly consistent profit.' With one path, one market, and post-hoc strategy selection, the superiority of strategy T3/T5 over T1/T2 is not statistically supported.
- [Table I, III-A] No baseline forecast is reported: the paper gives no persistence forecast, majority-class accuracy, LSTM or ARIMA comparison, or simple threshold rule on the raw spread, so the accuracy values (0.496-0.508) and the profit differences across models A-G cannot be attributed to the Transformer architecture. Moreover, a precision slightly above 50% for one class at one hour is not by itself evidence of profitability, because with a $5 uplift cost, losses on misclassified tail hours can exceed gains on correct tail hours; the paper should report the conditional per-trade P&L distribution. The gap between the best model-based hour-19 profit (T3 = 2482) and the ground-truth hour-19 profit (T7 = 14957) further shows that the backtest result is dominated by the assumed perfect foresight at hour 19.
- [II-D] The assumption that 'bids and offers always get cleared by placing the price at price cap or price floor' is load-bearing for converting forecast accuracy into profit, but it is not verified for ERCOT; ERCOT's virtual-bid screening and portfolio rules do not guarantee full clearing of cap/floor virtual bids, and the cited support is a PhD dissertation rather than ERCOT settlement documentation. The reported profits should be treated as an upper bound, and the paper should either justify the assumption with ERCOT market rules or provide a sensitivity analysis under partial clearing.
- [III-B] The strategy definitions in Section III-B are underspecified: for T1-T5 the paper does not state which predicted spread classes trigger a buy versus a sell virtual transaction, how the central class [-5,5) is handled, or the exact profit equation per trade (including the $5 uplift cost). Without this mapping, the cumulative profit numbers in Table I and Fig. 5 are not reproducible from the text alone.
minor comments (9)
- [Abstract] The phrase 'nearly consistent profit over the test period' overstates the results, since Section III-B reports a substantial loss on Aug. 20, 2024; 'positive cumulative profit' would be a more accurate summary.
- [II-A] The description of lagging is confusing: with lagging=2 the text says the inputs include days T-1, T, and T+1, but T+1 is the prediction target, so the sentence should clarify which features are known at bid-submission time.
- [II-A] The notation contrasting 'hour 0:00' with 'Hour Ending 1:00' should be defined once in the model formulation and used consistently throughout the paper.
- [II-B] The $12 threshold is described as 'about the median of the price spreads that is outside $5.00,' but the data window used to compute this median is not stated; if it includes 2024 data, this is an additional test-set-dependent choice.
- [II-D] 'Uplist cost' appears to be a typo for 'uplift cost.'
- [Fig. 4 caption] The caption 'Confusion matrix of the best model for each hour' does not identify which model among A-G is shown or whether the hourly matrices are aggregated over the weekly retraining windows; the text refers to model G, and the caption should match.
- [Table I] The abbreviations in the column headers (L, S, F, T1-T7) are defined only in footnotes placed after the table body; the footnote placement is easy to miss, and the table would benefit from an inline legend.
- [Data availability] The paper would benefit from a data/code availability statement and precise references for the ERCOT load, solar, and wind forecast products, with retrieval dates, since reproducibility of the profit numbers otherwise depends on unpublished details.
- [IV] Section IV lists sensible limitations, but the discussion of the Aug. 20, 2024 loss contradicts the abstract's 'nearly consistent profit' claim; the conclusion should be reworded to match the evidence.
Circularity Check
The headline profit is an in-sample selection: hour 19, the 50% precision rule, and model G are chosen from the test-period confusion matrix and profit table before being reported as the validated strategy.
-
fitted input called prediction
[Section III-A-B and Abstract]
"The results showed that for most hours the model predicted the correct class... except hour 19, where the precision for the class (−inf, −12) reaches or exceeds 50%. ... Based on the analysis above, hour 19 is selected to build most of the trading strategies... the most profitable strategy is from model G. ... The results show that the strategy of trading only at the peak hour with a precision score of over 50% produces nearly consistent profit over the test period."
The strategy parameters are fitted to the test set rather than fixed before it. Hour 19 is chosen because the test-period hourly confusion matrix (Fig. 4) shows precision above 50% at that hour, and the 50% threshold is an ex post reading of the same matrix; model G is reported because Table I, computed on the test period, shows it as the most profitable. The abstract then presents the profit of this selected hour/threshold/model as the paper's result. Walk-forward retraining only refreshes network weights; it does not constrain hour, threshold, strategy, or model selection. The reported 'nearly consistent profit' is therefore a maximum over many test-set comparisons (24 hours, 7 strategies, 7 model variants), not the out-of-sample profit of a pre-specified rule.
full rationale
The classification model itself is not circularly derived: its inputs (load, solar, wind, temporal features, historical price spread) do not encode the target spread labels, and the weekly walk-forward retraining is an appropriate leakage control for the network weights. The clearing-at-cap/floor assumption is imported from an external dissertation [9], not from the present authors, so it is a modeling assumption rather than a self-citation. The central profitability claim, however, is not independently grounded: hour 19, the precision>50% rule, and model G were all selected after examining the test-period confusion matrix and profit table, so the cumulative profit reported in Table I and Fig. 5 contains an in-sample selection component. This is a fitted-input-called-prediction pattern rather than a definitional identity, so the score is 6 rather than higher; the accuracy/precision evaluation of the classifier remains a valid walk-forward measurement, but the headline trading result does not.
Assumptions & free parameters
free parameters (4)
- Bin edges for spread classes =
-12, -5, 5, 12 USD/MWh
- Hour 19 strategy selection =
19
- Transformer hyperparameters (layers, dropout, learning rate, weight decay) =
not reported
- Prediction probability threshold for T5 =
0.50
assumptions (5)
- domain assumption ERCOT published load, solar, wind, and price-spread data used for training and evaluation are accurate and consistent.
- domain assumption Virtual bids clear whenever placed at the price cap or floor.
- domain assumption Bidding 1 MWh per day does not change the system lambda (price).
- domain assumption The five fixed spread ranges adequately capture the decision-relevant structure for virtual bidding.
- domain assumption A fixed $5/MWh uplift cost applies to all trades.
Cite this review
Pith. "Pith review of Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market." pith.science (2026). https://pith.science/paper/O7DW2S3U
@misc{pith2026241200062,
author = {Pith},
title = {Pith review of: Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market},
year = {2026},
howpublished = {\url{https://pith.science/paper/O7DW2S3U}},
note = {Machine review of arXiv:2412.00062}
}
read the original abstract
Virtual bidding plays an important role in two-settlement electric power markets, as it can reduce discrepancies between day-ahead and real-time markets. Renewable energy penetration increases volatility in electricity prices, making accurate forecasting critical for virtual bidders, reducing uncertainty and maximizing profits. This study presents a Transformer-based deep learning model to forecast the price spread between real-time and day-ahead electricity prices in the ERCOT (Electric Reliability Council of Texas) market. The proposed model leverages various time-series features, including load forecasts, solar and wind generation forecasts, and temporal attributes. The model is trained under realistic constraints and validated using a walk-forward approach by updating the model every week. Based on the price spread prediction results, several trading strategies are proposed and the most effective strategy for maximizing cumulative profit under realistic market conditions is identified through backtesting. The results show that the strategy of trading only at the peak hour with a precision score of over 50% produces nearly consistent profit over the test period. The proposed method underscores the importance of an accurate electricity price forecasting model and introduces a new method of evaluating the price forecast model from a virtual bidder's perspective, providing valuable insights for future research.
Figures
Reference graph
Works this paper leans on
-
[1]
11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...
- [2]
-
[3]
Z. Zhao, C. Wang, M. Nokleby, and C. J. Miller, ``Improving short-term electricity price forecasting using day-ahead lmp with arima models,'' in 2017 IEEE Power & Energy Society General Meeting. 1em plus 0.5em minus 0.4em IEEE, 2017, pp. 1--5
work page 2017
-
[4]
M. Pavi \'c evi \'c and T. Popovi \'c , ``Forecasting day-ahead electricity metrics with artificial neural networks,'' Sensors, vol. 22, no. 3, p. 1051, 2022
work page 2022
-
[5]
J. Trebbien, S. P \"u tz, B. Sch \"a fer, H. S. Nyg rd, L. R. Gorj \ a o, and D. Witthaut, ``Probabilistic forecasting of day-ahead electricity prices and their volatility with lstms,'' in 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE). 1em plus 0.5em minus 0.4em IEEE, 2023, pp. 1--5
work page 2023
-
[6]
K. Rawal and A. Ahmad, ``Day-ahead market electricity price prediction using time series forecasting,'' in 2022 1st International Conference on Sustainable Technology for Power and Energy Systems (STPES). 1em plus 0.5em minus 0.4em IEEE, 2022, pp. 1--6
work page 2022
-
[7]
R. Galarneau-Vincent, G. Gauthier, and F. Godin, ``Foreseeing the worst: Forecasting electricity dart spikes,'' Energy Economics, vol. 119, p. 106521, 2023
work page 2023
-
[8]
W. Wang and N. Yu, ``A machine learning framework for algorithmic trading with virtual bids in electricity markets,'' in 2019 IEEE power & energy society general meeting (PESGM). 1em plus 0.5em minus 0.4em IEEE, 2019, pp. 1--5
work page 2019
Show all 12 references
-
[9]
R. Das, R. Bo, H. Chen, W. U. Rehman, and D. Wunsch, ``Forecasting nodal price difference between day-ahead and real-time electricity markets using long-short term memory and sequence-to-sequence networks,'' IEEE Access, vol. 10, pp. 832--843, 2021
2021
-
[10]
Li, ``Data-driven modeling and algorithmic trading in electricity market,'' Ph.D
Y. Li, ``Data-driven modeling and algorithmic trading in electricity market,'' Ph.D. dissertation, UC Riverside, 2024
2024
-
[11]
Y. Li, N. Yu, and W. Wang, ``Machine learning-driven virtual bidding with electricity market efficiency analysis,'' IEEE Transactions on Power Systems, vol. 37, no. 1, pp. 354--364, 2021
2021
-
[12]
R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, and I. Stoica, ``Tune: A research platform for distributed model selection and training,'' arXiv preprint arXiv:1807.05118, 2018
2018 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.