Pith. sign in

REVIEW 5 major objections 9 minor 12 references

Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market

T0 review · 5 major / 9 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read A Transformer model that classifies ERCOT's day-ahead/real-time price spread into five ranges can drive a virtual-bidding strategy trading only at hour 19 that earns nearly consistent profit over a ten-month test period.

desk verdict A sensible ERCOT spread-forecasting setup and a useful profit-based evaluation idea, undermined by a test-period-selected trading rule that makes the headline profit in-sample. read the letter →

arxiv 2412.00062 v1 pith:O7DW2S3U submitted 2024-11-25 cs.LG q-fin.CP

classification cs.LGq-fin.CP
keywords virtualbiddingelectricitypriceforecastingERCOTTransformerspreadwalk-forwardvalidationtradingstrategydeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Virtual bidding profits from the gap between day-ahead and real-time electricity prices, and this paper tries to show that a Transformer-based classifier can forecast that gap well enough to trade on it in ERCOT. The model predicts the SCED-minus-DAM spread into five ranges for each hour of the next day, using load, solar, and wind forecasts and calendar features, and it is retrained every week. The authors backtest seven trading strategies and report that trading only at hour 19, where precision for the strongly negative spread class exceeds 50%, yields nearly consistent cumulative profit after a $5/MWh uplift cost over January–October 2024. The broader point is that forecast models for virtual bidding should be judged by realized trading profit, not by classification accuracy alone.

What carries the argument

The central object is a 24-hour-ahead probability distribution over five price-spread classes for each hour of day T+1, with boundaries at -12, -5, +5, and +12 dollars chosen so the inner pair reflects the $5/MWh uplift cost. A Transformer encoder with positional encoding maps the previous days' load, solar, wind, and calendar features to these probabilities. The mechanism that carries the profitability is the hour-19 precision: only at that peak hour does the model's precision for the lowest spread class reach or exceed 50%, and the trading strategy converts that precision into 1 MWh virtual bids placed at the price cap or floor, limited to one trade per day and assuming bids always clear.

What would settle it

Re-run the same weekly walk-forward procedure on the next full year of ERCOT data with hour 19, the five spread classes, and the $5 uplift cost fixed in advance; if cumulative net profit is not consistently positive over that year, the claim of nearly consistent profit from this strategy is falsified. A second check would be to examine actual ERCOT bid-clearing outcomes for cap/floor virtual bids to see whether any go unfilled, which would invalidate the backtest's fills.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single-hour trading rule built on a five-class spread forecast is profitable in ERCOT's volatile two-settlement market. At hour 19 the model's precision for the class where SCED price is more than $12 below DAM price reaches 50% or better, and acting on those hourly predictions produces positive cumulative net profit over the test period. The paper further finds that the most profitable model—weekly fine-tuned with a 90-day lookback—does not have the best accuracy or F1 score, so conventional classification metrics mis-rank models for trading use and should be supplemented with an end-to-end profit evaluation.

Load-bearing premise

The reported profit rests on the premise that virtual bids placed at the price cap or floor always clear without moving the market price, and that the choice to trade only at hour 19 was not itself made by looking at the test-period profit data.

Editorial extensions

If this is right

  • A virtual bidder following the hour-19 strategy with the weekly fine-tuned model earns the highest cumulative profit among the seven tested strategies over the ten-month test period.
  • Trading at all hours, or filtering by predicted probability, earns less than focusing on the single peak hour, so spread-forecast value is concentrated where the market's daily pattern is most predictable.
  • The best classifier by accuracy or F1 is not the best by profit, so evaluating forecast models with end-to-end backtests is necessary for virtual-bidding applications.
  • Weekly fine-tuning with a short 90-day lookback outperforms training on all historical data under the profit criterion, suggesting that adapting to recent market conditions matters more than raw data volume.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the choice of hour 19 and the strategy rules was made after inspecting the test-period confusion matrix and profit curves, so the reported profit is not a fully out-of-sample measure; a pre-registered replication on a later year is needed to confirm the strategy earns positive returns when the trading hour is fixed before seeing results.
  • Beyond the paper: if the hour-19 pattern persists, the model could be turned into a risk-managed trading rule that sizes bids by predicted probability rather than trading a fixed 1 MWh each day, which the paper does not test.
  • Beyond the paper: because virtual bidding is supposed to align day-ahead and real-time prices, a reliably profitable spread forecaster would, if widely used, erode the very spreads it predicts; the strategy's edge may not be stable as more participants adopt similar models.
  • Beyond the paper: the outer thresholds at plus or minus $12 are data-dependent, so a testable extension is sensitivity analysis on those thresholds to see whether hour-19 profitability survives if the outer band is moved to $10 or $15.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 9 minor

Summary. The paper proposes a Transformer-based classifier that maps ERCOT exogeneous features (load, solar, wind forecasts, calendar variables) to the next-day SCED-minus-DAM price-spread category among five bins, with weekly walk-forward retraining and a test period of Jan. 1 to Oct. 20, 2024. Seven virtual-bidding strategies (T1-T7) are backtested; the authors report that trading only at the peak hour (hour 19) with a precision score over 50% yields nearly consistent profit (Table I, Fig. 5), with T6/T7 serving as ground-truth upper bounds. The paper concludes that evaluating the forecast model by cumulative virtual-bidding profit is a valuable addition to conventional classification metrics.

Significance. The paper has genuine strengths: the weekly walk-forward retraining and the explicit one-day gap between validation and test respect bid-submission timing, and evaluating the model by net trading profit rather than by accuracy alone is a sensible task-aligned choice. The limitation section is candid about missing features and about the Aug. 20, 2024 misprediction. If the profitability claim were valid, this would be a useful case study of Transformer-based spread forecasting for ERCOT virtual bidding, a relatively under-studied market. However, the headline result is not a valid out-of-sample estimate: the trading hour, the probability threshold, and the 'best model' are selected from the test-period confusion matrix and the test-period profit curves, and the evaluation lacks baselines and uncertainty quantification. The central claim as stated is therefore not established by the reported experiments.

major comments (5)
  1. [III-A, Fig. 4, Table I] Section III-A selects hour 19 by inspecting the test-period hourly confusion matrix ('Based on the analysis above, hour 19 is selected'), and the precision-above-50% justification is read from that same figure. Because the test period (Jan. 1 to Oct. 20, 2024) is used both to choose among 24 hours, seven models (A-G), and several strategy variants and to report the cumulative profits in Table I and Fig. 5, the reported 'nearly consistent profit' is an in-sample fitted quantity rather than an out-of-sample estimate. The walk-forward procedure retrains only the network weights; it does not constrain the strategy selection. A valid evaluation would pre-specify the hour, the trading rule, and the probability threshold before observing 2024 data, or select them on data ending before Jan. 1, 2024, and then report profit on a fresh holdout.
  2. [III-B, Fig. 5] All profitability conclusions rest on a single ten-month cumulative-profit path with no confidence intervals, bootstrap resampling, or alternative test windows. The series is necessarily autocorrelated, and the paper itself reports a 'significant dip on Aug. 20, 2024' causing substantial losses, which is in tension with the abstract's 'nearly consistent profit.' With one path, one market, and post-hoc strategy selection, the superiority of strategy T3/T5 over T1/T2 is not statistically supported.
  3. [Table I, III-A] No baseline forecast is reported: the paper gives no persistence forecast, majority-class accuracy, LSTM or ARIMA comparison, or simple threshold rule on the raw spread, so the accuracy values (0.496-0.508) and the profit differences across models A-G cannot be attributed to the Transformer architecture. Moreover, a precision slightly above 50% for one class at one hour is not by itself evidence of profitability, because with a $5 uplift cost, losses on misclassified tail hours can exceed gains on correct tail hours; the paper should report the conditional per-trade P&L distribution. The gap between the best model-based hour-19 profit (T3 = 2482) and the ground-truth hour-19 profit (T7 = 14957) further shows that the backtest result is dominated by the assumed perfect foresight at hour 19.
  4. [II-D] The assumption that 'bids and offers always get cleared by placing the price at price cap or price floor' is load-bearing for converting forecast accuracy into profit, but it is not verified for ERCOT; ERCOT's virtual-bid screening and portfolio rules do not guarantee full clearing of cap/floor virtual bids, and the cited support is a PhD dissertation rather than ERCOT settlement documentation. The reported profits should be treated as an upper bound, and the paper should either justify the assumption with ERCOT market rules or provide a sensitivity analysis under partial clearing.
  5. [III-B] The strategy definitions in Section III-B are underspecified: for T1-T5 the paper does not state which predicted spread classes trigger a buy versus a sell virtual transaction, how the central class [-5,5) is handled, or the exact profit equation per trade (including the $5 uplift cost). Without this mapping, the cumulative profit numbers in Table I and Fig. 5 are not reproducible from the text alone.
minor comments (9)
  1. [Abstract] The phrase 'nearly consistent profit over the test period' overstates the results, since Section III-B reports a substantial loss on Aug. 20, 2024; 'positive cumulative profit' would be a more accurate summary.
  2. [II-A] The description of lagging is confusing: with lagging=2 the text says the inputs include days T-1, T, and T+1, but T+1 is the prediction target, so the sentence should clarify which features are known at bid-submission time.
  3. [II-A] The notation contrasting 'hour 0:00' with 'Hour Ending 1:00' should be defined once in the model formulation and used consistently throughout the paper.
  4. [II-B] The $12 threshold is described as 'about the median of the price spreads that is outside $5.00,' but the data window used to compute this median is not stated; if it includes 2024 data, this is an additional test-set-dependent choice.
  5. [II-D] 'Uplist cost' appears to be a typo for 'uplift cost.'
  6. [Fig. 4 caption] The caption 'Confusion matrix of the best model for each hour' does not identify which model among A-G is shown or whether the hourly matrices are aggregated over the weekly retraining windows; the text refers to model G, and the caption should match.
  7. [Table I] The abbreviations in the column headers (L, S, F, T1-T7) are defined only in footnotes placed after the table body; the footnote placement is easy to miss, and the table would benefit from an inline legend.
  8. [Data availability] The paper would benefit from a data/code availability statement and precise references for the ERCOT load, solar, and wind forecast products, with retrieval dates, since reproducibility of the profit numbers otherwise depends on unpublished details.
  9. [IV] Section IV lists sensible limitations, but the discussion of the Aug. 20, 2024 loss contradicts the abstract's 'nearly consistent profit' claim; the conclusion should be reworded to match the evidence.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline profit is an in-sample selection: hour 19, the 50% precision rule, and model G are chosen from the test-period confusion matrix and profit table before being reported as the validated strategy.

  1. fitted input called prediction [Section III-A-B and Abstract]
    "The results showed that for most hours the model predicted the correct class... except hour 19, where the precision for the class (−inf, −12) reaches or exceeds 50%. ... Based on the analysis above, hour 19 is selected to build most of the trading strategies... the most profitable strategy is from model G. ... The results show that the strategy of trading only at the peak hour with a precision score of over 50% produces nearly consistent profit over the test period."

    The strategy parameters are fitted to the test set rather than fixed before it. Hour 19 is chosen because the test-period hourly confusion matrix (Fig. 4) shows precision above 50% at that hour, and the 50% threshold is an ex post reading of the same matrix; model G is reported because Table I, computed on the test period, shows it as the most profitable. The abstract then presents the profit of this selected hour/threshold/model as the paper's result. Walk-forward retraining only refreshes network weights; it does not constrain hour, threshold, strategy, or model selection. The reported 'nearly consistent profit' is therefore a maximum over many test-set comparisons (24 hours, 7 strategies, 7 model variants), not the out-of-sample profit of a pre-specified rule.

full rationale

The classification model itself is not circularly derived: its inputs (load, solar, wind, temporal features, historical price spread) do not encode the target spread labels, and the weekly walk-forward retraining is an appropriate leakage control for the network weights. The clearing-at-cap/floor assumption is imported from an external dissertation [9], not from the present authors, so it is a modeling assumption rather than a self-citation. The central profitability claim, however, is not independently grounded: hour 19, the precision>50% rule, and model G were all selected after examining the test-period confusion matrix and profit table, so the cumulative profit reported in Table I and Fig. 5 contains an in-sample selection component. This is a fitted-input-called-prediction pattern rather than a definitional identity, so the score is 6 rather than higher; the accuracy/precision evaluation of the classifier remains a valid walk-forward measurement, but the headline trading result does not.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper's empirical claims rest on public ERCOT data quality, idealized clearing assumptions, data-chosen bin thresholds, and test-set-based strategy selection. No new physical or mathematical entities are introduced.

free parameters (4)
  • Bin edges for spread classes = -12, -5, 5, 12 USD/MWh
    Thresholds are data-dependent: $5 is the assumed uplift break-even and $12 is described as about the median of spreads outside $5, so the classification target is shaped by the data.
  • Hour 19 strategy selection = 19
    Chosen by inspecting the test-period hourly confusion matrix (Section III-A), making the strategy a function of the test set.
  • Transformer hyperparameters (layers, dropout, learning rate, weight decay) = not reported
    Selected per week via Ray Tune on validation loss, but final values are not listed, so the exact model is not recoverable.
  • Prediction probability threshold for T5 = 0.50
    Used to gate trades in strategy T5; the value is a rule choice and is not derived from first principles.
assumptions (5)
  • domain assumption ERCOT published load, solar, wind, and price-spread data used for training and evaluation are accurate and consistent.
    The whole empirical claim depends on the quality and correctness of the public ERCOT data feeds (Section II-B).
  • domain assumption Virtual bids clear whenever placed at the price cap or floor.
    Stated in Section III-D; this removes realistic clearing risk and can overstate profitability.
  • domain assumption Bidding 1 MWh per day does not change the system lambda (price).
    Assumed in Section III-D; valid only for price-taking behavior.
  • domain assumption The five fixed spread ranges adequately capture the decision-relevant structure for virtual bidding.
    The classification target is defined by these bins; the choice of bin edges is not validated against a continuous forecast.
  • domain assumption A fixed $5/MWh uplift cost applies to all trades.
    Used as the break-even threshold; real uplift costs vary with portfolio and market conditions (Section II-D).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market." pith.science (2026). https://pith.science/paper/O7DW2S3U

@misc{pith2026241200062,
  author       = {Pith},
  title        = {Pith review of: Deep Learning-Based Electricity Price Forecast for Virtual Bidding in Wholesale Electricity Market},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O7DW2S3U}},
  note         = {Machine review of arXiv:2412.00062}
}
read the original abstract

Virtual bidding plays an important role in two-settlement electric power markets, as it can reduce discrepancies between day-ahead and real-time markets. Renewable energy penetration increases volatility in electricity prices, making accurate forecasting critical for virtual bidders, reducing uncertainty and maximizing profits. This study presents a Transformer-based deep learning model to forecast the price spread between real-time and day-ahead electricity prices in the ERCOT (Electric Reliability Council of Texas) market. The proposed model leverages various time-series features, including load forecasts, solar and wind generation forecasts, and temporal attributes. The model is trained under realistic constraints and validated using a walk-forward approach by updating the model every week. Based on the price spread prediction results, several trading strategies are proposed and the most effective strategy for maximizing cumulative profit under realistic market conditions is identified through backtesting. The results show that the strategy of trading only at the peak hour with a precision score of over 50% produces nearly consistent profit over the test period. The proposed method underscores the importance of an accurate electricity price forecasting model and introduces a new method of evaluating the price forecast model from a virtual bidder's perspective, providing valuable insights for future research.

Figures

Figures reproduced from arXiv: 2412.00062 by the authors.

Figure 2
Figure 2. Model architecture, where the output Pt represent the probability distribution of the price spread of hour 0:00 (In ERCOT, it needs to be converted into Hour Ending 1:00) [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Annual solar generation Current Operation Plan (COP) trends from [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Confusion matrix of the best model for each hour. Precision, recall, and accuracy are best viewed when zoomed in. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Cumulation profit for the best model using different trading strategies. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 11 canonical work pages

  1. [1]

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

  2. [2]

    Xu and R

    J. Xu and R. Baldick, ``Day-ahead price forecasting in ercot market using neural network approaches,'' in Proceedings of the Tenth ACM International Conference on Future Energy Systems, 2019, pp. 486--491

  3. [3]

    Z. Zhao, C. Wang, M. Nokleby, and C. J. Miller, ``Improving short-term electricity price forecasting using day-ahead lmp with arima models,'' in 2017 IEEE Power & Energy Society General Meeting. 1em plus 0.5em minus 0.4em IEEE, 2017, pp. 1--5

  4. [4]

    Pavi \'c evi \'c and T

    M. Pavi \'c evi \'c and T. Popovi \'c , ``Forecasting day-ahead electricity metrics with artificial neural networks,'' Sensors, vol. 22, no. 3, p. 1051, 2022

  5. [5]

    u tz, B. Sch \

    J. Trebbien, S. P \"u tz, B. Sch \"a fer, H. S. Nyg rd, L. R. Gorj \ a o, and D. Witthaut, ``Probabilistic forecasting of day-ahead electricity prices and their volatility with lstms,'' in 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE). 1em plus 0.5em minus 0.4em IEEE, 2023, pp. 1--5

  6. [6]

    Rawal and A

    K. Rawal and A. Ahmad, ``Day-ahead market electricity price prediction using time series forecasting,'' in 2022 1st International Conference on Sustainable Technology for Power and Energy Systems (STPES). 1em plus 0.5em minus 0.4em IEEE, 2022, pp. 1--6

  7. [7]

    Galarneau-Vincent, G

    R. Galarneau-Vincent, G. Gauthier, and F. Godin, ``Foreseeing the worst: Forecasting electricity dart spikes,'' Energy Economics, vol. 119, p. 106521, 2023

  8. [8]

    Wang and N

    W. Wang and N. Yu, ``A machine learning framework for algorithmic trading with virtual bids in electricity markets,'' in 2019 IEEE power & energy society general meeting (PESGM). 1em plus 0.5em minus 0.4em IEEE, 2019, pp. 1--5

Show all 12 references
  1. [9]

    R. Das, R. Bo, H. Chen, W. U. Rehman, and D. Wunsch, ``Forecasting nodal price difference between day-ahead and real-time electricity markets using long-short term memory and sequence-to-sequence networks,'' IEEE Access, vol. 10, pp. 832--843, 2021

  2. [10]

    Li, ``Data-driven modeling and algorithmic trading in electricity market,'' Ph.D

    Y. Li, ``Data-driven modeling and algorithmic trading in electricity market,'' Ph.D. dissertation, UC Riverside, 2024

  3. [11]

    Y. Li, N. Yu, and W. Wang, ``Machine learning-driven virtual bidding with electricity market efficiency analysis,'' IEEE Transactions on Power Systems, vol. 37, no. 1, pp. 354--364, 2021

  4. [12]

    R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, and I. Stoica, ``Tune: A research platform for distributed model selection and training,'' arXiv preprint arXiv:1807.05118, 2018

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.