Pith. sign in

REVIEW 5 major objections 7 minor 17 references

Conformal Uncertainty Quantification of Electricity Price Predictions for Risk-Averse Storage Arbitrage

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Wrapping any price forecaster in conformal intervals and scheduling only on unanimous samples preserves roughly 93% of arbitrage profit while cutting purchases to under 35% of the point-forecast baseline.

desk verdict A transparent, small-scale application of conformal PID to storage arbitrage with a simple unanimous-vote risk-averse policy; the headline purchase reduction is real but only shown for synthetic Gaussian forecast errors. read the letter →

arxiv 2412.07075 v1 pith:AJZJAGLU submitted 2024-12-10 math.OC stat.ML

classification math.OCstat.ML MSC 62G1590C15
keywords conformalpredictionenergystoragearbitrageuncertaintyquantificationreal-timeelectricitypricesrisk-aversepolicylong-runcoveragedistribution-freeMonte-Carlosampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Real-time electricity prices are volatile enough that a storage operator who follows a point forecast can lose money for months at a time. This paper claims that wrapping any point-price forecaster in conformal prediction intervals, and then scheduling only when many sampled prices agree, keeps most of the profit while trading far less. On New York State real-time price data with synthetic forecasters, the conservative version earns $7,707.87 against $8,284.20 for the point forecast, while buying $3,596.60 of energy instead of $12,936.43. Under a deliberately bad forecaster, the same policy turns a $9,902.98 loss into a $3,697.08 profit. The importance, if true, is that uncertainty-aware bidding can make storage participation in real-time markets viable even when price forecasting is poor.

What carries the argument

The central object is the conformal control prediction interval, built from a signed-residual nonconformity score $s(\lambda,\hat\lambda)=\lambda-\hat\lambda$ and an online quantile estimate $q_{t+1}=\hat q_{t+1}+r_t\sum_{i=1}^{t}(\mathbf{1}\{\hat\lambda\notin\hat C\}-\alpha)$, which achieves long-run coverage $\frac{1}{T}\sum_{t=1}^{T}\mathbf{1}\{\hat\lambda\notin\hat C\}=\alpha+o(1)$ for any point forecaster. On top of this interval, the risk-averse policy draws $N$ uniform samples from the prediction set, solves the arbitrage LP for each sample, and keeps a charge or discharge decision only if all samples agree; the traded amount is then the minimum (conservative) or maximum (aggressive) across agreeing samples. This combination converts an uncertainty interval into a trading rule.

What would settle it

Train any real point forecaster on one year of New York State real-time prices, apply the same conformal control layer and conservative voting policy on the next year, and compare purchases and profit against the point-forecast baseline; if the conservative policy's purchase share exceeds 60% of the point-forecast level or its profit is less than 70% of the point-forecast profit, the synthetic-error assumption in Section III-B does not transfer.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that conformal control, a PID-style quantile update that achieves long-run coverage in non-exchangeable time series, produces prediction intervals that adapt to forecaster quality, and a unanimous-vote Monte-Carlo policy over those intervals removes almost all unprofitable trades. The authors demonstrate that with a good synthetic forecaster (ground truth plus Gaussian noise with $5$/MWh standard deviation), the conservative policy retains 93% of point-forecast profit while reducing purchases to 28% of the baseline, and with a bad forecaster ($40$/MWh noise) it converts a deeply negative point-forecast outcome into a positive one. The core quantity is the purchase reduction: the policy acts as a veto filter, acting only when all sampled price scenarios agree on direction, and choosing the minimum traded amount in the conservative mode.

Load-bearing premise

The evaluation treats synthetic forecasts (ground truth plus independent zero-mean Gaussian noise) as a faithful stand-in for real forecaster errors, including the $40$/MWh 'bad' forecaster; if real prediction errors are biased, clustered, or heteroscedastic, the interval widths and the unanimous-vote outcomes could differ materially.

Editorial extensions

If this is right

  • With the conservative policy, a storage operator can keep roughly 93% of point-forecast profit while cycling the battery less than a third as often, which extends lifetime and reduces transaction costs.
  • The framework is agnostic to the point forecaster, so any improvement in underlying price prediction directly narrows the conformal intervals and raises the arbitrage profit of the risk-averse policy.
  • Under a poor forecaster, the risk-averse policy avoids the large losses that a point-forecast policy would incur, making real-time market participation feasible for operators who cannot afford high-quality forecasting.
  • The aggressive mode offers a way to trade more profit for more market activity, giving operators a tunable risk dial between the conservative and point-forecast baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'less than 35% purchases' figure is dataset- and forecaster-specific; the structural claim that a unanimous-vote veto over conformal intervals removes the worst trades is the transferable insight, not the exact ratio.
  • Because real price errors are clustered during spikes, a natural testable extension is replacing uniform interval sampling with sampling from the empirical residual distribution, which would keep the interval semantics but sharpen the veto.
  • The same interval-plus-unanimous-vote construction could be applied to other sequential bidding or trading problems, such as day-ahead commitment with uncertain renewables, wherever a point forecast is the fallback baseline.
  • A fixed $\alpha=0.05$ coverage target may be too tight when losses are strongly asymmetric; an extension would be to set $\alpha$ by the acceptable loss probability rather than by coverage.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes a two-layer framework for risk-averse energy storage arbitrage in real-time electricity markets. The first layer is a point price forecast model, which is left abstract (Eq. 1); the second layer is a conformal control algorithm that produces prediction intervals with long-run coverage guarantees. A risk-averse policy then samples prices uniformly from these intervals (Eq. 7), solves the arbitrage optimization (Eq. 6) for each sample, and aggregates the resulting charge/discharge decisions using a unanimous-vote rule with conservative or aggressive variants (Eqs. 8–10). The method is evaluated on New York City zonal price data using synthetic point forecasts (Eq. 11) constructed as ground truth plus independent Gaussian noise with two target standard deviations ($5/MWh and $40/MWh). The headline results, summarized in Table I, are that the conservative mode achieves $7,707.87 profit versus $8,284.20 for the point forecast while purchasing only $3,596.60 versus $12,936.43 under the good forecaster, and that it avoids the large losses of the point forecast under the bad forecaster.

Significance. If the proposed approach is validated, it could provide a model-agnostic way to convert point forecasts into risk-aware storage schedules, leveraging the distribution-free coverage guarantees of conformal prediction. The paper's conceptual contribution is timely and the inclusion of public code is a strength. However, the evaluation rests entirely on synthetic Gaussian forecast errors, and the risk-averse aggregation policy is a heuristic with no theoretical justification or sensitivity analysis. The reported profit and purchase figures are therefore an existence proof for a stylized forecaster rather than a demonstration that the method transfers to production forecasting. The central idea deserves further study, but the current evidence is not sufficient to establish the headline claims.

major comments (5)
  1. [Section III-B, Eq. (11) and Table I] The headline results are generated exclusively from synthetic point forecasts defined as ground truth plus independent zero-mean Gaussian noise with fixed standard deviations. Real-time electricity price forecast errors are typically heteroscedastic, autocorrelated, and heavy-tailed, especially during price spikes. The paper does not train any actual point forecast model (Eq. 1 remains abstract), so the conformal intervals and the unanimous-vote policy are only tested against an i.i.d. Gaussian noise model. The claims in the abstract and conclusion that the framework 'can achieve good profit margins with less than 35% purchases' are therefore not supported for real forecasters. Please add experiments with a trained forecaster (e.g., an LSTM or the transformer-based model from the authors' prior work), or at least with more realistic error models such as ARMA/GARCH or bootstrapped residuals from a trained model, and report how the results vary with the noise characteristics.
  2. [Section II-C, Eq. (7)] Equation (7) is internally inconsistent and not reproducible. U_i is described as 'a uniformly distributed random variable with zero mean and one standard deviation,' but a uniform distribution on any symmetric interval [-a,a] has standard deviation a/√3, and if U_i is uniform on [0,1] its mean is 0.5, not zero. Moreover, the formula uses the interval [ˆλ, ˆλ] with identical endpoints, so the expression (ˆλ_t − ˆλ_t) is identically zero, making the sampling degenerate. The intended definition should use explicit lower and upper bounds of the conformal set, e.g., C_t = [L_t, U_t], with π_i = L_t + (U_t − L_t) U_i and U_i ~ Uniform(0,1). This is a load-bearing point because the entire risk-averse policy rests on these samples; as written, the experiments in Section III-C cannot be replicated from the text.
  3. [Section II-A, Eqs. (3)–(5)] The nonconformity score in Eq. (3) is the signed residual λ − ˆλ, which defines the one-sided set {λ : λ − ˆλ ≤ q} in Eq. (4). The paper consistently refers to 'confidence intervals' and plots them as two-sided intervals, and the later sampling in Eq. (7) presumes a two-sided interval. A two-sided conformal interval requires the absolute residual |λ − ˆλ| or separate lower and upper quantiles; the current formulation is inconsistent. Additionally, Eq. (5) contains the event 1{ˆλ /∈ Ĉ}, which should be 1{λ /∈ C} because coverage is about the ground truth λ, not the prediction ˆλ. These notation errors obscure the method and make the claimed coverage guarantee difficult to verify. Please correct the score function, the set definition, and the update rule, and state explicitly whether the conformal set is one- or two-sided.
  4. [Section II-C, Eqs. (8)–(10) and Section III-C] The unanimous-vote aggregation rule is a heuristic that has not been justified theoretically or empirically. The paper does not analyze how the number of Monte Carlo samples N, the sampling distribution, or the interval width affect the resulting schedule, and there is no evidence that unanimity is a good proxy for risk aversion. Algorithm 1 uses an undefined loop bound M, and Table I presents point estimates without error bars, seed variation, or sensitivity to N. Since the central claim is that this policy 'achieves good profit margins with less than 35% purchases,' the lack of any uncertainty quantification on the policy's outputs is a major gap. Please report the variance of the profit and purchase metrics over random seeds, vary N, and justify the choice of the unanimity rule, for example by comparing against alternative aggregation schemes.
  5. [Abstract and Section IV] The paper states that the framework is 'distribution-free' and 'without assuming any distribution on the dataset,' but Eq. (7) imposes a uniform distribution on the prices within the conformal interval. Conformal prediction guarantees coverage of the prediction set; it does not provide a probability distribution over the set. Uniform sampling is an arbitrary assumption that could materially affect arbitrage decisions and is inconsistent with the distribution-free claim. The authors should either justify uniform sampling theoretically or relax the claim, and they should test sensitivity to the sampling distribution (e.g., using triangular or normal-distribution sampling within the interval).
minor comments (7)
  1. [Section II-B, Eq. (6)] The notation B is used both as a set of bidding policies and as a variable in '{pt, bt} ∈ B; ∀t ∈ T', which is confusing. Please clarify the feasible set and use distinct notation, e.g., define a set of feasible actions A_t.
  2. [Section II-A, Eq. (2)] The data split notation D1 = {(xi, λi)} T1 i=1 is ambiguous; it would be clearer to write D1 = {(x_i, λ_i)}_{i=1}^{T1} and state that T1 is the number of calibration points. The text says 'two weeks in 2022 for calibration,' but the notation suggests a generic number of samples.
  3. [Section III-A] The storage efficiency is stated as '90% one-way efficiency.' It is not clear whether this is a round-trip efficiency or per-direction; this affects the arbitrage calculation. Please define the efficiency convention explicitly.
  4. [Figure 3] The y-axis label 'Coverage (1-α)' is incomplete, and no numerical coverage statistics are reported. Please provide the mean and worst-month coverage for both forecasters, and indicate the target 95% level on the plot.
  5. [Figure 5] The legend entries 'point forecast - Median' and 'ours-conservative - Median' are unclear. If these are medians over multiple simulations, please specify the number of simulations and display confidence bands rather than single lines.
  6. [Algorithm 1] The loop bound M is not defined. It presumably corresponds to the 100 randomized instances mentioned in Section III-C, but this should be stated explicitly, and the loop should read 'for i = 1, ..., M do'.
  7. [Section I and III-B] The comparison with adaptive conformal inference (ACI) is only qualitative and visual. It would strengthen the paper to report quantitative metrics such as mean interval width, average coverage, and the number of 'infinite width' episodes for both methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the conformal guarantee is imported from an external theorem, the calibration/testing split is proper, and the risk-averse policy is a transparent heuristic evaluated on synthetic forecasters rather than a derived claim.

full rationale

The paper's derivation chain is not circular. The conformal coverage guarantee is taken from Angelopoulos et al. (an external, non-overlapping source), and the paper does not rely on its own prior work for any load-bearing mathematical result. The calibration set D1 and testing set D2 are explicitly separated in Eq. (2), and the conformal quantile is estimated from calibration data only, so the reported coverage is not fitted to the test set. The risk-averse policy in Eqs. (7)-(10) is a stated heuristic — unanimous voting plus min/max aggregation — not a quantity derived from the simulation outcomes; the 'less than 35% purchases' figure is an empirical result under the paper's transparently synthetic forecasters (Eq. 11), not a mathematical identity forced by the definitions. The most serious weakness is external validity of the synthetic Gaussian forecast errors, which is a validity concern, not circularity. Self-citations [1], [3], [7], [14], and [16] provide market context and supporting constraints; none of them supplies the central conformal or arbitrage claims. Therefore no circular step can be exhibited from the paper's own equations or citation chain.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper's own methodological novelty is a heuristic; its guarantees come from an external conformal PID theorem, and its evaluation rests on synthetic forecasters, so the central claim carries several unverified assumptions.

free parameters (5)
  • Number of Monte Carlo samples N = 100
    Algorithm 1 uses 100 random price samples; sensitivity to N is not reported.
  • Conformal control saturation function r_t = not specified
    Eq (5) requires choosing a saturation function r_t and sub-linear h(t) from Angelopoulos et al.; the paper does not state its form, leaving an implementation parameter.
  • Prediction horizon T = 6 hours
    Chosen as a tradeoff between uncertainty and operational benefit based on Fig 4 and 'our experiments'; not justified with a formal selection procedure.
  • Target noise std for synthetic forecasters = $5/MWh and $40/MWh
    Defines good and bad forecasters; only two levels are tested.
  • Desired miscoverage alpha = 0.05
    User-specified confidence level (95% coverage), chosen by hand to reflect risk-averse settings.
assumptions (4)
  • standard math Conformal PID control achieves long-run coverage 1/T sum of 1{lambda_t not in C_t} = alpha + o(1) for non-exchangeable time series (Theorem 1 of Angelopoulos et al.)
    The paper's conformal layer relies entirely on this external theorem to guarantee coverage; conditions on r_t and h(t) are assumed satisfied.
  • domain assumption Price-taker storage model with linear SoC dynamics (Eq 6) is a valid representation of a self-scheduled storage participant
    The arbitrage optimization and all downstream results assume this market model.
  • ad hoc to paper Uniform sampling from the conformal interval (Eq 7) is a meaningful Monte Carlo representation of price uncertainty
    Conformal sets only carry coverage guarantees, not distributional shape; the paper does not justify uniform sampling as the true conditional distribution, yet the risk-averse policy's decisions depend on it.
  • ad hoc to paper The unanimous-vote aggregation rule (Eqs 8-10) produces a risk-averse schedule
    No formal risk measure or optimality is given; 'risk-averse' is asserted for the min/consensus heuristic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conformal Uncertainty Quantification of Electricity Price Predictions for Risk-Averse Storage Arbitrage." pith.science (2026). https://pith.science/paper/AJZJAGLU

@misc{pith2026241207075,
  author       = {Pith},
  title        = {Pith review of: Conformal Uncertainty Quantification of Electricity Price Predictions for Risk-Averse Storage Arbitrage},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AJZJAGLU}},
  note         = {Machine review of arXiv:2412.07075}
}
abstract

This paper proposes a risk-averse approach to energy storage price arbitrage, leveraging conformal uncertainty quantification for electricity price predictions. The method addresses the significant challenges posed by the inherent volatility and uncertainty of real-time electricity prices, which create substantial risks of financial losses for energy storage participants relying on future price forecasts to plan their operations. The framework comprises a two-layer prediction model to quantify real-time price uncertainty confidence intervals with high coverage. The framework is distribution-free and can work with any underlying point prediction model. We evaluate the quantification effectiveness through storage price arbitrage application by managing the risk of participating in the real-time market. We design a risk-averse policy for profit-maximization of energy storage arbitrage to find the safest storage schedule with very minimal losses. Using historical data from New York State and synthetic price predictions, our evaluations demonstrate that this framework can achieve good profit margins with less than $35\%$ purchases.

Figures

Figures reproduced from arXiv: 2412.07075 by the authors.

Figure 1
Figure 1. The pipeline for the proposed risk-averse storage arbitrage framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. The predicted interval coverage of the ground truth. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Comparison look-ahead window and the confidence interval. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Cumulative daily net profits and purchases following each policy under different scenarios. For the [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 15 canonical work pages

  1. [7]

    Energy storage arbitrage in two-settlement markets: A transformer-based approach,

    S. Alghumayjan, J. Han, N. Zheng, M. Yi, and B. Xu, “Energy storage arbitrage in two-settlement markets: A transformer-based approach,” in Proceedings of the Power System Computation Conference (PSCC) 2024, Paris, France, 2024

  2. [1]

    Impact of local transmission congestion on energy storage arbitrage opportunities,

    Y . Wang, Y . Dvorkin, R. Fern´andez-Blanco, B. Xu, and D. S. Kirschen, “Impact of local transmission congestion on energy storage arbitrage opportunities,” in 2017 IEEE Power & Energy Society General Meeting . IEEE, 2017, pp. 1–5

  3. [2]

    Battery storage in the united states: An update on market trends,

    US Energy Information Association, “Battery storage in the united states: An update on market trends,” 2023

  4. [3]

    The role of electricity market design for energy storage in cost-efficient decarbonization,

    X. Qin, B. Xu, I. Lestas, Y . Guo, and H. Sun, “The role of electricity market design for energy storage in cost-efficient decarbonization,” Joule, 2023

  5. [4]

    Forecasting spot electricity prices: Deep learning approaches and empirical comparison of tradi- tional algorithms,

    J. Lago, F. De Ridder, and B. De Schutter, “Forecasting spot electricity prices: Deep learning approaches and empirical comparison of tradi- tional algorithms,” Applied Energy, vol. 221, pp. 386–405, 2018

  6. [5]

    Day-ahead electricity price forecasting via the application of artificial neural network based models,

    I. P. Panapakidis and A. S. Dagoumas, “Day-ahead electricity price forecasting via the application of artificial neural network based models,” Applied Energy, vol. 172, pp. 132–151, 2016

  7. [6]

    Forecasting day-ahead elec- tricity prices: A comparison of time series and neural network models taking external regressors into account,

    M. Lehna, F. Scheller, and H. Herwartz, “Forecasting day-ahead elec- tricity prices: A comparison of time series and neural network models taking external regressors into account,” Energy Economics, vol. 106, p. 105742, 2022

  8. [8]

    Smart “predict, then optimize

    A. N. Elmachtoub and P. Grigas, “Smart “predict, then optimize”,” Management Science , vol. 68, no. 1, pp. 9–26, 2022

Show all 17 references
  1. [9]

    Machine-learning applica- tions of algorithmic randomness,

    V . V ovk, A. Gammerman, and C. Saunders, “Machine-learning applica- tions of algorithmic randomness,” 1999

  2. [10]

    V ovk, A

    V . V ovk, A. Gammerman, and G. Shafer, Algorithmic learning in a random world . Springer, 2005, vol. 29

  3. [11]

    Conformal pid control for time series prediction,

    A. Angelopoulos, E. Candes, and R. J. Tibshirani, “Conformal pid control for time series prediction,” Advances in neural information processing systems, vol. 36, 2024

  4. [12]

    Conformal prediction for stochastic decision-making of pv power in electricity markets,

    Y . Renkema, N. Brinkel, and T. Alskaif, “Conformal prediction for stochastic decision-making of pv power in electricity markets,” Electric Power Systems Research , vol. 234, p. 110750, 2024

  5. [13]

    End-to-end con- formal calibration for robust grid-scale battery storage optimization,

    C. Yeh, N. Christianson, A. Wierman, and Y . Yue, “End-to-end con- formal calibration for robust grid-scale battery storage optimization,” in NeurIPS 2024 Workshop on Tackling Climate Change with Machine Learning, Vancouver, Canada, 12 2024

  6. [14]

    Arbitraging variable efficiency energy storage using analytical stochastic dynamic programming,

    N. Zheng, J. Jaworski, and B. Xu, “Arbitraging variable efficiency energy storage using analytical stochastic dynamic programming,” IEEE Transactions on Power Systems , vol. 37, no. 6, pp. 4785–4795, 2022

  7. [15]

    Energy- storage modeling: State-of-the-art and future research directions,

    R. Sioshansi, P. Denholm, J. Arteaga, S. Awara, S. Bhattacharjee, A. Bot- terud, W. Cole, A. Cortes, A. De Queiroz, J. DeCarolis et al., “Energy- storage modeling: State-of-the-art and future research directions,” IEEE transactions on power systems , vol. 37, no. 2, pp. 860–875, 2021

  8. [16]

    Transferable energy storage bidder,

    Y . Baker, N. Zheng, and B. Xu, “Transferable energy storage bidder,” IEEE Transactions on Power Systems , 2023

  9. [17]

    Adaptive conformal inference under distribu- tion shift,

    I. Gibbs and E. Candes, “Adaptive conformal inference under distribu- tion shift,” Advances in Neural Information Processing Systems , vol. 34, pp. 1660–1672, 2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.