pith. sign in

arxiv: 2604.08309 · v1 · submitted 2026-04-09 · 📡 eess.SY · cs.SY

Bayesian Inference for Estimating Generation Costs in Electricity Markets

Matthias Pirlet , Adrien Bolland , Alexandre Huynen , Quentin Louveaux , Gilles Louppe , Damien Ernst This is my paper

Pith reviewed 2026-05-10 17:22 UTC · model grok-4.3

classification 📡 eess.SY cs.SY
keywords Bayesian inferencegeneration costselectricity marketslatent variable modelneural posterior estimationmarginal costsstart-up costsIEEE RTS-96
0
0 comments X

The pith

Bayesian inference recovers marginal generation costs from electricity production schedules with uncertainty quantification.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper frames the estimation of generation costs such as marginal and start-up costs from observed production schedules as Bayesian inference on a latent variable model. A prior encodes initial beliefs about the costs, and balanced neural posterior estimation learns the updated posterior from market data. This matters because reliable cost estimates support market simulations, strategic bidding, and system planning while also providing measures of uncertainty. Validation on the IEEE RTS-96 test system recovers marginal costs within narrow credible intervals, yet start-up costs prove largely unidentifiable from schedules alone. The approach is benchmarked against inverse optimization, which yields larger parameter errors and supplies no uncertainty information.

Core claim

Estimating generation costs from observed electricity market data is formulated as Bayesian inference on a latent variable model, where a prior distribution encodes an initial belief on parameters and balanced neural posterior estimation learns the posterior given observations, recovering marginal costs with narrow credible intervals on the IEEE RTS-96 test system while start-up costs remain largely unidentifiable from schedules alone.

What carries the argument

The latent variable model linking generation costs to production schedules, with balanced neural posterior estimation used to approximate the posterior distribution from data.

If this is right

  • Marginal costs can be recovered with narrow credible intervals directly from observed schedules.
  • Start-up costs cannot be reliably identified using schedules alone.
  • The method supplies uncertainty quantification absent from inverse-optimization benchmarks.
  • Parameter estimates exhibit smaller errors than those produced by the inverse-optimization comparator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If marginal costs are recoverable this way, market participants could use the posterior distributions to simulate bidding outcomes under cost uncertainty.
  • The difficulty identifying start-up costs points to the possible value of supplementing schedules with bid data or operational logs in future applications.
  • Extending the latent model to include network flow constraints or multi-period decisions might improve identifiability of additional cost components.

Load-bearing premise

The latent variable model accurately captures the relationship between generation costs and observed production schedules, and the chosen prior distribution does not unduly influence the posterior for the quantities of interest.

What would settle it

Generating synthetic schedules on the IEEE RTS-96 system from known true cost parameters and verifying whether the resulting credible intervals contain those true marginal costs at the expected frequency.

Figures

Figures reproduced from arXiv: 2604.08309 by Adrien Bolland, Alexandre Huynen, Damien Ernst, Gilles Louppe, Matthias Pirlet, Quentin Louveaux.

Figure 1
Figure 1. Figure 1: Latent variable model for cost inference. White circles represent latent [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Marginal (diagonal) and joint (off-diagonal) posterior distributions for [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Marginal (diagonal) and joint (off-diagonal) posterior distributions [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Empirical coverage probability versus nominal credibility level for [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Posterior predictive check. For a single observation [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
read the original abstract

Estimating generation costs from observed electricity market data is essential for market simulation, strategic bidding, and system planning. To that end, we model the relationship between generation costs and production schedules with a latent variable model. Estimating generation costs from observed schedules is then formulated as Bayesian inference. A prior distribution encodes an initial belief on parameters, and the inference consists of updating the belief with the posterior distribution given observations. We use balanced neural posterior estimation (BNPE) to learn this posterior. Validation on the IEEE RTS-96 test system shows that marginal costs are recovered with narrow credible intervals, while start-up costs remain largely unidentifiable from schedules alone. The method is benchmarked against an inverse-optimization algorithm that exhibits larger parameter errors without uncertainty quantification.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper models the mapping from generation costs to observed production schedules via a latent variable model and casts cost estimation as Bayesian posterior inference, implemented with balanced neural posterior estimation (BNPE). On the IEEE RTS-96 test system the method recovers marginal costs inside narrow credible intervals while finding start-up costs largely unidentifiable from schedules alone; it is benchmarked against an inverse-optimization baseline that yields larger point errors without uncertainty quantification.

Significance. If the validation is shown to be robust to realistic mismatches between the assumed forward model and actual market data, the work would supply a practical tool for recovering cost parameters together with credible intervals, offering a clear advantage over existing point-estimate inverse methods for market simulation and planning.

major comments (2)
  1. [Abstract] Abstract (validation paragraph): the reported recovery of marginal costs with narrow credible intervals on IEEE RTS-96 is obtained by generating schedules from the same latent-variable model used for inference. This setup verifies that BNPE can invert its own forward simulator but does not test performance when unmodeled effects (transmission losses, reserve requirements, or strategic bidding) are present; the central claim of practical utility therefore rests on an unverified assumption that the data-generating process matches the model exactly.
  2. [Abstract] Abstract: no specification is given for the latent-variable model (how commitment, dispatch, and network constraints enter the likelihood), the functional form of the prior, or any sensitivity checks on prior hyperparameters. Because the posterior is the central object of the method, absence of these details prevents assessment of whether the narrow credible intervals for marginal costs are driven by data or by prior regularization.
minor comments (1)
  1. [Abstract] The abstract states that start-up costs remain 'largely unidentifiable'; a quantitative measure (e.g., posterior variance relative to prior variance or effective sample size) would make this claim precise.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the scope of our validation and the need for greater transparency in the abstract. We address each point below and outline the revisions we will make to the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract (validation paragraph): the reported recovery of marginal costs with narrow credible intervals on IEEE RTS-96 is obtained by generating schedules from the same latent-variable model used for inference. This setup verifies that BNPE can invert its own forward simulator but does not test performance when unmodeled effects (transmission losses, reserve requirements, or strategic bidding) are present; the central claim of practical utility therefore rests on an unverified assumption that the data-generating process matches the model exactly.

    Authors: We agree that the reported experiments generate schedules from the identical latent-variable model used for inference, confirming that BNPE can recover parameters when the forward model is exact. This does not yet address robustness to realistic mismatches such as transmission losses, reserve requirements, or strategic bidding. We will revise the abstract to state explicitly that the narrow credible intervals are obtained under the assumption that the data-generating process matches the model, and we will add a dedicated limitations paragraph discussing the implications of model misspecification together with planned extensions to test performance under such mismatches. These changes will temper the claim of immediate practical utility while preserving the contribution as a validated inference method under ideal conditions. revision: yes

  2. Referee: [Abstract] Abstract: no specification is given for the latent-variable model (how commitment, dispatch, and network constraints enter the likelihood), the functional form of the prior, or any sensitivity checks on prior hyperparameters. Because the posterior is the central object of the method, absence of these details prevents assessment of whether the narrow credible intervals for marginal costs are driven by data or by prior regularization.

    Authors: The abstract is intentionally concise, but we acknowledge that it omits key modeling choices. The latent-variable model (with commitment, dispatch, and network constraints entering the likelihood via the DCOPF formulation) and the prior (independent Gaussian distributions on marginal and start-up costs) are fully specified in Section 2 and the supplementary material; sensitivity analyses to prior hyperparameters appear in Section 4.3. We will expand the abstract by one or two sentences to summarize the latent-variable structure and prior form, and we will add a cross-reference to the sensitivity results. This will allow readers to evaluate the relative influence of data versus prior without lengthening the abstract excessively. revision: yes

Circularity Check

0 steps flagged

No circularity in Bayesian inference formulation or validation

full rationale

The paper defines a latent variable model linking generation costs to observed schedules, then casts parameter recovery as standard Bayesian updating from a prior via BNPE. Validation consists of forward simulation on the external IEEE RTS-96 system with known costs followed by posterior recovery; this is a conventional consistency check rather than any reduction of the reported posteriors or credible intervals to quantities defined inside the paper's own fitted parameters or self-citations. No equations, ansatzes, or uniqueness claims are shown to collapse by construction to the inputs, and the derivation remains independent of the target results.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract-only review; the central claim rests on the assumption that a latent variable model plus BNPE can invert the cost-to-schedule mapping, with no explicit free parameters or invented entities listed.

axioms (1)
  • domain assumption The relationship between generation costs and production schedules can be represented by a latent variable model whose parameters have a prior distribution that can be updated via posterior inference.
    Stated in the abstract as the modeling choice for Bayesian inference.

pith-pipeline@v0.9.0 · 5432 in / 1165 out tokens · 56144 ms · 2026-05-10T17:22:58.564123+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages

  1. [1]

    A. J. Wood, B. F. Wollenberg, and G. B. Shebl ´e,Power generation, operation, and control. John wiley & sons, 2013

    work page 2013

  2. [2]

    D. S. Kirschen and G. Strbac,Fundamentals of power system economics. John Wiley & Sons, 2018

    work page 2018

  3. [3]

    Economic impact of electricity market price forecasting errors: A demand-side analysis,

    H. Zareipour, C. A. Canizares, and K. Bhattacharya, “Economic impact of electricity market price forecasting errors: A demand-side analysis,” IEEE Transactions on Power Systems, vol. 25, no. 1, pp. 254–262, 2009

    work page 2009

  4. [4]

    Tackling energy price volatility: A smarter approach to price forecasting,

    European Commission Joint Research Centre, “Tackling energy price volatility: A smarter approach to price forecasting,” 2025, accessed: 2025-09-10. [Online]. Available: https://joint- research-centre.ec.europa.eu/jrc-news-and-updates/tackling-energy- price-volatility-smarter-approach-price-forecasting-2025-03-13 en

    work page 2025

  5. [5]

    Unit commitment-a bibliographical survey,

    N. P. Padhy, “Unit commitment-a bibliographical survey,”IEEE Trans- actions on power systems, vol. 19, no. 2, pp. 1196–1205, 2004

    work page 2004

  6. [6]

    A computationally efficient mixed-integer linear formulation for the thermal unit commitment problem,

    M. Carri ´on and J. M. Arroyo, “A computationally efficient mixed-integer linear formulation for the thermal unit commitment problem,”IEEE Transactions on power systems, vol. 21, no. 3, pp. 1371–1378, 2006

    work page 2006

  7. [7]

    A tight mip formu- lation of the unit commitment problem with start-up and shut-down constraints,

    C. Gentile, G. Morales-Espana, and A. Ramos, “A tight mip formu- lation of the unit commitment problem with start-up and shut-down constraints,”EURO Journal on Computational Optimization, vol. 5, no. 1, pp. 177–201, 2017

    work page 2017

  8. [8]

    Mas-Colell, M

    A. Mas-Colell, M. D. Whinston, J. R. Greenet al.,Microeconomic theory. Oxford university press New York, 1995, vol. 1

    work page 1995

  9. [9]

    Linear complementarity models of nash-cournot competition in bilateral and poolco power markets,

    B. Hobbs, “Linear complementarity models of nash-cournot competition in bilateral and poolco power markets,”IEEE Transactions on power systems, vol. 16, no. 2, pp. 194–202, 2001

    work page 2001

  10. [10]

    Architecture of power markets,

    R. Wilson, “Architecture of power markets,”Econometrica, vol. 70, no. 4, pp. 1299–1340, 2002

    work page 2002

  11. [11]

    J. R. Birge and F. Louveaux,Introduction to stochastic programming. Springer, 1997

    work page 1997

  12. [12]

    A stochastic model for the unit commitment problem,

    S. Takriti, J. R. Birge, and E. Long, “A stochastic model for the unit commitment problem,”IEEE Transactions on Power Systems, vol. 11, no. 3, pp. 1497–1508, 1996

    work page 1996

  13. [13]

    Stochastic optimization for unit commitment—a review,

    Q. P. Zheng, J. Wang, and A. L. Liu, “Stochastic optimization for unit commitment—a review,”IEEE Transactions on Power Systems, vol. 30, no. 4, pp. 1913–1924, 2014

    work page 1913

  14. [14]

    Fundamentals and recent developments in stochastic unit commitment,

    M. H ˚aberg, “Fundamentals and recent developments in stochastic unit commitment,”International Journal of Electrical Power & Energy Systems, vol. 109, pp. 38–48, 2019

    work page 2019

  15. [15]

    Adaptive robust optimization for the security constrained unit commitment prob- lem,

    D. Bertsimas, E. Litvinov, X. A. Sun, J. Zhao, and T. Zheng, “Adaptive robust optimization for the security constrained unit commitment prob- lem,”IEEE transactions on power systems, vol. 28, no. 1, pp. 52–63, 2012

    work page 2012

  16. [16]

    Inverse optimization,

    R. K. Ahuja and J. B. Orlin, “Inverse optimization,”Operations research, vol. 49, no. 5, pp. 771–783, 2001

    work page 2001

  17. [17]

    Inverse optimization for the recovery of market structure from market outcomes: An application to the MISO electricity market,

    J. R. Birge, A. Hortac ¸su, and J. M. Pavlin, “Inverse optimization for the recovery of market structure from market outcomes: An application to the MISO electricity market,”Operations Research, vol. 65, no. 4, pp. 837–855, 2017

    work page 2017

  18. [18]

    Data-driven inverse optimization for marginal offer price recovery in electricity markets,

    Z. Liang and Y . Dvorkin, “Data-driven inverse optimization for marginal offer price recovery in electricity markets,” inProceedings of the 14th ACM International Conference on Future Energy Systems, 2023, pp. 497–509

    work page 2023

  19. [19]

    Development of the internal electricity market in europe,

    L. Meeus, K. Purchala, and R. Belmans, “Development of the internal electricity market in europe,”The Electricity Journal, vol. 18, no. 6, pp. 25–35, 2005

    work page 2005

  20. [20]

    The frontier of simulation- based inference,

    K. Cranmer, J. Brehmer, and G. Louppe, “The frontier of simulation- based inference,”Proceedings of the National Academy of Sciences, vol. 117, no. 48, pp. 30 055–30 062, 2020

    work page 2020

  21. [21]

    Cost estimation in unit commitment problems using simulation-based inference,

    M. Pirlet, A. Bolland, G. Louppe, and D. Ernst, “Cost estimation in unit commitment problems using simulation-based inference,” in NeurIPS 2024 Workshop on Data-driven and Differentiable Simulations, Surrogates, and Solvers, 2024

    work page 2024

  22. [22]

    Variational inference with normalizing flows,

    D. Rezende and S. Mohamed, “Variational inference with normalizing flows,” inInternational conference on machine learning. PMLR, 2015, pp. 1530–1538

    work page 2015

  23. [23]

    Fastε-free inference of simulation models with bayesian conditional density estimation,

    G. Papamakarios and I. Murray, “Fastε-free inference of simulation models with bayesian conditional density estimation,”Advances in neural information processing systems, vol. 29, 2016

    work page 2016

  24. [24]

    Flexible statistical inference for mechanistic models of neural dynamics,

    J.-M. Lueckmann, P. J. Goncalves, G. Bassetto, K. ¨Ocal, M. Nonnen- macher, and J. H. Macke, “Flexible statistical inference for mechanistic models of neural dynamics,”Advances in neural information processing systems, vol. 30, 2017

    work page 2017

  25. [25]

    Automatic posterior transformation for likelihood-free inference,

    D. Greenberg, M. Nonnenmacher, and J. Macke, “Automatic posterior transformation for likelihood-free inference,” inInternational conference on machine learning. PMLR, 2019, pp. 2404–2414

    work page 2019

  26. [26]

    The IEEE reliability test system-1996. a report prepared by the reliability test system task force of the application of probability methods subcommittee,

    C. Grigg, P. Wong, P. Albrecht, R. Allan, M. Bhavaraju, R. Billinton, Q. Chen, C. Fong, S. Haddad, S. Kurugantyet al., “The IEEE reliability test system-1996. a report prepared by the reliability test system task force of the application of probability methods subcommittee,”IEEE Transactions on power systems, vol. 14, no. 3, pp. 1010–1020, 1999

    work page 1996

  27. [27]

    An updated version of the IEEE rts 24-bus system for electricity market and power system operation studies

    C. Ordoudis, P. Pinson, J. M. M. Gonz ´alez, and M. Zugno, “An updated version of the IEEE rts 24-bus system for electricity market and power system operation studies.” 2016

    work page 2016

  28. [28]

    Build, compute, critique, repeat: Data analysis with latent variable models,

    D. M. Blei, “Build, compute, critique, repeat: Data analysis with latent variable models,”Annual Review of Statistics and Its Application, vol. 1, no. 1, pp. 203–232, 2014

    work page 2014

  29. [29]

    Delaunoy, B

    A. Delaunoy, B. K. Miller, P. Forr ´e, C. Weniger, and G. Louppe, “Bal- ancing simulation-based inference for conservative posteriors,”arXiv preprint arXiv:2304.10978, 2023

    work page arXiv 2023

  30. [30]

    R. T. Rockafellar,Convex analysis. Princeton University Press, 1970

    work page 1970

  31. [31]

    A comprehensive review of security-constrained unit commitment,

    N. Yang, Z. Dong, L. Wu, L. Zhang, X. Shen, D. Chen, B. Zhu, and Y . Liu, “A comprehensive review of security-constrained unit commitment,”Journal of Modern Power Systems and Clean Energy, vol. 10, no. 3, pp. 562–576, 2021

    work page 2021

  32. [32]

    Neural spline flows,

    C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios, “Neural spline flows,”Advances in neural information processing systems, vol. 32, 2019

    work page 2019

  33. [33]

    The pseudo-marginal approach for efficient monte carlo computations,

    C. Andrieu and G. O. Roberts, “The pseudo-marginal approach for efficient monte carlo computations,” 2009

    work page 2009

  34. [34]

    Markov chain monte carlo without likelihoods,

    P. Marjoram, J. Molitor, V . Plagnol, and S. Tavar´e, “Markov chain monte carlo without likelihoods,”Proceedings of the National Academy of Sciences, vol. 100, no. 26, pp. 15 324–15 328, 2003

    work page 2003

  35. [35]

    Flow matching for scalable simulation-based inference.arXiv preprint arXiv:2305.17161,

    M. Dax, J. Wildberger, S. Buchholz, S. R. Green, J. H. Macke, and B. Sch ¨olkopf, “Flow matching for scalable simulation-based inference,” arXiv preprint arXiv:2305.17161, 2023. VII. APPENDIX A. Derivation of the BNPE objective We derived the BNPE objective by minimizing the expected Kullback-Leibler (KL) divergence between the true posterior p(θ|x)and it...

    work page arXiv 2023