pith. sign in

arxiv: 2506.11765 · v2 · submitted 2025-06-13 · 🧮 math.OC

State constrained stochastic optimal control of a PV system with battery storage via Fokker-Planck and Hamilton-Jacobi-Bellman equations

Pith reviewed 2026-05-19 09:39 UTC · model grok-4.3

classification 🧮 math.OC
keywords stochastic optimal controlphotovoltaic systemsbattery storageHamilton-Jacobi-Bellman equationFokker-Planck equationdimension reductionday-ahead market
0
0 comments X

The pith

Decomposing state space into controllable and uncontrollable parts reduces the HJB-FP system for PV battery control

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper sets up a continuous-time stochastic optimal control problem for the joint real-time management of battery storage and day-ahead market bidding in a photovoltaic plant. Solar irradiance, electricity prices, and battery dynamics are represented by stochastic differential equations, producing a constrained control task whose solution requires a coupled Hamilton-Jacobi-Bellman and Fokker-Planck system. To lower the computational cost of the high-dimensional problem, the authors split the state space into controllable and uncontrollable components, obtaining reduced optimality systems that keep the dynamic-programming structure intact. Numerical tests confirm that the reduced models deliver large savings in runtime while maintaining performance close to the full formulation and outperforming both rule-based heuristics and stochastic model-predictive control.

Core claim

The paper shows that decomposing the full state space into controllable battery and bidding decisions and uncontrollable solar-price dynamics produces lower-dimensional coupled HJB-FP equations whose solution yields an optimal feedback policy that can be computed fast enough for real-time use without meaningful loss of economic value.

What carries the argument

Dimension-reduction strategy that separates the state space into controllable and uncontrollable components while preserving the continuous-time dynamic-programming optimality conditions.

If this is right

  • Substantial computational savings enable real-time applicability of the optimal policy.
  • The optimality of the resulting feedback policy is retained after reduction.
  • A favorable trade-off between economic performance and computational effort is achieved.
  • The framework supports day-ahead market participation with lower runtime than stochastic MPC.
  • Benchmark comparisons confirm superiority over rule-based strategies in both profit and speed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same controllable-uncontrollable split could be applied to other renewables-plus-storage problems whose exogenous drivers evolve independently of the control.
  • The reduced formulation opens the possibility of adding battery-degradation or network-constraint states without restoring the original dimensionality.
  • Frequent re-optimization inside the day-ahead window becomes feasible, potentially improving forecast use and reducing imbalance penalties.
  • The approach may extend to multi-asset portfolios once the uncontrollable variables remain separable from the controllable ones.

Load-bearing premise

Decomposing the state space into controllable and uncontrollable components yields lower-dimensional optimality systems while preserving the continuous-time Dynamic Programming structure and the optimality of the resulting policy.

What would settle it

A side-by-side simulation in which the reduced model produces markedly higher operating costs or bidding penalties than the full-dimensional solution under realistic high-variability irradiance and price paths.

Figures

Figures reproduced from arXiv: 2506.11765 by Alfredo Berm\'udez, Iago Pad\'in.

Figure 1
Figure 1. Figure 1: Scheme of system considered and feasible power flows. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Functions θZ(s) and θΠ(s). 0 5 10 15 20 25 Time (h) 0 100 200 300 400 500 G HI Irra dia n c e (W/m2 ) E[I] I(s) ICS(s) [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Expectation of Is and functions ICS (s) and I(s). along with the reverting profile I(s) and the Clear Sky Irradiance ICS (s). The expec￾tation of Is is computed based on the expected value of the first state variable using expression (1). Similarly, [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Expectation of Πs and Es . 0 5 10 15 20 25 Time (h) 1.00 0.75 0.50 0.25 0.00 0.25 0.50 0.75 1.00 C o ntrol Pbat (M W) E[Pbat] [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Expectation of Pbat,s and 10 realizations. with the grid, E[Pgrid,s], alongside the expected electricity price E[Πs]. This compar￾ison highlights the coordination between power dispatch decisions and price signals over time. Additionally, the bids that the PV producer submits to the day-ahead elec￾tricity market auctions are also displayed, P˜ grid(s). These bids are typically required to be piecewise cons… view at source ↗
Figure 6
Figure 6. Figure 6: Expectation of Pbat,s , Psolar,s , Pgrid,s and Πs . Optimal power bid at day-ahead electricity market auctions is included (capacity firming). 0 5 10 15 20 25 Time (h) 0 100 200 300 400 500 G HI Irra dia n c e (W/m2 ) E[I] I(s) ICS(s) [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Expectation of Is and functions ICS (s) and I(s) for 1D case. shows the expected solar irradiance, E[Is], and functions ICS (s) and I(s) [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Expectation of Πs and Es for 1D case. 0 5 10 15 20 25 Time (h) 8 6 4 2 0 2 4 6 Power (MW) 40 60 80 100 120 140 160 180 Price of electricity ( /MWh) E[P(I)] E[Pbat] E[Pgrid] Pgrid(s) E[ ] [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Expectation of Pbat,s , Psolar,s , Pgrid,s and Πs together with optimal power bid P˜ grid(s) for 1D case. various power flows along with E[Πs]. In this case, the problem reduces to solving 2D PDEs. As before, all imposed constraints are respected, and results are now closer to the 3D case. To conclude this section, we compare the performance of the three cases by evalu￾ating the objective function, J, and … view at source ↗
Figure 10
Figure 10. Figure 10: Expectation of Is and functions ICS (s) and I(s) for 2D case. 0 5 10 15 20 25 Time (h) 6 4 2 0 2 4 Energy (MWh) 40 60 80 100 120 140 160 180 Price of electricity ( /MWh) E[ bat] E[ ] [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Expectation of Πs and Es for 2D case. although the improvement is modest. This explains the close similarity observed in the expected control actions and battery energy levels between the 2D and 3D formula￾tions. 6. Conclusion This paper builds on the framework introduced in [19] for wind farms, extending it to the joint optimization of real-time energy management and day-ahead market partic￾ipation for P… view at source ↗
Figure 12
Figure 12. Figure 12: Expectation of Pbat,s , Psolar,s , Pgrid,s and Πs , and optimal power bid P˜ grid(s) for 2D case [PITH_FULL_IMAGE:figures/full_fig_p021_12.png] view at source ↗
read the original abstract

With the growing global emphasis on sustainability and the implementation of contemporary environmental policies, photovoltaic (PV) generation is playing an increasingly important role in modern power systems, while its intrinsic variability poses challenges for real-time operation and electricity market participation. This paper proposes a continuous-time stochastic optimal control framework for the joint optimization of real-time battery management and day-ahead market bidding of PV plants with energy storage. Solar irradiance, electricity prices, and battery dynamics are modeled through stochastic differential equations (SDEs), leading to a constrained stochastic control problem characterized by a coupled Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck (FP) formulation. To mitigate the associated computational burden, a dimension-reduction strategy is introduced by decomposing the state space into controllable and uncontrollable components, yielding lower-dimensional optimality systems while preserving the continuous-time Dynamic Programming structure. Numerical results show that the reduced formulations achieve substantial computational savings, enabling real-time applicability without significant loss of performance. The proposed methodology is benchmarked against two rule-based strategies and a stochastic Model Predictive Control (MPC) approach, highlighting a favorable trade-off in terms of economic performance, computational efficiency, and suitability for day-ahead market participation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The manuscript develops a continuous-time stochastic optimal control model for joint real-time battery management and day-ahead market bidding in a PV plant with storage. Solar irradiance, electricity prices, and battery SOC are modeled as SDEs, yielding a state-constrained problem whose solution is characterized by a coupled HJB-FP system. A dimension-reduction technique is introduced that splits the state into controllable (battery) and uncontrollable (irradiance/price) components, producing lower-dimensional optimality equations while preserving the dynamic-programming structure. Numerical experiments report substantial computational savings relative to the full system and competitive economic performance against rule-based heuristics and stochastic MPC.

Significance. The work targets a practically relevant problem in renewable integration and electricity-market participation. If the dimension reduction is shown to preserve optimality under the stated constraints, the approach would provide a scalable route to real-time stochastic control for high-dimensional energy systems. The explicit benchmarking against established methods supplies concrete evidence of the claimed trade-off between accuracy and speed.

major comments (2)
  1. [§4.2, Eq. (15)] §4.2, Eq. (15): The reduced HJB equation is obtained by integrating the joint density over the uncontrollable states. The manuscript does not demonstrate that this marginalization leaves the admissible control set unchanged when the state constraint (e.g., instantaneous power balance or SOC limits) depends non-separably on both battery and PV output; without such a proof the optimality claim for the original constrained problem is not yet established.
  2. [§5.1, Table 2] §5.1, Table 2: The reported performance gains and constraint-satisfaction rates are given only for the reduced model. A direct side-by-side comparison of the full versus reduced policies under identical active-constraint realizations (SOC bounds and power limits) is required to quantify any optimality gap introduced by the decomposition.
minor comments (3)
  1. [§3] The notation distinguishing the value function V from the probability density p in the coupled HJB-FP system should be made explicit in the first appearance of the optimality conditions.
  2. [Figure 4] Figure 4: The color scale for the value-function surface is not labeled; adding a color bar would improve readability of the reported policy surfaces.
  3. [§2.1] A short discussion of how the chosen SDE parameters for irradiance and price were calibrated (or taken from literature) would help readers assess reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed review. The comments highlight important aspects of the dimension-reduction approach and its validation. We address each major comment below and indicate the planned revisions.

read point-by-point responses
  1. Referee: [§4.2, Eq. (15)] The reduced HJB equation is obtained by integrating the joint density over the uncontrollable states. The manuscript does not demonstrate that this marginalization leaves the admissible control set unchanged when the state constraint (e.g., instantaneous power balance or SOC limits) depends non-separably on both battery and PV output; without such a proof the optimality claim for the original constrained problem is not yet established.

    Authors: We thank the referee for this observation. In our formulation the SOC bounds act only on the controllable battery state, while instantaneous power-balance constraints are enforced in expectation via the reduced FP equation; the control set is defined as a function of battery state alone and is therefore invariant under marginalization over the uncontrollable states. Nevertheless, the manuscript does not contain an explicit proposition establishing this invariance for the general non-separable case. We will add a short proposition in §4.2 together with the required separability assumptions on the constraint functions. revision: yes

  2. Referee: [§5.1, Table 2] The reported performance gains and constraint-satisfaction rates are given only for the reduced model. A direct side-by-side comparison of the full versus reduced policies under identical active-constraint realizations (SOC bounds and power limits) is required to quantify any optimality gap introduced by the decomposition.

    Authors: We agree that a quantitative assessment of the optimality gap is desirable. Solving the full-dimensional coupled HJB-FP system at the same spatial-temporal resolution and scenario count used for the reduced model exceeds available computational resources, which is the central motivation for the reduction. We will therefore augment §5.1 with a limited but direct comparison performed on a shorter time horizon and coarser grid for a representative subset of scenarios, allowing policy evaluation under identical active-constraint realizations and providing an estimate of the gap. revision: partial

Circularity Check

0 steps flagged

No circularity: derivation rests on standard stochastic control theory with explicit dimension reduction

full rationale

The paper introduces a dimension-reduction strategy by splitting the state into controllable (battery) and uncontrollable (irradiance/price) components, then solves the resulting lower-dimensional HJB-FP system. This is presented as a modeling choice that preserves the continuous-time dynamic programming structure and optimality of the policy. No equations are shown that define a quantity in terms of itself, no fitted parameters are relabeled as predictions, and no load-bearing step reduces to a self-citation whose validity depends on the present work. The approach is benchmarked against external rule-based and MPC strategies, indicating the central claims are not forced by construction from the paper's own inputs. The skeptic concern about constraint coupling is a question of mathematical correctness rather than circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The framework rests on standard domain assumptions of stochastic differential equations for irradiance, prices, and battery dynamics together with the dynamic-programming principle; no new free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption Solar irradiance, electricity prices, and battery dynamics are adequately modeled by stochastic differential equations.
    Explicitly stated as the modeling step that leads to the constrained stochastic control problem.
  • domain assumption The continuous-time dynamic programming principle remains valid after state-space decomposition into controllable and uncontrollable components.
    Invoked to justify that the reduced optimality systems still solve the original problem.

pith-pipeline@v0.9.0 · 5747 in / 1327 out tokens · 49275 ms · 2026-05-19T09:39:56.447089+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

  1. [1]

    Energy Technology Perspectives 2024,

    “Energy Technology Perspectives 2024,” IEA, Paris, France,

  2. [2]

    Available: https://www.iea.org/reports/ energy-technology-perspectives-2024

    [Online]. Available: https://www.iea.org/reports/ energy-technology-perspectives-2024

  3. [3]

    Renewables 2024,

    “Renewables 2024,” IEA, Paris, France, 2024. [Online]. Available: https:// www.iea.org/reports/renewables-2024

  4. [4]

    Grid Integration Challenges and Solution Strategies for Solar PV Systems: A Review,

    M. Shafiullah, S.D. Ahmed and F.A. Al-Sulaiman, “Grid Integration Challenges and Solution Strategies for Solar PV Systems: A Review,” IEEE Access, vol. 10, pp. 52233–52257, 2022, doi: 10.1109/ACCESS.2022.3174555

  5. [5]

    Energy Management and Control System for Laboratory Scale Microgrid Based Wind-PV-Battery,

    A. Merabet, K. Tawfique Ahmed, H. Ibrahim, R. Beguenane and A. M. Y . M. Ghias, “Energy Management and Control System for Laboratory Scale Microgrid Based Wind-PV-Battery,”IEEE Trans. Sustain. Energy, vol. 8, no. 1, pp. 145–154, Jan. 2017, doi: 10.1109/TSTE.2016.2587828

  6. [6]

    Dynamic Modeling and Operation Strategy for a Mi- crogrid With Wind and Photovoltaic Resources,

    S. Bae and A. Kwasinski, “Dynamic Modeling and Operation Strategy for a Mi- crogrid With Wind and Photovoltaic Resources,” IEEE Trans. Smart Grid, vol. 3, no. 4, pp. 1867–1876, Dec. 2012, doi: 10.1109/TSG.2012.2198498

  7. [7]

    A Novel Holistic Energy Management System Incorporating PV Generation and Battery Storage for Commercial Customers,

    M. A. Efstratiadi, S. Tsakanikas, P. Papadopoulos and D. R. Salinas,“A Novel Holistic Energy Management System Incorporating PV Generation and Battery Storage for Commercial Customers,” IEEE Trans. Sustain. Energy, vol. 15, no. 3, pp. 1475–1485, Jul. 2024, doi: 10.1109/TSTE.2024.3353224. 22

  8. [8]

    Stochastic programming approach for optimal day-ahead market bid- ding curves of a microgrid,

    R. Herding, E. Ross, W. R. Jones, V . M. Charitopoulos and L. G. Papageor- giou, “Stochastic programming approach for optimal day-ahead market bid- ding curves of a microgrid,” Applied Energy , vol. 336, p. 120847, 2023, doi: 10.1016/j.apenergy.2023.120847

  9. [9]

    Optimal Power Flow Manage- ment for Grid Connected PV Systems With Batteries,

    Y . Riffonneau, S. Bacha, F. Barruel and S. Ploix,“Optimal Power Flow Manage- ment for Grid Connected PV Systems With Batteries,” IEEE Trans. Sustain. En- ergy, vol. 2, no. 3, pp. 309–320, Jul. 2011, doi: 10.1109/TSTE.2011.2114901

  10. [10]

    Scenario-based multi-objective optimization strategy for rural PV-battery systems,

    Y . Zhi and X. Yang, “Scenario-based multi-objective optimization strategy for rural PV-battery systems,” Applied Energy , vol. 345, p. 121314, 2023, doi: 10.1016/j.apenergy.2023.121314

  11. [11]

    Probabilistic forecasts of solar irradiance using stochastic differential equations,

    E. B. Iversen, J. M. Morales, J. K. Møller and H. Madsen, “Probabilistic forecasts of solar irradiance using stochastic differential equations,”Environmetrics, vol. 25, no. 3, pp. 152–164, 2014, doi: 10.1002/env.2267

  12. [12]

    Day-Ahead Probabilistic Fore- cast of Solar Irradiance: A Stochastic Di fferential Equation Approach,

    J. Badosa, E. Gobet, M. Grangereau and D. Kim, “Day-Ahead Probabilistic Fore- cast of Solar Irradiance: A Stochastic Di fferential Equation Approach,” in Re- newable Energy: Forecasting and Risk Management (FRM 2017) , P. Drobinski, M. Mougeot, D. Picard, R. Plougonven, and P. Tankov, Eds., Springer Proceed- ings in Mathematics & Statistics, vol 254., pp....

  13. [13]

    Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics,

    Q. Chen, Z. Kuang, X. Liu and T. Zhang, “Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics,”Applied Energy, vol. 364, p. 123163, 2024, doi:10.1016/j.apenergy.2024.123163

  14. [14]

    Comparative study of the dy- namic programming-based and rule-based operation strategies for grid-connected PV-battery systems of office buildings,

    B. Zou, J. Peng, S. Li, Y . Li, J. Yan and H. Yang, “Comparative study of the dy- namic programming-based and rule-based operation strategies for grid-connected PV-battery systems of office buildings,”Applied Energy, vol. 305, p. 117875, 2022, doi: 10.1016/j.apenergy.2021.117875

  15. [15]

    Optimal energy management for a con- nected microgrid using dynamic programming method,

    W. Chouaf, A. Abbou and A. Agga, “Optimal energy management for a con- nected microgrid using dynamic programming method,” in 2022 8th International Conference on Optimization and Applications (ICOA), Genoa, Italy, 2022, pp. 1-6, doi: 10.1109/ICOA55659.2022.9934437

  16. [16]

    A stochastic optimal control solution to the energy management of a microgrid with storage and renewables,

    A. Belloni, L. Piroddi and M. Prandini, “A stochastic optimal control solution to the energy management of a microgrid with storage and renewables,” in 2016 American Control Conference (ACC) , Boston, MA, USA, 2016, pp. 2340-2345, doi: 10.1109/ACC.2016.7525267

  17. [17]

    Bardi, and I

    M. Bardi, and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solu- tions of Hamilton-Jacobi-Bellman Equations. Boston: Birkh ¨auser, 1997, doi: 10.1007/978-0-8176-4755-1. 23

  18. [18]

    On the Connection be- tween the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks,

    M. Annunziato, A. Borz `ı, F. Nobile and R. Tempone, “On the Connection be- tween the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks,” Applied Mathematics, vol. 5, pp. 2476–2484, 2014. doi: 10.4236/am.2014.516239

  19. [19]

    State Constrained Optimal Control via the Fokker-Planck Equation,

    P. Rutquist, T. Wik and C. Breitholtz, “State Constrained Optimal Control via the Fokker-Planck Equation,”IFAC-PapersOnLine, vol. 50, no. 1, pp. 6303–6307, 2017, doi: 10.1016/j.ifacol.2017.08.893

  20. [20]

    Stochastic control with state constraints via the Fokker-Planck equation. Application to renewable energy plants with batteries,

    A. Berm ´udez and I. Pad ´ın, “Stochastic control with state constraints via the Fokker-Planck equation. Application to renewable energy plants with batteries,” Comptes Rendus. M´ ecanique, The scientific legacy of Roland Glowinski, vol. 351, no. S1, pp. 89–110, 2023, doi: 10.5802/crmeca.236

  21. [21]

    Risken, The Fokker-Planck Equation: Methods of Solution and Applications, 2nd ed., Springer Series in Synergetics, vol

    H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications, 2nd ed., Springer Series in Synergetics, vol. 18, Springer, 1996

  22. [22]

    Partial di fferential equation methods for stochastic dynamic optimization: an application to wind power generation with energy storage,

    P. Johnson, S. Howell and P. Duck, “Partial di fferential equation methods for stochastic dynamic optimization: an application to wind power generation with energy storage,” Philos. Trans. A Math. Phys. Eng. Sci. , vol. 375, no. 2100, p. 20160301, 2017, doi: 10.1098/rsta.2016.0301

  23. [23]

    Fortin and R

    M. Fortin and R. Glowinski, Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems,Amsterdam: North Holland, 1983

  24. [24]

    Chance-Constrained Programming,

    A. Charnes and W. W. Cooper, “Chance-Constrained Programming,” Manage. Sci., vol. 6, no. 1, pp. 73–79, 1959

  25. [25]

    Duality methods for solving variational inequal- ities,

    A. Berm ´udez and C. Moreno, “Duality methods for solving variational inequal- ities,” Comput. Math. Appl. , vol. 7, no. 1, pp. 43–58, 1981, doi: 10.1016 /0898- 1221(81)90006-7

  26. [26]

    R. T. Rockafellar, Convex Analysis, Princeton: Princeton University Press, 1997

  27. [27]

    Three ways to solve partial di fferential equa- tions with neural networks - A review,

    J. Blechschmidt and O. G. Ernst, “Three ways to solve partial di fferential equa- tions with neural networks - A review,” GAMM-Mitteilungen, vol. 44, no. 2, e202100006, 2021, doi: 10.1002/gamm.202100006

  28. [28]

    OMIE - Operador del Mercado Ib ´erico de Energ ´ıa, Daily market results - Hourly prices (21 Dec. 2022). Available: https://www.omie.es/es/ market-results/daily/daily-market/daily-hourly-price?scope= daily&date=2022-12-21 [Accessed: 6 Jun. 2025]. 24