State constrained stochastic optimal control of a PV system with battery storage via Fokker-Planck and Hamilton-Jacobi-Bellman equations
Pith reviewed 2026-05-19 09:39 UTC · model grok-4.3
The pith
Decomposing state space into controllable and uncontrollable parts reduces the HJB-FP system for PV battery control
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper shows that decomposing the full state space into controllable battery and bidding decisions and uncontrollable solar-price dynamics produces lower-dimensional coupled HJB-FP equations whose solution yields an optimal feedback policy that can be computed fast enough for real-time use without meaningful loss of economic value.
What carries the argument
Dimension-reduction strategy that separates the state space into controllable and uncontrollable components while preserving the continuous-time dynamic-programming optimality conditions.
If this is right
- Substantial computational savings enable real-time applicability of the optimal policy.
- The optimality of the resulting feedback policy is retained after reduction.
- A favorable trade-off between economic performance and computational effort is achieved.
- The framework supports day-ahead market participation with lower runtime than stochastic MPC.
- Benchmark comparisons confirm superiority over rule-based strategies in both profit and speed.
Where Pith is reading between the lines
- The same controllable-uncontrollable split could be applied to other renewables-plus-storage problems whose exogenous drivers evolve independently of the control.
- The reduced formulation opens the possibility of adding battery-degradation or network-constraint states without restoring the original dimensionality.
- Frequent re-optimization inside the day-ahead window becomes feasible, potentially improving forecast use and reducing imbalance penalties.
- The approach may extend to multi-asset portfolios once the uncontrollable variables remain separable from the controllable ones.
Load-bearing premise
Decomposing the state space into controllable and uncontrollable components yields lower-dimensional optimality systems while preserving the continuous-time Dynamic Programming structure and the optimality of the resulting policy.
What would settle it
A side-by-side simulation in which the reduced model produces markedly higher operating costs or bidding penalties than the full-dimensional solution under realistic high-variability irradiance and price paths.
Figures
read the original abstract
With the growing global emphasis on sustainability and the implementation of contemporary environmental policies, photovoltaic (PV) generation is playing an increasingly important role in modern power systems, while its intrinsic variability poses challenges for real-time operation and electricity market participation. This paper proposes a continuous-time stochastic optimal control framework for the joint optimization of real-time battery management and day-ahead market bidding of PV plants with energy storage. Solar irradiance, electricity prices, and battery dynamics are modeled through stochastic differential equations (SDEs), leading to a constrained stochastic control problem characterized by a coupled Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck (FP) formulation. To mitigate the associated computational burden, a dimension-reduction strategy is introduced by decomposing the state space into controllable and uncontrollable components, yielding lower-dimensional optimality systems while preserving the continuous-time Dynamic Programming structure. Numerical results show that the reduced formulations achieve substantial computational savings, enabling real-time applicability without significant loss of performance. The proposed methodology is benchmarked against two rule-based strategies and a stochastic Model Predictive Control (MPC) approach, highlighting a favorable trade-off in terms of economic performance, computational efficiency, and suitability for day-ahead market participation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript develops a continuous-time stochastic optimal control model for joint real-time battery management and day-ahead market bidding in a PV plant with storage. Solar irradiance, electricity prices, and battery SOC are modeled as SDEs, yielding a state-constrained problem whose solution is characterized by a coupled HJB-FP system. A dimension-reduction technique is introduced that splits the state into controllable (battery) and uncontrollable (irradiance/price) components, producing lower-dimensional optimality equations while preserving the dynamic-programming structure. Numerical experiments report substantial computational savings relative to the full system and competitive economic performance against rule-based heuristics and stochastic MPC.
Significance. The work targets a practically relevant problem in renewable integration and electricity-market participation. If the dimension reduction is shown to preserve optimality under the stated constraints, the approach would provide a scalable route to real-time stochastic control for high-dimensional energy systems. The explicit benchmarking against established methods supplies concrete evidence of the claimed trade-off between accuracy and speed.
major comments (2)
- [§4.2, Eq. (15)] §4.2, Eq. (15): The reduced HJB equation is obtained by integrating the joint density over the uncontrollable states. The manuscript does not demonstrate that this marginalization leaves the admissible control set unchanged when the state constraint (e.g., instantaneous power balance or SOC limits) depends non-separably on both battery and PV output; without such a proof the optimality claim for the original constrained problem is not yet established.
- [§5.1, Table 2] §5.1, Table 2: The reported performance gains and constraint-satisfaction rates are given only for the reduced model. A direct side-by-side comparison of the full versus reduced policies under identical active-constraint realizations (SOC bounds and power limits) is required to quantify any optimality gap introduced by the decomposition.
minor comments (3)
- [§3] The notation distinguishing the value function V from the probability density p in the coupled HJB-FP system should be made explicit in the first appearance of the optimality conditions.
- [Figure 4] Figure 4: The color scale for the value-function surface is not labeled; adding a color bar would improve readability of the reported policy surfaces.
- [§2.1] A short discussion of how the chosen SDE parameters for irradiance and price were calibrated (or taken from literature) would help readers assess reproducibility.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed review. The comments highlight important aspects of the dimension-reduction approach and its validation. We address each major comment below and indicate the planned revisions.
read point-by-point responses
-
Referee: [§4.2, Eq. (15)] The reduced HJB equation is obtained by integrating the joint density over the uncontrollable states. The manuscript does not demonstrate that this marginalization leaves the admissible control set unchanged when the state constraint (e.g., instantaneous power balance or SOC limits) depends non-separably on both battery and PV output; without such a proof the optimality claim for the original constrained problem is not yet established.
Authors: We thank the referee for this observation. In our formulation the SOC bounds act only on the controllable battery state, while instantaneous power-balance constraints are enforced in expectation via the reduced FP equation; the control set is defined as a function of battery state alone and is therefore invariant under marginalization over the uncontrollable states. Nevertheless, the manuscript does not contain an explicit proposition establishing this invariance for the general non-separable case. We will add a short proposition in §4.2 together with the required separability assumptions on the constraint functions. revision: yes
-
Referee: [§5.1, Table 2] The reported performance gains and constraint-satisfaction rates are given only for the reduced model. A direct side-by-side comparison of the full versus reduced policies under identical active-constraint realizations (SOC bounds and power limits) is required to quantify any optimality gap introduced by the decomposition.
Authors: We agree that a quantitative assessment of the optimality gap is desirable. Solving the full-dimensional coupled HJB-FP system at the same spatial-temporal resolution and scenario count used for the reduced model exceeds available computational resources, which is the central motivation for the reduction. We will therefore augment §5.1 with a limited but direct comparison performed on a shorter time horizon and coarser grid for a representative subset of scenarios, allowing policy evaluation under identical active-constraint realizations and providing an estimate of the gap. revision: partial
Circularity Check
No circularity: derivation rests on standard stochastic control theory with explicit dimension reduction
full rationale
The paper introduces a dimension-reduction strategy by splitting the state into controllable (battery) and uncontrollable (irradiance/price) components, then solves the resulting lower-dimensional HJB-FP system. This is presented as a modeling choice that preserves the continuous-time dynamic programming structure and optimality of the policy. No equations are shown that define a quantity in terms of itself, no fitted parameters are relabeled as predictions, and no load-bearing step reduces to a self-citation whose validity depends on the present work. The approach is benchmarked against external rule-based and MPC strategies, indicating the central claims are not forced by construction from the paper's own inputs. The skeptic concern about constraint coupling is a question of mathematical correctness rather than circularity.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Solar irradiance, electricity prices, and battery dynamics are adequately modeled by stochastic differential equations.
- domain assumption The continuous-time dynamic programming principle remains valid after state-space decomposition into controllable and uncontrollable components.
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/ArithmeticFromLogic.leanabsolute_floor_iff_bare_distinguishability unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
decomposing the state space into controllable and uncontrollable components, yielding lower-dimensional optimality systems while preserving the continuous-time Dynamic Programming structure
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
the FP equation ... and the HJB equation ... optimality system
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Energy Technology Perspectives 2024,
“Energy Technology Perspectives 2024,” IEA, Paris, France,
work page 2024
-
[2]
Available: https://www.iea.org/reports/ energy-technology-perspectives-2024
[Online]. Available: https://www.iea.org/reports/ energy-technology-perspectives-2024
work page 2024
-
[3]
“Renewables 2024,” IEA, Paris, France, 2024. [Online]. Available: https:// www.iea.org/reports/renewables-2024
work page 2024
-
[4]
Grid Integration Challenges and Solution Strategies for Solar PV Systems: A Review,
M. Shafiullah, S.D. Ahmed and F.A. Al-Sulaiman, “Grid Integration Challenges and Solution Strategies for Solar PV Systems: A Review,” IEEE Access, vol. 10, pp. 52233–52257, 2022, doi: 10.1109/ACCESS.2022.3174555
-
[5]
Energy Management and Control System for Laboratory Scale Microgrid Based Wind-PV-Battery,
A. Merabet, K. Tawfique Ahmed, H. Ibrahim, R. Beguenane and A. M. Y . M. Ghias, “Energy Management and Control System for Laboratory Scale Microgrid Based Wind-PV-Battery,”IEEE Trans. Sustain. Energy, vol. 8, no. 1, pp. 145–154, Jan. 2017, doi: 10.1109/TSTE.2016.2587828
-
[6]
Dynamic Modeling and Operation Strategy for a Mi- crogrid With Wind and Photovoltaic Resources,
S. Bae and A. Kwasinski, “Dynamic Modeling and Operation Strategy for a Mi- crogrid With Wind and Photovoltaic Resources,” IEEE Trans. Smart Grid, vol. 3, no. 4, pp. 1867–1876, Dec. 2012, doi: 10.1109/TSG.2012.2198498
-
[7]
M. A. Efstratiadi, S. Tsakanikas, P. Papadopoulos and D. R. Salinas,“A Novel Holistic Energy Management System Incorporating PV Generation and Battery Storage for Commercial Customers,” IEEE Trans. Sustain. Energy, vol. 15, no. 3, pp. 1475–1485, Jul. 2024, doi: 10.1109/TSTE.2024.3353224. 22
-
[8]
Stochastic programming approach for optimal day-ahead market bid- ding curves of a microgrid,
R. Herding, E. Ross, W. R. Jones, V . M. Charitopoulos and L. G. Papageor- giou, “Stochastic programming approach for optimal day-ahead market bid- ding curves of a microgrid,” Applied Energy , vol. 336, p. 120847, 2023, doi: 10.1016/j.apenergy.2023.120847
-
[9]
Optimal Power Flow Manage- ment for Grid Connected PV Systems With Batteries,
Y . Riffonneau, S. Bacha, F. Barruel and S. Ploix,“Optimal Power Flow Manage- ment for Grid Connected PV Systems With Batteries,” IEEE Trans. Sustain. En- ergy, vol. 2, no. 3, pp. 309–320, Jul. 2011, doi: 10.1109/TSTE.2011.2114901
-
[10]
Scenario-based multi-objective optimization strategy for rural PV-battery systems,
Y . Zhi and X. Yang, “Scenario-based multi-objective optimization strategy for rural PV-battery systems,” Applied Energy , vol. 345, p. 121314, 2023, doi: 10.1016/j.apenergy.2023.121314
-
[11]
Probabilistic forecasts of solar irradiance using stochastic differential equations,
E. B. Iversen, J. M. Morales, J. K. Møller and H. Madsen, “Probabilistic forecasts of solar irradiance using stochastic differential equations,”Environmetrics, vol. 25, no. 3, pp. 152–164, 2014, doi: 10.1002/env.2267
-
[12]
J. Badosa, E. Gobet, M. Grangereau and D. Kim, “Day-Ahead Probabilistic Fore- cast of Solar Irradiance: A Stochastic Di fferential Equation Approach,” in Re- newable Energy: Forecasting and Risk Management (FRM 2017) , P. Drobinski, M. Mougeot, D. Picard, R. Plougonven, and P. Tankov, Eds., Springer Proceed- ings in Mathematics & Statistics, vol 254., pp....
-
[13]
Q. Chen, Z. Kuang, X. Liu and T. Zhang, “Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics,”Applied Energy, vol. 364, p. 123163, 2024, doi:10.1016/j.apenergy.2024.123163
-
[14]
B. Zou, J. Peng, S. Li, Y . Li, J. Yan and H. Yang, “Comparative study of the dy- namic programming-based and rule-based operation strategies for grid-connected PV-battery systems of office buildings,”Applied Energy, vol. 305, p. 117875, 2022, doi: 10.1016/j.apenergy.2021.117875
-
[15]
Optimal energy management for a con- nected microgrid using dynamic programming method,
W. Chouaf, A. Abbou and A. Agga, “Optimal energy management for a con- nected microgrid using dynamic programming method,” in 2022 8th International Conference on Optimization and Applications (ICOA), Genoa, Italy, 2022, pp. 1-6, doi: 10.1109/ICOA55659.2022.9934437
-
[16]
A. Belloni, L. Piroddi and M. Prandini, “A stochastic optimal control solution to the energy management of a microgrid with storage and renewables,” in 2016 American Control Conference (ACC) , Boston, MA, USA, 2016, pp. 2340-2345, doi: 10.1109/ACC.2016.7525267
-
[17]
M. Bardi, and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solu- tions of Hamilton-Jacobi-Bellman Equations. Boston: Birkh ¨auser, 1997, doi: 10.1007/978-0-8176-4755-1. 23
-
[18]
On the Connection be- tween the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks,
M. Annunziato, A. Borz `ı, F. Nobile and R. Tempone, “On the Connection be- tween the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks,” Applied Mathematics, vol. 5, pp. 2476–2484, 2014. doi: 10.4236/am.2014.516239
-
[19]
State Constrained Optimal Control via the Fokker-Planck Equation,
P. Rutquist, T. Wik and C. Breitholtz, “State Constrained Optimal Control via the Fokker-Planck Equation,”IFAC-PapersOnLine, vol. 50, no. 1, pp. 6303–6307, 2017, doi: 10.1016/j.ifacol.2017.08.893
-
[20]
A. Berm ´udez and I. Pad ´ın, “Stochastic control with state constraints via the Fokker-Planck equation. Application to renewable energy plants with batteries,” Comptes Rendus. M´ ecanique, The scientific legacy of Roland Glowinski, vol. 351, no. S1, pp. 89–110, 2023, doi: 10.5802/crmeca.236
-
[21]
H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications, 2nd ed., Springer Series in Synergetics, vol. 18, Springer, 1996
work page 1996
-
[22]
P. Johnson, S. Howell and P. Duck, “Partial di fferential equation methods for stochastic dynamic optimization: an application to wind power generation with energy storage,” Philos. Trans. A Math. Phys. Eng. Sci. , vol. 375, no. 2100, p. 20160301, 2017, doi: 10.1098/rsta.2016.0301
-
[23]
M. Fortin and R. Glowinski, Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems,Amsterdam: North Holland, 1983
work page 1983
-
[24]
Chance-Constrained Programming,
A. Charnes and W. W. Cooper, “Chance-Constrained Programming,” Manage. Sci., vol. 6, no. 1, pp. 73–79, 1959
work page 1959
-
[25]
Duality methods for solving variational inequal- ities,
A. Berm ´udez and C. Moreno, “Duality methods for solving variational inequal- ities,” Comput. Math. Appl. , vol. 7, no. 1, pp. 43–58, 1981, doi: 10.1016 /0898- 1221(81)90006-7
work page 1981
-
[26]
R. T. Rockafellar, Convex Analysis, Princeton: Princeton University Press, 1997
work page 1997
-
[27]
Three ways to solve partial di fferential equa- tions with neural networks - A review,
J. Blechschmidt and O. G. Ernst, “Three ways to solve partial di fferential equa- tions with neural networks - A review,” GAMM-Mitteilungen, vol. 44, no. 2, e202100006, 2021, doi: 10.1002/gamm.202100006
-
[28]
OMIE - Operador del Mercado Ib ´erico de Energ ´ıa, Daily market results - Hourly prices (21 Dec. 2022). Available: https://www.omie.es/es/ market-results/daily/daily-market/daily-hourly-price?scope= daily&date=2022-12-21 [Accessed: 6 Jun. 2025]. 24
work page 2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.