Regress-Later Monte Carlo for Optimal Inventory Control with applications in energy

We develop a Monte-Carlo based numerical method for solving discrete-time stochastic optimal control problems with inventory. These are optimal control problems in which the control affects only a deterministically evolving inventory process on a compact state space while the random underlying process manifests itself through the objective functional. We propose a Regress Later modification of the traditional Regression Monte Carlo which allows to decouple inventory levels in two successive time steps and to include in the basis functions of the regression the dependence on the inventory levels. We develop a backward construction of trajectories for the inventory which enables us to use policy iteration of Longstaff-Schwartz type avoiding nested simulations. Our algorithm improves on the grid discretisation procedure largely used in literature and practice, and on the recently proposed control randomisation by [Kharroubi et al. (2014) Monte Carlo Methods and Applications, 20(2), pp. 145-165]. We validate our approach on three numerical examples: a benchmark problem of energy arbitrage used to compare different methods available in literature; a multi-dimensional problem of control of two connected water reservoirs; and a high-dimensional problem of the management of a battery with the purpose of assisting the operations of a wind turbine in providing electricity to a group of buildings in a cost effective way.