pith. sign in

arxiv: 2506.10506 · v4 · submitted 2025-06-12 · 🧮 math.OC · cs.NA· math.NA

On a mean-field Pontryagin minimum principle for stochastic optimal control

Manfred Opper , Sebastian Reich This is my paper

Pith reviewed 2026-05-19 09:59 UTC · model grok-4.3

classification 🧮 math.OC cs.NAmath.NA
keywords mean-fieldPontryagin minimum principlestochastic optimal controlHamiltonian structuregauge freedomMcKean-Vlasovboundary value problemmean-field ODE
0
0 comments X

The pith

A deterministic mean-field Pontryagin minimum principle for stochastic optimal control is derived using auxiliary functions with gauge freedom for decoupling.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a mean-field deterministic version of the Pontryagin minimum principle that applies to stochastic optimal control problems. This formulation avoids forward-backward stochastic differential equations by introducing a pair of auxiliary functions that create a Hamiltonian structure. Gauge freedom in one auxiliary function allows the forward and reverse time equations to be decoupled, which simplifies solving the associated boundary value problem. For infinite-horizon discounted cost problems, the approach reduces computation of the optimal control to a pair of forward mean-field ordinary differential equations. The method is illustrated numerically on controlled inverted pendulum, Lorenz-63, and Lorenz-96 systems and extends in principle to more general mean-field control settings.

Core claim

The McKean-Pontryagin minimum principle is a deterministic mean-field type extension of the classical Pontryagin minimum principle to stochastic optimal control. It is realized by introducing a pair of auxiliary functions that recover Hamiltonian structure, where a gauge freedom in the choice of one function is used to decouple the forward and reverse time equations and thereby simplify the solution of the underlying boundary value problem. In the infinite-horizon discounted case the mean-field formulation converts the task of finding the optimal control law into the solution of a pair of forward mean-field ordinary differential equations.

What carries the argument

A pair of auxiliary functions that recover the Hamiltonian structure in the mean-field formulation, with gauge freedom in one function used to decouple forward and reverse time equations.

If this is right

  • Stochastic optimal control boundary value problems become solvable after decoupling via the gauge choice.
  • Infinite-horizon discounted problems reduce to solving a pair of forward mean-field ordinary differential equations.
  • The formulation applies to linear-quadratic problems and extends to general mean-field type control problems.
  • Numerical solution is feasible for low- and moderate-dimensional systems such as controlled pendulums and Lorenz attractors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same auxiliary-function construction and gauge choice could be explored in mean-field games or other stochastic differential game settings.
  • Adaptation of the auxiliary functions might allow the method to handle nonlinear costs or state constraints beyond the linear-quadratic case.
  • The reduction to forward-only equations could enable faster real-time implementations in applications with continuous uncertainty.

Load-bearing premise

The existence and suitable regularity of the pair of auxiliary functions that recover the Hamiltonian structure and permit gauge choice for decoupling in the mean-field formulation.

What would settle it

No pair of auxiliary functions with the required regularity exists for a simple stochastic linear-quadratic control problem, or the proposed decoupling fails to produce accurate optimal controls in numerical tests on the inverted pendulum or Lorenz systems.

Figures

Figures reproduced from arXiv: 2506.10506 by Manfred Opper, Sebastian Reich.

Figure 1
Figure 1. Figure 1: Upper right panel: Time evolution of the ensemble mean angle and ensemble mean velocity. Upper left panel: Control law (82) at final time as function of (θ, v). Lower right panel: Control law (82) at final time as function of θ. Lower left panel: Control law (82) at final time as function of v. with equations of motion ˙θt = vt (80a) , v˙ (80b) t = sin(θt) − σvt + cos(θt)Ut and σ = 5. Consider the running … view at source ↗
Figure 2
Figure 2. Figure 2: Controlled Lorenz-63 model: Displayed is the three￾dimensional trajectory of the particle {X (i) t } with i = 1 over the time interval t ∈ [0, 100]. After an initial transient, the trajectory enters a quasi-period orbit. The discrete Schr¨odinger bridge based formulation from Section 6.2 has been imple￾mented with ensemble size M = 200 and ε = 2∆t. The resulting evolution equations (47) have been simulated… view at source ↗
Figure 3
Figure 3. Figure 3: Controlled Lorenz-63 model: Displayed is the dependence of the computed controls U (i) tend as a function of the three components of X (i) tend at final time tend = 100 for i = 1, . . . , M, M = 100. McKean–Pontryagin formulation is able to stabilize the unstable equilibrium (0, 0)T. The remaining fluctuations in the ensemble are due to the non-vanishing diffusion which guarantee an exploration of state sp… view at source ↗
Figure 4
Figure 4. Figure 4: NMPC for inverted pendulum: Displayed are the time evo￾lution of the ensemble of angles and velocities as a function of time. Starting from an ensemble centered at (1, 0)T, the controlled ensemble eventually samples the vicinity of the unstable equilibrium (0, 0)T. The magnitude of the fluctuations depends on the magnitude of the added diffusivity. Those fluctuations decrease as Σ → 0. ensemble Kalman filt… view at source ↗
read the original abstract

This paper outlines a novel extension of the classical Pontryagin minimum (maximum) principle to stochastic optimal control problems. Contrary to the well-known stochastic Pontryagin minimum principle involving forward-backward stochastic differential equations, the proposed formulation is deterministic and of mean-field type. We denote it by the McKean-Pontryagin minimum principle. The Hamiltonian structure of the proposed McKean-Pontryagin minimum principle is achieved via the introduction of a pair of auxiliary functions. A gauge freedom in the choice of one of these two functions can be used to decouple the forward and reverse time equations; hence simplifying the solution of the underlying boundary value problem. We also consider infinite horizon discounted cost optimal control problems. In this case, the mean-field formulation allows one to convert the computation of the desired optimal control law into solving a pair of forward mean-field ordinary differential equations. The McKean-Pontryagin minimum principle is tested numerically for a controlled inverted pendulum, a controlled Lorenz-63 system, and a controlled Lorenz-96 system. Although the focus is on linear-quadratic control problems, the proposed methodology is extendable to more general problems including mean-field type control formulations.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript proposes a McKean-Pontryagin minimum principle as a deterministic mean-field extension of the classical Pontryagin minimum principle for stochastic optimal control. The Hamiltonian structure is recovered via a pair of auxiliary functions, with gauge freedom in one function used to decouple the forward and reverse-time equations. For infinite-horizon discounted problems the formulation reduces the optimal control law to a pair of forward mean-field ODEs. The approach is tested numerically on linear-quadratic problems for a controlled inverted pendulum and controlled Lorenz-63/96 systems, and is claimed to extend to general mean-field type control.

Significance. If the auxiliary functions can be shown to exist with the required regularity, the result would supply a deterministic alternative to classical stochastic Pontryagin principles that rely on forward-backward SDEs, potentially simplifying boundary-value problems in stochastic and mean-field control. The gauge-freedom decoupling is a technically attractive feature. Numerical illustrations on the inverted pendulum and Lorenz systems provide preliminary evidence of applicability, though without quantitative error analysis or convergence rates. The explicit treatment of the infinite-horizon discounted case is a positive aspect.

major comments (1)
  1. Derivation of the McKean-Pontryagin minimum principle (sections following the abstract): the central claim rests on the existence and suitable regularity of the pair of auxiliary functions introduced to recover the Hamiltonian structure and to permit a gauge choice that decouples the forward and reverse equations. No existence theorem, fixed-point argument, or regularity conditions (Lipschitz continuity, growth bounds, etc.) are supplied to guarantee that these functions are well-defined for the underlying stochastic processes or in the infinite-horizon discounted setting. Without such justification the asserted simplification of the boundary-value problem and the deterministic mean-field reduction remain unsecured.
minor comments (2)
  1. Numerical experiments section: the tests on LQ problems and Lorenz systems report no quantitative error analysis, convergence rates, or direct comparison against standard FBSDE solvers, which limits assessment of practical accuracy and computational advantage.
  2. References: the manuscript would benefit from additional citations to recent literature on mean-field stochastic control and infinite-dimensional Pontryagin principles to better situate the contribution.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and constructive feedback on our manuscript. We address the major comment point by point below, with a commitment to strengthening the theoretical foundations where needed.

read point-by-point responses

  1. Referee: Derivation of the McKean-Pontryagin minimum principle (sections following the abstract): the central claim rests on the existence and suitable regularity of the pair of auxiliary functions introduced to recover the Hamiltonian structure and to permit a gauge choice that decouples the forward and reverse equations. No existence theorem, fixed-point argument, or regularity conditions (Lipschitz continuity, growth bounds, etc.) are supplied to guarantee that these functions are well-defined for the underlying stochastic processes or in the infinite-horizon discounted setting. Without such justification the asserted simplification of the boundary-value problem and the deterministic mean-field reduction remain unsecured.

    Authors: We acknowledge that the manuscript introduces the auxiliary functions to recover the Hamiltonian structure and enable gauge-based decoupling but does not supply a general existence theorem or fixed-point argument with explicit regularity conditions such as Lipschitz continuity or growth bounds. In the linear-quadratic examples, the functions are constructed explicitly via the associated mean-field Riccati equations and ODEs, which are well-posed under standard matrix assumptions. For the general case and infinite-horizon discounted setting, we agree that additional justification is required. In the revised manuscript we will add a subsection stating sufficient conditions (Lipschitz drift/diffusion, linear growth, and discounting for contraction) and sketching a fixed-point argument for existence of the auxiliary functions as solutions to the deterministic mean-field equations. This will secure the claimed simplification of the boundary-value problem. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation builds on classical Pontryagin and mean-field principles

full rationale

The paper extends the classical Pontryagin minimum principle to a deterministic mean-field formulation for stochastic optimal control by introducing a pair of auxiliary functions to recover Hamiltonian structure and exploiting gauge freedom to decouple forward and reverse equations. This is framed as a novel but direct extension without any reduction of the central McKean-Pontryagin principle to a fitted parameter, self-defined quantity, or self-citation chain by construction. Numerical tests on the inverted pendulum and Lorenz systems function as external validation rather than internal forcing. The derivation remains self-contained against the stated classical benchmarks and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The formulation rests on the introduction of auxiliary functions whose existence is assumed to achieve the Hamiltonian structure; no free parameters or invented physical entities are apparent from the abstract.

axioms (1)
  • domain assumption Existence of auxiliary functions that recover Hamiltonian structure in the mean-field setting
    Invoked to define the McKean-Pontryagin principle and enable decoupling.

pith-pipeline@v0.9.0 · 5740 in / 1134 out tokens · 21666 ms · 2026-05-19T09:59:54.562575+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Digital Twins: McKean-Pontryagin Control for Partially Observed Physical Twins

    math.OC 2025-10 unverdicted novelty 6.0

    The authors derive forward mean-field equations that integrate ensemble Kalman filtering with McKean-Pontryagin control to enable simultaneous online data assimilation and optimal control for partially observed stocha...

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Stochastic Control of Partially Observable Systems

    Alain Bensoussan. Stochastic Control of Partially Observable Systems . Cambridge University Press, Cambridge, 1992

    work page 1992

  2. [2]

    Estimation and Control of Dynamical Systems

    Alain Bensoussan. Estimation and Control of Dynamical Systems . Springer, Cham, 2018

    work page 2018

  3. [3]

    Herman J. Bierens. The Nadaraya–Watson kernel regressi on function estimator. In Topics in Advanced Econometrics, page 212–247. Cambridge University Press, New York, 1994

    work page 1994

  4. [4]

    Edoardo Calvello, Sebastian Reich, and Andrew M. Stuart . Ensemble Kalman methods: A mean field perspective. Acta Numerica, 34:123–291, 2025

    work page 2025

  5. [5]

    SIAM, Philadelphia, 2016

    Ren´ e Carmona.Lectures on BSDEs, Stochastic Control, and Stochastic Differ ential Games with Financial Applications . SIAM, Philadelphia, 2016

    work page 2016

  6. [6]

    Crandall and Pierre-Louis Lions

    Michael G. Crandall and Pierre-Louis Lions. Viscosity s olutions of Hamilton-Jacobi equations. Transactions of the American Mathematical Society , 277, 1983

    work page 1983

  7. [7]

    Paul A.M. Dirac. Generalized Hamiltonian dynamics. Can. J. Math. , 2:129–148, 1950

    work page 1950

  8. [8]

    Deep learning- based numerical methods for high-dimensional parabolic partial differential equati ons and backward stochastic differential equations

    Weinan E, Jiequn Han, and Arnulf Jentzen. Deep learning- based numerical methods for high-dimensional parabolic partial differential equati ons and backward stochastic differential equations. Communications in Mathematics and Statistics , 5:349–380, 2017. 20 MANFRED OPPER AND SEBASTIAN REICH

    work page 2017

  9. [9]

    Algorithms for solving high-dimensional PDEs: From nonlinear Monte Carlo to machine learning

    Weinan E, Jiequn Han, and Arnulf Jentzen. Algorithms for solving high-dimensional PDEs: From nonlinear Monte Carlo to machine learning. Nonlinearity, 35:278, 2021

    work page 2021

  10. [10]

    Vossepoel, and Peter Jan

    Geir Evensen, Femke C. Vossepoel, and Peter Jan. van Lee uwen. Data Assimila- tion Fundamentals: A unified Formulation of the State and Par ameter Estimation Problem. Springer Nature Switzerland AG, Cham, Switzerland, 2022

    work page 2022

  11. [11]

    An introduction to nonlinear model predictive control

    Rolf Findeisen and Frank Allg¨ ower. An introduction to nonlinear model predictive control. In A. G. Jager, de and H. J. Zwart, editors, Systems and control : 21th Benelux meeting 2002 , pages 119–141. Technische Universiteit Eindhoven, 2002

    work page 2002

  12. [12]

    Gottwald, Fengyi Li, Sebastian Reich, and Yous sef Marzouk

    Georg A. Gottwald, Fengyi Li, Sebastian Reich, and Yous sef Marzouk. Stable gen- erative modeling using Schr¨ odinger bridges. Phil. Trans. R. Soc. A , 383:20240332, 2025

    work page 2025

  13. [13]

    Gottwald, Shuigen Liu, Youssef Marzouk, Sebas tian Reich, and Xin T

    Georg A. Gottwald, Shuigen Liu, Youssef Marzouk, Sebas tian Reich, and Xin T. Tong. Localized diffusion models for high dimensional distri butions generation. Technical report, arXiv:2505.04417, 2025

    work page arXiv 2025

  14. [14]

    Geomeric Mechanics

    Darryl Holm. Geomeric Mechanics. Part I: Dynamics and Symmetry . World Sci- entific Publishing, Singapore, 2nd edition, 2008

    work page 2008

  15. [15]

    An efficient on-policy deep learning framework for stochastic optimal control

    Mengjian Hua, Mathieu Lauriere, and Eric Vanden-Eijnd en. An efficient on-policy deep learning framework for stochastic optimal control. In Frontiers in Probabilistic Inference: Learning meets Sampling , 2025

    work page 2025

  16. [16]

    Joshi, Amirhossein Taghvaei, Prashant G

    Anant A. Joshi, Amirhossein Taghvaei, Prashant G. Meht a, and Sean P. Meyn. Con- trolled interacting particle algorithms for simulation-b ased reinforcement learning. Systems & Control Letters , 170:105392, 2022

    work page 2022

  17. [17]

    Joshi, Amirhossein Taghvaei, Prashant G

    Anant A. Joshi, Amirhossein Taghvaei, Prashant G. Meht a, and Sean P. Meyn. Dual ensemble Kalman filter for stochastic optimal control. In 2024 IEEE 63rd Conference on Decision and Control (CDC) , pages 1917–1922, 2024

    work page 2024

  18. [18]

    Leading the Lor enz 63 system towards the prescribed regime by model predictive control coupled w ith data assimilation

    Fumitoshi Kawasaki and Shunji Kotsuki. Leading the Lor enz 63 system towards the prescribed regime by model predictive control coupled w ith data assimilation. Nonlin. Processes Geophys., 31:319–333, 2024

    work page 2024

  19. [19]

    Deterministic nonperiodic flow

    Edward N Lorenz. Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20(2):130–141, 1963

    work page 1963

  20. [20]

    Deterministic part icle flows for constraining stochastic nonlinear systems

    Dimitra Maoutsa and Manfred Opper. Deterministic part icle flows for constraining stochastic nonlinear systems. Phys. Rev. Res. , 4:043035, 2022

    work page 2022

  21. [21]

    I nteracting particle solu- tions of Fokker–Planck equations through gradient-log-de nsity estimation

    Dimitra Maoutsa, Sebastian Reich, and Manfred Opper. I nteracting particle solu- tions of Fokker–Planck equations through gradient-log-de nsity estimation. Entropy, 22(8), 2020

    work page 2020

  22. [22]

    Marshall and Ronald R

    Nicholas F. Marshall and Ronald R. Coifman. Manifold le arning with bi-stochastic kernels. IMA J. Appl. Maths. , 84:455–482, 2019

    work page 2019

  23. [23]

    Mehta and Sean P

    Prashant G. Mehta and Sean P. Meyn. A feedback particle fi lter-based approach to optimal control with partial observations. In 52nd IEEE Conference on Decision and Control, pages 3121–3127, 2013

    work page 2013

  24. [24]

    Control Systems and Reinforcement Learning

    Sean Meyn. Control Systems and Reinforcement Learning . Cambridge University Press, Cambridge, 2022

    work page 2022

  25. [25]

    Nikolas N¨ usken and Lorenz Richter. Solving high-dime nsional Hamilton-Jacobi- Bellman PDEs using neural networks: Perspectives from the t heory of controlled ON A MEAN-FIELD PONTRYAGIN MINIMUM PRINCIPLE 21 diffusions and measures on path space. Partial Differential Equations and Applica- tions, 2:48, 2021

    work page 2021

  26. [26]

    Pavliotis

    Grigorios A. Pavliotis. Stochastic Processes and Applications. Springer Verlag, New York, 2016

    work page 2016

  27. [27]

    Backward stochastic differential equations and applications to optimal control

    Shige Peng. Backward stochastic differential equations and applications to optimal control. Applied Mathematics and Optimization , 27:125–144, 1993

    work page 1993

  28. [28]

    Pointryagin, V.G

    L.S. Pointryagin, V.G. Boltyanskii, R.V. Gamkrelidze , and E.F. Mihchenko. The Mathematical Theory of Optimal Processes . John Wiley & Sons, New York, 1962

    work page 1962

  29. [29]

    Optimal control of Markov processes with incomple te state information

    Karl Johan ˚ Astr¨ om. Optimal control of Markov processes with incomple te state information. Journal of Mathematical Analysis and Applications , 10:174–205, 1965

    work page 1965

  30. [30]

    Rawlings, David Q

    James B. Rawlings, David Q. Mayne, and Moritz M. Diehl. Model Predictive Con- trol: Theory, Computation, and Design . Nob Hill Publishing, Madison, 2nd edition, 2018

    work page 2018

  31. [31]

    Ensemble Kalman-Bucy filtering for no nlinear model predictive control

    Sebastian Reich. Ensemble Kalman-Bucy filtering for no nlinear model predictive control. Technical report, arXiv:2503.12474, 2025

    work page arXiv 2025

  32. [32]

    Particle-based algorithms for stoch astic optimal control

    Sebastian Reich. Particle-based algorithms for stoch astic optimal control. In Bertrand Chapron, Dan Crisan, Darryl Holm, Etienne M´ emin, and Jane-Lisa Coughlan, editors, Stochastic Transport in Upper Ocean Dynamics III , pages 243–

    work page

  33. [33]

    Springer Nature Switzerland, Cham, 2025

    work page 2025

  34. [34]

    Probabilistic Forecasting and Bayesian Data Assimilation

    Sebastian Reich and Colin Cotter. Probabilistic Forecasting and Bayesian Data Assimilation. Cambridge University Press, 2015

    work page 2015

  35. [35]

    Hamiltonian fluid mechanics

    Rick Salmon. Hamiltonian fluid mechanics. Ann. Rev. Fluid Mech. , 20:225–256, 1988

    work page 1988

  36. [36]

    Wormell and Sebastian Reich

    Caroline L. Wormell and Sebastian Reich. Spectral conv ergence of diffusion maps: Improved error bounds and an alternative normalisation. SIAM J. Numer. Anal. , 59:1687–1734, 2021

    work page 2021

  37. [37]

    Belief space planning: A covariance steering ap proach

    Dongliang Zheng, Jack Ridderhof, Panagiotis Tsiotras , and Ali-akbar Agha- mohammadi. Belief space planning: A covariance steering ap proach. In 2022 In- ternational Conference on Robotics and Automation (ICRA) , pages 11051–11057, 2022. Institut f ¨ur Softw aretechnik und Theoretische Informatik, Technisc he Universit ¨at Berlin, Marchstraße 23, 10587 Be...

    work page 2022