pith. sign in

arxiv: 2510.00937 · v3 · submitted 2025-10-01 · 🧮 math.OC · cs.NA· math.NA

Digital Twins: McKean-Pontryagin Control for Partially Observed Physical Twins

Manfred Opper , Sebastian Reich This is my paper

Pith reviewed 2026-05-18 10:39 UTC · model grok-4.3

classification 🧮 math.OC cs.NAmath.NA
keywords digital twinsoptimal controlensemble Kalman filterMcKean-Pontryaginmean-field equationspartial observationsdata assimilationstochastic processes
0
0 comments X

The pith

Forward-evolving mean-field equations enable simultaneous online data assimilation and control computation for partially observed processes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper addresses optimal control for processes that are only partially observed, a key requirement for digital twin setups where a physical system is monitored incompletely and control must be applied in real time. It integrates the ensemble Kalman filter for assimilating incoming observations with the McKean-Pontryagin method for stochastic optimal control. The result is a set of forward-evolving mean-field equations governing both the system states and their co-states. These equations permit updating the estimated state distribution and calculating control inputs continuously as new data arrives. Readers interested in practical control of uncertain systems would find this relevant because it avoids the need for full state knowledge or offline backward computations.

Core claim

We derive forward evolving mean-field evolution equations for states and co-states which simultaneously allow for an online assimilation of data as well as an online computation of control laws. The proposed methodology is therefore perfectly suited for real time applications of digital twins. We present numerical results for controlled Lorenz-63 and Lorenz-96 systems as well as an inverted pendulum.

What carries the argument

McKean-Pontryagin approach to stochastic optimal control combined with the ensemble Kalman filter, which replaces the full state distribution to yield tractable forward mean-field equations under partial observations.

If this is right

  • Real-time control laws can be computed while assimilating data online.
  • The method applies to partially observed diffusion processes in digital twin contexts.
  • Numerical demonstrations on Lorenz systems and inverted pendulum confirm feasibility.
  • Control is achieved without requiring complete state information at each step.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Extensions to other filtering techniques could broaden applicability to different observation models.
  • Scalability to very high-dimensional systems remains to be verified through further experiments.
  • Potential integration with model predictive control variants for hybrid approaches.

Load-bearing premise

The McKean-Pontryagin formulation remains valid and stable when the state distribution is replaced by the ensemble Kalman filter approximation under partial observations.

What would settle it

A simulation of the inverted pendulum or Lorenz system where the approximated control laws lead to instability or poor performance compared to full-observation baselines as observation noise increases.

Figures

Figures reproduced from arXiv: 2510.00937 by Manfred Opper, Sebastian Reich.

Figure 1
Figure 1. Figure 1: Controlled Lorenz-63 model. Displayed is the thre [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Controlled Lorenz-63 model. First component of th [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Controlled Lorenz-63 model. Estimation error; [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Controlled Lorenz-63 model. Same as Figure 1 but fo [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Controlled Lorenz-63 model. Same as Figure 2 but fo [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Controlled Lorenz-63 model. Same as Figure 3 but fo [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Inverted pendulum. Left panel: Controlled angle, [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
read the original abstract

Optimal control for fully observed diffusion processes is well established and has led to numerous numerical implementations based on, for example, Bellman's principle, model free reinforcement learning, Pontryagin's maximum principle, and model predictive control. In contrast, much fewer algorithms are available for optimal control of partially observed processes. However, this scenario is central to the digital twin paradigm, where a physical twin is partially observed and control laws are derived based on a digital twin. In this paper, we contribute to this challenge by combining data assimilation in the form of the ensemble Kalman filter with the recently proposed McKean-Pontryagin approach to stochastic optimal control. We derive forward evolving mean-field evolution equations for states and co-states which simultaneously allow for an online assimilation of data as well as an online computation of control laws. The proposed methodology is therefore perfectly suited for real time applications of digital twins. We present numerical results for controlled Lorenz-63 and Lorenz-96 systems as well as an inverted pendulum.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes combining the ensemble Kalman filter for data assimilation with the McKean-Pontryagin maximum principle to derive forward-evolving mean-field equations for both states and co-states. These equations enable simultaneous online assimilation of partial observations and computation of control laws for partially observed diffusion processes, with the goal of supporting real-time digital twin applications. The approach is illustrated through numerical experiments on controlled Lorenz-63 and Lorenz-96 systems as well as an inverted pendulum.

Significance. If the central derivation is valid and the EnKF approximation preserves the required adjoint dynamics, the work would offer a practical advance for online stochastic optimal control under partial observations. The forward-only structure is well-suited to real-time settings, and testing on chaotic attractors such as Lorenz-63/96 provides a relevant stress test for digital-twin scenarios.

major comments (2)
  1. [§3] §3 (derivation of the combined mean-field system): the claim that the EnKF empirical measure serves as a sufficient statistic that closes the co-state equation relies on preservation of the necessary conditional expectations and martingale property. For nonlinear observation operators (as in the Lorenz examples), the standard EnKF update introduces bias and sampling variance that can violate this closure; the manuscript does not supply a rigorous justification, error bound, or counter-example analysis for this step.
  2. [Numerical results] Numerical results section (Lorenz-63/96 experiments): the reported trajectories and control performance lack quantitative error analysis, stability margins, or comparison against a fully observed baseline or alternative assimilation schemes. Without these, it is difficult to confirm that the method remains stable on chaotic attractors under partial observations.
minor comments (2)
  1. [Abstract] The abstract states the main contribution but supplies no key equations or brief statement of the approximation error; adding one or two representative equations would improve accessibility.
  2. [§2] Notation for the empirical measure and the EnKF update step should be introduced consistently before its use in the mean-field equations.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive comments on our manuscript. We address each major point below and describe the revisions we intend to incorporate.

read point-by-point responses

  1. Referee: [§3] §3 (derivation of the combined mean-field system): the claim that the EnKF empirical measure serves as a sufficient statistic that closes the co-state equation relies on preservation of the necessary conditional expectations and martingale property. For nonlinear observation operators (as in the Lorenz examples), the standard EnKF update introduces bias and sampling variance that can violate this closure; the manuscript does not supply a rigorous justification, error bound, or counter-example analysis for this step.

    Authors: We acknowledge that the EnKF provides only an approximation to the conditional law, and that finite-ensemble bias and variance are present for nonlinear observations. The derivation in §3 proceeds by substituting the EnKF empirical measure directly into the McKean-Pontryagin co-state equation, thereby obtaining a closed forward system. While a complete error analysis lies beyond the scope of the present work, we will revise §3 to include an explicit discussion of this approximation step, citing relevant consistency results for the EnKF and noting the conditions under which the martingale property is approximately preserved. A short remark on potential propagation of sampling error into the control law will also be added. revision: yes

  2. Referee: [Numerical results] Numerical results section (Lorenz-63/96 experiments): the reported trajectories and control performance lack quantitative error analysis, stability margins, or comparison against a fully observed baseline or alternative assimilation schemes. Without these, it is difficult to confirm that the method remains stable on chaotic attractors under partial observations.

    Authors: We agree that additional quantitative diagnostics would strengthen the numerical section. In the revised manuscript we will report RMSE values for both state reconstruction and accumulated control cost, include a direct comparison against the fully observed McKean-Pontryagin controller (serving as an ideal baseline), and add a brief comparison with a particle-filter-based alternative where ensemble size permits. We will also extend the time horizons shown for the Lorenz-63/96 attractors and comment on observed stability margins and any instances of control degradation under partial observations. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation is a novel combination of established methods

full rationale

The paper derives forward-evolving mean-field equations for states and co-states by combining the ensemble Kalman filter (for online data assimilation under partial observations) with the McKean-Pontryagin maximum principle. This is presented as a new methodology suited for digital twins, with numerical validation on Lorenz-63/96 and inverted pendulum examples. No quoted steps reduce the claimed equations to fitted parameters by construction, nor does the central claim rest on a self-citation chain that itself lacks independent content. The combination is treated as an original synthesis rather than a renaming or self-referential fit, making the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The approach rests on standard assumptions of stochastic optimal control and ensemble Kalman filtering; no new free parameters, ad-hoc axioms, or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption The underlying state process is a diffusion whose law can be approximated by an ensemble of particles.
    Invoked when replacing the true filtering distribution by the ensemble Kalman filter output.
  • domain assumption The McKean-Pontryagin optimality conditions extend to the partially observed case via mean-field evolution.
    Central modeling choice stated in the abstract.

pith-pipeline@v0.9.0 · 5703 in / 1235 out tokens · 31058 ms · 2026-05-18T10:39:29.953721+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 50 canonical work pages · 1 internal anchor

  1. [1]

    Meyn, Control Systems and Reinforcement Learning, Ca mbridge Uni- versity Press, Cambridge, 2022

    S. Meyn, Control Systems and Reinforcement Learning, Ca mbridge Uni- versity Press, Cambridge, 2022

    work page 2022

  2. [2]

    T. M. Moerland, J. Broekens, A. Plaat, C. M. Jonker, Model -based Reinforcement Learning: A Survey, Now Publishers, 2023

    work page 2023

  3. [3]

    Reich, C

    S. Reich, C. Cotter, Probabilistic Forecasting and Baye sian Data As- similation, Cambridge University Press, 2015

    work page 2015

  4. [4]

    K. Law, A. Stuart, K. Zygalakis, Data Assimilation, Cham , Switzerland: Springer 214

    work page

  5. [5]

    K. J. Åström, Optimal control of Markov processes with in complete state information, Journal of Mathematical Analysis and Ap plications 10 (1965) 174–205

    work page 1965

  6. [6]

    Bensoussan, Stochastic Control of Partially Observa ble Systems, Cambridge University Press, Cambridge, 1992

    A. Bensoussan, Stochastic Control of Partially Observa ble Systems, Cambridge University Press, Cambridge, 1992

    work page 1992

  7. [7]

    van Handel, Stochastic Calculus, Filtering, and Stoc hastic Control, Lecture Notes, Caltech, 2007

    R. van Handel, Stochastic Calculus, Filtering, and Stoc hastic Control, Lecture Notes, Caltech, 2007

    work page 2007

  8. [8]

    Nisio, Stochastic Control Theory, Springer, Tokyo, 2 nd edn., 2015

    M. Nisio, Stochastic Control Theory, Springer, Tokyo, 2 nd edn., 2015

    work page 2015

  9. [9]

    Silver, J

    D. Silver, J. Veness, Monte-Carlo Planning in Large POMD Ps, in: J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, A. Culot ta (Eds.), Advances in Neural Information Processing Systems, vol. 23 , Curran Associates, Inc., 2010

    work page 2010

  10. [10]

    van de Water, J

    H. van de Water, J. C. Willems, The certainty equivalenc e property in stochastic control theory, IEEE Trans. Autom. Contr. AC- 26 (1981) 1080–1087

    work page 1981

  11. [11]

    M. L. Littman, A. R. Cassandra, L. P. Kaelbling, in: Inte rnational Conference on Machine Learning, 362–370, 1995

    work page 1995

  12. [12]

    S. Ross, J. Pineau, S. Paquet, B. Chaib-draa, Online Pla nning algo- rithms for POMDPs, Journal of Artificial Inelligence Resear ch 32 (2008) 663–704. 21

    work page 2008

  13. [13]

    Zheng, J

    D. Zheng, J. Ridderhof, P. Tsiotras, A.-A. Agha-Mohamm adi, Belief Space Planning: A Covariance Steering Approach, in: 2022 In terna- tional Conference on Robotics and Automation (ICRA), 11051 –11057, 2022

    work page 2022

  14. [14]

    Zheng, J

    D. Zheng, J. Ridderhof, Z. Zhang, P. Tsiotras, A.-A. Agh a-Mohammadi, CS-BRM: A Probabilistic RoadMap for Consistent Belief Spac e Plan- ning With Reachability Guarantees, IEEE Transactions on Ro botics (2024) 1630–1649

    work page 2024

  15. [15]

    Kappen, Path integrals and symmetry breaking for opt imal con- trol theory, J Statistical Mechanics: Theory and Experimen ts 11 (2005) 11011

    H. Kappen, Path integrals and symmetry breaking for opt imal con- trol theory, J Statistical Mechanics: Theory and Experimen ts 11 (2005) 11011

    work page 2005

  16. [16]

    H. J. Kappen, V. Gómez, M. Opper, Optimal control as a gra phical model inference problem, Machine Learning 87 (2012) 159–18 2

    work page 2012

  17. [17]

    Abdulsamad, S

    H. Abdulsamad, S. Iqbal, S. Särkkä, Sequential Monte Ca rlo for pol- icy optimization in continuous POMDPs, Tech. Rep., arXiv:2 505.16732, 2025

    work page 2025

  18. [18]

    Kawasaki, S

    F. Kawasaki, S. Kotsuki, Leading the Lorenz 63 system to wards the prescribed regime by model predictive control coupled with data assim- ilation, Nonlin. Processes Geophys. 31 (2024) 319–333

    work page 2024

  19. [19]

    Ensemble Kalman-Bucy filtering for no nlinear model predictive control

    S. Reich, Ensemble Kalman-Bucy filtering for nonlinear model predictive control, Tech. Rep., arXiv:2503.12474, 2025

    work page arXiv 2025

  20. [20]

    Evensen, F

    G. Evensen, F. C. Vossepoel, P. J. van Leeuwen, Data Assi milation Fundamentals: A unified Formulation of the State and Paramet er Esti- mation Problem, Springer Nature Switzerland AG, Cham, Swit zerland, 2022

    work page 2022

  21. [21]

    On a mean-field Pontryagin minimum principle for stochastic optimal control

    M. Opper, S. Reich, On a mean-field Pontryagin minimum pr inciple for stochastic optimal control, Tech. Rep., arXiv:2506.10506 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    Calvello, S

    E. Calvello, S. Reich, A. M. Stuart, Ensemble Kalman met hods: A mean field perspective, Acta Numerica 34 (2025) 123–291

    work page 2025

  23. [23]

    Bergemann, S

    K. Bergemann, S. Reich, An ensemble Kalman-Bucy filter f or continuous data assimilation, Meteorologische Zeitschrift 21 (3) (20 12) 213. 22

    work page

  24. [24]

    Chopin, O

    N. Chopin, O. Papaspiliopoulos, An Introduction to Seq uential Monte Carlo, Springer Nature Switzerland AG, Cham, Switzerland, 2020

    work page 2020

  25. [25]

    Snyder, T

    C. Snyder, T. Bengtsson, P. Bickel, J. Anderson, Obstac les to High- Dimensional Particle Filtering, Monthly Weather Review 13 6 (2008) 4629–4640

    work page 2008

  26. [26]

    Bergemann, S

    K. Bergemann, S. Reich, A mollified ensemble Kalman filte r, Quarterly Journal of the Royal Meteorological Society 136 (651) (2010 ) 1636–1643

    work page 2010

  27. [27]

    Carmona, Lectures on BSDEs, Stochastic Control, and Stochastic Differential Games with Financial Applications, SIAM, Phil adelphia, 2016

    R. Carmona, Lectures on BSDEs, Stochastic Control, and Stochastic Differential Games with Financial Applications, SIAM, Phil adelphia, 2016

    work page 2016

  28. [28]

    G. A. Pavliotis, Stochastic Processes and Application s, Springer Verlag, New York, 2016

    work page 2016

  29. [29]

    T. Yang, P. G. Mehta, S. P. Meyn, Feedback particle filter , IEEE Trans. Automat. Control 58 (10) (2013) 2465–2480, ISSN 0018-9286

    work page 2013

  30. [30]

    Pointryagin, V

    L. Pointryagin, V. Boltyanskii, R. Gamkrelidze, E. Mih chenko, The Mathematical Theory of Optimal Processes, John Wiley & Sons , New York, 1962

    work page 1962

  31. [31]

    Bensoussan, Estimation and Control of Dynamical Sys tems, Springer, Cham, 2018

    A. Bensoussan, Estimation and Control of Dynamical Sys tems, Springer, Cham, 2018

    work page 2018

  32. [32]

    P. G. Mehta, S. P. Meyn, A feedback particle filter-based approach to optimal control with partial observations, in: 52nd IEEE Co nference on Decision and Control, 3121–3127, 2013

    work page 2013

  33. [33]

    H. J. Bierens, The Nadaraya-Watson kernel regression f unction estima- tor, in: Topics in Advanced Econometrics, Cambridge Univer sity Press, New York, 212–247, 1994

    work page 1994

  34. [34]

    C. L. Wormell, S. Reich, Spectral convergence of diffusi on maps: Im- proved error bounds and an alternative normalisation, SIAM J. Numer. Anal. 59 (2021) 1687–1734

    work page 2021

  35. [35]

    G. A. Gottwald, F. Li, S. Reich, Y. Marzouk, Stable gener ative modeling using Schrödinger bridges, Phil. Trans. R. Soc. A 383 (2025) 20240332. 23

    work page 2025

  36. [36]

    N. F. Marshall, R. R. Coifman, Manifold learning with bi -stochastic kernels, IMA J. Appl. Maths. 84 (2019) 455–482

    work page 2019

  37. [37]

    E. N. Lorenz, Deterministic nonperiodic flow, Journal o f the Atmo- spheric Sciences 20 (2) (1963) 130–141

    work page 1963

  38. [38]

    Findeisen, F

    R. Findeisen, F. Allgöwer, An Introduction to Nonlinea r Model Pre- dictive Control, in: A. G. Jager, de, H. J. Zwart (Eds.), Syst ems and control : 21th Benelux meeting 2002, Technische Universite it Eindhoven, 119–141, 2002

    work page 2002

  39. [39]

    J. B. Rawlings, D. Q. Mayne, M. M. Diehl, Model Predictiv e Control: Theory, Computation, and Design, Nob Hill Publishing, Madi son, 2nd edn., 2018

    work page 2018

  40. [40]

    Bauer, A

    P. Bauer, A. Thorpe, G. Brunet, The quiet revolution of n umerical weather prediction, Nature 525 (2015) 47–55

    work page 2015

  41. [41]

    Bauer, B

    P. Bauer, B. Stevens, W. Hazeleger, A digital twin of Ear th for the green transition, Nature Climate Change 11 (2021) 80–83

    work page 2021

  42. [42]

    Asch, A Toolbox for Digital Twins: From Model-Based t o Data- Driven, SIAM, Philadelphia, 2022

    M. Asch, A Toolbox for Digital Twins: From Model-Based t o Data- Driven, SIAM, Philadelphia, 2022

    work page 2022

  43. [43]

    K. Sel, A. Hawkins-Daarud, A. Chaudhuri, D. Osman, A. Ba hai, D. Pay- darfar, K. Willcox, C. Chung, R. Jafari, Survey and perspect ive on ver- ification, validation, and uncertainty quantification of di gital twins for precision medicine, npj Digital Medicine 8 (2025) 40

    work page 2025

  44. [44]

    S. A. Niederer, M. S. Sacks, M. Girolami, K. Willcox, Sca ling digital twins from the artisanal to the industrial, Nature Computat ional Science 1 (2021) 313–320

    work page 2021

  45. [45]

    Henao-Garcia, M

    S. Henao-Garcia, M. Kapteyn, K. E. Willcox, M. Tezzele, M. Castroviejo-Fernandez, T. Kim, M. Ambrosino, I. Kolmano vsky, H. Basu, P. Jirwankar, R. Sanfelice, Digital-Twin-Enabled Multi- Spacecraft On-Orbit Operations, AIAA SCITECH 2025 Forum, d oi: \bibinfo{doi}{10.2514/6.2025-1432}, 2025

    work page doi:10.2514/6.2025-1432 2025

  46. [46]

    Rep., SIAM Publishing, Philadelphia, 2024

    SIAM, Report on the Future of Computational Science, Te ch. Rep., SIAM Publishing, Philadelphia, 2024. 24

    work page 2024

  47. [47]

    Rep., Washington, DC: T he National Academies Press, 2024

    National Academy of Sciences, Foundational Research G aps and Future Directions for Digital Twins, Tech. Rep., Washington, DC: T he National Academies Press, 2024

    work page 2024

  48. [48]

    G. A. Gottwald, S. Reich, Localized Schrödinger Bridge Sampler, Tech. Rep., arXiv:2409.07968, 2024

    work page arXiv 2024

  49. [49]

    Maoutsa, S

    D. Maoutsa, S. Reich, M. Opper, Interacting Particle So lutions of Fokker–Planck Equations Through Gradient-Log-Density Es timation, Entropy 22 (8) (2020) 802

    work page 2020

  50. [50]

    G. A. Gottwald, S. Liu, Y. Marzouk, S. Reich, X. T. Tong, L ocalized Diffusion Models for High Dimensional Distributions Genera tion, Tech. Rep., arXiv:2505.04417, 2025. 25

    work page arXiv 2025