REVIEW 2 major objections 5 minor 21 references
Neural Hamiltonian Operator
T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that high-dimensional stochastic control can be solved as a statistical inference problem: training the Neural Hamiltonian Operator—neural networks for the feedback control and the value-function gradient—on the terminal…
desk verdict Clean operator-theoretic framing of deep FBSDE, but the central recovery claim fails: the terminal-mismatch loss has a huge zero set that does not single out the optimal Hamiltonian operator. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Neural Hamiltonian Operator $L_\Psi$, the infinitesimal generator of the forward SDE for the extended state $X_t=(S_t,\tilde p_t)$. On a test function $g$, it acts as $(L_\Psi g)(t,x)=\nabla_x g(x)^\top b_\Psi(t,s)+\frac{1}{2}\mathrm{Tr}(D_\Psi(t,s)\nabla_x^2 g(x))$, where $b_\Psi$ stacks the drift of the state and the candidate adjoint, and $D_\Psi=\Sigma_\Psi C_t \Sigma_\Psi^\top$ is built from the diffusion blocks $\Sigma_\Psi=(\sigma(\cdot,\alpha_\omega)^\top, q_\Psi^\top)^\top$ with $q_\Psi=(\nabla_s\Phi_\xi)\sigma(\cdot,\alpha_\omega)$. This operator carries the whole argument because its coefficients are exactly the feedback control $\alpha_\omega$ and the decoupling-field ansatz $\Phi_\xi\approx\nabla_s V$, so minimizing the terminal mismatch over $L_\Psi$ is equivalent to searching over candidate optimal dynamics.
What would settle it
On a scalar problem with an even terminal payoff, so that $G(s)=G(-s)$, train two NHOs from different initializations until $J(\Psi)$ reaches machine zero; if the two parameter sets imply measurably different controls or state paths from the same initial state, the identifiability assumption behind Theorem 4.3 is false. A complementary check is to run the same training on a problem whose known value function is only a viscosity solution, not classically differentiable, and compare the implied control with the known optimal one.
Extended reading notes
Core claim
The paper's central claim is that the optimal feedback control of a stochastic control problem coincides with the dynamics of the unique Neural Hamiltonian Operator that satisfies the terminal condition of the PMP. Concretely, for network parameters $\Psi=(\omega,\xi)$, the candidate adjoint is $\tilde p_t$, initialized as $\Phi_\xi(t_0,s_0)$ and evolved forward through the SDE whose generator is $L_\Psi$, while the terminal condition requires $\tilde p_T=\nabla G(S_T)$. Training minimizes $J(\Psi)=\mathbb{E}[\|\tilde p_T-\nabla G(S_T)\|^2]$, and the paper asserts that a minimizer of this loss gives an operator whose coefficients match those of the true optimal Hamiltonian system. Theorem 4.1 states that the NHO family is dense in the space of true Hamiltonian operators in coefficient sup-norm for any continuous square-integrable martingale driver, and Theorems 4.3 and 4.6 state that SGD reaches the optimum in expectation under identifiability, $L$-smoothness, and global or local Polyak-Lojasiewicz conditions. The extensions to ergodic control add a loss based on the variance of the Hamiltonian along paths, whose zero minimizer is shown to imply a constant Hamiltonian almost surely under the invariant measure.
Load-bearing premise
The load-bearing premise is that zero terminal mismatch can only be produced by the true optimal dynamics: no other parameter setting can satisfy the end condition by coincidence, and the value function is assumed to be genuinely differentiable so that the adjoint process really is its gradient. If either part fails, a network can drive the loss to zero while generating dynamics that are not the optimal control.
Editorial extensions
If this is right
- Under Theorem 4.1, any sufficiently smooth optimal Hamiltonian system can be matched to arbitrary precision by an NHO, so in the smooth regime the method carries no irreducible approximation error.
- When the identifiability, smoothness, and Polyak-Lojasiewicz assumptions hold, SGD on the terminal mismatch converges in expectation, making the deep FBSDE training procedure a provable statistical estimator rather than a heuristic.
- Because the approximation and operator definitions only require a continuous square-integrable martingale driver, the same training scheme applies to models whose noise is not Brownian, including drivers with stochastic covariance.
- In the ergodic infinite-horizon setting, a zero of the Hamiltonian-variance loss forces the learned Hamiltonian to be constant almost surely under the invariant measure, giving a necessary-condition check for learned policies.
- The reported $d=50$ experiments on a known analytic control, a double-well potential, and portfolio liquidation show the terminal-loss training reaching small values while the implied control matches reference behavior.
Reading between the lines
- Beyond the paper, a trained NHO could be reused as an initialization for a different horizon or terminal cost, because the generator is local in time; the paper does not test this transfer.
- Beyond the paper, disagreement among multiple near-zero-loss initializations would be a practical red flag for the identifiability assumption, suggesting the terminal condition alone is too weak and the initial-state distribution should be widened.
- Beyond the paper, the classical-solution assumption implies the method is most fragile in viscosity regimes; smoothing a non-differentiable terminal cost and watching whether the learned control tracks the unique optimal one would test that boundary.
- Beyond the paper, the Lyapunov regularizer in the ergodic loss can serve as an online stability certificate: estimating the drift of $\|S_t\|^2$ from simulated paths and checking that it stays negative would certify ergodicity of a learned policy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper formulates high-dimensional stochastic control through a Neural Hamiltonian Operator (NHO), a parameterized generator for the FBSDE system associated with Pontryagin's maximum principle. The parameters Ψ=(ω,ξ) define a candidate control 𝛼𝜔 and a decoupling-field ansatz Φ𝜉; the coupled system (3)-(4) is simulated forward, and training minimizes the terminal mismatch loss (5). Section 4 proves a universal approximation theorem for the NHO coefficients (Theorem 4.1) and two SGD convergence theorems (Theorems 4.3 and 4.6) under smoothness, Polyak-Łojasiewicz, and identifiability assumptions. Section 5 adapts the framework to ergodic control with a Hamiltonian-variance objective and Lyapunov regularization, and Section 6 reports d=50 numerical experiments on three problems.
Significance. The universal approximation theorem is a solid and useful result: it reduces the problem to C1 approximation of the optimal control, the value gradient, and its Jacobian, and it is independent of the training procedure. The operator-theoretic framing of deep FBSDE solvers is a helpful organizing perspective, and the numerical experiments demonstrate that the terminal-mismatch objective can be minimized in high dimensions. However, the central claim that the NHO recovers the optimal feedback control and value-gradient dynamics is not established. The loss (5) enforces only the terminal condition of the PMP; it does not enforce the Hamiltonian-maximization condition. The identifiability assumption that is needed to bridge this gap is not proved and is generically false in the idealized function class used by the approximation theorem. The convergence theorems are therefore conditional on a condition that fails in the intended setting, and the paper's main claim is unsupported.
major comments (2)
- [§3.3, Eq. (5); §4.1, Assumption 1] The terminal mismatch loss (5) contains no term that enforces the PMP Hamiltonian-maximization condition. For any fixed admissible Markov feedback control α, the Markovian BSDE with generator ∇_sH(t,s,α(t,s),v,∇_s v σ) and terminal condition v(T,s)=∇G(s) has a decoupling field v^α under the smoothness assumptions of Section 2. Taking Φξ=v^α and 𝛼𝜔=α in (3)-(4) yields p̃_T=v^α(T,S_T)=∇G(S_T), so J(Ψ)=0 even though L_Ψ is the operator associated with the non-optimal control α. This is not a measure-zero 'spurious' phenomenon: it holds for every admissible α, including the zero control in a linear-quadratic problem. Thus Assumption 1, which the paper calls 'subtle yet crucial' in Remark 4.4, is false in the continuum limit of the network class used by Theorem 4.1, and the observability remark does not supply a replacement condition. The paper needs either a proof of identifiability under explicit conditions or a different objective that includes the maximization condition.
- [§4.1, Theorem 4.3; §4.2, Theorem 4.6] Both convergence theorems conclude that the expected loss tends to the minimum value, lim_k E[J(Ψ_k)]=0 or lim_k E[J_λ(Ψ_k)]=J_λ(Ψ*). These are statements about the training objective, not about the learned operator. The abstract and Section 1 interpret the minimizer as recovering the optimal feedback control and value-gradient dynamics; that interpretation requires the injectivity asserted in Assumption 1. Since the construction in the previous comment shows that J=0 is attained by non-optimal operators, the convergence results do not substantiate the recovery claim. The theorems should be stated as conditional results about loss minimization under the stated assumptions, and the paper should not claim recovery of the optimal control unless an identifiability result is proved.
minor comments (5)
- [Theorem 4.1] The statement approximates 𝛼𝜔 to 𝛼∗ but does not specify how the network output is constrained to the compact set K. A projection onto K or a suitable output-layer design should be included so that μ, σ, and f are evaluated on their domain; otherwise the coefficients bΨ and ΣΨ are not even defined for arbitrary network outputs.
- [§3.3, Eq. (5)] The initial-state distribution for the expectation in J is not specified. Remark 4.4's identifiability argument explicitly relies on a rich distribution over initial states; the definition of J should state the distribution (e.g., a sampling measure ν on R^d).
- [Remark 6.1] The smoothing parameter ε used to approximate the non-smooth cost |α|^(3/2) is not reported; include the numerical value for reproducibility.
- [Figures 2, 4, and 6] The legend entries 'Reg(0.1·||p||²)' and 'Reg(0.01·||p||²)' do not match the regularizer in Eq. (6), which is an integrated penalty on ||∇_s Φ_ξ||²_F; align the notation or explain the abbreviation.
- [Section 7] The conclusion states the method is validated 'in up to 100 dimensions,' but all reported experiments are in d=50; correct the discrepancy.
Circularity Check
Zero terminal loss is achievable by the decoupling field of any admissible feedback control, so the claimed recovery of the optimal NHO is imposed by the unproved identifiability assumption rather than derived from the training objective.
-
self definitional
[Section 4.1, Theorem 4.3 Assumption 1 / Remark 4.4]
"Assumption 1 (Identifiability): This is a subtle yet crucial assumption, postulating that the map from the operator's coefficients to the terminal condition is injective. A spurious operator LΨ≠L∗ could theoretically achieve J(Ψ) = 0 if it happens to satisfy the terminal condition by coincidence."
The loss J(Ψ)=E[||p̃_T−∇G(S_T)||^2] enforces only the PMP terminal boundary; it contains no Hamiltonian-maximization term. Under the paper's own smoothness assumptions, fix any admissible Markov control α and let v^α be the decoupling field of the BSDE with terminal condition ∇G and generator ∇_sH(t,s,α,·,·). Taking Φ_ξ=v^α and q_Ψ=(∇_s v^α)σ makes the candidate forward system (3)-(4) satisfy p̃_T=∇G(S_T), hence J(Ψ)=0, while α need not maximize H and LΨ≠L∗. Therefore the asserted injectivity 'J(Ψ)=0 ⇒ coefficients equal L∗' is not a preliminary fact; it is logically equivalent to the paper's central claim that terminal-loss minimization identifies the optimal Hamiltonian operator.
-
fitted input called prediction
[Section 3.3, Step 3 (Eq. (5))]
"Minimizing J(Ψ) with stochastic gradient descent corresponds to searching for a parameter set Ψ∗ such that the operator LΨ∗ generates dynamics consistent with the necessary conditions of optimality."
The PMP's necessary conditions consist of the adjoint BSDE with terminal condition plus the Hamiltonian-maximization condition. Eq. (5) penalizes only the terminal mismatch ||p̃_T−∇G(S_T)||^2; there is no term for H(·,α,p,q)=max_ã H(·,ã,p,q). Consequently the sentence equates minimization of a terminal fit with recovery of an optimality-consistent operator. This equivalence is exactly the identifiability assumption of Theorem 4.3, which is stated without proof and fails in the smooth cases admitted by Assumption 2.1 because any admissible control's decoupling field makes the terminal loss zero. Calling the zero-loss minimizer 'the optimal NHO' is thus the fitted value being presented as a prediction, rather than an inference from the objective.
full rationale
The universal-approximation component (Theorem 4.1 and Lemma A.1) is self-contained: it reduces to Hornik-type C^1 density and is external, so no circularity attaches there. The SGD convergence theorems are standard P-L/Robbins-Siegmund arguments conditional on stated assumptions. The self-citations to Qi (2025a,b) are inspirational and are not used to prove Theorem 4.1, so they are not load-bearing. The circularity is concentrated in the identification step: the loss J of Eq. (5) is a terminal-mismatch fit, and the paper's central assertion that minimizing it recovers the optimal feedback control and value-gradient dynamics is not derived from the PMP system. It is installed as Assumption 1 of Theorem 4.3, which postulates that zero terminal loss forces the operator coefficients to equal those of the true optimal operator. Remark 4.4 concedes that 'a spurious operator LΨ≠L∗ could theoretically achieve J(Ψ)=0' and merely assumes such coincidences away. Because any admissible Markov control has a decoupling field that zeroes J, the identification claim is generically the same statement as the recovery claim. The paper is transparent about the assumption being 'crucial', but the abstract's 'optimal NHO can be found by training' is not supported by the loss alone; it rests on the assumed injectivity. This is a partial circularity (score 6): the training objective is a fit, and the optimality conclusion is forced by the identifiability assumption rather than by the terminal condition.
Assumptions & free parameters
free parameters (2)
- Regularization weight λ =
0.1 and 0.01 used in experiments
- Smoothing parameter ε for non-smooth cost =
not specified
assumptions (6)
- domain assumption Existence of a unique classical C^{1,2} solution to the optimal control problem
- ad hoc to paper Identifiability: zero terminal loss implies L_Ψ = L* almost everywhere
- ad hoc to paper Polyak-Lojasiewicz condition holds for the NHO loss
- domain assumption Ergodicity of the state process under the learned operator
- standard math C1 universal approximation by single-hidden-layer neural networks
- domain assumption Dissipativity condition of the underlying dynamics
Cite this review
Pith. "Pith review of Neural Hamiltonian Operator." pith.science (2026). https://pith.science/paper/SHR2VPIU
@misc{pith2026250701313,
author = {Pith},
title = {Pith review of: Neural Hamiltonian Operator},
year = {2026},
howpublished = {\url{https://pith.science/paper/SHR2VPIU}},
note = {Machine review of arXiv:2507.01313}
}
read the original abstract
Stochastic control problems in high dimensions are notoriously difficult to solve due to the curse of dimensionality. An alternative to traditional dynamic programming is Pontryagin's Maximum Principle (PMP), which recasts the problem as a system of Forward-Backward Stochastic Differential Equations (FBSDEs). In this paper, we introduce a formal framework for solving such problems with deep learning by defining a \textbf{Neural Hamiltonian Operator (NHO)}. This operator parameterizes the coupled FBSDE dynamics via neural networks that represent the feedback control and an ansatz for the value function's spatial gradient. We show how the optimal NHO can be found by training the underlying networks to enforce the consistency conditions dictated by the PMP. By adopting this operator-theoretic view, we situate the deep FBSDE method within the rigorous language of statistical inference, framing it as a problem of learning an unknown operator from simulated data. This perspective allows us to prove the universal approximation capabilities of NHOs under general martingale drivers and provides a clear lens for analyzing the significant optimization challenges inherent to this class of models.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION format.url url empty "" url if FUNCTION article output.bibitem format.authors "author" output.check author format.key output output.year.check new.block format.title "title" output.check new.block crossref missing format.jour.vol output format.article.crossref output.nonnull format.pages output if ne...
-
[2]
binproceedings [author] Amos , Brandon B. , Xu , Lei L. Kolter , J. Zico J. Z. ( 2017 ). Input Convex Neural Networks . In Proceedings of the 34th International Conference on Machine Learning . Proceedings of Machine Learning Research 70 146--155 . PMLR . binproceedings
work page 2017
-
[3]
barticle [author] Barles , Guy G. Souganidis , Panagiotis E. P. E. ( 1991 ). Convergence of approximation schemes for fully nonlinear second order equations . Asymptotic Analysis 4 271--283 . barticle
work page 1991
-
[4]
barticle [author] Beck , Christian C. , E , Weinan W. Jentzen , Arnulf A. ( 2019 ). Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations . Journal of Nonlinear Science 29 1563--1619 . barticle
work page 2019
-
[5]
barticle [author] Bender , Christian C. Denk , Robert R. ( 2007 ). A forward scheme for backward SDEs . Stochastic Processes and their Applications 117 1793--1812 . barticle
work page 2007
-
[6]
barticle [author] Crandall , Michael G. M. G. , Ishii , Hitoshi H. Lions , Pierre-Louis P.-L. ( 1992 ). User's guide to viscosity solutions of second order partial differential equations . Bulletin of the American Mathematical Society 27 1--67 . barticle
work page 1992
-
[7]
barticle [author] E , Weinan W. , Han , Jiequn J. Jentzen , Arnulf A. ( 2017 ). Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations . Communications in Mathematics and Statistics 5 349--380 . barticle
work page 2017
-
[8]
bbook [author] Fleming , Wendell H. W. H. Soner , H. Mete H. M. ( 2006 ). Controlled Markov Processes and Viscosity Solutions , 2nd ed. Stochastic Modelling and Applied Probability 25 . Springer , New York . bbook
work page 2006
Show all 21 references
-
[9]
, Jentzen , Arnulf A
barticle [author] Han , Jiequn J. , Jentzen , Arnulf A. E , Weinan W. ( 2018 ). Solving high-dimensional partial differential equations using deep learning . Proceedings of the National Academy of Sciences 115 8505--8510 . barticle
2018
-
[10]
( 1990 )
barticle [author] Hornik , Kurt K. ( 1990 ). Approximation capabilities of multilayer feedforward networks . Neural Networks 4 251--257 . barticle
1990
-
[11]
bbook [author] Khasminskii , Rafail Z. R. Z. ( 2011 ). Stochastic Stability of Differential Equations , 2nd ed. Stochastic Modelling and Applied Probability 66 . Springer , Heidelberg . bbook
2011
-
[12]
, Li , Zongyi Z
barticle [author] Kovachki , Nikola N. , Li , Zongyi Z. , Liu , Burigede B. , Nabian , M. A. M. A. , P \'e rez , J. A. J. A. , Spadanacio , L. L. , Zhu , K. K. , Azizzadenesheli , Kamyar K. , Bhattacharya , Kaushik K. , Stuart , Andrew A. Anandkumar , Anima A. ( 2023 ). Neural...
2023
-
[13]
, Kovachki , Nikola N
binproceedings [author] Li , Zongyi Z. , Kovachki , Nikola N. , Azizzadenesheli , Kamyar K. , Liu , Burigede B. , Bhattacharya , Kaushik K. , Stuart , Andrew A. Anandkumar , Anima A. ( 2021 ). Fourier Neural Operator for Parametric Partial Differential Equations . In Internati...
2021
-
[14]
, Jin , Pengzhan P
barticle [author] Lu , Luyu L. , Jin , Pengzhan P. Karniadakis , George Em G. E. ( 2021 ). DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators . Nature Machine Intelligence 3 218--229 . barticle
2021
-
[15]
bbook [author] Meyn , Sean P. S. P. Tweedie , Richard L. R. L. ( 2009 ). Markov Chains and Stochastic Stability , 2nd ed. Cambridge University Press , Cambridge . bbook
2009
-
[16]
barticle [author] Polyak , Boris T. B. T. ( 1963 ). Gradient methods for minimizing functionals . Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki 3 643--653 . barticle
1963
-
[17]
( 2025 a)
bmisc [author] Qi , Qian Q. ( 2025 a). Universal Approximation Theorem for Deep Q-Learning via FBSDE System . bmisc
2025
-
[18]
( 2025 b)
binproceedings [author] Qi , Qian Q. ( 2025 b). Universal Approximation Theorem of Deep Q-Networks . In International Conference on Machine Learning . PMLR . binproceedings
2025
-
[19]
, Perdikaris , Paris P
barticle [author] Raissi , Maziar M. , Perdikaris , Paris P. Karniadakis , George E. G. E. ( 2019 ). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations . Journal of Computation...
2019
-
[20]
Spiliopoulos , Konstantinos K
barticle [author] Sirignano , Justin J. Spiliopoulos , Konstantinos K. ( 2018 ). DGM: A deep learning algorithm for solving partial differential equations . Journal of Computational Physics 375 1339--1364 . barticle
2018
-
[21]
Zhou , Xun Yu X
bbook [author] Yong , Jiongmin J. Zhou , Xun Yu X. Y. ( 1999 ). Stochastic Controls: Hamiltonian Systems and HJB Equations . Applications of Mathematics 43 . Springer , New York . bbook
1999
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.