pith. sign in

arxiv: 2603.01084 · v2 · pith:YMM5XUKInew · submitted 2026-03-01 · 🧮 math.DS · cs.NA· math.NA· math.OC

Kernel-Based LMI Approaches to Solving the Hamilton-Jacobi-Bellman Equation and Nonlinear Optimal Control

Pith reviewed 2026-05-21 12:07 UTC · model grok-4.3

classification 🧮 math.DS cs.NAmath.NAmath.OC
keywords Hamilton-Jacobi-Bellman equationnonlinear optimal controllinear matrix inequalityreproducing kernel Hilbert spaceRiccati equationsuboptimality bound
0
0 comments X

The pith

An explicit Riccati-Hessian equality constraint at equilibrium lets a kernel LMI formulation approximate solutions to the Hamilton-Jacobi-Bellman equation without collapsing to the trivial solution.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a convex optimization method for finding approximate solutions to the Hamilton-Jacobi-Bellman equation that governs nonlinear optimal control. The gradient of the value function is represented in a reproducing kernel Hilbert space, and the quadratic Hamilton-Jacobi-Bellman inequality is rewritten as a linear matrix inequality that remains linear in the kernel coefficients. Adding a Riccati-Hessian equality constraint at the equilibrium forces the approximation to agree with the algebraic Riccati equation of the linearized system. This construction produces a semidefinite program whose solution yields a feedback law whose closed-loop cost stays within a bound of the optimal cost; the bound depends on the residual of the inequality and on problem data rather than on further details of the approximant.

Core claim

Representing the gradient of the value function in a reproducing kernel Hilbert space and converting the Hamilton-Jacobi-Bellman inequality into a Schur-complement linear matrix inequality yields a convex semidefinite program in the kernel coefficients. The novel Riccati-Hessian equality constraint imposed at the equilibrium removes the trivial zero solution and enforces consistency with the algebraic Riccati equation of the linearized dynamics. The resulting approximation satisfies the suboptimality relation J(x0; û) − V*(x0) ≤ ε T(x0), where T(x0) is determined solely by the system data and the working domain.

What carries the argument

The Riccati-Hessian equality constraint at the equilibrium point, which forces the Hessian of the kernel approximant to satisfy the algebraic Riccati equation of the linearized system.

If this is right

  • The semidefinite program is convex and can be solved by standard interior-point solvers.
  • The suboptimality bound remains valid even when the exact value function is not contained in the reproducing kernel Hilbert space.
  • On benchmark problems the method recovers exact polynomial solutions to machine precision when they lie in the chosen space and produces residuals smaller than those of several competing techniques on the Van der Pol oscillator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same LMI-plus-constraint structure could be applied to other reproducing kernel spaces or to polynomial sum-of-squares bases for systems whose value functions admit known structure.
  • Because the bound T(x0) depends only on data and domain size, the approach may remain useful for real-time receding-horizon implementations where exact dynamic programming is intractable.
  • Scaling the number of kernel centers while monitoring both residual and actual closed-loop cost offers a practical diagnostic for when further refinement ceases to improve performance.

Load-bearing premise

The gradient of the value function lies sufficiently close to the chosen reproducing kernel Hilbert space that the linear matrix inequality and the Riccati-Hessian constraint together produce an approximation whose suboptimality stays bounded.

What would settle it

Compute the closed-loop cost starting from a chosen initial state and verify whether it exceeds the predicted quantity ε T(x0) by more than numerical tolerance; separately, check whether the Hessian of the obtained approximant at the equilibrium matches the algebraic Riccati solution of the linearized dynamics.

Figures

Figures reproduced from arXiv: 2603.01084 by Boumediene Hamzi, Umesh Vaidya.

Figure 1
Figure 1. Figure 1: Comprehensive numerical results for the 1D scalar system with multiple initial condi [PITH_FULL_IMAGE:figures/full_fig_p017_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comprehensive numerical results for the 2D radially symmetric system with multiple [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comprehensive numerical results for the Van der Pol oscillator ( [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
read the original abstract

We present a kernel-based linear matrix inequality (LMI) approach for the approximate solution of Hamilton--Jacobi--Bellman (HJB) equations arising in nonlinear optimal control. The method represents the gradient of the value function in a reproducing kernel Hilbert space (RKHS) and uses a Schur-complement reformulation to convert the quadratic HJB inequality into an LMI that is linear in the kernel coefficients, yielding a convex semidefinite program. The novel ingredient is an explicit Riccati--Hessian \emph{equality} constraint at the equilibrium, which removes the trivial solution and forces the Hessian of the approximation to match the algebraic Riccati equation solution of the linearised system. We give a suboptimality bound $J(x_0;\hat u) - V^*(x_0)\le \varepsilon\,T(x_0)$ in which $T(x_0)$ depends only on the problem data and the working domain (not on the approximation), and an RKHS approximation rate. Numerical experiments on a corrected 1D polynomial benchmark and on the Van der Pol oscillator measure $\varepsilon$, the RKHS approximation error, and the closed-loop cost $J(x_0;\hat u)$ versus the optimal value $V^*(x_0)$. On the 1D problem with $V^*$ in the polynomial-kernel RKHS the method recovers $V^*$ to within $3\times10^{-7}$ and achieves $0.000\%$ suboptimality. On Van der Pol it achieves the smallest HJB residual ($\varepsilon\approx 2.62$) of any method tested, beats LQR on every initial condition, and is within $0.42\%$ of the best per-IC cost (Albrekht order 6). When $V^*$ is not in the chosen RKHS, the method degrades gracefully: residuals stop improving with more centres but suboptimality remains bounded ($\le 13\%$ on the 1D test).

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The paper proposes a kernel-based LMI approach to approximately solve the Hamilton-Jacobi-Bellman equation for nonlinear optimal control. The gradient of the value function is represented in an RKHS, the quadratic HJB inequality is converted to a convex LMI via Schur complement, and a novel Riccati-Hessian equality constraint is imposed at the equilibrium to exclude the trivial zero solution and enforce consistency with the linearized algebraic Riccati equation. A suboptimality bound J(x0; û) - V*(x0) ≤ ε T(x0) is derived where T(x0) depends only on problem data and domain, together with an RKHS approximation rate. Numerical results are reported on a corrected 1D polynomial system (recovery to 3e-7, 0% suboptimality) and the Van der Pol oscillator (smallest HJB residual among tested methods, within 0.42% of best per-IC cost).

Significance. If the central derivations hold, the work supplies a convex SDP formulation for nonlinear optimal control with an explicit, data-dependent suboptimality guarantee that remains valid even when the true value function lies outside the chosen RKHS. This combination of convexity, a non-triviality constraint that anchors the approximation to the linearization, and a bound independent of the particular kernel coefficients is a meaningful contribution to approximate dynamic programming and could enable reliable controller synthesis for systems where exact HJB solution is intractable.

minor comments (3)
  1. [Abstract] Abstract: the phrase 'corrected 1D polynomial benchmark' should briefly indicate what the original benchmark was and what correction was applied, so that readers can immediately place the reported 3e-7 recovery in context.
  2. [Numerical experiments] Numerical experiments section: the specific kernel (e.g., polynomial degree or Gaussian bandwidth) and the placement rule for the kernel centers should be stated explicitly, together with the number of centers used in each example, to support reproducibility.
  3. [Numerical experiments] The suboptimality percentages (0.000% on the 1D case, 0.42% on Van der Pol) are given relative to different references; a short clarifying sentence on how each percentage is computed would remove any ambiguity.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary of our work, for recognizing the combination of convexity, the Riccati-Hessian anchoring constraint, and the data-dependent suboptimality bound as a meaningful contribution, and for recommending minor revision. The referee's description of the method and results is accurate.

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained

full rationale

The paper derives its LMI formulation by representing the value-function gradient in an RKHS, applying a Schur-complement transformation to the HJB inequality, and imposing an explicit linear Riccati-Hessian equality at the origin to exclude the zero solution. The suboptimality bound is stated to depend only on problem data and domain (not on the kernel coefficients or approximation error), and the numerical examples measure residuals and costs directly against known optima without reducing any central claim to a fitted input or self-referential definition. No load-bearing self-citation chains, uniqueness theorems imported from the authors, or ansatzes smuggled via prior work are used to justify the core steps; the approach remains convex and internally consistent even when V* lies outside the RKHS.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The method rests on standard properties of reproducing kernel Hilbert spaces and the existence of a solution to the linearised algebraic Riccati equation; no free parameters or invented entities are explicitly introduced in the abstract.

axioms (2)
  • domain assumption The gradient of the optimal value function lies in the reproducing kernel Hilbert space spanned by the chosen kernel and centres.
    Invoked when representing the gradient in RKHS to obtain the LMI in kernel coefficients.
  • standard math The linearised system around the equilibrium admits a stabilising solution to the algebraic Riccati equation.
    Required for the Riccati-Hessian equality constraint to be well-defined.

pith-pipeline@v0.9.0 · 5918 in / 1546 out tokens · 52840 ms · 2026-05-21T12:07:52.425358+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages · 1 internal anchor

  1. [1]

    Alexander and D

    R. Alexander and D. Giannakis. Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques.Physica D: Nonlinear Phenomena, 409:132520, 2020

  2. [2]

    Batlle, Y

    P. Batlle, Y. Chen, B. Hosseini, H. Owhadi, and A. M. Stuart. Error analysis of kernel/GP methods for nonlinear and parametric PDEs. arXiv preprint arXiv:2305.04962, 2023

  3. [3]

    D. P. Bertsekas.Dynamic Programming and Optimal Control, volume 1. Athena Scientific, 2012

  4. [4]

    Bittracher, S

    A. Bittracher, S. Klus, B. Hamzi, P. Koltai, and C. Sch¨ utte. Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifolds. arXiv preprint arXiv:1904.08622, 2019

  5. [5]

    Bouvrie and B

    J. Bouvrie and B. Hamzi. Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space. In2010 48th Annual Allerton Conference on Communication, Control, and Computing, pages 294–301, 2010

  6. [6]

    Bouvrie and B

    J. Bouvrie and B. Hamzi. Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure. InProc. of the 2012 American Control Conference, pages 294–301, 2012

  7. [7]

    Bouvrie and B

    J. Bouvrie and B. Hamzi. Kernel methods for the approximation of nonlinear systems.SIAM J. Control and Optimization, 55(4):2460–2492, 2017

  8. [8]

    Bouvrie and B

    J. Bouvrie and B. Hamzi. Kernel methods for the approximation of some key quantities of nonlinear systems.Journal of Computational Dynamics, 4(1):1–19, 2017

  9. [9]

    Boyd and L

    S. Boyd and L. Vandenberghe.Convex Optimization. Cambridge University Press, 2004

  10. [10]

    Y. Chen, B. Hosseini, H. Owhadi, and A. M. Stuart. Solving and learning nonlinear PDEs with Gaussian processes. arXiv preprint arXiv:2103.12959, 2021. 23

  11. [11]

    Cucker and S

    F. Cucker and S. Smale. On the mathematical foundations of learning.Bulletin of the American Mathematical Society, 39:1–49, 2002

  12. [12]

    G. E. Fasshauer and M. J. McCourt.Kernel-based Approximation Methods using MATLAB. World Scientific Publishing Company, 2015

  13. [13]

    Giesl.Construction of Global Lyapunov Functions Using Radial Basis Functions, volume 1904 ofLecture Notes in Mathematics

    P. Giesl.Construction of Global Lyapunov Functions Using Radial Basis Functions, volume 1904 ofLecture Notes in Mathematics. Springer, Berlin Heidelberg, 2007

  14. [14]

    Giesl, B

    P. Giesl, B. Hamzi, M. Rasmussen, and K. Webster. Approximation of Lyapunov functions from noisy data.Journal of Computational Dynamics, 7:57–81, 2019

  15. [15]

    Gros and M

    S. Gros and M. Zanon. Data-driven control of nonlinear systems: An approximate dynamic programming approach using kernel methods.IEEE Transactions on Automatic Control, 65(10):4048–4063, 2020

  16. [16]

    Haasdonk, B

    B. Haasdonk, B. Hamzi, G. Santin, and D. Wittwar. Greedy kernel methods for center manifold approximation. InNumerical Mathematics and Advanced Applications ENUMATH 2019, pages 95–106. Springer, 2020

  17. [17]

    Haasdonk, B

    B. Haasdonk, B. Hamzi, G. Santin, and D. Wittwar. Kernel methods for center manifold approximation and a weak data-based version of the center manifold theorems.Physica D: Nonlinear Phenomena, 427:133007, 2021

  18. [18]

    Hamzi and F

    B. Hamzi and F. Colonius. Kernel methods for the approximation of discrete-time linear autonomous and control systems.SN Applied Sciences, 1(7):1–12, 2019

  19. [19]

    Hamzi, A

    B. Hamzi, A. Jafarian, H. Owhadi, and L. Paillet. A note on microlocal kernel design for some slow-fast stochastic differential equations with critical transitions and application to EEG signals. arXiv preprint arXiv:2206.06312, 2022

  20. [20]

    Hamzi, C

    B. Hamzi, C. Kuehn, and S. Mohamed. A note on kernel methods for multiscale systems with critical transitions.Mathematical Methods in the Applied Sciences, 42(3):907–917, 2019

  21. [21]

    Hamzi, U

    B. Hamzi, U. G. Vaidya, and H. Owhadi. Kernel methods for some transport equations with application to learning kernels for the approximation of Koopman eigenfunctions: A unified approach via variational methods, Green’s functions, and RKHS. Preprint, ResearchGate, May 2025

  22. [22]

    B. Hou, A. R. R. Matavalam, S. Bose, and U. Vaidya. Propagating uncertainty through system dynamics in reproducing kernel Hilbert space.Physica D: Nonlinear Phenomena, page 134168, 2024

  23. [23]

    Huang and U

    B. Huang and U. Vaidya. A convex approach to data-driven optimal control via Perron- Frobenius and Koopman operators.IEEE Transactions on Automatic Control, 67(9):4778– 4785, 2022

  24. [24]

    Jalalian, J

    Y. Jalalian, J. F. O. Ramirez, A. Hsu, B. Hosseini, and H. Owhadi. Data-efficient kernel methods for learning differential equations and their solution operators: Algorithms and error analysis. arXiv preprint arXiv:2203.12610, 2025

  25. [25]

    Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences

    M. Kanagawa, P. Hennig, D. Sejdinovic, and B. K. Sriperumbudur. Gaussian processes and kernel methods: A review on connections and equivalences. arXiv preprint arXiv:1807.02582, 2018. 24

  26. [26]

    S. Klus, F. N¨ uske, and B. Hamzi. Kernel-based approximation of the Koopman generator and Schr¨ odinger operator.Entropy, 22(7):722, 2020

  27. [27]

    J. B. Lasserre. Semidefinite programming relaxations for semialgebraic problems.Mathematical Programming, 112(1):65–92, 2008

  28. [28]

    J. Lee, B. Hamzi, B. Hou, H. Owhadi, G. Santin, and U. Vaidya. Kernel methods for the ap- proximation of the eigenfunctions of the Koopman operator.Physica D: Nonlinear Phenomena, 476:134662, 2025

  29. [29]

    J. Lee, B. Hamzi, Y. Kevrekidis, and H. Owhadi. Gaussian processes simplify differential equations. arXiv preprint, September 2024

  30. [30]

    J. Lee, H. Owhadi, B. Hamzi, and U. G. Vaidya. A note on kernel methods for the construction of Lyapunov functions using Koopman eigenfunctions. Preprint, ResearchGate, December 2024

  31. [31]

    Lengyel, B

    D. Lengyel, B. Hamzi, H. Owhadi, and P. Parpas. Kernel sum of squares for data adapted kernel learning of dynamical systems from data: A global optimization approach. arXiv preprint arXiv:2408.06465, 2024

  32. [32]

    D. Long, N. Mrvaljevic, S. Zhe, and B. Hosseini. A kernel approach for PDE discovery and operator learning. arXiv preprint arXiv:2305.16397, 2023

  33. [33]

    Moyalan, H

    J. Moyalan, H. Choi, Y. Chen, and U. Vaidya. Data-driven optimal control via linear transfer operators: A convex approach.Automatica, 150:110841, 2023

  34. [34]

    H. Owhadi. Computational graph completion.Research in the Mathematical Sciences, 9(2):27, 2022

  35. [35]

    P. A. Parrilo.Structured Semidefinite Programs and Semialgebraic Geometry Methods in Ro- bustness and Optimization. PhD thesis, California Institute of Technology, 2000

  36. [36]

    P. A. Parrilo. A sum of squares optimization approach to control design.Automatica, 39(2):185–201, 2003

  37. [37]

    Raghunathan and U

    A. Raghunathan and U. Vaidya. Optimal stabilization using Lyapunov measures.IEEE Transactions on Automatic Control, 59(5):1316–1321, 2014

  38. [38]

    Santin and B

    G. Santin and B. Haasdonk. Kernel methods for surrogate modeling. InSystem and Data- Driven Methods and Algorithms. De Gruyter, 2019

  39. [39]

    Smirnov, B

    A. Smirnov, B. Hamzi, and H. Owhadi. Mean-field limits of trained weights in deep learning: A dynamical systems perspective.Dolomites Research Notes on Approximation, 15(3):89–101, 2022

  40. [40]

    U. Vaidya. When Koopman meets Hamilton and Jacobi.IEEE Transactions on Automatic Control, 2025

  41. [41]

    Vaidya and D

    U. Vaidya and D. Tellez-Castro. Data-driven stochastic optimal control with safety constraints using linear transfer operators.IEEE Transactions on Automatic Control, 68(10):6017–6032, 2023

  42. [42]

    Wendland.Scattered Data Approximation

    H. Wendland.Scattered Data Approximation. Cambridge University Press, 2004. 25