REVIEW 2 major objections 4 minor 48 references
This paper claims that minimizing a single second-order stochastic residual — one scalar space–time network trained on one-step Brownian residuals plus terminal penalties — provably controls the full jet (value, gradient, Hessian) of fully
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 20:10 UTC pith:WXLCC57H
load-bearing objection This is the first population-level full-jet convergence theory for a deep Hessian-dependent parabolic solver, and the main theorems check out; the advertised O(h^1/2) rate, however, rests on an O(h) best-approximation premise that the paper proves only qualitatively. the 2 major comments →
A Deep Second-Order Stochastic Residual Method for Fully Nonlinear Parabolic PDEs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim: the D2SRM population objective is both reliable and attainable. Reliability (Theorem 3.8): under globally Lipschitz data, identity diffusion, and small-gain condition 2√2L₂<1, every admissible candidate satisfies PE²_h ≤ C(h + L_h(θ)) — a small objective forces the whole jet (value, gradient, Hessian) to track the true solution in the Brownian occupation law. Attainability (Theorem 3.9): approximate population minimizers obey PE²_h ≤ C(h + inf_θ A_h(U_θ) + ε_opt,h), separating time discretization, representation error, and optimizer suboptimality; with the latter two O(h), the full-jet occupation norm is O(h^{1/2}). The chain runs through a source-Picard contractio
What carries the argument
The load-bearing objects are the Brownian occupation space H_β = L²(ν_β) with weight r_β(t)p_t(x) dt dx — the norm in which Hessian recovery is proved stable — and the source-update map Φ(F) = f(t,x,u_F,∇u_F,D²u_F) with Lipschitz constant λ(β) = L₀/β + L₁/√β + 2√2L₂. The 2√2 factor comes from a Gaussian affine-complement coercivity: on the Gaussian law at time t, the projected generator A^μ_t Q^aff_t is bounded by √2 times the Hessian norm, so taking β large makes Φ a contraction and yields a unique occupation mild solution. On the grid the argument shifts to an implicit reference scheme whose conditional Gaussian projections P⁰, P¹, P² extract value, first-chaos, and second-chaos channels;
Load-bearing premise
The load-bearing premise is the small-gain condition 2√2 L₂ < 1 on the driver's Hessian-Lipschitz constant (Assumption 3.1); it drives the source-Picard contraction, the stability of Theorem 6.4, and hence both main theorems. For the natural diagonal-Hessian driver f = αΣ|γ_kk| it reads α < 1/√(8d) (Remark 6.5) — dimension-hostile and narrower than the monotone-scheme regime α < 1/2. If it fails, the paper gives no bound; Section 9 also places finite-sample generalization and
What would settle it
Within the assumptions (2√2 L₂ < 1, identity diffusion, global Lipschitz driver), find one admissible Markov candidate with small population objective L_h(θ) but large full-jet occupation error PE²_h; the theorem says none exists. A concrete program: take the d = 1, f = g = 0 case of Remark 6.6 and rebuild the counterexample with σ(X_i)-measurable state-local variables instead of the path-dependent second-chaos variable Q₀ — Theorem 6.4 asserts this cannot be done, so any Markov triple with bounded J_h and divergent first-node Hessian error as h→0 would refute it. Alternatively, in the proved
If this is right
- A posteriori certification at the population level: the training objective doubles as an error certificate — small L_h(θ) provably means small value, gradient, and Hessian error in the Brownian occupation norm, with no extra regularity required of the candidate.
- Error separability: for approximate minimizers the bound is O(h) + best-representation error + optimizer gap, so the three error sources can be studied and reduced independently; O(h) in the last two yields O(h^{1/2}) full-jet accuracy.
- Well-posedness by contraction: the exact source-Picard iteration converges geometrically in the occupation space under weak Hessian coupling, identifying the solution as the fixed point the network is implicitly trying to match.
- The Markov/local comparison class is structurally necessary: the counterexample of Remark 6.6 (d = 1, f = g = 0) shows no h-independent bound can hold for arbitrary adapted path-dependent triples, delimiting any L²-residual-based analysis of second-order solvers.
- The proved regime is narrower than monotone-scheme regimes: for f = αΣ|γ_kk| the small-gain requirement α < 1/√(8d) is dimension-hostile, and the paper's own experiments show residual informativeness degrading as α grows toward the boundary.
Where Pith is reading between the lines
- If the reliability bound is right, the empirical loss could serve as a practical, conservative monitor of full-jet accuracy during training — with the caveat that the theorem is population-level and says nothing about the SGD gap or finite-sample noise.
- The small-gain boundary points at the fixed Brownian occupation law as the bottleneck rather than the network class; a testable extension is to train against a small mixture of occupation laws and check whether jet error improves in the strong-coupling regime the paper leaves open.
- The additive, linear-in-h discretization term suggests an adaptive time-stepping heuristic: local residual contributions could flag where to refine, since discretization error enters the final bound separately from representation and optimization terms.
- Because the attainability proof relies on C³ activations (centered Softplus, tanh) and qualitative density in a third-order derivative norm, a natural empirical question is whether common smooth-but-non-C³ activations still reach the proved O(h^{1/2}) jet rate — the theory would not cover them.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces D2SRM, a deep learning method for fully nonlinear parabolic PDEs in which a single scalar space-time network supplies value, gradient, and Hessian through derivative-consistent jets and is trained by minimizing one-step second-order Brownian residuals plus terminal penalties. The theoretical core is a new Brownian occupation-space framework. Under global Lipschitz assumptions on the driver and a small-gain condition 2√2 L2 < 1, Theorem 3.7 proves existence and uniqueness of a nonlinear occupation mild solution via a contracting source Picard map, with the explicit contraction constant λ(β)=L0/β+L1/√β+2√2 L2. Theorem 3.8 gives the a posteriori reliability bound PE_h^2 ≤ C(h+L_h(θ)) for admissible candidates. Theorem 3.9 gives the population convergence bound PE_h^2 ≤ C(h+inf_θ A_h(U_θ)+ε_opt,h), and, when the approximation and optimization terms are O(h), the full-jet occupation norm is O(h^{1/2}). Experiments on a 100-dimensional manufactured equation compare terminal treatments and Hessian couplings and report decreasing errors with decreasing step size, with code provided.
Significance. If the conditional claims are accepted, this is a substantial step: it is one of the first analyses that converts a small L2 learning residual into control of the full jet (value, gradient, Hessian) for Hessian-dependent fully nonlinear parabolic PDEs, and it separates time-discretization, representation, and optimization errors. The paper is unusually honest: the small-gain boundary is stated explicitly, the population-level scope is acknowledged, and Section E.4 explicitly says that the best-approximation term is only shown to vanish qualitatively. The explicit constants in Lemma 4.1, Theorem 4.2, and Theorem 3.7 are useful and appear internally consistent. The main weakness is that the advertised O(h^{1/2}) rate is conditional on an O(h) approximation premise that no concrete network family is shown to satisfy, and the numerical time-discretization study is run mostly outside the proved small-gain regime. These are limitations rather than contradictions of the formal statements, but they materially affect the paper's headline claims.
major comments (2)
- [§3.3, Eq. (3.6); §E.4; abstract] The advertised O(h^{1/2}) rate is not realized by any demonstrated network sequence. Theorem 3.9 gives PE_h^2≤C(h+inf_{θ∈Θ_h}A_h(U_θ)+ε_opt,h), so the rate requires inf A_h(U_θ)=O(h). Proposition E.3 proves only qualitative vanishing, inf A_h→0, and the text explicitly states that a coefficient-dependent O(h) rate requires additional estimates. Thus the unconditional consequence of Theorem 3.9 is convergence with no rate, and the O(h^{1/2}) statement is a conditional corollary with an unverified premise. The paper acknowledges this, but because the abstract's rate claim is a central selling point, the abstract and introduction should state clearly that the rate is not known to be attained by any concrete neural family, or a positive approximation-rate result should be supplied.
- [Assumption 3.1; Remark 6.5; §8.4] The small-gain condition 2√2 L2<1 is load-bearing for Theorems 3.7, 6.4, 3.8, and 3.9. For the benchmark driver f_α=α∑|γ_kk|, it reads α<1/√(8d), which for d=100 is about 0.035. Experiment 3, the time-discretization study whose fitted exponents are compared with the theoretical O(h^{1/2}) rate, uses α=0.25, which is outside the proved regime. Consequently the numerical evidence for the headline rate is not covered by the theorem. The paper does say there is 'no guarantee' outside the regime, but the presentation should make explicit that Experiment 3 is a heuristic probe, not a verification of Theorem 3.9. Either add a time-convergence run inside the proved range or qualify the comparison in the text and figure captions.
minor comments (4)
- [Assumptions 3.2 and 3.3] Assumption 3.2 begins 'Theorem 3.1 holds' and Assumption 3.3 begins 'Theorem 3.2 holds'; these should refer to Assumptions 3.1 and 3.2, respectively.
- [Remarks 6.5, 6.8; §8.2] Several cross-references are wrong: 'Theorem 6.5' and 'Theorem 6.8' should be Remark 6.5 and Remark 6.8, and 'σcsp defined in Theorem 7.2' should refer to Proposition E.3 or §E.4.
- [Definition 2.1 and Theorem 3.9] The parameter sets Θ_h are allowed to depend on h, but this is not made explicit in the definitions. For the infimum in Theorem 3.9 to vanish, the network family must generally grow with h; the text should state that no fixed finite architecture is claimed to achieve the approximation bound.
- [Eq. (8.3)] The denominator '+ε_rel^2 B N' is ambiguous: it is unclear whether the regularizer is ε_rel^2/(B N) or ε_rel^2 multiplied by B N. Please clarify the displayed formula.
Circularity Check
No significant circularity: the bounds are conditional a posteriori stability results, no fitted constants feed the derivation, and the self-citations are not load-bearing.
full rationale
The derivation chain is self-contained and not circular. The paper defines the population objective L_h and the path error PE2_h, then proves in Theorem 3.8 that PE2_h(Y^theta,Z^theta,Gamma^theta) <= C(h+L_h(theta)) by inserting the auxiliary implicit reference scheme and using exact-grid consistency (Proposition 5.8), mesh-uniform residual stability (Theorem 6.4), and convergence of the reference scheme (Corollary 6.7). Proposition 7.1 bounds L_h by C(h+A_h), so Theorem 3.9 follows as an upper bound, not by construction. Nothing is fitted to the numerical results: the small-gain condition 2*sqrt(2)*L2 < 1 and the rate constants are derived before Section 8, and the experimental log-log slopes are presented as empirical checks rather than as inputs. The conditional O(h^{1/2}) assertion is the direct square-root of inequality (3.6) under the stated O(h) hypotheses; Section E.4 explicitly proves only qualitative vanishing of the best-approximation term and says a quantitative O(h) rate requires additional estimates, which is an acknowledged unverified premise rather than a circular reduction. The self-citations ([6], [17], [19]) appear in related-work and forward-looking remarks and are not load-bearing for the main theorems; no uniqueness theorem by the same authors is invoked to force the choice of method or objective. I therefore cannot exhibit any specific equation or fitted parameter that makes a supposed prediction equal to its own input.
Axiom & Free-Parameter Ledger
free parameters (3)
- beta (occupation weight) =
not fixed numerically; any sufficiently large value with lambda(beta) < 1 (no data fitting)
- alpha (benchmark Hessian coupling) =
0.025, 1/sqrt(800) ~ 0.035, 0.25, 0.45
- network/training hyperparameters (width 64x3, batch 256, LR schedules) =
standard choices from the deep-PDE-solver literature
axioms (6)
- domain assumption Global Lipschitz continuity of f in (y,z,gamma) with constants L0, L1, L2 and the small-gain condition 2*sqrt(2)*L2 < 1
- domain assumption g in H^2(mu_T), spatial Lipschitz continuity of f, time 1/2-Holder and Brownian-state regularity (Lt, LB)
- domain assumption Identity diffusion X_t = x0 + W_t with a fixed deterministic start
- standard math Standard Gaussian analytic toolkit: Poincare inequality, Gaussian Bochner/OU identity, Hermite-Mehler expansions, Ito formula, Schur test and Hardy-Copson weighted inequalities
- domain assumption Markov comparison class C_h: the sampled jet must be sigma(X_i)-measurable at each node
- standard math Hornik-type C3 derivative density of the centered-Softplus ridge class with respect to the finite measure nu_h
invented entities (1)
-
Nonlinear occupation mild solution (source fixed point F* in H_beta)
independent evidence
read the original abstract
We introduce the Deep Second-Order Stochastic Residual Method (D2SRM) for high-dimensional, Hessian-dependent fully nonlinear parabolic PDEs. A single scalar space--time network generates derivative-consistent approximations of the solution, gradient, and Hessian, which are trained jointly through second-order Brownian one-step residuals and terminal value and gradient penalties. For globally Lipschitz equations with identity diffusion and sufficiently weak Hessian coupling, we establish well-posedness in a Brownian occupation space and develop a population-level convergence theory. Under additional regularity, an a posteriori estimate bounds the squared full-jet occupation error of any admissible candidate by the time step and its population objective. For approximate population minimizers, the error bound separates time discretization, neural approximation, and population suboptimality; when the latter two terms are $O(h)$, the full-jet occupation norm is $O(h^{1/2})$. Experiments on a 100-dimensional manufactured benchmark compare terminal treatments, probe Hessian couplings inside and outside the proved small-gain range, and show decreasing errors as the time step decreases. The code is available at https://github.com/ZZHPKU/D2SRM.
Reference graph
Works this paper leans on
-
[1]
and Long, J
Han, J. and Long, J. , title =. Probab. Uncertain. Quant. Risk , volume =. 2020 , eid =
2020
-
[2]
and Han, J
E, W. and Han, J. and Jentzen, A. , title =. Commun. Math. Stat. , volume =. 2017 , pages =
2017
-
[3]
Deep Backward Schemes for High-Dimensional Nonlinear
Hur. Deep Backward Schemes for High-Dimensional Nonlinear. Math. Comp. , volume =. 2020 , pages =
2020
-
[4]
and Touzi, N
Fahim, A. and Touzi, N. and Warin, X. , title =. Ann. Appl. Probab. , volume =. 2011 , pages =
2011
-
[5]
and Zhang, J
Guo, W. and Zhang, J. and Zhuo, J. , title =. Ann. Appl. Probab. , volume =. 2015 , pages =
2015
-
[6]
Adapted Solution of a Backward Stochastic Differential Equation , journal =
Pardoux,. Adapted Solution of a Backward Stochastic Differential Equation , journal =. 1990 , pages =
1990
-
[7]
, title =
Janson, S. , title =
-
[8]
, title =
Nualart, D. , title =
-
[9]
and Peccati, G
Nourdin, I. and Peccati, G. , title =
-
[10]
and Persson, L.-E
Kufner, A. and Persson, L.-E. and Samko, N. , title =. 2017 , doi =
2017
-
[11]
Adams, R. A. and Fournier, J. J. F. , title =
-
[12]
, title =
Hornik, K. , title =. Neural Networks , volume =. 1991 , pages =
1991
-
[13]
Error Bounds for Approximations with Deep
G. Error Bounds for Approximations with Deep. Anal. Appl. (Singap.) , volume =. 2020 , pages =
2020
-
[14]
2021 , pages =
On the Approximation of Functions by Tanh Neural Networks , journal =. 2021 , pages =
2021
-
[15]
and Schmocker, P
Neufeld, A. and Schmocker, P. , title =. Anal. Appl. (Singap.) , volume =. 2026 , pages =
2026
-
[16]
and Yang, H
Hon, S. and Yang, H. , title =. Neural Networks , volume =. 2022 , pages =
2022
-
[17]
Aronson, D. G. , title =. Bull. Amer. Math. Soc. , volume =. 1967 , pages =
1967
-
[18]
, title =
Sheu, S.-J. , title =. Ann. Probab. , volume =. 1991 , pages =
1991
-
[19]
, title =
Ledoux, M. , title =. Ann. Fac. Sci. Toulouse Math. (6) , volume =. 2000 , pages =
2000
-
[20]
Stroock, D. W. and Turetsky, J. , title =. Comm. Anal. Geom. , volume =. 1998 , pages =
1998
-
[21]
and Long, J
Chen, H. and Long, J. and Wu, L. , title =. Ann. Statist. , volume =. 2025 , pages =
2025
-
[22]
Halmos, P. R. , title =. A Hilbert Space Problem Book , edition =. 1982 , pages =
1982
-
[23]
and Shreve, S
Karatzas, I. and Shreve, S. E. , title =. 1991 , doi =
1991
-
[24]
and Soner, H
Cheridito, P. and Soner, H. M. and Touzi, N. and Victoir, N. , title =. Comm. Pure Appl. Math. , volume =. 2007 , pages =
2007
-
[25]
and Hornung, F
Grohs, P. and Hornung, F. and Jentzen, A. and von Wurstemberger, P. , title =. Mem. Amer. Math. Soc. , volume =. 2023 , eid =
2023
-
[26]
and Jentzen, A
Hutzenthaler, M. and Jentzen, A. and Kruse, T. and Nguyen, T. A. , title =. Partial Differ. Equ. Appl. , volume =. 2020 , eid =
2020
-
[27]
and Shukla, K
Hu, Z. and Shukla, K. and Karniadakis, G. E. and Kawaguchi, K. , title =. Neural Networks , volume =. 2024 , eid =
2024
-
[28]
and Shi, Z
Hu, Z. and Shi, Z. and Karniadakis, G. E. and Kawaguchi, K. , title =. Comput. Methods Appl. Mech. Engrg. , volume =. 2024 , eid =
2024
-
[29]
and Guillin, A
Cattiaux, P. and Guillin, A. , title =. S. 2014 , pages =
2014
-
[30]
and Li, X.-M
Chen, X. and Li, X.-M. and Wu, B. , title =. Ann. Probab. , volume =. 2023 , pages =
2023
-
[31]
Time-Uniform
Monmarch. Time-Uniform. Electron. J. Probab. , volume =. 2024 , eid =
2024
-
[32]
and Jentzen, A
Han, J. and Jentzen, A. and E, W. , title =. Proc. Natl. Acad. Sci. USA , volume =. 2018 , pages =
2018
-
[33]
and Han, J
E, W. and Han, J. and Jentzen, A. , title =. Nonlinearity , volume =. 2022 , pages =
2022
-
[34]
and Souganidis, P
Barles, G. and Souganidis, P. E. , title =. Asymptot. Anal. , volume =. 1991 , pages =
1991
-
[35]
Soner, H. M. and Touzi, N. and Zhang, J. , title =. Probab. Theory Related Fields , volume =. 2012 , pages =
2012
-
[36]
and E, W
Beck, C. and E, W. and Jentzen, A. , title =. J. Nonlinear Sci. , volume =. 2019 , pages =
2019
-
[37]
and Spiliopoulos, K
Sirignano, J. and Spiliopoulos, K. , title =. J. Comput. Phys. , volume =. 2018 , pages =
2018
-
[38]
and Perdikaris, P
Raissi, M. and Perdikaris, P. and Karniadakis, G. E. , title =. J. Comput. Phys. , volume =. 2019 , pages =
2019
-
[39]
and Hu, W
Han, J. and Hu, W. and Long, J. and Zhao, Y. , title =. SIAM J. Sci. Comput. , volume =. 2026 , pages =
2026
-
[40]
and Ouyang, D
Jia, Y. and Ouyang, D. and Pham, H. and Zhou, X. Y. , title =. 2026 , eprint =
2026
-
[41]
and Warin, X
Pham, H. and Warin, X. and Germain, M. , title =. Partial Differ. Equ. Appl. , volume =. 2021 , eid =
2021
-
[42]
, title =
Tan, X. , title =. Electron. J. Probab. , volume =. 2013 , eid =
2013
-
[43]
and Molinaro, R
Mishra, S. and Molinaro, R. , title =. IMA J. Numer. Anal. , volume =. 2023 , pages =
2023
-
[44]
and Darbon, J
Shin, Y. and Darbon, J. and Karniadakis, G. E. , title =. Commun. Comput. Phys. , volume =. 2020 , pages =
2020
-
[45]
Nguwi, J. Y. and Penent, G. and Privault, N. , title =. J. Comput. Phys. , volume =. 2024 , eid =
2024
-
[46]
and Loeper, G
Lefebvre, W. and Loeper, G. and Pham, H. , title =. Digital Finance , volume =. 2023 , pages =
2023
-
[47]
and Han, J
Zhou, M. and Han, J. and Lu, J. , title =. SIAM J. Sci. Comput. , volume =. 2021 , number =
2021
-
[48]
and Lu, J
Zhou, M. and Lu, J. , title =. SIAM J. Control Optim. , volume =. 2025 , number =
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.