Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling

Haley Rosso; Khachik Sargsyan; Lars Ruthotto

REVIEW 4 major objections 5 minor 34 references

Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that Legendre polynomial bases are the right choice for parameterizing time-dependent weights in continuous-time neural networks, because their orthogonality keeps training stable and cheap while preserving accuracy.

desk verdict Useful efficiency comparison, but the neural-ODE accuracy claim is contradicted by the paper's own Table 5 on two benchmarks. read the letter →

arxiv 2507.22045 v1 pith:7GVFHCZZ submitted 2025-07-29 cs.LG math.OC

Haley Rosso , Lars Ruthotto , Khachik Sargsyan This is my paper

classification cs.LGmath.OC MSC 68T0765F3565L06

keywords neuralODEsweightparameterizationsurrogatemodelingresidualnetworksLegendrepolynomialsmonomialbasisorthogonalpolynomialdiscretize-then-optimize

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Continuous-time neural networks — neural ODEs and residual networks — are attractive surrogates for expensive physical simulators, but their time-varying weights are costly and unstable to train. This paper asks whether constraining those weights to a low-dimensional polynomial subspace makes training easier, and which polynomial basis should be used. The central claim is that an orthogonal basis matters: Legendre-parameterized weights train more stably, need fewer function evaluations, and match or beat both monomial parameterizations and unconstrained weights in accuracy. This is tested in both discretize-then-optimize and optimize-then-discretize settings across three high-dimensional surrogate benchmarks. If correct, the practical message is to reach for orthogonal polynomial bases when parameterizing time-dependent network weights.

What carries the argument

The central object is the polynomial weight parameterization $\theta_P(t)=\sum_{i=1}^d p_i(t)\theta_i$, evaluated at $N$ discrete times through the basis matrix $A$ with $A_{ij}=p_i(t_j)$, so the network becomes $F(y,\Theta A)$ in discretize-then-optimize and gradients are projected onto the basis by integrals in optimize-then-discretize. The mechanism that carries the argument is the conditioning of the Vandermonde matrix built from the basis: monomials give a Vandermonde matrix whose condition number grows fast with degree, while Legendre polynomials' orthogonality keeps the matrix well-conditioned. This conditioning difference is what the paper identifies as the reason monomial-parameterized training stalls and Legendre-parameterized training converges stably and cheaply.

What would settle it

Rerun the DCR neural-ODE comparison with monomial parameterization using per-basis tuned learning rates, schedules, and a rescaled time interval, alongside a direct measurement of the monomial and Legendre Vandermonde condition numbers on the actual time grid; if monomial convergence and function-evaluation counts become comparable to Legendre's, the claimed conditioning advantage is not decisive.

Watch

Extended reading notes

Core claim

The paper's discovery is that replacing monomial basis functions with Legendre polynomials in the weight parameterization $\theta_P(t)=\sum_i p_i(t)\theta_i$ changes the numerical conditioning of the training problem enough to determine whether convergence happens. Because the monomial Vandermonde matrix relating basis coefficients to time samples is ill-conditioned, small errors in evaluating the basis at time points become large weight errors, and training stalls; the orthogonal Legendre Vandermonde matrix stays well-conditioned, so gradients are projected cleanly onto the basis. In ResNets, third-order Legendre parameterization matches unparameterized accuracy while reducing the number of trainable weights, and it pairs especially well with Hamiltonian/Verlet architectures. In neural ODEs, where an adaptive integrator's function-evaluation count dominates runtime, Legendre parameterization cuts function evaluations by roughly an order of magnitude (about 8x on the DCR benchmark) while reaching similar or better loss. The authors' conclusion is that parameterization is a favorable expressivity-efficiency trade-off only when the basis is orthogonal.

Load-bearing premise

The load-bearing premise is that the identical optimizer settings used for both bases are fair, so the monomial basis's poor convergence reflects the basis itself rather than a mismatch between monomials and those fixed hyperparameters.

Editorial extensions

If this is right

A Legendre-parameterized ResNet can match the accuracy of an unparameterized ResNet with fewer trainable weights, reducing memory and training cost in memory-constrained settings.
In neural ODEs, switching from monomial to Legendre weight bases can reduce the number of adaptive-integrator function evaluations by nearly an order of magnitude, which dominates runtime.
Higher-degree Legendre bases (up to degree 6 in the reported experiments) increase expressivity and approach or beat unparameterized baselines, at the price of more parameters.
Both major training paradigms—discretize-then-optimize and optimize-then-discretize—benefit from the same orthogonal-basis parameterization, suggesting a unified default for continuous-time surrogates.
The Hamiltonian/Verlet architecture is a particularly good match for Legendre parameterization, since smooth orthogonal weights align with energy-preserving time integration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural testable extension is to swap Legendre for Chebyshev or Fourier bases; if orthogonality is the real driver, similar efficiency gains should appear, and the paper's conditioning story would be confirmed.
Because all bases were compared under identical optimizer settings, an ablation that tunes learning rate and schedule separately for monomials would reveal whether monomials are intrinsically worse or just need different hyperparameters.
The conditioning mechanism predicts that as polynomial degree grows, monomial parameterization should degrade in stability faster than Legendre; a degree scan on a small synthetic ODE would quantify this directly.
In low-data surrogate regimes, parameterization acts as a form of regularization, so Legendre bases may also improve generalization as training set size shrinks; varying the training set size would test this.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: the efficiency story is real, the accuracy story is not as written. The paper compares monomial vs Legendre polynomial parameterization of time-varying weights in ResNets, Hamiltonian networks, and neural ODEs on three surrogate modeling tasks, under both discretize-then-optimize and optimize-then-discretize training. The side-by-side comparison across architectures and training paradigms is new, even though the parameterization itself is essentially NANODE's. The computational savings are substantial: in neural ODEs, Legendre cuts function evaluations by about an order of magnitude at the same polynomial degree, on all three benchmarks. That is a practically useful result, and the paper is clearly written.

The soft spot is load-bearing. Table 5, the paper's only seed-level statistics, says Legendre is worse than monomial on CDR (validation 326 vs 298) and far worse on DCR (validation 0.281 vs 0.037, std 0.607 vs 0.056). But the text and Figure 6 claim the opposite for DCR: Legendre reaches about 0.28 while monomial never goes below 0.6. These cannot both be right. Either the table is mislabeled or the figure is from a different configuration. As submitted, the headline claim of 'comparable or better accuracy' is unsupported for two of three benchmarks. The efficiency claim survives; the accuracy claim needs seed-level reconciliation.

There is also an incorrect statement in Section 4.2: constant-weight neural ODEs do not need time parameterization to compute gradients; the adjoint method in Chen et al. handles that fine. It is not central to the experiments, but it is wrong. A smaller issue: the ResNet experiments fix the same learning rate, batch size, and epoch count for both bases, so the monomial failure, while plausibly due to Vandermonde conditioning, is not checked against different optimization settings. And the ResNet figures lack error bars.

Who is this for? Practitioners in scientific ML who want a cheaper default weighting scheme in continuous-time surrogates. The efficiency result is probably real and worth adopting. But the paper needs a major revision: reconcile Table 5 with the text/figure, release seed-level curves, and correct the Section 4.2 claim. I would send it to peer review because the comparison is useful and the efficiency claim is likely correct, despite the current reporting problems.

Referee Report

4 major / 5 minor

Summary. The manuscript studies polynomial weight-parameterization for continuous-time deep networks: it expands time-dependent weights in monomial and Legendre bases and applies both to ResNet-style (discretize-then-optimize) and neural-ODE (optimize-then-discretize) training on three surrogate benchmarks (ELM, CDR, DCR). The main claim is that Legendre bases give more stable training, lower computational cost, and accuracy comparable to or better than monomial or unconstrained weights. The function-evaluation reductions in Table 5 support the cost part of this claim, but the accuracy and stability parts are not consistently supported by the paper's own exhibits, and the DCR neural-ODE results in Table 5 and Figure 6 are mutually inconsistent.

Significance. If the reported Legendre advantages held, the paper would provide a simple, useful default choice for time-dependent weight parameterization in neural-ODE surrogates. The manuscript has a clear experimental design, and the efficiency evidence is strong: across all three datasets, Legendre reduces the number of function evaluations by roughly an order of magnitude (Table 5). However, the accuracy and stability claims are currently not established: for CDR and DCR, Table 5 shows monomial with lower mean error and lower variance, and the DCR figure contradicts the DCR table. The paper provides neither code nor seed-level data, so the reported numbers cannot be checked externally. The conditioning explanation in Section 4.3 is plausible but is asserted rather than demonstrated. With corrected exhibits, per-seed data, and a fairer hyperparameter protocol, the comparison could be made convincing.

major comments (4)

[§5.3, Table 5 and Figure 6] The DCR rows of Table 5 and the DCR panel of Figure 6 report opposite accuracy orderings. Table 5 gives Legendre mean training/validation errors of 0.282/0.281 with standard deviations 0.617/0.607, and monomial errors of 0.036/0.037 with standard deviations 0.055/0.056, so monomial is about 7.6x more accurate and far more stable across the five seeds. Figure 6 and the accompanying text instead say the Legendre model reaches a validation loss near 0.28 while monomial fails to drop below 0.6. Both cannot be correct. The abstract's 'accuracy comparable to or better' and 'more stable training dynamics' claims rest on these exhibits; moreover, Table 5 does not report the non-parameterized neural-ODE error values needed to quantify the 'comparable to or better than unconstrained' part of the claim. Please reconcile the exhibits and provide seed-level curves and non-parameterized numeric errors.
[§5.2–5.3 and §4.3] The comparison uses identical hyperparameters for monomial and Legendre bases (learning rate 0.001, batch size 32, up to 1000 epochs, ADAM and GNvpro), so the paper's attribution of monomial failure to Vandermonde ill-conditioning is not established. A basis that requires a smaller learning rate or more epochs would look unfairly bad under fixed settings. Please either report a per-basis hyperparameter search, show that monomial still fails under well-tuned settings, or provide direct condition-number/spectral diagnostics for the two Vandermonde matrices in Eqs. (11) and (13). Without this, the observed Legendre advantage may be an optimization-protocol artifact rather than a property of the basis.
[§5.3, Table 5 (CDR rows)] For CDR, Table 5 reports Legendre mean training/validation errors of 282.2/326.4 versus monomial 264.9/297.9, with Legendre standard deviations about four times larger (104/92 versus 26/28). The text in §5.3 states only that 'Legendre remains the most efficient' and does not acknowledge that monomial achieves lower mean error and lower variance on this benchmark. This contradicts the 'accuracy comparable to or better' and 'more stable training dynamics' claims for a second dataset. Please report the seed-level results, investigate whether the large Legendre variance reflects failed runs, and either revise the accuracy/stability claims or provide an explanation (e.g., a different convergence tolerance).
[§5.3] The paper claims that Legendre parameterization 'introduces negligible overhead in terms of implementation or FLOPs,' but the only cost evidence provided is the number of function evaluations in Table 5. Function evaluations are not the same as FLOPs, wall-clock time, or memory, and the polynomial basis evaluation and the adjoint integration can change the per-evaluation cost. Please report measured runtime/FLOP/memory comparisons, or rephrase the efficiency claim to refer specifically to the reported function-evaluation counts.

minor comments (5)

[§5.1 and §1] The acronym DCR is used inconsistently: Section 1 calls it 'direct current resistivity,' while Section 5.1 labels it 'Diffusion-Convection.' Please make the acronym and model description consistent.
[Data availability] The data availability section contains the duplicated sentence 'The CDR dataset can be downloaded here.' with no actual URL. Provide working links for all datasets and, ideally, the code used to produce Tables 2–5 and Figures 2–6.
[Table 2 caption] The caption of Table 2 refers to the ELM surrogate model, but the table contains rows for DCR and CDR as well; correct the caption and clarify the apparent '634' entry in the CDR non-parameterized ADAM row (T=1).
[§3.2 and Eq. (9)] Equation (9) and the gradient formula in Section 3.2 contain notational problems: the displayed integral '∫ a(t)^T ∇θ f(...) a(t) dt' appears to have an extra 'a(t)' factor, and Eq. (8) writes '∂u(t)/∂θP(t)' as if it were a scalar derivative of a vector-valued function. Please use full Jacobian notation.
[Throughout] Typographical issues: 'Picard-Linderlof' should be 'Picard-Lindelöf,' and the caption of Figure 6 should state what the colored lines correspond to and explain why the legend order changes between panels.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the Legendre-vs-monomial accuracy and efficiency claims are empirical benchmark comparisons, not derivations that reduce to their inputs.

full rationale

The paper's central claims are experimental: Legendre parameterization yields more stable training dynamics, fewer function evaluations, and comparable or better accuracy than monomial or unconstrained weights. These are supported by training and validation error curves, function-evaluation counts, and convergence tables (Tables 2-5, Figures 2-6), all of which are external benchmarks (ELM, CDR, DCR). The parameterization formula in Eq. (4), θP(t)=Σ pi(t)θi, is a representation, not a result that presumes the conclusion; no fitted parameter is relabeled as a prediction, and no equation is equivalent to another by construction. The Vandermonde conditioning discussion in Section 4.3 is a standard mathematical statement used to explain, not to define, the observed monomial failures. The paper does cite several works by co-author Ruthotto ([10], [16], [26]) for the ResNet/Hamiltonian architecture, the GNvpro optimizer, and the discretize-then-optimize versus optimize-then-discretize framing, but these citations shape the experimental setup rather than provide the load-bearing evidence for the Legendre advantage; the benchmark results stand independently of those cited results. One internal consistency issue should be flagged as a correctness risk rather than circularity: Table 5 reports DCR neural-ODE mean errors of 0.282/0.281 for Legendre and 0.036/0.037 for monomial, while Section 5.3 and Figure 6 state that the Legendre model reaches a validation loss of about 0.28 and the monomial case fails to drop below 0.6. This contradiction affects the strength of the accuracy claim as written, but it does not make any derivation circular. No circular step can be exhibited with a quote and a specific reduction, so the circularity score is 0.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The empirical comparison rests on the chosen hyperparameters (polynomial degree, depth, optimizer settings) and on the representativeness of the three benchmarks. No new physical or mathematical entities are postulated. The conditioning explanation for monomial failure is an assumption, not a proven mechanism.

free parameters (6)

polynomial degree d = 3 (default), 4-6 in one experiment
Hand-chosen basis order that controls model expressivity. The paper's default comparison uses d=3, and the central claim depends on this choice.
number of time steps N / depth T = 12 steps (stated in Section 5.2); Table 2 tabulates depths T=1,5,10
The paper states 12 time steps and 15 channels, but Table 2 reports training loss at depths T=1,5,10; the inconsistent reporting affects the parameter-count comparison.
learning rate = 0.001
Fixed for the ResNet experiments and apparently for the neural ODE experiments. No tuning is reported, and this choice may disadvantage the monomial basis.
batch size = 32
Fixed for ResNet training; not reported for the neural ODE experiments.
regularization coefficient alpha = not specified numerically
Equation (2) defines alpha but its value is never given. The loss landscape and hence the comparison between parameterizations depend on it.
number of channels = 15
Fixed width of the hidden state in all experiments; not varied or justified.

assumptions (3)

standard math Right-hand side f of the neural ODE is Lipschitz continuous in the state
Invoked via the Picard-Lindelof theorem (Theorem 1 in Section 3.2.1) to guarantee local existence and uniqueness of the initial value problem. Standard for neural ODE analysis.
domain assumption The three benchmarks (ELM, CDR, DCR) are representative surrogate modeling problems
The paper generalizes from three datasets to a broad claim about weight parameterization. Representativeness is asserted, not established.
ad hoc to paper Monomial Vandermonde conditioning explains the observed convergence failure
Section 4.3 argues that the ill-conditioned Vandermonde matrix causes monomials to fail, but the direct mechanism linking conditioning to the observed optimization stall is not demonstrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling." pith.science (2026). https://pith.science/paper/7GVFHCZZ

@misc{pith2026250722045,
  author       = {Pith},
  title        = {Pith review of: Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7GVFHCZZ}},
  note         = {Machine review of arXiv:2507.22045}
}

read the original abstract

Continuous-time deep learning models, such as neural ordinary differential equations (ODEs), offer a promising framework for surrogate modeling of complex physical systems. A central challenge in training these models lies in learning expressive yet stable time-varying weights, particularly under computational constraints. This work investigates weight parameterization strategies that constrain the temporal evolution of weights to a low-dimensional subspace spanned by polynomial basis functions. We evaluate both monomial and Legendre polynomial bases within neural ODE and residual network (ResNet) architectures under discretize-then-optimize and optimize-then-discretize training paradigms. Experimental results across three high-dimensional benchmark problems show that Legendre parameterizations yield more stable training dynamics, reduce computational cost, and achieve accuracy comparable to or better than both monomial parameterizations and unconstrained weight models. These findings elucidate the role of basis choice in time-dependent weight parameterization and demonstrate that using orthogonal polynomial bases offers a favorable tradeoff between model expressivity and training efficiency.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 23 canonical work pages

[1]

In: Summer School of the German Research School for Simula tion Sciences (2019)

Sudret, B.: Surrogate models for uncertainty quantiﬁcation an d design optimiza- tion. In: Summer School of the German Research School for Simula tion Sciences (2019). https://doi.org/10.3929/ethz-b-000359599

work page doi:10.3929/ethz-b-000359599 2019
[2]

: Tgcnn: An eﬃcient surrogate for real-time data assimila- tion in subsurface ﬂow

Kovachki, N.B., et al. : Tgcnn: An eﬃcient surrogate for real-time data assimila- tion in subsurface ﬂow. Computers & Mathematics with Applications 81, 336–354 (2021) https://doi.org/10.1016/j.camwa.2020.12.019

work page doi:10.1016/j.camwa.2020.12.019 2021
[3]

The Innovat ion Energy 2(2), 100087–1 (2025)

Wang, N., Chen, Y., Zhang, D.: A comprehensive review of physics- informed deep learning and its applications in geoenergy development. The Innovat ion Energy 2(2), 100087–1 (2025)

work page 2025
[4]

Journal of Computationa l physics 378, 686–707 (2019)

Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neur al networks: 31 A deep learning framework for solving forward and inverse problems involving nonlinear partial diﬀerential equations. Journal of Computationa l physics 378, 686–707 (2019)

work page 2019
[5]

Nature machine intelligence 3(3), 218–229 (2021)

Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E.: Learning nonlin ear oper- ators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence 3(3), 218–229 (2021)

work page 2021
[6]

arXiv (2018)

Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duve- naud, D.: Neural Ordinary Diﬀerential Equations. arXiv (2018). https://doi.org/10.48550/ARXIV.1806.07366 . https://arxiv.org/abs/1806.07366

arXiv doi:10.48550/arxiv.1806.07366 2018
[7]

Ruthotto, L.: Diﬀerential Equations for Continuous-Time Deep L earning (2024)

work page 2024
[8]

Spline parameterization of neural network controls for deep learning

G¨ unther, S., Pazner, W., Qi, D.: Spline parameterization of neura l network con- trols for deep learning. arXiv preprint arXiv:2103.00301 abs/2103.00301 (2021) 2103.00301

work page Pith review arXiv 2021

Show all 34 references

[9]

Communications in Mathematics and Statistics 5(1), 1–11 (2017) https://doi.org/10.1007/s40304-017-0103-z

E, W.: A Proposal on Machine Learning via Dynamical Systems. Communications in Mathematics and Statistics 5(1), 1–11 (2017) https://doi.org/10.1007/s40304-017-0103-z

2017 doi
[10]

Inverse Problems 34(1), 014004 (2017) https://doi.org/10.1088/1361-6420/aa9a90 1705.03341

Haber, E., Ruthotto, L.: Stable architectures for deep neura l networks. Inverse Problems 34(1), 014004 (2017) https://doi.org/10.1088/1361-6420/aa9a90 1705.03341

2017 arXiv
[11]

Ott, K., Katiyar, P., Hennig, P., Tiemann, M.: When are neural ODE solutions proper odes? CoRR abs/2007.15386 (2020) 2007.15386

2020 arXiv
[12]

https://arxiv.org/abs/2205.14612

Sander, M.E., Ablin, P., Peyr´ e, G.: Do Residual Neural Networksdiscretize Neural Ordinary Diﬀerential Equations? (2022). https://arxiv.org/abs/2205.14612

2022 arXiv
[13]

CoRR abs/2005.01906 (2020) 2005.01906

Davis, J.Q., Choromanski, K., Varley, J., Lee, H., Slotine, J.E., Likho sterov, V., Weller, A., Makadia, A., Sindhwani, V.: Time dependence in non-autonom ous neural odes. CoRR abs/2005.01906 (2020) 2005.01906

2020 arXiv
[14]

CoRR abs/1912.10382 (2019) 1912.10382

Li, Q., Lin, T., Shen, Z.: Deep learning via dynamical systems: An ap proximation perspective. CoRR abs/1912.10382 (2019) 1912.10382

2019 arXiv
[15]

Journal of Computational Dynamics 6(2), 171–198 (2019)

Benning, M., Celledoni, E., Ehrhardt, M.J., Owren, B., Sch¨ onlieb, C .-B.: Deep learning as optimal control problems: Models and numerical method s. Journal of Computational Dynamics 6(2), 171–198 (2019)

2019
[16]

CoRR abs/2007.13171 (2020) 2007.13171 32

Newman, E., Ruthotto, L., Hart, J.L., Bloemen Waanders, B.G.: Tr ain like a (var)pro: Eﬃcient training of neural networks with variable proje ction. CoRR abs/2007.13171 (2020) 2007.13171 32

2020 arXiv
[17]

arXiv preprint arXiv:1909.12077 (2019)

Zhong, Y.D., Dey, B., Chakraborty, A.: Symplectic ode-net: Lea rning hamiltonian dynamics with control. arXiv preprint arXiv:1909.12077 (2019)

2019 arXiv
[18]

In: ICLR 2024 Workshop on AI4Diﬀeren tialEquations In Science (2024)

Nair, A., Barwey, S., Pal, P., Maulik, R.: Investigation of latent time -scales in neu- ral ODE surrogate models. In: ICLR 2024 Workshop on AI4Diﬀeren tialEquations In Science (2024). https://openreview.net/forum?id=zLMeuYXUve

2024
[19]

Machine Learning: Science and Technology 6(2), 025069 (2025) https://doi.org/10.1088/2632-2153/ade4ee

Vermari¨ en, G., Bisbas, T.G., Viti, S., Zhao, Y., Tang, X., Ravichan dran, R.: Neuralpdr: neural diﬀerential equations as surrogate models for photodissoci- ation regions. Machine Learning: Science and Technology 6(2), 025069 (2025) https://doi.org/10.1088/2632-2153/ade4ee

2025 doi
[20]

Computer Methods in Applied Mec hanics and Engineering 441, 117990 (2025) https://doi.org/10.1016/j.cma.2025.117990

Zhou, A., Barati Farimani, A.: Predicting change, not states: A n alternate frame- work for neural pde surrogates. Computer Methods in Applied Mec hanics and Engineering 441, 117990 (2025) https://doi.org/10.1016/j.cma.2025.117990

2025
[21]

https://arxiv.org/abs/2209.10633

Yu, D., Miao, H., Wu, H.: Neural Generalized Ordinary Diﬀerential E quations with Layer-varying Parameters (2022). https://arxiv.org/abs/2209.10633

2022 arXiv
[22]

CoRR abs/2002.08071 (2020) 2002.08071

Massaroli, S., Poli, M., Park, J., Yamashita, A., Asama, H.: Dissectin g neural odes. CoRR abs/2002.08071 (2020) 2002.08071

2020 arXiv
[23]

CoRR abs/1512.03385 (2015) 1512.03385

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for imag e recognition. CoRR abs/1512.03385 (2015) 1512.03385

2015 arXiv
[24]

CoRR abs/1709.06079 (2017) 1709.06079

Huang, L., Liu, X., Lang, B., Yu, A.W., Li, B.: Orthogonal weight nor malization: Solution to optimization over multiple dependent stiefel manifolds in de ep neural networks. CoRR abs/1709.06079 (2017) 1709.06079

2017 arXiv
[25]

CoRR abs/1702.00071 (2017) 1702.00071

Vorontsov, E., Trabelsi, C., Kadoury, S., Pal, C.: On orthogonalit y and learning recurrent networks with long term dependencies. CoRR abs/1702.00071 (2017) 1702.00071

2017 arXiv
[26]

optimize-discr etize for time- series regression and continuous normalizing ﬂows

Onken, D., Ruthotto, L.: Discretize-optimize vs. optimize-discr etize for time- series regression and continuous normalizing ﬂows. CoRR abs/2005.13420 (2020) 2005.13420

2020 arXiv
[27]

In: Leitmann, G

Kopp, R.E.: Pontryagin maximum principle. In: Leitmann, G. (ed.) O ptimiza- tion Techniques. Mathematics in Science and Engineering, vol. 5, pp. 255–279. Elsevier, New York (1962). https://doi.org/10.1016/S0076-5392(08)62095-0 . https://www.sciencedirect.com/science/article/pi...

1962 doi
[28]

CoRR abs/1902.10298 (2019) 1902.10298

Gholami, A., Keutzer, K., Biros, G.: ANODE: unconditionally accura te memory- eﬃcient gradients for neural odes. CoRR abs/1902.10298 (2019) 1902.10298

2019 arXiv
[29]

Applied Mathematical Sciences

Abraham, R., Marsden, J.E., Ratiu, T.: Manifolds, Tensor Analysis , and Applications. Applied Mathematical Sciences. Springer, New York (1 993). 33 https://books.google.com/books?id=dWHet zgyCAC
[30]

https://arxiv.org/abs/math/0305207

Calcaterra, C., Boldt, A.: Lipschitz Flow-box Theorem (2006). https://arxiv.org/abs/math/0305207

2006 arXiv
[31]

Khalil, H.K.: Lyapunov Stability, 3rd edn., pp. 112–140. Prentice H all, ??? (2002)

2002
[32]

YouTube, NeurIPS 2020 Workshop on Diﬀerentiable Programming (2 020)

Ruthotto, L.: A Numerical Analysis Perspective on Deep Neural Networks. YouTube, NeurIPS 2020 Workshop on Diﬀerentiable Programming (2 020). https://www.youtube.com/watch?v=xL2KZZMrPwA

2020
[33]

arXiv preprint arXiv:1412.6980 (2014) arXiv:1412.6980 [cs.LG]

Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) arXiv:1412.6980 [cs.LG]

2014 arXiv
[34]

In: Pattern Recognition and Computer Vision: Se c- ond Chinese Conference, PRCV 2019, Xi’an, China, November 8-11, 2019, Proceedings, Part I, pp

He, L., Xie, X., Lin, Z.: Neural ordinary diﬀerential equations with envo- lutionary weights. In: Pattern Recognition and Computer Vision: Se c- ond Chinese Conference, PRCV 2019, Xi’an, China, November 8-11, 2019, Proceedings, Part I, pp. 598–610. Springer, Berlin, Heidelberg...

2019 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

In: Summer School of the German Research School for Simula tion Sciences (2019)

Sudret, B.: Surrogate models for uncertainty quantiﬁcation an d design optimiza- tion. In: Summer School of the German Research School for Simula tion Sciences (2019). https://doi.org/10.3929/ethz-b-000359599

work page doi:10.3929/ethz-b-000359599 2019

[2] [2]

: Tgcnn: An eﬃcient surrogate for real-time data assimila- tion in subsurface ﬂow

Kovachki, N.B., et al. : Tgcnn: An eﬃcient surrogate for real-time data assimila- tion in subsurface ﬂow. Computers & Mathematics with Applications 81, 336–354 (2021) https://doi.org/10.1016/j.camwa.2020.12.019

work page doi:10.1016/j.camwa.2020.12.019 2021

[3] [3]

The Innovat ion Energy 2(2), 100087–1 (2025)

Wang, N., Chen, Y., Zhang, D.: A comprehensive review of physics- informed deep learning and its applications in geoenergy development. The Innovat ion Energy 2(2), 100087–1 (2025)

work page 2025

[4] [4]

Journal of Computationa l physics 378, 686–707 (2019)

Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neur al networks: 31 A deep learning framework for solving forward and inverse problems involving nonlinear partial diﬀerential equations. Journal of Computationa l physics 378, 686–707 (2019)

work page 2019

[5] [5]

Nature machine intelligence 3(3), 218–229 (2021)

Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E.: Learning nonlin ear oper- ators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence 3(3), 218–229 (2021)

work page 2021

[6] [6]

arXiv (2018)

Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duve- naud, D.: Neural Ordinary Diﬀerential Equations. arXiv (2018). https://doi.org/10.48550/ARXIV.1806.07366 . https://arxiv.org/abs/1806.07366

arXiv doi:10.48550/arxiv.1806.07366 2018

[7] [7]

Ruthotto, L.: Diﬀerential Equations for Continuous-Time Deep L earning (2024)

work page 2024

[8] [8]

Spline parameterization of neural network controls for deep learning

G¨ unther, S., Pazner, W., Qi, D.: Spline parameterization of neura l network con- trols for deep learning. arXiv preprint arXiv:2103.00301 abs/2103.00301 (2021) 2103.00301

work page Pith review arXiv 2021