Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

A Quasilinear Algorithm for Computing Higher-Order Derivatives of Deep Feed-Forward Neural Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that n-TangentProp computes exact high-order input derivatives of dense feed-forward networks in quasilinear time, reducing the exponential cost it attributes to autodifferentiation and making derivative-heavy PINN…

desk verdict A correct but well-known recurrence is marketed as a novel algorithm against a strawman baseline; the PINN engineering results are real, but the complexity claims do not survive contact with Taylor-mode AD. read the letter →

arxiv 2412.09752 v1 pith:BJLQXN6P submitted 2024-12-12 cs.LG stat.ML

Kyle R. Chickering This is my paper · ORCID
classification cs.LGstat.ML
keywords higher-orderderivativesautomaticdifferentiationFaàdiBruno'sformulaphysics-informedneuralnetworksfeed-forwardquasilinearalgorithmTangentProppartitionfunction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the exponential cost of computing high-order derivatives during physics-informed neural network (PINN) training is not inherent. It proposes n-TangentProp, an extension of the TangentProp formalism, which computes the exact derivative $d^n/dx^n f(x)$ of a dense feed-forward network with a smooth, parameter-free activation in a single forward pass. The claimed runtime is quasilinear, $O(e^{\sqrt{n}}M)$, against $O(e^{\sqrt{n}}/n\,M^n)$ for autodifferentiation, with memory reduced from exponential to linear. If the claim holds, derivative-heavy PINN losses—Sobolev training, high-order Burgers profiles, and similar PDE residual terms—become substantially cheaper to evaluate.

What carries the argument

The load-bearing object is Faà di Bruno's formula, the chain rule generalized to $n$-th derivatives of a composition. The paper inserts it into the TangentProp forward pass: instead of differentiating the network by building and traversing a computational graph, each activation layer directly produces all derivatives $(\xi^{(m)})$ of its input, and the weight layers combine them linearly. The sum over integer partitions of $n$ with Bell-polynomial coefficients $C_p$ is what couples lower-order derivatives into the $n$-th one, and the derivatives must be computed in increasing order because $\xi^{(n)}$ depends on all $\xi^{(k)}$ with $k<n$. The partition function $p(n)$ therefore sets the combinatorial size of each activation step, and its asymptotic behavior is exactly what turns the recurrence into a quasilinear algorithm.

What would settle it

Run a Taylor-mode automatic differentiation implementation that propagates truncated Taylor series layer by layer on the same dense feed-forward benchmark and compare its runtime for $d^n/dx^n f(x)$ with n-TangentProp across $n$ and $M$; a polynomial-in-$(M,n)$ runtime comparable to or faster than n-TangentProp's would falsify the paper's claim of an exponential-to-quasilinear improvement over autodifferentiation.

Watch

Extended reading notes

Core claim

The paper's central claim, stated on its own terms, is that applying Faà di Bruno's formula layer by layer converts the forward pass into a complete derivative computation. For each layer, the activation's $n$-th derivative is obtained by summing over all integer partitions of $n$: $(\xi^{(n)})^{\ell-1}_j = \sum_p C_p \sigma^{(|p|)}(a^\ell_j) \prod_{m=1}^n (\xi^{(m)})^\ell_j{}^{p_m}$, then the linear weights propagate it by $(\gamma^{(n)})^\ell_i = \sum_j w^\ell_{ij}(\xi^{(n)})^{\ell-1}_j$. Since the number of partition terms is the partition number $p(n)$, and the classical bound $p(n)=O(e^{\sqrt{n}}/n)$ holds, the total cost for a network with $M$ parameters is $O(n p(n) M)=O(e^{\sqrt{n}}M)$. The authors emphasize that the method is exact, computes only the input derivatives actually needed, and that autodifferentiation instead implicitly pays an $O(e^{\sqrt{n}}/n\,M^n)$ cost because it builds all partial derivatives.

Load-bearing premise

The speedup claim rests on the assumption that autodifferentiation must compute and store all partial derivatives with respect to every input and parameter, making $O(M^n)$ time and memory unavoidable; if a polynomial-time method such as Taylor-mode automatic differentiation already computes the same input derivatives, the exponential baseline being beaten is not the true state of the art.

Editorial extensions

If this is right

  • Physics-informed training with Sobolev losses can use higher derivative orders, such as $m=4$ or more, at practical cost instead of stopping at one or two extra derivatives.
  • The third and fourth self-similar smooth Burgers profiles become computable on a single GPU; the paper reports the third profile in under an hour and projects the autodifferentiation route to exceed 25 hours.
  • Memory use drops from exponential, $O(M^n)$, to linear, $O(nM)$, so more derivatives fit on the same hardware than autodifferentiation can store.
  • Because the method is exact, it can replace autodifferentiation for derivative computation inside a PINN training loop without changing the loss definition or the solution accuracy.
  • For optimizers such as L-BFGS, which evaluate many forward passes during line search, the cheap forward pass compounds into large end-to-end training speedups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the asymptotic comparison is against reverse-mode computational-graph autodifferentiation as implemented in common deep learning libraries; Taylor-mode automatic differentiation, which propagates truncated Taylor series layer by layer, also computes input derivatives without materializing all $O(M^n)$ partials, so a direct benchmark against Taylor-mode AD would likely shrink the headlin
  • Inference: the recurrence is stated for a scalar input and parameter-free activations, but the same Faà di Bruno summation should extend to mixed partial derivatives and multi-dimensional inputs, which would broaden the method to general PDE residuals.
  • Inference: the practical crossover point depends on implementation constants; the reported advantage already appears around three derivatives for a small PINN architecture, and a compiled or fully vectorized implementation would likely make the method win at lower derivative orders and larger batch sizes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes n-TangentProp, an extension of the TangentProp formalism that computes the n-th derivative of a dense feed-forward neural network with respect to its input in a single forward pass. The derivation applies Faà di Bruno's formula layer-by-layer, leading to the recurrence in Eq. (5b) and Algorithm 1, with a claimed runtime of O(e^{sqrt(n)} M) and memory O(nM). The authors contrast this with an alleged O(e^{sqrt(n)}/n M^n) cost for autodifferentiation, and they present empirical scaling experiments and an end-to-end PINN application to unstable self-similar Burgers profiles, with code available on GitHub.

Significance. If the quasilinear claim were correct, the algorithm would be a significant contribution to physics-informed neural networks, where high-order input derivatives are a bottleneck. The Faà di Bruno-based recurrence is exact and self-contained, and the released code is a concrete engineering artifact. The empirical forward-pass timings for moderate n are consistent with the paper's own complexity formula when compared against PyTorch reverse-mode autodifferentiation. However, the central theoretical significance is undermined by the fact that Taylor-mode forward automatic differentiation already computes input-only derivatives in polynomial time per layer; against that state of the art, the proposed algorithm is asymptotically slower, not faster. The paper's contribution is therefore best understood as an empirical acceleration over a specific implementation (PyTorch reverse-mode), not as a quasilinear alternative to automatic differentiation in general.

major comments (4)
  1. [Abstract; Section III-A] The claimed exponential baseline for autodifferentiation is not the relevant baseline for input-only derivatives. Taylor-mode forward automatic differentiation (univariate jets) propagates a truncated Taylor series through each elementary operation and computes d^n f/dx^n in O(n^2 M) time for a network with M parameters, without enumerating all partial derivatives. The paper's assertion that repeated autodifferentiation necessarily costs O(M^n) because there are O(M^n) n-th order partial derivatives conflates the number of possible derivatives with the cost of computing one specific input derivative. Against this true polynomial baseline, the proposed O(e^{O(sqrt(n))} M) runtime is asymptotically slower, so the central 'quasilinear instead of exponential' claim is unsupported and likely false. The authors should either compare against Taylor-mode AD explicitly or substantially reframe the contribution as an engineering acceleration over a particular reverse-mode implementation.
  2. [Section III-B, Eq. (5b)] The displayed recurrence is not index-consistent. The left-hand side is (xi^(n))^{ell-1}_j, while the right-hand side evaluates sigma and xi at layer ell (sigma^{(|p|)}(a^ell_j) and (xi^(m))^ell_j). Following Eq. (3b), the correct recurrence should use the layer-(ell-1) quantities, namely sigma^{(|p|)}(a^{ell-1}_j) and the derivatives of a^{ell-1}_j, not those of a^ell_j. As printed, the equations do not directly imply the update steps in Algorithm 1; the indexing convention must be corrected or explicitly defined before the derivation can be verified.
  3. [Section III-B, Hardy-Ramanujan bound] The Hardy-Ramanujan asymptotic is misstated. The correct classical bound is p(n) = O(exp(pi sqrt(2n/3))/n), which gives n p(n) = O(exp(2.565 sqrt(n))), not O(exp(sqrt(n))). Consequently the refined runtime O(np(n)M) is O(exp(2.565 sqrt(n)) M), and the term 'quasilinear' is inaccurate for a superpolynomial (though subexponential) function. Even with the corrected constant, the runtime is asymptotically worse than any polynomial in n, which further undermines the comparison with Taylor-mode AD.
  4. [Section IV-B and IV-C] The empirical evaluation compares the proposed method only against PyTorch reverse-mode autodifferentiation. Since the theoretical baseline is invalid, these experiments do not establish an improvement over the state of the art in higher-order AD. The observed speedups for n >= 3 may still support a narrower engineering claim, but the paper should clearly state that the comparison is against a specific implementation and should not imply that the algorithm outperforms all existing exact higher-order differentiation methods.
minor comments (4)
  1. [Section I, contributions] The phrase 'for it's implementation' should be 'for its implementation'.
  2. [Throughout] There are several typographical issues, including 'classifictation' in Section III-B and the broken formatting of 'Fa `a di Bruno' in the same section.
  3. [Section IV-C1 and Appendix A] The main text states that the fourth profile was 'able to compute' and reports timings, while Appendix A says the method 'failed to adequately converge' to the fourth profile. This contradiction should be resolved and clearly reported.
  4. [Algorithm 1] The variable names y, a, and the use of 'BELL' are only loosely described in the text; a short explanation of the data structures and of how the Bell coefficients are precomputed would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: n-TangentProp is a direct Faà di Bruno forward-pass derivation; the exponential-autodiff baseline is a correctness concern, not a circular step.

full rationale

The derivation chain is self-contained. Section III-B starts from Faà di Bruno's formula (Eq. 4) and applies it to the layer activations, producing the recurrence (5a)–(5b); the claimed runtime O(n p(n) M) follows by counting the partition sum and the number of layers. No fitted parameter, target derivative value, or external benchmark is used to construct the formula, so the central claim is not circular. The paper's only self-citation ([7], Chickering et al.) appears as related work on self-similar Burgers perturbations and is not load-bearing. The genuinely weak step is Section III-A's assertion that autodifferentiation costs O(M^n) because there are O(M^n) n-th order partial derivatives; this conflates the number of possible partials with the cost of computing one input derivative and is contradicted by Taylor-mode forward AD, which computes input derivatives in polynomial time. That is a correctness/novelty risk, not circularity: the n-TangentProp recurrence does not assume the exponential baseline. The appendix's admitted limitations (not reproducing [40], not converging the fourth profile) are honest reports rather than circular reasoning. The Hardy–Ramanujan bound is also misstated as O(e^{√ n}/n) without the constant π√(2/3), but this is a mathematical error, not a circular step. Accordingly, no circular step can be exhibited and the score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The algorithm itself has no fitted parameters and introduces no new physical or mathematical entities. The main assumptions are standard mathematical results (Faà di Bruno, Hardy-Ramanujan) and a baseline complexity assumption about autodifferentiation that is not independently verified.

assumptions (4)
  • standard math Faà di Bruno's formula applies to the composition of the network's smooth activation and linear layers
    Used in Eq. (5b); it is a classical theorem, and the paper assumes the activation (tanh) is C^n.
  • domain assumption Automatic differentiation for the n-th input derivative has runtime O(M^n) and memory O(M^n)
    Stated in Sections II and III-A; this is the baseline the paper's improvement is measured against. It is load-bearing for the significance claim but not for the correctness of the algorithm.
  • standard math Hardy-Ramanujan asymptotic p(n) = O(e^{sqrt(n)}/n)
    Used in Section III-B to derive the claimed quasilinear bound.
  • domain assumption The partition-sum product in Eq. (5b) costs O(|p|) and the Bell tables are precomputed
    The stated complexity O(n p(n) M) assumes the product over partitions is computed with cost proportional to the number of parts; a naive implementation may be slower.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Quasilinear Algorithm for Computing Higher-Order Derivatives of Deep Feed-Forward Neural Networks." pith.science (2026). https://pith.science/paper/BJLQXN6P

@misc{pith2026241209752,
  author       = {Pith},
  title        = {Pith review of: A Quasilinear Algorithm for Computing Higher-Order Derivatives of Deep Feed-Forward Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BJLQXN6P}},
  note         = {Machine review of arXiv:2412.09752}
}
abstract

The use of neural networks for solving differential equations is practically difficult due to the exponentially increasing runtime of autodifferentiation when computing high-order derivatives. We propose $n$-TangentProp, the natural extension of the TangentProp formalism \cite{simard1991tangent} to arbitrarily many derivatives. $n$-TangentProp computes the exact derivative $d^n/dx^n f(x)$ in quasilinear, instead of exponential time, for a densely connected, feed-forward neural network $f$ with a smooth, parameter-free activation function. We validate our algorithm empirically across a range of depths, widths, and number of derivatives. We demonstrate that our method is particularly beneficial in the context of physics-informed neural networks where \ntp allows for significantly faster training times than previous methods and has favorable scaling with respect to both model size and loss-function complexity as measured by the number of required derivatives. The code for this paper can be found at https://github.com/kyrochi/n\_tangentprop.

Figures

Figures reproduced from arXiv: 2412.09752 by the authors.

Figure 1
Figure 1. Average runtime for a combined forward and backwards pass using [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. Backwards pass times for the model shown in Figure 1. The top [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. The ratio of forward pass run times between autodifferentiation and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Results from training a PINN to find the first smooth profile for [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Results from training a PINN to solve (7) with [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Results from training a PINN to solve (7) with [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Results from training a PINN to solve (7) with [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Results from training a PINN to solve (7) with [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SplineNet: An Isogeometric Deep Learning Method for Complex Shells

    cs.LG 2026-07 conditional novelty 6.0 of 10

    SplineNet constructs a neural network whose architecture exactly reproduces isogeometric spline basis functions via Bézier extraction, enabling both data-free PDE solving and operator learning for complex shell structures.

Reference graph

Works this paper leans on

42 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    A NTONION , X

    K. A NTONION , X. WANG , M. R AISSI , AND L. J OSHIE , Machine learn- ing through physics–informed neural networks: Progress and challenges, Academic Journal of Science and Technology, 9 (2024), pp. 46–49

    work page 2024

  2. [2]

    B ARENBLATT AND Y

    G. B ARENBLATT AND Y. B. Z EL’D OVICH , Self-similar solutions as intermediate asymptotics, Annual Review of Fluid Mechanics, 4 (1972), pp. 285–312

    work page 1972

  3. [3]

    A. G. B AYDIN , B. A. P EARLMUTTER , A. A. R ADUL , AND J. M. SISKIND , Automatic differentiation in machine learning: a survey , Journal of machine learning research, 18 (2018), pp. 1–43

    work page 2018

  4. [4]

    B IASI , Self-similar solutions to the compressible euler equations and their instabilities, Communications in Nonlinear Science and Numerical Simulation, 103 (2021), p

    A. B IASI , Self-similar solutions to the compressible euler equations and their instabilities, Communications in Nonlinear Science and Numerical Simulation, 103 (2021), p. 106014

    work page 2021

  5. [5]

    B ISCHOF AND M

    R. B ISCHOF AND M. K RAUS , Multi-objective loss balancing for physics-informed deep learning , arXiv preprint arXiv:2110.09813, (2021)

    arXiv 2021

  6. [6]

    R. A. B RUALDI , Introductory Combinatorics, Pearson, 5 ed., 2010

    work page 2010

  7. [7]

    K. R. C HICKERING , R. C. M ORENO -VASQUEZ , AND G. P ANDYA, Asymptotically self-similar shock formation for 1d fractal burgers’ equation, SIAM Journal on Mathematical Analysis, 55 (2023), pp. 7328– 7360

    work page 2023

  8. [8]

    C HIU , J

    P.-H. C HIU , J. C. W ONG , C. O OI, M. H. D AO, AND Y.-S. O NG, Can-pinn: A fast physics-informed neural network based on coupled- automatic–numerical differentiation method, Computer Methods in Ap- plied Mechanics and Engineering, 395 (2022), p. 114909

    work page 2022

Show all 42 references
  1. [9]

    C YBENKO , Approximation by superpositions of a sigmoidal function, Mathematics of control, signals and systems, 2 (1989), pp

    G. C YBENKO , Approximation by superpositions of a sigmoidal function, Mathematics of control, signals and systems, 2 (1989), pp. 303–314

    1989

  2. [10]

    W. M. C ZARNECKI , S. O SINDERO , M. J ADERBERG , G. S WIRSZCZ , AND R. P ASCANU , Sobolev training for neural networks , Advances in neural information processing systems, 30 (2017)

    2017

  3. [11]

    D E RYCK , F

    T. D E RYCK , F. B ONNET , S. M ISHRA , AND E. DE B ´EZENAC , An operator preconditioning perspective on training in physics-informed machine learning, arXiv preprint arXiv:2310.05801, (2023)

    2023 arXiv

  4. [12]

    E GGERS AND M

    J. E GGERS AND M. A. F ONTELOS , Singularities: formation, structure, and propagation, vol. 53, Cambridge University Press, 2015

    2015

  5. [13]

    L. C. E VANS, Partial differential equations, vol. 19, American Mathe- matical Society, 2022

    2022

  6. [14]

    F AREA , O

    A. F AREA , O. Y LI-HARJA , AND F. E MMERT-STREIB , Understanding physics-informed neural networks: Techniques, applications, trends, and challenges, AI, 5 (2024), pp. 1534–1557

    2024

  7. [15]

    H ABERMAN , Elementary applied partial differential equations: with Fourier series and boundary value problems , Prentice-Hall Englewood Cliffs, NJ, 1987

    R. H ABERMAN , Elementary applied partial differential equations: with Fourier series and boundary value problems , Prentice-Hall Englewood Cliffs, NJ, 1987

    1987

  8. [16]

    G. H. H ARDY AND S. R AMANUJAN , Asymptotic formulae for the distribution of integers of various types , Proceedings of the London Mathematical Society, 2 (1917), pp. 112–132

    1917

  9. [17]

    H ORNIK , Approximation capabilities of multilayer feedforward net- works, Neural networks, 4 (1991), pp

    K. H ORNIK , Approximation capabilities of multilayer feedforward net- works, Neural networks, 4 (1991), pp. 251–257

    1991

  10. [18]

    J AHANI -NASAB AND M

    M. J AHANI -NASAB AND M. A. B IJARCHI , Enhancing convergence speed with feature enforcing physics-informed neural networks using boundary conditions as prior knowledge , Scientific Reports, 14 (2024), p. 23836

    2024

  11. [19]

    K RISHNAPRIYAN , A

    A. K RISHNAPRIYAN , A. G HOLAMI , S. Z HE, R. K IRBY, AND M. W. MAHONEY , Characterizing possible failure modes in physics-informed neural networks , Advances in neural information processing systems, 34 (2021), pp. 26548–26560

    2021

  12. [20]

    M ADDU , D

    S. M ADDU , D. S TURM , C. L. M ¨ULLER , AND I. F. S BALZARINI , Inverse dirichlet weighting enables reliable training of physics informed neural networks, Machine Learning: Science and Technology, 3 (2022), p. 015026

    2022

  13. [21]

    M ARKIDIS , The old and the new: Can physics-informed deep- learning replace traditional linear solvers? , Frontiers in big Data, 4 (2021), p

    S. M ARKIDIS , The old and the new: Can physics-informed deep- learning replace traditional linear solvers? , Frontiers in big Data, 4 (2021), p. 669097

    2021

  14. [22]

    M CGREIVY AND A

    N. M CGREIVY AND A. H AKIM , Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations, Nature Machine Intelligence, (2024), pp. 1–14

    2024

  15. [23]

    M. A. N ABIAN , R. J. G LADSTONE , AND H. M EIDANI , Efficient training of physics-informed neural networks via importance sam- pling, Computer-Aided Civil and Infrastructure Engineering, 36 (2021), pp. 962–977

    2021

  16. [24]

    N OCEDAL AND S

    J. N OCEDAL AND S. J. W RIGHT , Numerical optimization , Springer, 1999

    1999

  17. [25]

    O H AND F

    S.-J. O H AND F. PASQUALOTTO , Gradient blow-up for dispersive and dissipative perturbations of the burgers equation , Archive for Rational Mechanics and Analysis, 248 (2024), p. 54

    2024

  18. [26]

    P ASZKE , S

    A. P ASZKE , S. G ROSS , F. M ASSA , A. L ERER , J. B RADBURY , G. C HANAN , T. K ILLEEN , Z. L IN, N. G IMELSHEIN , L. A NTIGA , ET AL ., Pytorch: An imperative style, high-performance deep learning library, Advances in neural information processing systems, 32 (2019)

    2019

  19. [27]

    P ENWARDEN , A

    M. P ENWARDEN , A. D. J AGTAP, S. Z HE, G. E. K ARNIADAKIS , AND R. M. K IRBY , A unified scalable framework for causal sweeping strategies for physics-informed neural networks (pinns) and their tem- poral decompositions , Journal of Computational Physics, 493 (2023), p. 112464

    2023

  20. [28]

    R AHAMAN , A

    N. R AHAMAN , A. B ARATIN , D. A RPIT , F. D RAXLER , M. L IN, F. HAMPRECHT , Y. BENGIO , AND A. C OURVILLE , On the spectral bias of neural networks , in International conference on machine learning, PMLR, 2019, pp. 5301–5310

    2019

  21. [29]

    R AISSI , P

    M. R AISSI , P. P ERDIKARIS , AND G. E. K ARNIADAKIS , Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational physics, 378 (2019), pp. 686–707

    2019

  22. [30]

    R ATHORE , W

    P. R ATHORE , W. L EI, Z. F RANGELLA , L. L U, AND M. U DELL , Chal- lenges in training pinns: A loss landscape perspective , arXiv preprint arXiv:2402.01868, (2024)

    2024 arXiv

  23. [31]

    R OMAN , The formula of faa di bruno , The American Mathematical Monthly, 87 (1980), pp

    S. R OMAN , The formula of faa di bruno , The American Mathematical Monthly, 87 (1980), pp. 805–809

    1980

  24. [32]

    S HARMA AND V

    R. S HARMA AND V. S HANKAR , Accelerated training of physics- informed neural networks (pinns) using meshless discretizations , Ad- vances in Neural Information Processing Systems, 35 (2022), pp. 1034– 1046

    2022

  25. [33]

    H.-J. M. S HI AND D. M UDIGERE , Pytorch l-bfgs. https://github.com/ hjmshi/PyTorch-LBFGS, 2020

    2020

  26. [34]

    S IMARD , B

    P. S IMARD , B. V ICTORRI , Y. LECUN, AND J. D ENKER , Tangent prop- a formalism for specifying selected invariances in an adaptive network , Advances in neural information processing systems, 4 (1991)

    1991

  27. [35]

    H. S ON, J. W. J ANG , W. J. H AN, AND H. J. H WANG, Sobolev training for physics informed neural networks , arXiv preprint arXiv:2101.08932, (2021)

    2021 arXiv

  28. [36]

    W ANG , Y

    H. W ANG , Y. C AO, Z. H UANG , Y. L IU, P. H U, X. L UO, Z. S ONG , W. Z HAO, J. L IU, J. S UN, ET AL ., Recent advances on machine learning for computational fluid dynamics: A survey , arXiv preprint arXiv:2408.12171, (2024). 9

    2024 arXiv

  29. [37]

    W ANG , S

    S. W ANG , S. S ANKARAN , H. WANG , AND P. PERDIKARIS , An expert’s guide to training physics-informed neural networks , arXiv preprint arXiv:2308.08468, (2023)

    2023 arXiv

  30. [38]

    W ANG , Y

    S. W ANG , Y. T ENG , AND P. P ERDIKARIS , Understanding and miti- gating gradient pathologies in physics-informed neural networks , arXiv preprint arXiv:2001.04536, (2020)

    2020 arXiv

  31. [39]

    W ANG , X

    S. W ANG , X. Y U, AND P. P ERDIKARIS , When and why pinns fail to train: A neural tangent kernel perspective , Journal of Computational Physics, 449 (2022), p. 110768

    2022

  32. [40]

    W ANG , C.-Y

    Y. W ANG , C.-Y. L AI, J. G ´OMEZ -SERRANO , AND T. B UCKMASTER , Asymptotic self-similar blow-up profile for three-dimensional axisym- metric euler equations using neural networks , Physical Review Letters, 130 (2023), p. 244002

    2023

  33. [41]

    Z.-Q. J. X U, Y. Z HANG , T. L UO, Y. X IAO, AND Z. M A, Frequency principle: Fourier analysis sheds light on deep neural networks , arXiv preprint arXiv:1901.06523, (2019)

    2019 arXiv

  34. [42]

    Y ANG AND J

    Y. Y ANG AND J. H E, Deeper or wider: A perspective from optimal generalization error with sobolev loss, arXiv preprint arXiv:2402.00152, (2024). APPENDIX A. Additional Details for the Self-Similar Burgers Experiments We report the results from running the self-similar Burgers...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.