pith. sign in

arxiv: 2606.12735 · v1 · pith:XSZALDYJnew · submitted 2026-06-10 · 💻 cs.LG

Physics-Informed Neural Networks and Radial Basis Functions for PDEs with Dirac Delta Sources

Pith reviewed 2026-06-27 09:44 UTC · model grok-4.3

classification 💻 cs.LG
keywords Physics-Informed Neural NetworksRadial Basis FunctionsDirac DeltaWeak FormResidual Least SquaresGroundwater TransportInverse ProblemsPartial Differential Equations
0
0 comments X

The pith

Weak-form integration of Dirac deltas lets RBF-RLS solve PDEs accurately while PINN residuals fail to reach zero.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that Physics-Informed Neural Networks cannot drive residuals to zero on PDEs containing Dirac delta sources even after weak-form integration, whereas Radial Basis Function Residual Least Squares consistently yields accurate forward and inverse solutions. The difference is demonstrated on linear transport equations for groundwater flow and rivers using synthetic data, noisy data, and real measurements. Neural Tangent Kernel theory is invoked to account for the contrasting convergence behavior between the two Residual Least Squares formulations. A reader would care because singular sources appear in many physical models, and reliable machine-learning solvers for such cases would reduce the need for ad-hoc smoothing approximations.

Core claim

The paper claims that integrating the weak-form equation allows direct treatment of Dirac delta terms without smooth surrogates; under this treatment RBF-RLS produces good forward and inverse solutions to transport problems while PINNs do not make residuals converge to zero, with the distinction explained by Neural Tangent Kernel theory.

What carries the argument

Weak-form integration of the Dirac delta inside the Residual Least Squares loss, applied once to PINNs and once to an RBF expansion.

If this is right

  • RBF-RLS can be applied directly to inverse problems that fit real-world measurements without introducing modeling error from delta approximations.
  • The same weak-form treatment extends to other linear PDEs that represent flow and transport in porous media.
  • NTK analysis supplies a diagnostic tool for choosing among different Residual Least Squares architectures when singular sources are present.
  • Forward solutions remain stable under both synthetic and noisy data once the delta is integrated exactly.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar weak-form handling might improve other neural or kernel methods on problems with point sources or discontinuities.
  • The observed gap suggests testing whether hybrid RBF-neural architectures inherit the convergence advantages of pure RBF-RLS.
  • The result raises the question of whether the same integration step improves performance on nonlinear transport equations.

Load-bearing premise

Neural Tangent Kernel theory correctly predicts why residuals converge under RBF-RLS but not under PINNs after the same weak-form integration of the delta.

What would settle it

Run both PINN and RBF-RLS on the same one-dimensional linear transport equation with a single Dirac delta source, integrate the weak form once, and check whether the PINN residual norm actually reaches machine zero or remains bounded away from zero.

Figures

Figures reproduced from arXiv: 2606.12735 by Alexandre Tartakovsky, Manuel Reyna.

Figure 1
Figure 1. Figure 1: RBF–RLS forward solution of the ADE with an instantaneous point release. (a) Solution [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of RBF–RLS and PINN forward and inverse solutions of the ADE with a step [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: RBF–RLS inverse solution of the ADE with first order exchange to an immobile zone fitted [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: PINN results for the DE with point source forcing. a) Evolution of the losses in PINN. [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
read the original abstract

Physics-Informed Neural Networks (PINNs) are a machine learning method for solving forward and inverse Partial Differential Equations (PDEs). When applied to PDEs with Dirac delta functions in the forcing terms, boundary conditions, or initial conditions, PINNs require approximating them with smooth surrogate functions, a practice that can introduce significant modeling errors. In this work, we exploit the interpretation of PINNs as Residual Least Squares (RLS) methods and show that this perspective enables direct treatment of Dirac delta terms by integrating the weak-form equation. Among RLS formulations other than PINN, we focus on the Radial Basis Function (RBF) expansion (also known as a single-layer RBF Network). We show that while integrating out the Dirac delta in PINNs causes residuals to fail to converge to zero, RBF-RLS consistently provides good forward and inverse solutions to transport problems. We explain this finding using the Neural Tangent Kernel (NTK) theory. We test both approaches on linear PDEs that represent groundwater flow and transport in porous media and rivers. We solve inverse problems to fit synthetic data, noisy synthetic data, and real-world measurements.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that standard PINNs, when Dirac delta sources are handled by integrating the weak-form residual, fail to drive the training loss (residual) to zero, whereas RBF expansions within the same RLS framework succeed at both forward and inverse solutions for linear transport PDEs; the distinction is attributed to properties of the Neural Tangent Kernel, with numerical demonstrations on groundwater flow and river transport models using synthetic, noisy, and real measurements.

Significance. If the empirical distinction and its NTK grounding hold after verification, the work would identify a concrete regime where deep PINN residuals stagnate on singular sources while shallow RBF-RLS remains effective, offering guidance for method selection in applications with point sources. The inclusion of inverse problems on real data strengthens applicability; explicit spectral analysis of the NTK (if added) would constitute a useful theoretical contribution.

major comments (2)
  1. [§4] §4 (NTK explanation): the assertion that NTK theory accounts for PINN residual non-convergence versus RBF-RLS success after weak-form delta integration is not supported by explicit computation of the relevant NTK eigenvalues or gradient-flow spectrum for the integrated loss operator; without this, the explanation cannot be distinguished from confounding factors such as network depth (deep vs. single-layer) or optimizer behavior.
  2. [Results] Results section (convergence plots and tables): the central claim that PINN residuals fail to reach zero while RBF-RLS succeeds requires demonstration that identical weak-form integration, quadrature, and collocation strategy are applied to both methods; any post-hoc differences in these implementation choices would undermine the attribution to the RLS formulation class.
minor comments (2)
  1. [§2] §2 (weak-form definition): clarify the precise quadrature rule used to integrate the Dirac delta term and confirm that the same rule is stated for both PINN and RBF-RLS.
  2. [Figures] Figure captions: add explicit labels for which curves correspond to PINN versus RBF-RLS and report the final residual norms numerically rather than only visually.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful comments, which help clarify the contributions and limitations of our work. We address the major comments point by point below.

read point-by-point responses
  1. Referee: [§4] §4 (NTK explanation): the assertion that NTK theory accounts for PINN residual non-convergence versus RBF-RLS success after weak-form delta integration is not supported by explicit computation of the relevant NTK eigenvalues or gradient-flow spectrum for the integrated loss operator; without this, the explanation cannot be distinguished from confounding factors such as network depth (deep vs. single-layer) or optimizer behavior.

    Authors: We acknowledge that our manuscript does not include explicit computation of NTK eigenvalues or the gradient-flow spectrum for the weak-form loss. The NTK explanation in §4 is based on established properties from the literature regarding the behavior of deep networks versus shallow linear models under least-squares losses. Specifically, deep PINNs can suffer from spectral bias and poor conditioning when the loss involves integrated singular terms, while the RBF expansion corresponds to a kernel method with more favorable properties for this setting. The comparison intentionally contrasts the standard deep PINN architecture with the single-layer RBF, as these represent typical choices in each RLS class. We will revise the text in §4 to emphasize that the NTK provides a qualitative rationale rather than a quantitative proof, and to explicitly discuss the role of network depth as part of the method distinction. No additional numerical NTK analysis will be added, as it would require substantial new theoretical work beyond the paper's scope. revision: partial

  2. Referee: [Results] Results section (convergence plots and tables): the central claim that PINN residuals fail to reach zero while RBF-RLS succeeds requires demonstration that identical weak-form integration, quadrature, and collocation strategy are applied to both methods; any post-hoc differences in these implementation choices would undermine the attribution to the RLS formulation class.

    Authors: The manuscript applies the same weak-form integration and RLS framework to both methods, as described in Sections 2 and 3. The quadrature and collocation points are chosen identically for the integrated residual in both cases, with the only difference being the parametrization of the solution (neural network weights versus RBF coefficients). To address this concern, we will add explicit statements and possibly a comparison table in the Results section confirming that the integration scheme, number of quadrature points, and collocation strategy are the same for PINN and RBF-RLS experiments. This will be supported by reference to the shared code repository if available. revision: yes

Circularity Check

0 steps flagged

No circularity in derivation or claims

full rationale

The paper's core claim rests on an empirical observation that weak-form integration of Dirac deltas causes PINN residuals to fail to converge while RBF-RLS succeeds, with NTK theory invoked as an external explanatory framework. No equations, parameters, or results are shown to reduce by construction to fitted inputs, self-definitions, or self-citation chains; the distinction is presented as a tested difference rather than a tautological renaming or forced prediction. The derivation chain is self-contained against external benchmarks and does not rely on load-bearing self-references for its validity.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the domain assumption that PINNs are equivalent to residual least-squares methods and that NTK theory applies to the observed residual behavior; no free parameters or new entities are introduced in the abstract.

axioms (2)
  • domain assumption PINNs can be interpreted as Residual Least Squares (RLS) methods
    Invoked to justify direct weak-form treatment of Dirac deltas without smoothing
  • domain assumption Neural Tangent Kernel (NTK) theory explains the difference in residual convergence between PINNs and RBF-RLS
    Used to account for why one method succeeds and the other fails after weak-form integration

pith-pipeline@v0.9.1-grok · 5730 in / 1411 out tokens · 23634 ms · 2026-06-27T09:44:36.910448+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 29 canonical work pages · 1 internal anchor

  1. [1]

    Raissi, P

    M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving non- linear partial differential equations, Journal of Computational Physics 378 (2019) 686–707

  2. [2]

    G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, L. Yang, Physics-informed machine learning, Nature Reviews Physics 3 (6) (2021) 422–440

  3. [3]

    Cuomo, V

    S. Cuomo, V. S. di Cola, F. Giampaolo, G. Rozza, M. Raissi, F. Piccialli, Scientific machine learning through physics-informed neural networks: Where we are and what’s next (2022). doi:10.48550/ARXIV.2201.05624

  4. [4]

    A. M. Tartakovsky, C. O. Marrero, P. Perdikaris, G. D. Tartakovsky, D. Barajas- Solano, Physics-informed deep neural networks for learning parameters and consti- tutive relationships in subsurface flow problems, Water Resources Research 56 (5) (2020) e2019WR026731

  5. [5]

    Q. He, A. M. Tartakovsky, Physics-informed neural network method for forward and backward advection-dispersion equations, Water Resources Research 57 (7) (2021) e2020WR029479. 22

  6. [6]

    X. Zhao, H. Liu, F. Leng, W. Yang, X. Yin, Y. Yi, Source trac- ing for pollutants in river channels based on a physics-informed neu- ral network, Water Resources Research 62 (2) (2026) e2025WR040846. doi:https://doi.org/10.1029/2025WR040846

  7. [7]

    Huang, H

    X. Huang, H. Liu, B. Shi, Z. Wang, K. Yang, Y. Li, B. Weng, M. Wang, H. Chu, J. Zhou, F. Yu, B. Hua, L. Chen, B. Dong, Solving partial differential equations with point source based on physics-informed neural networks (2021). URLarxiv.org/abs/2111.01394

  8. [8]

    W. E, B. Yu, The deep ritz method: A deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics 6 (1) (2018) 1–12. doi:10.1007/s40304-018-0127-z. URLarxiv.org/pdf/1710.00211

  9. [9]

    C. Wang, H. Guo, X. Yan, Z.-L. Shi, Y. Yang, Improved physics-informed neural networks for the reinterpreted discrete fracture model, Journal of Computational Physics 520 (2025) 113491. doi:10.1016/j.jcp.2024.113491

  10. [10]

    Z. Xu, Y. Yang, The hybrid dimensional representation of permeability ten- sor: A reinterpretation of the discrete fracture model and its extension on nonconforming meshes, Journal of Computational Physics 415 (2020) 109523. doi:10.1016/j.jcp.2020.109523

  11. [11]

    H. Park, G. Jo, A physics-informed neural network based method for the nonlin- ear poisson-boltzmann equation and its error analysis, Journal of Computational Physics 522 (2024) 113579–113579. doi:10.1016/j.jcp.2024.113579

  12. [12]

    M. D. Rosa, L. Pompameo, A. Litvinenko, S. Cuomo, Homo-pinn: Hyperparam- eter optimization of a multi-output physics-informed neural network, Operations Research Forum 6 (4) (Oct 2025). doi:10.1007/s43069-025-00561-7

  13. [13]

    Kapoor, H

    T. Kapoor, H. Wang, A. Núñez, R. Dollevoet, Physics-informed machine learning for moving load problems, Journal of Physics: Conference Series 2647 (15) (2024) 152003. doi:10.1088/1742-6596/2647/15/152003

  14. [14]

    A. Liu, J. Li, J. Bi, Z. Chen, Y. Wang, C. Lu, Y. Jin, B. Lin, A novel reservoir simulation model based on physics informed neural networks, Physics of Fluids 36 (11) (Nov. 2024). doi:10.1063/5.0239376

  15. [15]

    Suarez, G

    J.-P. Suarez, G. Jacobs, Regularization of singularities in the weighted summation of dirac-delta functions for the spectral solution of hyperbolic conservation laws? (2016). URLarxiv.org/abs/1611.05510

  16. [16]

    E. D. Eason, A review of least-squares methods for solving partial differential equa- tions, International Journal for Numerical Methods in Engineering 10 (5) (1976) 1021–1046. doi:10.1002/nme.1620100505. 23

  17. [17]

    Karniadakis

    E. Kharazmi, Z. Zhang, G. E. M. Karniadakis, hp-VPINNs: Varia- tional physics-informed neural networks with domain decomposition, Com- puter Methods in Applied Mechanics and Engineering 374 (2021) 113547. doi:10.1016/j.cma.2020.113547

  18. [18]

    Hanke, R

    M. Hanke, R. März, Convergence analysis of least-squares collocation methods for nonlinear higher-index differential–algebraic equations, Journal of Computational and Applied Mathematics 387 (2021) 112514

  19. [19]

    J. N. Reddy, An introduction to the finite element method, McGraw-Hill Inc, 1993

  20. [20]

    S. Wang, X. Yu, P. Perdikaris, When and why pinns fail to train: A neural tan- gent kernel perspective, Journal of Computational Physics 449 (2022) 110768. doi:https://doi.org/10.1016/j.jcp.2021.110768

  21. [21]

    D. Lowe, D. Broomhead, Multivariable functional interpolation and adaptive net- works, Complex systems 2 (3) (1988) 321–355

  22. [22]

    Schwenker, H

    F. Schwenker, H. A. Kestler, G. Palm, Three learning phases for radial-basis- function networks, Neural Networks 14 (4) (2001) 439–458. doi:10.1016/S0893- 6080(01)00027-2

  23. [23]

    Ghosh, A

    J. Ghosh, A. Nag, An overview of radial basis function networks, in: R. J. Howlett, L. C. Jain (Eds.), Radial Basis Function Networks 2: New Advances in Design, Physica-Verlag, Heidelberg, 2001, pp. 1–36

  24. [24]

    E. J. Kansa, Multiquadrics—A scattered data approximation scheme with ap- plications to computational fluid-dynamics—I surface approximations and partial derivative estimates, Computers & Mathematics with Applications 19 (8) (1990) 127–145

  25. [25]

    E. J. Kansa, Multiquadrics—A scattered data approximation scheme with applica- tions to computational fluid-dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations, Computers & Mathematics with Applications 19 (8) (1990) 147–161

  26. [26]

    T.-O. Kwok, L. Ling, On Convergence of a Least-Squares Kansa’s Method for the Modified Helmholtz Equations, Adv. Appl. Math. Mech. (2009)

  27. [27]

    M. M. Alqezweeni, V. I. Gorbachenko, M. V. Zhukov, M. S. Jaafar, Efficient solving of boundary value problems using radial basis function networks learned by trust region method, International Journal of Mathematics and Mathematical Sciences 2018 (1) (2018) 9457578. doi:10.1155/2018/9457578

  28. [28]

    Bai, G.-R

    J. Bai, G.-R. Liu, A. Gupta, L. Alzubaidi, X.-Q. Feng, Y. Gu, Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear partial differential equations, Computer Methods in Applied Mechanics and Engineering 415 (2023) 116290. doi:10.1016/j.cma.2023.116290. 24

  29. [29]

    V. I. Gorbachenko, M. M. Alqezweeni, M. S. Jaafar, V. Z. Orbachenko, Application of parametric identification method and radial basis function networks for solution of inverse boundary value problems, in: 2017 Annual Conference on New Trends in Information & Communications Technology Applications (NTICT), 2017, pp. 18–21. doi:10.1109/NTICT.2017.7976151

  30. [30]

    V. I. Gorbachenko, M. V. Zhukov, Solving boundary value problems of mathemat- ical physics using radial basis function networks, Computational Mathematics and Mathematical Physics 57 (1) (2017) 145–155. doi:10.1134/S0965542517010079

  31. [31]

    V. I. Gorbachenko, D. A. Stenkin, Physics-Informed Radial Basis-Function Net- works, Technical Physics (10 2023). doi:10.1134/S1063784223050018

  32. [32]

    B. A. Finlayson, Introduction, Society for Industrial and Applied Mathematics, 2014, Ch. 1, pp. 3–14. doi:10.1137/1.9781611973242.ch1

  33. [33]

    Nguyen, J

    H. Nguyen, J. Reynen, A space-time least-square finite element scheme for advection-diffusion equations, Computer Methods in Applied Mechanics and En- gineering 42 (3) (1984) 331–342. doi:10.1016/0045-7825(84)90012-4

  34. [34]

    Nguyen, J

    H. Nguyen, J. Reynen, A space-time least-square finite element scheme for advection-diffusion equations, Computer Methods in Applied Mechanics and En- gineering 42 (3) (1984) 331–342

  35. [35]

    Tartakovsky, D

    A. Tartakovsky, D. Barajas-Solano, Q. He, Physics-informed machine learning with conditional karhunen-loève expansions, Journal of Computational Physics 426 (2021) 109904

  36. [36]

    Mai-Duy, T

    N. Mai-Duy, T. Tran-Cong, Approximation of function and its derivatives using radial basis function networks, Applied Mathematical Modelling 27 (3) (2003) 197–

  37. [37]

    doi:10.1016/S0307-904X(02)00101-4

  38. [38]

    Virtanen, R

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Courna- peau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, et al., Scipy 1.0: fun- damental algorithms for scientific computing in python, Nature methods 17 (3) (2020) 261–272

  39. [39]

    N. J. Quinlan, M. Basa, M. Lastiwka, Truncation error in mesh-free particle meth- ods, International Journal for Numerical Methods in Engineering 66 (13) (2006) 2064–2085. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.1617, doi:10.1002/nme.1617

  40. [40]

    M. Levy, B. Berkowitz, Measurement and analysis of non-fickian dispersion in heterogeneous porous media, Journal of contaminant hydrology 64 (3-4) (2003) 203–226

  41. [41]

    F. R. de Hoog, J. H. Knight, A. N. Stokes, An Improved Method for Numeri- cal Inversion of Laplace Transforms, SIAM Journal on Scientific and Statistical Computing 3 (3) (1982) 357–366. doi:10.1137/0903022. 25

  42. [42]

    P. Wang, O. A. Cirpka, Surface Transient Storage Under Low-Flow Conditions in Streams With Rough Bathymetry, Water Resources Research 57 (12) (2021) e2021WR029899. doi:10.1029/2021WR029899

  43. [43]

    Hollenbeck, Invlap.m: A matlab function for numerical inversion of laplace transforms by the de hoog algorithm [software] (1998)

    K. Hollenbeck, Invlap.m: A matlab function for numerical inversion of laplace transforms by the de hoog algorithm [software] (1998). URLhttp://www.isva.dtu.dk/staff/karl/invlap.htm

  44. [44]

    M. M. Reyna, A. M. Tartakovsky, Parameter estimation in river transport models with immobile phase exchange using dimensional analysis and reduced-order mod- els (2025). URLarxiv.org/abs/2510.19664

  45. [45]

    I. W. Seo, T. S. Cheong, Moment-Based Calculation of Parameters for the Storage Zone Model for River Dispersion, Journal of Hydraulic Engineering 127 (6) (2001) 453–465. doi:10.1061/(ASCE)0733-9429(2001)127:6(453)

  46. [46]

    González-Pinzón, R

    R. González-Pinzón, R. Haggerty, M. Dentz, Scaling and predicting solute trans- port processes in streams, Water Resources Research 49 (7) (2013) 4071–4088. doi:10.1002/wrcr.20280

  47. [47]

    Aghababaei, T

    M. Aghababaei, T. R. Ginn, Temporal moments of one-dimensional advective- dispersive transport with exchange represented via memory function models: Ap- plication to river corridor transport, Advances in Water Resources 172 (2023) 104383. doi:10.1016/j.advwatres.2023.104383

  48. [48]

    C. F. Nordin, G. V. Sabol, Empirical data on longitudinal dispersion in rivers, Tech. Rep. 74-20, U.S. Geological Survey, publication Title: Water-Resources In- vestigations Report (1974). doi:10.3133/wri7420

  49. [49]

    Rodríguez, P

    L. Rodríguez, P. Tunby, A. Abusang, A. Tartakovsky, K. Carroll, T. Ginn, R. González-Pinzón, Tierras tracer injection experiments in rivers and streams (2.0) [data set] (2025). doi:10.5281/zenodo.15794259

  50. [50]

    Jacot, F

    A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: Convergence and general- ization in neural networks, Advances in neural information processing systems 31 (2018) 8571–8580

  51. [51]

    ep” is a point source in the PDE, “ei

    H. Baty, Solving stiff ordinary differential equations using physics informed neu- ral networks (pinns): simple recipes to improve training of vanilla-pinns (2023). arXiv:2304.08289. 26 Appendix A. General formulation of Dirac delta source terms We can include point or instantaneous sources by adding Dirac delta terms to the right hand side of the PDE or ...