Physics-Informed Neural Networks and Radial Basis Functions for PDEs with Dirac Delta Sources
Pith reviewed 2026-06-27 09:44 UTC · model grok-4.3
The pith
Weak-form integration of Dirac deltas lets RBF-RLS solve PDEs accurately while PINN residuals fail to reach zero.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper claims that integrating the weak-form equation allows direct treatment of Dirac delta terms without smooth surrogates; under this treatment RBF-RLS produces good forward and inverse solutions to transport problems while PINNs do not make residuals converge to zero, with the distinction explained by Neural Tangent Kernel theory.
What carries the argument
Weak-form integration of the Dirac delta inside the Residual Least Squares loss, applied once to PINNs and once to an RBF expansion.
If this is right
- RBF-RLS can be applied directly to inverse problems that fit real-world measurements without introducing modeling error from delta approximations.
- The same weak-form treatment extends to other linear PDEs that represent flow and transport in porous media.
- NTK analysis supplies a diagnostic tool for choosing among different Residual Least Squares architectures when singular sources are present.
- Forward solutions remain stable under both synthetic and noisy data once the delta is integrated exactly.
Where Pith is reading between the lines
- Similar weak-form handling might improve other neural or kernel methods on problems with point sources or discontinuities.
- The observed gap suggests testing whether hybrid RBF-neural architectures inherit the convergence advantages of pure RBF-RLS.
- The result raises the question of whether the same integration step improves performance on nonlinear transport equations.
Load-bearing premise
Neural Tangent Kernel theory correctly predicts why residuals converge under RBF-RLS but not under PINNs after the same weak-form integration of the delta.
What would settle it
Run both PINN and RBF-RLS on the same one-dimensional linear transport equation with a single Dirac delta source, integrate the weak form once, and check whether the PINN residual norm actually reaches machine zero or remains bounded away from zero.
Figures
read the original abstract
Physics-Informed Neural Networks (PINNs) are a machine learning method for solving forward and inverse Partial Differential Equations (PDEs). When applied to PDEs with Dirac delta functions in the forcing terms, boundary conditions, or initial conditions, PINNs require approximating them with smooth surrogate functions, a practice that can introduce significant modeling errors. In this work, we exploit the interpretation of PINNs as Residual Least Squares (RLS) methods and show that this perspective enables direct treatment of Dirac delta terms by integrating the weak-form equation. Among RLS formulations other than PINN, we focus on the Radial Basis Function (RBF) expansion (also known as a single-layer RBF Network). We show that while integrating out the Dirac delta in PINNs causes residuals to fail to converge to zero, RBF-RLS consistently provides good forward and inverse solutions to transport problems. We explain this finding using the Neural Tangent Kernel (NTK) theory. We test both approaches on linear PDEs that represent groundwater flow and transport in porous media and rivers. We solve inverse problems to fit synthetic data, noisy synthetic data, and real-world measurements.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that standard PINNs, when Dirac delta sources are handled by integrating the weak-form residual, fail to drive the training loss (residual) to zero, whereas RBF expansions within the same RLS framework succeed at both forward and inverse solutions for linear transport PDEs; the distinction is attributed to properties of the Neural Tangent Kernel, with numerical demonstrations on groundwater flow and river transport models using synthetic, noisy, and real measurements.
Significance. If the empirical distinction and its NTK grounding hold after verification, the work would identify a concrete regime where deep PINN residuals stagnate on singular sources while shallow RBF-RLS remains effective, offering guidance for method selection in applications with point sources. The inclusion of inverse problems on real data strengthens applicability; explicit spectral analysis of the NTK (if added) would constitute a useful theoretical contribution.
major comments (2)
- [§4] §4 (NTK explanation): the assertion that NTK theory accounts for PINN residual non-convergence versus RBF-RLS success after weak-form delta integration is not supported by explicit computation of the relevant NTK eigenvalues or gradient-flow spectrum for the integrated loss operator; without this, the explanation cannot be distinguished from confounding factors such as network depth (deep vs. single-layer) or optimizer behavior.
- [Results] Results section (convergence plots and tables): the central claim that PINN residuals fail to reach zero while RBF-RLS succeeds requires demonstration that identical weak-form integration, quadrature, and collocation strategy are applied to both methods; any post-hoc differences in these implementation choices would undermine the attribution to the RLS formulation class.
minor comments (2)
- [§2] §2 (weak-form definition): clarify the precise quadrature rule used to integrate the Dirac delta term and confirm that the same rule is stated for both PINN and RBF-RLS.
- [Figures] Figure captions: add explicit labels for which curves correspond to PINN versus RBF-RLS and report the final residual norms numerically rather than only visually.
Simulated Author's Rebuttal
We thank the referee for their thoughtful comments, which help clarify the contributions and limitations of our work. We address the major comments point by point below.
read point-by-point responses
-
Referee: [§4] §4 (NTK explanation): the assertion that NTK theory accounts for PINN residual non-convergence versus RBF-RLS success after weak-form delta integration is not supported by explicit computation of the relevant NTK eigenvalues or gradient-flow spectrum for the integrated loss operator; without this, the explanation cannot be distinguished from confounding factors such as network depth (deep vs. single-layer) or optimizer behavior.
Authors: We acknowledge that our manuscript does not include explicit computation of NTK eigenvalues or the gradient-flow spectrum for the weak-form loss. The NTK explanation in §4 is based on established properties from the literature regarding the behavior of deep networks versus shallow linear models under least-squares losses. Specifically, deep PINNs can suffer from spectral bias and poor conditioning when the loss involves integrated singular terms, while the RBF expansion corresponds to a kernel method with more favorable properties for this setting. The comparison intentionally contrasts the standard deep PINN architecture with the single-layer RBF, as these represent typical choices in each RLS class. We will revise the text in §4 to emphasize that the NTK provides a qualitative rationale rather than a quantitative proof, and to explicitly discuss the role of network depth as part of the method distinction. No additional numerical NTK analysis will be added, as it would require substantial new theoretical work beyond the paper's scope. revision: partial
-
Referee: [Results] Results section (convergence plots and tables): the central claim that PINN residuals fail to reach zero while RBF-RLS succeeds requires demonstration that identical weak-form integration, quadrature, and collocation strategy are applied to both methods; any post-hoc differences in these implementation choices would undermine the attribution to the RLS formulation class.
Authors: The manuscript applies the same weak-form integration and RLS framework to both methods, as described in Sections 2 and 3. The quadrature and collocation points are chosen identically for the integrated residual in both cases, with the only difference being the parametrization of the solution (neural network weights versus RBF coefficients). To address this concern, we will add explicit statements and possibly a comparison table in the Results section confirming that the integration scheme, number of quadrature points, and collocation strategy are the same for PINN and RBF-RLS experiments. This will be supported by reference to the shared code repository if available. revision: yes
Circularity Check
No circularity in derivation or claims
full rationale
The paper's core claim rests on an empirical observation that weak-form integration of Dirac deltas causes PINN residuals to fail to converge while RBF-RLS succeeds, with NTK theory invoked as an external explanatory framework. No equations, parameters, or results are shown to reduce by construction to fitted inputs, self-definitions, or self-citation chains; the distinction is presented as a tested difference rather than a tautological renaming or forced prediction. The derivation chain is self-contained against external benchmarks and does not rely on load-bearing self-references for its validity.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption PINNs can be interpreted as Residual Least Squares (RLS) methods
- domain assumption Neural Tangent Kernel (NTK) theory explains the difference in residual convergence between PINNs and RBF-RLS
Reference graph
Works this paper leans on
-
[1]
Raissi, P
M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving non- linear partial differential equations, Journal of Computational Physics 378 (2019) 686–707
2019
-
[2]
G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, L. Yang, Physics-informed machine learning, Nature Reviews Physics 3 (6) (2021) 422–440
2021
-
[3]
S. Cuomo, V. S. di Cola, F. Giampaolo, G. Rozza, M. Raissi, F. Piccialli, Scientific machine learning through physics-informed neural networks: Where we are and what’s next (2022). doi:10.48550/ARXIV.2201.05624
-
[4]
A. M. Tartakovsky, C. O. Marrero, P. Perdikaris, G. D. Tartakovsky, D. Barajas- Solano, Physics-informed deep neural networks for learning parameters and consti- tutive relationships in subsurface flow problems, Water Resources Research 56 (5) (2020) e2019WR026731
2020
-
[5]
Q. He, A. M. Tartakovsky, Physics-informed neural network method for forward and backward advection-dispersion equations, Water Resources Research 57 (7) (2021) e2020WR029479. 22
2021
-
[6]
X. Zhao, H. Liu, F. Leng, W. Yang, X. Yin, Y. Yi, Source trac- ing for pollutants in river channels based on a physics-informed neu- ral network, Water Resources Research 62 (2) (2026) e2025WR040846. doi:https://doi.org/10.1029/2025WR040846
- [7]
-
[8]
W. E, B. Yu, The deep ritz method: A deep learning-based numerical algorithm for solving variational problems, Communications in Mathematics and Statistics 6 (1) (2018) 1–12. doi:10.1007/s40304-018-0127-z. URLarxiv.org/pdf/1710.00211
work page internal anchor Pith review Pith/arXiv arXiv doi:10.1007/s40304-018-0127-z 2018
-
[9]
C. Wang, H. Guo, X. Yan, Z.-L. Shi, Y. Yang, Improved physics-informed neural networks for the reinterpreted discrete fracture model, Journal of Computational Physics 520 (2025) 113491. doi:10.1016/j.jcp.2024.113491
-
[10]
Z. Xu, Y. Yang, The hybrid dimensional representation of permeability ten- sor: A reinterpretation of the discrete fracture model and its extension on nonconforming meshes, Journal of Computational Physics 415 (2020) 109523. doi:10.1016/j.jcp.2020.109523
-
[11]
H. Park, G. Jo, A physics-informed neural network based method for the nonlin- ear poisson-boltzmann equation and its error analysis, Journal of Computational Physics 522 (2024) 113579–113579. doi:10.1016/j.jcp.2024.113579
-
[12]
M. D. Rosa, L. Pompameo, A. Litvinenko, S. Cuomo, Homo-pinn: Hyperparam- eter optimization of a multi-output physics-informed neural network, Operations Research Forum 6 (4) (Oct 2025). doi:10.1007/s43069-025-00561-7
-
[13]
T. Kapoor, H. Wang, A. Núñez, R. Dollevoet, Physics-informed machine learning for moving load problems, Journal of Physics: Conference Series 2647 (15) (2024) 152003. doi:10.1088/1742-6596/2647/15/152003
-
[14]
A. Liu, J. Li, J. Bi, Z. Chen, Y. Wang, C. Lu, Y. Jin, B. Lin, A novel reservoir simulation model based on physics informed neural networks, Physics of Fluids 36 (11) (Nov. 2024). doi:10.1063/5.0239376
-
[15]
J.-P. Suarez, G. Jacobs, Regularization of singularities in the weighted summation of dirac-delta functions for the spectral solution of hyperbolic conservation laws? (2016). URLarxiv.org/abs/1611.05510
Pith/arXiv arXiv 2016
-
[16]
E. D. Eason, A review of least-squares methods for solving partial differential equa- tions, International Journal for Numerical Methods in Engineering 10 (5) (1976) 1021–1046. doi:10.1002/nme.1620100505. 23
-
[17]
E. Kharazmi, Z. Zhang, G. E. M. Karniadakis, hp-VPINNs: Varia- tional physics-informed neural networks with domain decomposition, Com- puter Methods in Applied Mechanics and Engineering 374 (2021) 113547. doi:10.1016/j.cma.2020.113547
-
[18]
Hanke, R
M. Hanke, R. März, Convergence analysis of least-squares collocation methods for nonlinear higher-index differential–algebraic equations, Journal of Computational and Applied Mathematics 387 (2021) 112514
2021
-
[19]
J. N. Reddy, An introduction to the finite element method, McGraw-Hill Inc, 1993
1993
-
[20]
S. Wang, X. Yu, P. Perdikaris, When and why pinns fail to train: A neural tan- gent kernel perspective, Journal of Computational Physics 449 (2022) 110768. doi:https://doi.org/10.1016/j.jcp.2021.110768
-
[21]
D. Lowe, D. Broomhead, Multivariable functional interpolation and adaptive net- works, Complex systems 2 (3) (1988) 321–355
1988
-
[22]
F. Schwenker, H. A. Kestler, G. Palm, Three learning phases for radial-basis- function networks, Neural Networks 14 (4) (2001) 439–458. doi:10.1016/S0893- 6080(01)00027-2
-
[23]
Ghosh, A
J. Ghosh, A. Nag, An overview of radial basis function networks, in: R. J. Howlett, L. C. Jain (Eds.), Radial Basis Function Networks 2: New Advances in Design, Physica-Verlag, Heidelberg, 2001, pp. 1–36
2001
-
[24]
E. J. Kansa, Multiquadrics—A scattered data approximation scheme with ap- plications to computational fluid-dynamics—I surface approximations and partial derivative estimates, Computers & Mathematics with Applications 19 (8) (1990) 127–145
1990
-
[25]
E. J. Kansa, Multiquadrics—A scattered data approximation scheme with applica- tions to computational fluid-dynamics—II solutions to parabolic, hyperbolic and elliptic partial differential equations, Computers & Mathematics with Applications 19 (8) (1990) 147–161
1990
-
[26]
T.-O. Kwok, L. Ling, On Convergence of a Least-Squares Kansa’s Method for the Modified Helmholtz Equations, Adv. Appl. Math. Mech. (2009)
2009
-
[27]
M. M. Alqezweeni, V. I. Gorbachenko, M. V. Zhukov, M. S. Jaafar, Efficient solving of boundary value problems using radial basis function networks learned by trust region method, International Journal of Mathematics and Mathematical Sciences 2018 (1) (2018) 9457578. doi:10.1155/2018/9457578
-
[28]
J. Bai, G.-R. Liu, A. Gupta, L. Alzubaidi, X.-Q. Feng, Y. Gu, Physics-informed radial basis network (PIRBN): A local approximating neural network for solving nonlinear partial differential equations, Computer Methods in Applied Mechanics and Engineering 415 (2023) 116290. doi:10.1016/j.cma.2023.116290. 24
-
[29]
V. I. Gorbachenko, M. M. Alqezweeni, M. S. Jaafar, V. Z. Orbachenko, Application of parametric identification method and radial basis function networks for solution of inverse boundary value problems, in: 2017 Annual Conference on New Trends in Information & Communications Technology Applications (NTICT), 2017, pp. 18–21. doi:10.1109/NTICT.2017.7976151
-
[30]
V. I. Gorbachenko, M. V. Zhukov, Solving boundary value problems of mathemat- ical physics using radial basis function networks, Computational Mathematics and Mathematical Physics 57 (1) (2017) 145–155. doi:10.1134/S0965542517010079
-
[31]
V. I. Gorbachenko, D. A. Stenkin, Physics-Informed Radial Basis-Function Net- works, Technical Physics (10 2023). doi:10.1134/S1063784223050018
-
[32]
B. A. Finlayson, Introduction, Society for Industrial and Applied Mathematics, 2014, Ch. 1, pp. 3–14. doi:10.1137/1.9781611973242.ch1
-
[33]
H. Nguyen, J. Reynen, A space-time least-square finite element scheme for advection-diffusion equations, Computer Methods in Applied Mechanics and En- gineering 42 (3) (1984) 331–342. doi:10.1016/0045-7825(84)90012-4
-
[34]
Nguyen, J
H. Nguyen, J. Reynen, A space-time least-square finite element scheme for advection-diffusion equations, Computer Methods in Applied Mechanics and En- gineering 42 (3) (1984) 331–342
1984
-
[35]
Tartakovsky, D
A. Tartakovsky, D. Barajas-Solano, Q. He, Physics-informed machine learning with conditional karhunen-loève expansions, Journal of Computational Physics 426 (2021) 109904
2021
-
[36]
Mai-Duy, T
N. Mai-Duy, T. Tran-Cong, Approximation of function and its derivatives using radial basis function networks, Applied Mathematical Modelling 27 (3) (2003) 197–
2003
-
[37]
doi:10.1016/S0307-904X(02)00101-4
-
[38]
Virtanen, R
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Courna- peau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, et al., Scipy 1.0: fun- damental algorithms for scientific computing in python, Nature methods 17 (3) (2020) 261–272
2020
-
[39]
N. J. Quinlan, M. Basa, M. Lastiwka, Truncation error in mesh-free particle meth- ods, International Journal for Numerical Methods in Engineering 66 (13) (2006) 2064–2085. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.1617, doi:10.1002/nme.1617
-
[40]
M. Levy, B. Berkowitz, Measurement and analysis of non-fickian dispersion in heterogeneous porous media, Journal of contaminant hydrology 64 (3-4) (2003) 203–226
2003
-
[41]
F. R. de Hoog, J. H. Knight, A. N. Stokes, An Improved Method for Numeri- cal Inversion of Laplace Transforms, SIAM Journal on Scientific and Statistical Computing 3 (3) (1982) 357–366. doi:10.1137/0903022. 25
-
[42]
P. Wang, O. A. Cirpka, Surface Transient Storage Under Low-Flow Conditions in Streams With Rough Bathymetry, Water Resources Research 57 (12) (2021) e2021WR029899. doi:10.1029/2021WR029899
-
[43]
Hollenbeck, Invlap.m: A matlab function for numerical inversion of laplace transforms by the de hoog algorithm [software] (1998)
K. Hollenbeck, Invlap.m: A matlab function for numerical inversion of laplace transforms by the de hoog algorithm [software] (1998). URLhttp://www.isva.dtu.dk/staff/karl/invlap.htm
1998
-
[44]
M. M. Reyna, A. M. Tartakovsky, Parameter estimation in river transport models with immobile phase exchange using dimensional analysis and reduced-order mod- els (2025). URLarxiv.org/abs/2510.19664
arXiv 2025
-
[45]
I. W. Seo, T. S. Cheong, Moment-Based Calculation of Parameters for the Storage Zone Model for River Dispersion, Journal of Hydraulic Engineering 127 (6) (2001) 453–465. doi:10.1061/(ASCE)0733-9429(2001)127:6(453)
-
[46]
R. González-Pinzón, R. Haggerty, M. Dentz, Scaling and predicting solute trans- port processes in streams, Water Resources Research 49 (7) (2013) 4071–4088. doi:10.1002/wrcr.20280
-
[47]
M. Aghababaei, T. R. Ginn, Temporal moments of one-dimensional advective- dispersive transport with exchange represented via memory function models: Ap- plication to river corridor transport, Advances in Water Resources 172 (2023) 104383. doi:10.1016/j.advwatres.2023.104383
-
[48]
C. F. Nordin, G. V. Sabol, Empirical data on longitudinal dispersion in rivers, Tech. Rep. 74-20, U.S. Geological Survey, publication Title: Water-Resources In- vestigations Report (1974). doi:10.3133/wri7420
-
[49]
L. Rodríguez, P. Tunby, A. Abusang, A. Tartakovsky, K. Carroll, T. Ginn, R. González-Pinzón, Tierras tracer injection experiments in rivers and streams (2.0) [data set] (2025). doi:10.5281/zenodo.15794259
-
[50]
Jacot, F
A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: Convergence and general- ization in neural networks, Advances in neural information processing systems 31 (2018) 8571–8580
2018
-
[51]
ep” is a point source in the PDE, “ei
H. Baty, Solving stiff ordinary differential equations using physics informed neu- ral networks (pinns): simple recipes to improve training of vanilla-pinns (2023). arXiv:2304.08289. 26 Appendix A. General formulation of Dirac delta source terms We can include point or instantaneous sources by adding Dirac delta terms to the right hand side of the PDE or ...
arXiv 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.