Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Interpretability and Generalization Bounds for Learning Spatial Physics

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that for linear differential equations, gradient-descent training drives a model only onto a projection of the true solution operator, so the subspace spanned by training data, not data quantity or grid fineness…

desk verdict A useful paper on a real problem, with one theorem overstated; the projection barrier is solid, the finite-difference bias formula needs a caveat. read the letter →

arxiv 2506.15199 v4 pith:TYQPUM73 submitted 2025-06-18 cs.LG stat.ML

Alejandro Francisco Queiruga , Theo Gutman-Solo , Shuai Jiang This is my paper · ORCID
classification cs.LGstat.ML MSC 68T0765N0665N12
keywords operatorlearninggeneralizationboundsGreen'sfunctionfinitedifferencesfunction-spacephysics-informedneuralnetworksmechanisticinterpretabilityout-of-distribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks why machine-learning models of physics can show near-zero training error and still fail completely on new inputs. Its answer, proved for the simplest linear case and demonstrated across many architectures, is that the function space the training data fill, not just how much data there is or how fine the grid is, sets the ceiling on what the model can learn. A linear model trained by gradient descent converges to a projection of the true Green's-function operator onto the training subspace; the leftover part of the operator is invisible to training and cannot be recovered by more data or finer discretization. For a finite-difference parameter fit, the paper proves an irreducible bias of order $\Delta x^q$ that actually grows with the polynomial degree of the training data. The same subspace barrier appears empirically in deep linear networks, MLPs, DeepONets, Fourier neural operators, and physics-informed variants, along with an interpretability method that extracts the learned Green's function from the weights.

What carries the argument

The load-bearing object is the discretized Green's function matrix $A_{ij} = \int_0^1 G(x_i,s)\psi_j(s)\,ds$, defined by expanding the forcing in the grid basis $\psi_j$; it is the exact solution operator the model is trying to learn. The argument runs through the subspace-projection identity $W^* = A U U^T + W_0(I-UU^T)$, where $U$ comes from the SVD of the basis-evaluation matrix $B$, together with the Taylor-expansion coefficients of finite-difference stencils that produce the $\Delta x^q$ bias terms. These two pieces turn training dynamics and parameter fitting into linear algebra, which is what makes the generalization ceiling computable a priori.

What would settle it

Train a linear model on many samples of degree-$p$ polynomial forcings at a fixed fine grid, then test on a sine forcing of higher frequency: the theorem predicts the test MSE cannot fall below the squared projection residual $\|A - A U U^T\|_F^2$ no matter how many samples, so finding a test MSE below that bound would refute Theorem 3.2.

Watch

Extended reading notes

Core claim

The paper's central discovery is an exact characterization of what a linear model learns from solution-forcing pairs. When forcing functions are drawn from a $p$-dimensional subspace with basis matrix $B$, gradient descent on the mean-squared-error loss converges to $W^* = A U U^T + W_0(I - U U^T)$, where $A$ is the discretized Green's-function solution operator and $U$ is the left orthogonal basis of $B$. The learned operator is the true operator projected onto the training subspace, plus the part of the initial weights orthogonal to that subspace; the projection residual $A - A U U^T$ is invisible to the training loss and cannot be removed by adding more samples or refining the grid. In parallel, for a finite-difference parameter fit the paper proves that the relative parameter error carries an irreducible $O(\Delta x^q)$ stencil-truncation term that grows as the polynomial degree of the training data increases, so richer training data can actively hurt. The same function-space barrier is then shown empirically across deep linear networks, MLPs, DeepONets, Fourier neural operators, and physics-informed variants, together with a mechanistic check that recovers Green's-function structure from the weights of models that do generalize.

Load-bearing premise

The exact formulas assume the training forcing functions are random combinations of basis functions with zero mean, independent coefficients, and identical variance across all basis degrees; real data with correlated or decaying spectra would shift the constants, and the transfer of the mechanism to nonlinear networks is demonstrated empirically rather than proved.

Editorial extensions

If this is right

  • Near-zero training error is not evidence that a model has learned the underlying physics; a converged model can be blind to every component of the operator that lies outside the training subspace.
  • Finer grids do not automatically help operator learning, because the training subspace is a smaller slice of a larger space of possible forcings, while finite-difference parameter error falls only at the stencil's truncation rate $\Delta x^q$.
  • Richer training data can be harmful: for a finite-difference fit, each polynomial degree beyond the stencil order adds another positive $\Delta x^m$ term to the parameter bias.
  • A cheap cross-subspace evaluation, training on one function family and testing on the others, reveals whether a model has subspace generalization, producing the block lower-triangular error structure seen in the experiments.
  • Extracting Green's functions by one-hot impulse probes gives a visual and algebraic diagnostic: generalized models show the correct kernel structure, and their learned inverse recovers a local stencil.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical consequence the authors leave implicit is that data collection for scientific machine learning should target broad-spectrum or piecewise-rough forcings rather than smooth analytic families; their FEM-dataset results suggest such data can recover the full operator.
  • For real applications where forcing coefficients are correlated or spectrally decaying, the exact constants in Theorem 3.1 will change, but the qualitative claim that the training function space sets the generalization ceiling should persist, and this is testable by rerunning the heatmaps with real-world spectra.
  • The projection view suggests a diagnostic for large neural operators: compute the numerical rank of the training input distribution; if it is far below the spatial resolution, out-of-distribution failure is expected no matter how low the training loss.
  • By replacing the linear operator with the Jacobian of a nonlinear map, the projection identity would predict which tangent directions are learned, connecting the result to transfer learning for nonlinear dynamics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies generalization of machine-learning models for linear differential equations on 1D/2D Poisson and biharmonic problems. It proposes two theoretical results: (i) a finite-difference parameter fit has an irreducible bias of order Delta x^q that grows with the polynomial degree p of training data (Theorem 3.1), and (ii) a linear model trained by gradient descent converges to the projection of the true discrete solution operator onto the training-data subspace (Theorem 3.2). It then presents extensive cross-evaluation heatmaps for seven model families, showing that out-of-subspace test errors are large even when training MSE is near zero; introduces a Green's-function-based interpretability diagnostic; and recommends cross-set validation as a benchmark.

Significance. Theorem 3.2 is the strongest result: it is a parameter-free exact characterization of the learned weights, and it cleanly explains why near-zero training error does not imply operator recovery. The empirical comparison across finite-difference fits, linear models, MLPs, DeepONets, FNOs, PINNs, and PI-DeepONets is unusually broad, and the proposed cross-evaluation methodology is a practical benchmarking tool. The Green's-function extraction from trained weights is a useful diagnostic. The main intellectual contribution---that the training function space, not data quantity alone, sets the generalization ceiling for linear models---is significant for the SciML community, provided the issues with Theorem 3.1 are corrected.

major comments (3)
  1. [App. A.1, Theorem 3.1] The proof of Theorem 3.1 replaces the discrete grid loss of Eq. (7) with the continuous integral in Eq. (20) without justification. The minimizer of the discrete loss is governed by discrete moments such as sum_i x_i^k, not by the continuous values 1/(m+j+1); therefore the closed forms in Eqs. (28)-(31) and the claim that each added polynomial degree strictly increases |w-k| are not proven for the actual training loss at finite Delta x. The asymptotic O(Delta x^q) floor is plausible from Taylor truncation, but the theorem is stated as an exact equality and is overstated as written. Please restate Theorem 3.1 as an asymptotic result or provide a discrete proof.
  2. [App. A.1, Eq. (18)] The expansion FD_q(u,Delta x) = sum_{m=0}^{q-1} c_m x^m + sum_{m=q}^p c_m(1+zeta_m Delta x^m)x^m does not describe the action of a standard centered stencil on monomials. For the three-point stencil applied to u=x^4, the truncation error is 2 Delta x^2, an additive constant, not a perturbation of the x^4 coefficient; in general the error contains lower-degree terms. Consequently Eq. (31) is not the exact minimizer of Eq. (7), which is consistent with the overestimation reported in Fig. 2. The theorem should be re-derived from the actual stencil expansion, or explicitly presented as a leading-order approximation.
  3. [Sec. 4.1 vs. App. B] Section 4.1 says the linear model is trained with W0=0 and no regularization, but App. B states that all models use AdamW with weight decay lambda=0.01. If the linear-model runs used weight decay, they minimize a regularized loss rather than the unregularized loss (10) in Theorem 3.2, so the near-machine-precision agreement in Fig. 2 (left) would not be expected from the theorem. Please clarify the configuration actually used; if weight decay was present, either report unregularized runs or quantify the effect of the regularization term.
minor comments (5)
  1. [Eq. (16)] The summation index starts at m=-2; it should start at m=0 for the stated polynomial solutions.
  2. [App. A.3] The phrase 'Assuming the singular values of B are identical' is unnecessary and generally false for Vandermonde matrices; the convergence argument holds componentwise for any positive diagonal D^2 and the assumption should be removed.
  3. [Sec. 3.2] The preamble says the relative error 'can be bounded from above,' but Theorem 3.1 is stated as an equality; align the wording after the theorem is revised.
  4. [Fig. 2 caption] The phrase 'the analytical assumptions overestimate the error' undercuts the equality claim in Theorem 3.1; clarify whether the curves are bounds or exact predictions.
  5. [Reproducibility] Consider adding a code/data availability statement for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the theoretical predictions are derived from stated sampling assumptions and are compared to, not fitted from, the measured errors.

full rationale

The paper's central claims (Theorem 3.1 and Theorem 3.2) are derived in the appendices from explicit assumptions about the sampling model (f(n) = B c(n) with iid coefficients) and about the finite-difference stencil's Taylor truncation. The predicted quantities—the parameter bias |w-k|/|k| in Eq. (8) and the projected operator W* = A U U^T + W0(I - U U^T) in Eq. (11)—are computed from the derived closed forms, not obtained by fitting to the experimental errors. Where the analytical curves and measurements disagree (Fig. 2), the paper reports the disagreement and attributes it to worst-case truncation constants, which is the opposite of circular reasoning. The empirical verification uses independently generated datasets and reports honest mismatches, including the SINDy floating-point artifact in Fig. 13. The only self-citations (Queiruga et al., 2020; Krishnapriyan et al., 2023) occur in the background and are not load-bearing for the theorems; no uniqueness claim or ansatz is imported from them. A reviewer concern that the proof of Theorem 3.1 replaces the discrete grid loss (Eq. 7) with a continuous integral (Eq. 20) is a technical correctness issue about the exact equality at finite Δx, not a circularity: the asymptotic O(Δx^q) floor and Theorem 3.2's projection result stand independently of that substitution, and the paper's own Fig. 2 shows the resulting overestimate rather than hiding it.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central results are nearly self-contained: the only inputs are the analytical Green's function of the target linear operator, the function-space sampling model (f = B c with iid c), and standard numerical-analysis facts (Taylor truncation of stencils, normal equations). No constants are fitted to data; the truncation coefficients in Theorem 3.1 are derived from the fixed stencil. The main domain assumptions are that training and test data are exact analytical solutions of the same operator A, and that the linear-model mechanism transfers to nonlinear architectures, which is asserted empirically rather than proven.

assumptions (5)
  • domain assumption Training and test data are exact analytical solutions u = A f of a fixed linear operator A (the Poisson Green's function, Eqs. 2-3 and 9), with the same A for all function spaces.
    This is the ground truth for Theorems 3.1-3.2 and the benchmark (Sec. 3.1). If real data come from a mis-specified or varying operator, the clean projection structure and quantitative bounds do not apply; Fig. 7 is the only partial relaxation (additive noise).
  • domain assumption Forcing coefficients c are iid with zero mean, and in Theorem 3.1 with identical second moments E[c_m^2] for every polynomial degree m.
    App. A.1 states: 'By assumption that the expectations on the second moment are identical for each coefficient c_m.' This collapses the loss to diagonal terms (Eq. 26) and gives the closed-form w (Eqs. 28-31); correlated or spectrally decaying coefficients would change the constants.
  • standard math Gradient descent on the convex MSE converges to the unregularized least-squares minimizer.
    Normal-equations and convex-optimization theory (App. A.3). The experiments opt in by using W_0 = 0, no regularization, full-batch training, and 10,000 samples.
  • standard math A finite-difference stencil of order q satisfies FD_q(u, Δx) = u'' + O(Δx^q) with Taylor-boundable coefficients.
    Textbook numerical analysis (Eqs. 17-18); the basis of Theorem 3.1's error expansion.
  • ad hoc to paper The linear-model projection barrier transfers symptomatically to non-convex architectures (DeepONet, FNO, MLP, PINN, PI-DeepONet) for which no analogous theorem is given.
    The paper labels these results empirical ('not guaranteed', 'suggests'); this transfer is the main inductive step behind the broad conclusion in Sec. 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpretability and Generalization Bounds for Learning Spatial Physics." pith.science (2026). https://pith.science/paper/TYQPUM73

@misc{pith2026250615199,
  author       = {Pith},
  title        = {Pith review of: Interpretability and Generalization Bounds for Learning Spatial Physics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TYQPUM73}},
  note         = {Machine review of arXiv:2506.15199}
}
read the original abstract

While there are many applications of ML to scientific problems that look promising, visuals can be deceiving. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain ML models applied to linear differential equations for parameter discovery or solution finding. Beyond the quantity and discretization of data, we identify that the function space of the data is critical to the generalization of the model. A similar lack of generalization is empirically demonstrated for commonly used models, including physics-specific techniques. Counterintuitively, we find that different classes of models can exhibit opposing generalization behaviors. Based on our theoretical analysis, we also introduce a new mechanistic interpretability lens on scientific models whereby Green's function representations can be extracted from the weights of black-box models. Our results inform a new cross-validation technique for measuring generalization in physical systems, which can serve as a benchmark.

Figures

Figures reproduced from arXiv: 2506.15199 by the authors.

Figure 1
Figure 1. Expectations when learning a black box linear model, u = W f. Despite some visual similarity to the Green’s function operator (top), naive approaches are not guaranteed to converge to the true operator. The issue lies in the sampling procedure of the training data. The naive approach of using polynomial analytical solutions (illustrated as cubic polynomials) can achieve machine precision MSE loss, but even with infi… view at source ↗
Figure 2
Figure 2. Comparison of numerical experiments to analytically derived ML parameters from Theorems 3.1 and 3.2 trained on polynomial spaces with Ngrid = 22. (Left) The observed error for the linear matrix matches predictions for lower order p. (Middle and right) For the finite difference model, the analytical assumptions overestimate the error, but the trend matches with observations. Poly Sine Cosine 123456781234567812345678 … view at source ↗
Figure 3
Figure 3. Learning a parameter using a PINN inverse problem (mean of five runs; shaded region is the min–max spread). Mir￾roring Theorem 3.1, the error grows with the polynomial degree p of the training data, as the truncation error of the PDE loss is absorbed into the learned w. The proof is in App. A.1, and empirically verified in Sec￾tion 4.1. In other words, the error is irreducible, even with noiseless and infinite data … view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Cross-evaluation error heatmaps for four model types. Rows: training set; columns: test set. Dashed lines separate function class families; cells left of each cell are subspaces. Color shows log-scale MSE (sharp transitions ≈ ×1010). The blue lower triangular blocks in…
Figure 5
Figure 5. Figure 5: Generalization to out-of-distribution test datasets for the DeepONet and Fourier Neural Operator. The generalization patterns are similar to the Deep Linear models in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The generalization pattern for the Physics-Informed DeepONet is similar to the Deep ONet in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Poisson equation with training data corrupted by noise. Effect of Noise We investigate the effect of measurement noise on the generalization boundaries by adding normally distributed noise to the 1D Poisson equation training data of u and f. The results are displayed i…
Figure 8
Figure 8. Figure 8: Generalization to out-of-distribution test datasets constructed for the 2D Poisson and Biharmonic equations learned with a linear model. 4.3. Extracting Green’s Functions From Black-Box Models The Green’s function is defined as the impulse response of the solution oper…
Figure 9
Figure 9. Figure 9: Visualization of the parameter matrix W of the linear model as the dataset order increases. The top row displays one row Wi1 of the matrix, the row vector corresponding to the impulse at point x = ∆x. It can be seen to converge to the shape of a Green’s function. The m…
Figure 10
Figure 10. Figure 10: Visualization of test function evaluations, Model(ei), on black-box models can reveal Green’s function structure (bottom row). The top row shows one test function evaluation at the node x = ∆x, and the bottom row shows a matrix constructed from model evaluations for a…
Figure 11
Figure 11. Figure 11: Generalization errors for all models with five different seeds for each training run. For the Finite Difference SINDy model, the lower y limit is truncated to 10−12; training on Poly 1, Sine 1 and Cosine 1 achieves error ≈ 10−38 . 16 [PITH_FULL_IMAGE:figures/full_fig…
Figure 12
Figure 12. Figure 12: Convergence of the weight matrix W as the grid size increases. The matrix structure is invariant to the discretization, however, each parameter scales proportional to ∆x (Eq. 9). 17 [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Experiments on SINDy equation discovery as the grid size (x-axis) and training data polynomial function order (line color in legend) is changed. 13a: training data MSE (observable by the scientist). 13b : difference from true parameter (not observable). After converge…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 17 canonical work pages

  1. [1]

    On the optimization of deep networks: Implicit acceleration by overparameterization

    Sanjeev Arora, Nadav Cohen, and Elad Hazan. On the optimization of deep networks: Implicit acceleration by overparameterization. In International conference on machine learning, pp.\ 244--253. PMLR, 2018

    work page 2018

  2. [2]

    Baratta, Joseph P

    Igor A. Baratta, Joseph P. Dean, Jørgen S. Dokken, Michal Habera, Jack S. Hale, Chris N. Richardson, Marie E. Rognes, Matthew W. Scroggs, Nathan Sime, and Garth N. Wells. Dolfinx: The next generation fenics problem solving environment, December 2023. URL https://doi.org/10.5281/zenodo.10447666

    doi:10.5281/zenodo.10447666 2023

  3. [3]

    Data-driven discovery of green’s functions with human-understandable deep learning

    Nicolas Boull \'e , Christopher J Earls, and Alex Townsend. Data-driven discovery of green’s functions with human-understandable deep learning. Scientific reports, 12 0 (1): 0 4824, 2022

    work page 2022

  4. [4]

    Elliptic pde learning is provably data-efficient

    Nicolas Boull \'e , Diana Halikias, and Alex Townsend. Elliptic pde learning is provably data-efficient. Proceedings of the National Academy of Sciences, 120 0 (39): 0 e2303904120, 2023

    work page 2023

  5. [5]

    Choose a transformer: Fourier or galerkin

    Shuhao Cao. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems, 34: 0 24924--24940, 2021

    work page 2021

  6. [6]

    The finite element method for elliptic problems

    Philippe G Ciarlet. The finite element method for elliptic problems. SIAM, 2002

    2002

  7. [7]

    Kutz, and Steven Brunton

    Brian de Silva, Kathleen Champion, Markus Quade, Jean-Christophe Loiseau, J. Kutz, and Steven Brunton. Pysindy: A python package for the sparse identification of nonlinear dynamical systems from data. Journal of Open Source Software, 5 0 (49): 0 2104, 2020. doi:10.21105/joss.02104. URL https://doi.org/10.21105/joss.02104

    doi:10.21105/joss.02104 2020

  8. [8]

    Deepgreen: deep learning of green’s functions for nonlinear boundary value problems

    Craig R Gin, Daniel E Shea, Steven L Brunton, and J Nathan Kutz. Deepgreen: deep learning of green’s functions for nonlinear boundary value problems. Scientific reports, 11 0 (1): 0 21614, 2021

    work page 2021

Show all 38 references
  1. [9]

    Learning to drive from a world model

    Mitchell Goff, Greg Hogan, George Hotz, Armand du Parc Locmaria, Kacper Raczy, Harald Sch \"a fer, Adeeb Shihadeh, Weixing Zhang, and Yassine Yousfi. Learning to drive from a world model. arXiv preprint arXiv:2504.19077, 2025

    2025 arXiv

  2. [10]

    Recurrent world models facilitate policy evolution

    David Ha and J \"u rgen Schmidhuber. Recurrent world models facilitate policy evolution. Advances in neural information processing systems, 31, 2018

    2018

  3. [11]

    Learning physical models that can respect conservation laws

    Derek Hansen, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, and Michael W Mahoney. Learning physical models that can respect conservation laws. In International Conference on Machine Learning, pp.\ 12469--12510. PMLR, 2023 a

  4. [12]

    Maddix, Shima Alizadeh, Gaurav Gupta, and Michael W

    Derek Hansen, Danielle C. Maddix, Shima Alizadeh, Gaurav Gupta, and Michael W. Mahoney. Learning physical models that can respect conservation laws. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of...

  5. [13]

    Kaptanoglu, Brian M

    Alan A. Kaptanoglu, Brian M. de Silva, Urban Fasel, Kadierdan Kaheman, Andy J. Goldschmidt, Jared Callaham, Charles B. Delahunt, Zachary G. Nicolaou, Kathleen Champion, Jean-Christophe Loiseau, J. Nathan Kutz, and Steven L. Brunton. Pysindy: A comprehensive python package for ...

    2022 doi

  6. [14]

    A library for learning neural operators, 2024

    Jean Kossaifi, Nikola Kovachki, Zongyi Li, Davit Pitt, Miguel Liu-Schiaffini, Robert Joseph George, Boris Bonev, Kamyar Azizzadenesheli, Julius Berner, and Anima Anandkumar. A library for learning neural operators, 2024

    2024

  7. [15]

    Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew M

    Nikola B. Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew M. Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces. CoRR, abs/2108.08481, 2021

    2021 arXiv

  8. [16]

    Characterizing possible failure modes in physics-informed neural networks

    Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advances in neural information processing systems, 34: 0 26548--26560, 2021

    2021

  9. [17]

    Learning continuous models for continuous physics

    Aditi S Krishnapriyan, Alejandro F Queiruga, N Benjamin Erichson, and Michael W Mahoney. Learning continuous models for continuous physics. Communications Physics, 6 0 (1): 0 319, 2023

    2023

  10. [18]

    Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems

    Randall J LeVeque. Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007

    2007

  11. [19]

    Transformers handle endogeneity in in-context linear regression

    Haodong Liang, Krishnakumar Balasubramanian, and Lifeng Lai. Transformers handle endogeneity in in-context linear regression. arXiv preprint arXiv:2410.01265, 2024

    2024 arXiv

  12. [20]

    Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, and George Em Karniadakis. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019

    1910 arXiv

  13. [21]

    Learning nonlinear operators via deeponet based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3 0 (3): 0 218--229, 2021 a

    2021

  14. [22]

    DeepXDE : A deep learning library for solving differential equations

    Lu Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. DeepXDE : A deep learning library for solving differential equations. SIAM Review, 63 0 (1): 0 208--228, 2021 b . doi:10.1137/19M1274067

    2021 doi

  15. [23]

    Progress measures for grokking via mechanistic interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=9XFSbDPmdW

    2023

  16. [24]

    Fourierformer: Transformer meets generalized fourier integral theorem

    Tan Nguyen, Minh Pham, Tam Nguyen, Khai Nguyen, Stanley Osher, and Nhat Ho. Fourierformer: Transformer meets generalized fourier integral theorem. Advances in Neural Information Processing Systems, 35: 0 29319--29335, 2022

    2022

  17. [25]

    Resnet after all: Neural odes and their numerical solution

    Katharina Ott, Prateek Katiyar, Philipp Hennig, and Michael Tiemann. Resnet after all: Neural odes and their numerical solution. In 9th International Conference on Learning Representations (ICLR), 2021

    2021

  18. [26]

    Thermodynamically consistent physics-informed neural networks for hyperbolic systems

    Ravi G Patel, Indu Manickam, Nathaniel A Trask, Mitchell A Wood, Myoungkyu Lee, Ignacio Tomas, and Eric C Cyr. Thermodynamically consistent physics-informed neural networks for hyperbolic systems. Journal of Computational Physics, 449: 0 110754, 2022

    2022

  19. [27]

    Grokking: Generalization beyond overfitting on small algorithmic datasets

    Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprint arXiv:2201.02177, 2022

    2022 arXiv

  20. [28]

    Continuous-in-depth neural networks

    Alejandro F Queiruga, N Benjamin Erichson, Dane Taylor, and Michael W Mahoney. Continuous-in-depth neural networks. arXiv preprint arXiv:2008.02389, 2020

    2008 arXiv

  21. [29]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

    Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378: 0 686--707, 2019

    2019

  22. [30]

    Data-driven discovery of partial differential equations

    Samuel H Rudy, Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Data-driven discovery of partial differential equations. Science advances, 3 0 (4): 0 e1602614, 2017

    2017

  23. [31]

    Do residual neural networks discretize neural ordinary differential equations? In S

    Michael Sander, Pierre Ablin, and Gabriel Peyr\' e . Do residual neural networks discretize neural ordinary differential equations? In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 3...

    2022

  24. [32]

    Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

    Andrew M Saxe, James L McClelland, and Surya Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120, 2013

    2013 arXiv

  25. [33]

    General-purpose foundation models for increased autonomy in robot-assisted surgery

    Samuel Schmidgall, Ji Woong Kim, Alan Kuntz, Ahmed Ezzat Ghazi, and Axel Krieger. General-purpose foundation models for increased autonomy in robot-assisted surgery. Nature Machine Intelligence, pp.\ 1--9, 2024

    2024

  26. [34]

    Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs

    Nathaniel Trask, Andy Huang, and Xiaozhe Hu. Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs. Journal of Computational Physics, 456: 0 110969, 2022

    2022

  27. [35]

    Ai feynman: A physics-inspired method for symbolic regression

    Silviu-Marian Udrescu and Max Tegmark. Ai feynman: A physics-inspired method for symbolic regression. Science Advances, 6 0 (16): 0 eaay2631, 2020

    2020

  28. [36]

    Nonlocal attention operator: Materializing hidden knowledge towards interpretable physics discovery

    Yue Yu, Ning Liu, Fei Lu, Tian Gao, Siavash Jafarzadeh, and Stewart Silling. Nonlocal attention operator: Materializing hidden knowledge towards interpretable physics discovery. arXiv preprint arXiv:2408.07307, 2024

    2024 arXiv

  29. [37]

    Trained transformers learn linear models in-context

    Ruiqi Zhang, Spencer Frei, and Peter L Bartlett. Trained transformers learn linear models in-context. Journal of Machine Learning Research, 25 0 (49): 0 1--55, 2024

    2024

  30. [38]

    On numerical integration in neural ordinary differential equations

    Aiqing Zhu, Pengzhan Jin, Beibei Zhu, and Yifa Tang. On numerical integration in neural ordinary differential equations. In International Conference on Machine Learning, pp.\ 27527--27547. PMLR, 2022

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.