Pith. sign in

REVIEW 4 major objections 6 minor 35 references

Mask-PINNs: Mitigating Internal Covariate Shift in Physics-Informed Neural Networks

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A learnable mask stabilizes PINN training and cuts solution error.

desk verdict A simple mask layer that empirically stabilizes PINN training, but the paper's central ICS explanation is not proven by the local Lipschitz analysis. read the letter →

arxiv 2505.06331 v3 pith:PHIQ4TAX submitted 2025-05-09 cs.LG cs.AI

Feilong Jiang , Xiaonan Hou , Jianqiao Ye , Min Xia This is my paper · ORCID
classification cs.LGcs.AI
keywords physics-informedneuralnetworksinternalcovariateshiftlearnablemaskfunctionactivationpartialdifferentialequationsfeaturedistributionstabilizationtangentkernelwide
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Mask-PINNs claims that a single learnable, input-conditioned mask placed at every hidden layer can tame the internal covariate shift that destabilizes physics-informed neural networks, without using the batch or feature statistics that break PINNs' pointwise physical constraints. The mask multiplies each pre-activation by $F(z)=1-\exp(-\alpha^2 z^2)$, a smooth inverted-Gaussian curve that contracts large activations and, close to zero, behaves like $\alpha^2 z^2$, lowering the local sensitivity of the activation. The paper argues this keeps feature distributions centered and compact, and reports consistently lower relative L2 errors than vanilla, ResNet, weight-normalized, LAAF, PirateNet, and ABU baselines on convection, wave, and Helmholtz equations across Tanh, GELU, SiLU, and SoftPlus. It also reports that masked networks keep improving as width grows, where vanilla PINNs degrade. If right, the method is a simple architectural change that widens the usable design space of PINNs.

What carries the argument

The load-bearing object is the mask function $F(u)=1-\exp(-\alpha^2 u^2)$, applied pointwise to pre-activations and multiplied before the activation, so each hidden layer computes $H=F(z)\sigma(z)$. Since $0\le F\le 1$, it globally contracts activation magnitudes; near $u=0$ the Taylor expansion $F(u)=\alpha^2 u^2+O(u^4)$ makes the derivative of the masked activation vanish faster than the bare activation's, which Proposition 1 turns into a reduction of the local minimal Lipschitz constant. That local contraction, combined with the globally bounded magnitude, is the mechanism the paper uses to explain why feature distributions stay centered and compact, and why wider networks do not saturate.

What would settle it

Train a 30-layer Mask-PINN on the convection equation, logging per-layer pre-activation mean, variance, and gradient norms every 1,000 iterations: if deeper layers' distributions drift away from zero or gradient norms collapse while $\alpha$ stays positive, then the local contraction proved near initialization does not guarantee the global stability the paper claims.

Watch

Extended reading notes

Core claim

The paper's central claim is that inserting the mask $H = F(z)\sigma(z)$, with $F(z)=1-\exp(-\alpha^2 z^2)$ and $\alpha$ learnable, into each hidden layer counteracts the drift in pre-activation distributions that makes PINNs hard to train. The mask is pointwise and deterministic, so the PDE constraints, which require derivatives of the network output with respect to inputs, remain intact—unlike batch or layer normalization, which couple outputs across samples or force every feature to the same statistics. The paper proves that near zero the mask reduces the activation's local Lipschitz constant by an order: for activations with $\sigma(0)=0$ the leading term scales as $r^3$ versus $r$ for vanilla, and for $\sigma(0)\neq 0$ as $r^2$ versus $r$. Empirically, Mask-PINN reports the best relative L2 error under every activation on the convection and wave benchmarks and on Helmholtz, and is the only method that avoids failure on that problem for SiLU and SoftPlus. The authors interpret the results as evidence that stabilizing feature distributions is itself a sufficient mechanism for improved PINN training, and they support this with pre-activation histograms, loss-landscape smoothness, and NTK eigenvalue spectra that stay higher and converge to a steady shape earlier than vanilla PINN.

Load-bearing premise

The central assumption is that the mask's provable contraction of activation sensitivity near zero, established locally at initialization, continues to hold over the entire training trajectory and at every depth, so that feature distributions never drift far enough to reintroduce saturation or vanishing gradients.

Editorial extensions

If this is right

  • If the central claim holds, PINNs can be made wider without the usual degradation, because the mask keeps pre-activations in a Gaussian-like band even at large width.
  • Training becomes robust to activation choice: the mask restores competitive accuracy for Tanh, GELU, SiLU, and SoftPlus, including cases where vanilla PINNs fail entirely.
  • The reported NTK analysis implies faster convergence and a stable optimization trajectory, so Mask-PINNs should reach lower loss with fewer iterations in other PDE settings.
  • Because the mask is pointwise and deterministic, it can be dropped into existing PINN code without changing the loss formulation or sampling strategy.
  • The authors suggest the same mask may transfer to other coordinate-based networks such as implicit neural representations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the mask's benefit likely compounds with existing cures for PINN training pathologies—adaptive activations, loss balancing, or residual connections—because it acts on a different failure axis (distribution drift) than those methods.
  • Editorial inference: the mask's dependence on the initialization of $\alpha$, which the paper flags as impactful, suggests a principled initialization rule (e.g. tied to layer width or activation curvature) could be derived; the reported experiments tune $\alpha$ per activation, and a rule would remove that tuning.
  • Editorial inference: the same mechanism may transfer to other coordinate-based MLP tasks such as neural radiance fields or signed distance functions, where internal covariate shift and saturation also limit training, but this is not tested in the paper.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Mask-PINNs, a modification of standard PINN architectures in which a learnable pointwise mask F(z)=1-exp(-alpha^2 z^2) multiplies the activation output at each hidden layer. The mask is intended to reduce internal covariate shift (ICS) while preserving the deterministic, pointwise input-output mapping required by PDE-based losses. The authors provide a local Lipschitz analysis near zero (Proposition 1), then evaluate the method on convection, wave, and Helmholtz equations with Tanh, GELU, SiLU, and SoftPlus activations, reporting consistent relative-L2-error improvements over several baselines. They also report that Mask-PINNs allow wider networks to be trained more effectively and present an NTK eigenvalue-spectrum comparison on Burgers' equation. The central empirical claim is that Mask-PINNs improve accuracy, convergence stability, and robustness across PDE benchmarks and activations.

Significance. If the empirical results hold, Mask-PINNs would be a simple, inexpensive architectural change with broad applicability to PINN training. The paper's strengths are its systematic benchmarking across three PDEs and four activation functions, the straightforward and physically pointwise mechanism, and the demonstration that wider networks remain trainable. The authors also make a genuine attempt at a theoretical justification, although, as detailed below, that justification is only local and does not currently support the global ICS-stability claim. The consistent improvements over strong baselines, even without error bars, make this a useful contribution to the literature.

major comments (4)
  1. [Section 3.2, Proposition 1 and Remark 2] The theoretical support for the central claim is incomplete. Proposition 1 shows only that, as r->0, the local Lipschitz constant of H=F*sigma on [-r,r] is asymptotically smaller than that of sigma. Since F(z)=1-exp(-alpha^2 z^2) saturates to 1 for |z| large relative to 1/alpha, the contraction is confined to a neighborhood of zero. Remark 2 asserts without proof that under Xavier initialization pre-activations concentrate near zero and that this local contraction keeps outputs stable throughout training. No bound on ||z_l(t)|| or on feature distributions over the training trajectory is provided, and no analysis connects the local Lipschitz bound to gradient flow or to the PDE loss landscape. The paper should either supply a global, training-time argument (for example, a mean-field or dynamical-isometry analysis that bounds pre-activation variances under the masked activation) or substantially soften the theoretical claims and present the mechanism as an empirical regularizer.
  2. [Section 4, Tables 1-3] All quantitative results are reported as averages over 5 independent trials, but no standard deviations, confidence intervals, or individual trial values are given. For claims of 'consistent and significant improvements', the absence of variance information is a problem, especially for close entries such as Table 2, GELU row, where Mask-PINN gives 2.90e-3 versus LAAF's 4.37e-3. Please report mean +/- std or provide the per-seed results, and state the random seed handling.
  3. [Section 4.3 and Section 6] The mask scaling parameter alpha is initialized differently across problems and activations: 1.0 for most settings, 5.0 for Tanh and 2.0 for SoftPlus in the Helmholtz experiments. The conclusion explicitly acknowledges that the initial value of alpha has a considerable impact on training dynamics. As alpha controls the sharpness of the mask, the claimed robustness is conditional on this hyperparameter. The authors should provide a sensitivity study over alpha initializations or a principled, task-independent initialization strategy.
  4. [Section 5, Eqs. (44)-(50)] The NTK analysis is not derived for the masked network. The kernel formulas in Eqs. (44)-(50) are written for a generic network and are not specialized to Mask-PINNs; the text does not show how F enters the kernel, and no analytical claim about eigenvalues is proved. The eigenvalue-spectrum plots are empirical observations. Either derive the NTK of the masked architecture and connect its spectral properties to the mask, or explicitly present this section as an empirical diagnostic rather than a theoretical analysis.
minor comments (6)
  1. [Section 3.2, proof of Proposition 1] For the case sigma(0)=0, the expansion of H'(u) appears to be H'(u)=3 alpha^2 sigma'(0) u^2 + O(u^3), not alpha^2 sigma'(0) u^2 + O(u^3); the asymptotic order is unchanged, but the displayed equation should be corrected.
  2. [Remark 2] Remark 2 refers to 'Proposition 2' when the relevant result is Proposition 1; please correct the cross-reference.
  3. [Section 3.3] The claim that placing the mask before the activation function 'can therefore reintroduce ICS' is not supported by an experiment or by a quantitative argument. A small ablation comparing mask-before-activation, mask-after-activation, and the proposed side-by-side placement would make the design choice convincing.
  4. [Section 4, Figures 4-9] The pre-activation distribution figures and variance curves would be more informative with axis labels, units, and error bars or multiple trial overlays; as presented, they are qualitative.
  5. [Section 4.4] The wider-network experiment uses only 3 hidden layers and does not compare against other wide-network methods (e.g., PirateNets or residual architectures). The claim that Mask-PINNs 'enable effective use of wider networks' would be stronger with a comparison to the best-performing baseline at the same widths.
  6. [General] No code or data availability statement is provided. Releasing the implementation and random seeds would substantially improve reproducibility of the reported five-trial averages.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Mask-PINNs is proposed a priori and validated on external PDE benchmarks.

full rationale

The derivation chain of Mask-PINNs is self-contained rather than circular. The mask function F(z)=1-exp(-alpha^2 z^2) is introduced as a novel architectural component (Eq. 15–17) without being defined in terms of the ICS measure it is claimed to mitigate. Proposition 1 derives a local Lipschitz contraction from Taylor expansions of F and sigma; this is a genuine mathematical statement, not a restatement of the empirical results. The empirical claims are tested on convection, wave, and Helmholtz equations against vanilla PINNs, ResNet, WN, LAAF, PirateNets, and ABU-PINN, with prediction errors reported on held-out analytical solutions. The per-activation choices of alpha initialization (e.g., 5.0 for Tanh in Helmholtz) are hyperparameter settings and are openly acknowledged as sensitivity in the conclusion; they do not constitute fitting a parameter to the reported test errors. The only self-citation, Ref. [17] by the authors, is used as related work on deeper PINNs, but the width-degradation motivation is independently demonstrated in Fig. 8 by comparing vanilla PINNs and Mask-PINNs across widths, so the self-citation is not load-bearing. The theoretical gap that the local Lipschitz analysis does not by itself prove global training-time stability is a limitation of the supporting argument, but it is not circular reasoning: the mask mechanism is proposed before the analysis, and the analysis is not derived from the desired empirical outcome. No fitted input is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known result is merely relabeled.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on an unproven link between local activation-function properties and global training stability, as well as on hand-tuned α initializations. No new physical entities are introduced.

free parameters (1)
  • α initialization = 1.0 for convection/wave; for Helmholtz: 1.0 (GELU/SiLU), 5.0 (Tanh), 2.0 (SoftPlus)
    The mask scaling parameter α is learned during training, but its initialization is chosen by hand per benchmark and activation function. The paper admits in the Conclusion that α initialization has a considerable impact on training dynamics, making it a tuned experimental choice rather than a principled parameter.
assumptions (4)
  • standard math Smoothness and differentiability of activation functions and the mask for Taylor expansions
    Proposition 1 in Section 3.2 uses local Taylor expansions of F and σ near 0, requiring smoothness.
  • domain assumption Xavier initialization concentrates pre-activations near zero
    Remark 2 in Section 3.2 invokes this to apply the local Lipschitz contraction to the training regime, but no empirical check of this concentration is provided for the trained networks.
  • ad hoc to paper Local sensitivity reduction near zero implies global mitigation of internal covariate shift during training
    The paper does not prove that the pointwise contraction of the Lipschitz constant translates into bounded feature distributions over training iterations and network depth. This is the unproved causal bridge between the theoretical analysis and the empirical claims.
  • domain assumption Batch and layer normalization inherently violate pointwise physical consistency in PINNs
    Section 2.2 argues that BN and LN mix information across samples or features, which is problematic for derivative-based PINN losses. The paper demonstrates this empirically for one heat conduction problem but does not provide a general proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mask-PINNs: Mitigating Internal Covariate Shift in Physics-Informed Neural Networks." pith.science (2026). https://pith.science/paper/PHIQ4TAX

@misc{pith2026250506331,
  author       = {Pith},
  title        = {Pith review of: Mask-PINNs: Mitigating Internal Covariate Shift in Physics-Informed Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PHIQ4TAX}},
  note         = {Machine review of arXiv:2505.06331}
}
read the original abstract

Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical laws directly into the loss function. However, as a fundamental optimization issue, internal covariate shift (ICS) hinders the stable and effective training of PINNs by disrupting feature distributions and limiting model expressiveness. Unlike standard deep learning tasks, conventional remedies for ICS -- such as Batch Normalization and Layer Normalization -- are not directly applicable to PINNs, as they distort the physical consistency required for reliable PDE solutions. To address this issue, we propose Mask-PINNs, a novel architecture that introduces a learnable mask function to regulate feature distributions while preserving the underlying physical constraints of PINNs. We provide a theoretical analysis showing that the mask suppresses the expansion of feature representations through a carefully designed modulation mechanism. Empirically, we validate the method on multiple PDE benchmarks -- including convection, wave propagation, and Helmholtz equations -- across diverse activation functions. Our results show consistent improvements in prediction accuracy, convergence stability, and robustness. Furthermore, we demonstrate that Mask-PINNs enable the effective use of wider networks, overcoming a key limitation in existing PINN frameworks.

Figures

Figures reproduced from arXiv: 2505.06331 by the authors.

Figure 2
Figure 2. Illustration of the architecture of the Mask-PINNs. Given an input vector x, a mask function can be expressed as: ( ) ( ) 2 F = − − 1 exp   x αx (15) where α is a learnable vector that controls the sharpness of the mask function. α is initialized as a positive vector [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Mask function with different α. With the mask inserted, the feed-forward process of layer l can be expressed as: ( ) ( ) ( ) , l l l z w H b =+ (16) ( ) ( ) ( ) ( ) ( ) ( ), l l l l H F z z =  (17) mbe ing nputs [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Convection Equation: pre-activation distributions of the vanilla and proposed methods [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: 1D wave propagation equation: variances of pre-activations of vanilla and proposed methods with Tanh activation. 4.3 Helmholtz Equation We consider the Helmholtz equation expressed as below: ( )     22 2 22 , 0, 1, 1 , 1, 1 , uu k u q x y x y xy  + + − =  −  − …
Figure 6
Figure 6. Figure 6: Helmholtz equation: averaged loss curves of different models with GELU as activation function [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Helmholtz equation: loss landscapes of vanilla method and the proposed method with GELU as activation function. To further explore the reason behind such difference, [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Relative L 2 errors of vanilla PINNs and the proposed method across various network widths. To further explore the underlying reasons behind this improvement, we present the pre￾activation distributions of the vanilla and proposed methods in [PITH_FULL_IMAGE:figures/f…
Figure 10
Figure 10. Figure 10: Evolution of the NTK eigenvalue spectra during training [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Loss and relative L 2 error curves during training. 6 Conclusion In this work, we focus on internal covariate shift, a well-known problem in deep learning that remains unresolved in PINNs and significantly undermines training stability and solution accuracy. We propos…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 29 canonical work pages

  1. [1]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,

    M. Raissi, P. Perdikaris, and G. E. Karniadakis, "Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations," Journal of Computational physics, vol. 378, pp. 686-707, 2019

    work page 2019

  2. [2]

    Physics guided RNNs for modeling dynamical systems: A case study in simulating lake temperature profiles,

    X. Jia et al., "Physics guided RNNs for modeling dynamical systems: A case study in simulating lake temperature profiles," in Proceedings of the 2019 SIAM international conference on data mining, 2019: SIAM, pp. 558-566

    work page 2019

  3. [3]

    Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations,

    M. Raissi, A. Yazdani, and G. E. Karniadakis, "Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations," Science, vol. 367, no. 6481, pp. 1026-1030, 2020

    work page 2020

  4. [4]

    Artificial intelligence velocimetry and microaneurysm -on-a-chip for three-dimensional analysis of blood flow in physiology and disease,

    S. Cai et al. , "Artificial intelligence velocimetry and microaneurysm -on-a-chip for three-dimensional analysis of blood flow in physiology and disease," Proceedings of the National Academy of Sciences, vol. 118, no. 13, p. e2100697118, 2021

    work page 2021

  5. [5]

    Three -dimensional spatiotemporal wind field reconstruction based on physics-informed deep learning,

    J. Zhang and X. Zhao, "Three -dimensional spatiotemporal wind field reconstruction based on physics-informed deep learning," Applied Energy, vol. 300, p. 117390, 2021

    work page 2021

  6. [6]

    Physics -informed neural networks (PINNs) for fluid mechanics: A review,

    S. Cai, Z. Mao, Z. Wang, M. Yin, and G. E. Karniadakis, "Physics -informed neural networks (PINNs) for fluid mechanics: A review," Acta Mechanica Sinica, vol. 37, no. 12, pp. 1727-1738, 2021

    work page 2021

  7. [7]

    Physics -informed neural networks for heat transfer problems,

    S. Cai, Z. Wang, S. Wang, P. Perdikaris, and G. E. Karniadakis, "Physics -informed neural networks for heat transfer problems," Journal of Heat Transfer, vol. 143, no. 6, p. 060801, 2021

    work page 2021

  8. [8]

    Physics -informed Neural Networks (PINN) for computational solid mechanics: Numerical frameworks and applications,

    H. Hu, L. Qi, and X. Chao, "Physics -informed Neural Networks (PINN) for computational solid mechanics: Numerical frameworks and applications," Thin- Walled Structures, p. 112495, 2024

    work page 2024

Show all 35 references
  1. [9]

    Physics-Informed neural network solver for numerical analysis in geoengineering,

    X.-X. Chen, P. Zhang, and Z. -Y . Yin, "Physics-Informed neural network solver for numerical analysis in geoengineering," Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards, vol. 18, no. 1, pp. 33-51, 2024

    2024

  2. [10]

    The application of physics - informed machine learning in multiphysics modeling in chemical engineering,

    Z. Wu, H. Wang, C. He, B. Zhang, T. Xu, and Q. Chen, "The application of physics - informed machine learning in multiphysics modeling in chemical engineering," Industrial & Engineering Chemistry Research, vol. 62, no. 44, pp. 18178-18204, 2023

    2023

  3. [11]

    The application of improved physics -informed neural network (IPINN) method in finance,

    Y . Bai, T. Chaolu, and S. Bilige, "The application of improved physics -informed neural network (IPINN) method in finance," Nonlinear Dynamics, vol. 107, no. 4, pp. 3655-3667, 2022

    2022

  4. [12]

    Physics-informed machine learning: case studies for weather and climate modelling,

    K. Kashinath et al., "Physics-informed machine learning: case studies for weather and climate modelling," Philosophical Transactions of the Royal Society A, vol. 379, no. 2194, p. 20200093, 2021

    2021

  5. [13]

    Learning Specialized Activation Functions for Physics-Informed Neural Networks,

    H. Wang, L. Lu, and G. Huang, "Learning Specialized Activation Functions for Physics-Informed Neural Networks," Communications in Computational Physics, vol. 34, no. 4, pp. 869-906, 2023

    2023

  6. [14]

    Simple yet effective adaptive activation functions for physics - informed neural networks,

    J. Zhang and C. Ding, "Simple yet effective adaptive activation functions for physics - informed neural networks," Computer Physics Communications, vol. 307, p. 109428, 2025

    2025

  7. [15]

    PirateNets: Physics -informed Deep Learning with Residual Adaptive Networks,

    S. Wang, B. Li, Y . Chen, and P. Perdikaris, "PirateNets: Physics -informed Deep Learning with Residual Adaptive Networks," arXiv preprint arXiv:2402.00326, 2024

    2024 arXiv

  8. [16]

    HyResPINNs: Adaptive Hybrid Residual Networks for Learning Optimal Combinations of Neural and RBF Components for Physics -Informed Modeling,

    M. Cooley, R. M. Kirby, S. Zhe, and V . Shankar, "HyResPINNs: Adaptive Hybrid Residual Networks for Learning Optimal Combinations of Neural and RBF Components for Physics -Informed Modeling," arXiv preprint arXiv:2410.03573, 2024

  9. [17]

    Element -wise multiplication based deeper physics - informed neural networks,

    F. Jiang, X. Hou, and M. Xia, "Element -wise multiplication based deeper physics - informed neural networks," arXiv preprint arXiv:2406.04170, 2024

    2024 arXiv

  10. [18]

    Implicit neural representations with periodic activation functions,

    V . Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, "Implicit neural representations with periodic activation functions," Advances in neural information processing systems, vol. 33, pp. 7462-7473, 2020

    2020

  11. [19]

    Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis,

    H. woo Lee, H. Choi, and H. Kim, "Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis," in The Thirteenth International Conference on Learning Representations

  12. [20]

    Multi -objective loss balancing for physics -informed deep learning,

    R. Bischof and M. A. Kraus, "Multi -objective loss balancing for physics -informed deep learning," Computer Methods in Applied Mechanics and Engineering, vol. 439, p. 117914, 2025

    2025

  13. [21]

    Config: Towards conflict -free training of physics informed neural networks,

    Q. Liu, M. Chu, and N. Thuerey, "Config: Towards conflict -free training of physics informed neural networks," arXiv preprint arXiv:2408.11104, 2024

    2024 arXiv

  14. [22]

    Gradient Alignment in Physics - informed Neural Networks: A Second -Order Optimization Perspective,

    S. Wang, A. K. Bhartari, B. Li, and P. Perdikaris, "Gradient Alignment in Physics - informed Neural Networks: A Second -Order Optimization Perspective," arXiv preprint arXiv:2502.00604, 2025

    2025

  15. [23]

    Understanding and mitigating gradient flow pathologies in physics -informed neural networks,

    S. Wang, Y . Teng, and P. Perdikaris, "Understanding and mitigating gradient flow pathologies in physics -informed neural networks," SIAM Journal on Scientific Computing, vol. 43, no. 5, pp. A3055-A3081, 2021

    2021

  16. [24]

    PhyGeoNet: Physics -informed geometry-adaptive convolutional neural networks for solving parameterized steady -state PDEs on irregular domain,

    H. Gao, L. Sun, and J. -X. Wang, "PhyGeoNet: Physics -informed geometry-adaptive convolutional neural networks for solving parameterized steady -state PDEs on irregular domain," Journal of Computational Physics, vol. 428, p. 110079, 2021

    2021

  17. [25]

    A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks,

    K. Shukla, J. D. Toscano, Z. Wang, Z. Zou, and G. E. Karniadakis, "A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks," Computer Methods in Applied Mechanics and Engineering, vol. 431, p. 117290, 2024

    2024

  18. [26]

    f -PICNN: a physics -informed convolutional neural network for partial differential equations with space -time domain,

    B. Yuan, H. Wang, A. Heitor, and X. Chen, "f -PICNN: a physics -informed convolutional neural network for partial differential equations with space -time domain," Journal of Computational Physics, vol. 515, p. 113284, 2024

    2024

  19. [27]

    Batch normalization: accelerating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, "Batch normalization: accelerating deep network training by reducing internal covariate shift," presented at the Proceedings of the 32nd International Conference on International Conference on Machine Learning - V olume 37, Lille, France, 2015

    2015

  20. [28]

    Activation functions: Comparison of trends in practice and research for deep learning,

    C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall, "Activation functions: Comparison of trends in practice and research for deep learning," arXiv preprint arXiv:1811.03378, 2018

    2018 arXiv

  21. [29]

    Mathematical analysis and performance evaluation of the gelu activation function in deep learning,

    M. Lee, "Mathematical analysis and performance evaluation of the gelu activation function in deep learning," Journal of Mathematics, vol. 2023, no. 1, p. 4229924, 2023

    2023

  22. [30]

    Activation functions in deep learning: A comprehensive survey and benchmark,

    S. R. Dubey, S. K. Singh, and B. B. Chaudhuri, "Activation functions in deep learning: A comprehensive survey and benchmark," Neurocomputing, vol. 503, pp. 92 -108, 2022

    2022

  23. [31]

    Layer normalization,

    J. L. Ba, "Layer normalization," arXiv preprint arXiv:1607.06450, 2016

    2016 arXiv

  24. [32]

    Weight normalization: A simple reparameterization to accelerate training of deep neural networks,

    T. Salimans and D. P. Kingma, "Weight normalization: A simple reparameterization to accelerate training of deep neural networks," Advances in neural information processing systems, vol. 29, 2016

    2016

  25. [33]

    Locally adaptive activation functions with slope recovery for deep and physics -informed neural networks,

    A. D. Jagtap, K. Kawaguchi, and G. Em Karniadakis, "Locally adaptive activation functions with slope recovery for deep and physics -informed neural networks," Proceedings of the Royal Society A, vol. 476, no. 2239, p. 20200334, 2020

    2020

  26. [34]

    Physics -informed neural networks for mesh deformation with exact boundary enforcement,

    A. Aygun, R. Maulik, and A. Karakus, "Physics -informed neural networks for mesh deformation with exact boundary enforcement," Engineering Applications of Artificial Intelligence, vol. 125, p. 106660, 2023

    2023

  27. [35]

    Deepsdf: Learning continuous signed distance functions for shape representation,

    J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, "Deepsdf: Learning continuous signed distance functions for shape representation," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 165-174

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.