pith. sign in

arxiv: 2604.21761 · v1 · submitted 2026-04-23 · 💻 cs.LG · cs.CE· physics.comp-ph

Transferable Physics-Informed Representations via Closed-Form Head Adaptation

Jian Cheng Wong , Isaac Yin Chung Lai , Pao-Hsiung Chiu , Chin Chun Ooi , Abhishek Gupta , Yew-Soon Ong This is my paper

Pith reviewed 2026-05-09 22:15 UTC · model grok-4.3

classification 💻 cs.LG cs.CEphysics.comp-ph
keywords physics-informed neural networksPINNstransferable representationspseudoinverse adaptationpartial differential equationsmulti-task learninggeneralizationclosed-form head adaptation
0
0 comments X

The pith

Pi-PINN learns a shared physics-informed embedding space that lets new PDEs be solved by closed-form pseudoinverse adaptation of the output head without retraining or new data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Pi-PINN to address the slow per-instance training and poor generalization of standard physics-informed neural networks. It trains a backbone on multiple PDEs to build a transferable representation in one embedding space, then solves both familiar and entirely new PDEs by adapting only the final linear head through a single least-squares pseudoinverse step that enforces the PDE constraints. This combination yields predictions 100-1000 times faster than conventional PINN training while producing lower error than purely data-driven models even when given only two samples. The work also examines how to combine multi-task data-driven losses with physics-informed losses to improve overall accuracy across equation families.

Core claim

Pi-PINN learns a transferable physics-informed representation in a shared embedding space from training PDEs and enables rapid solving of both known and unknown PDE instances via closed-form head adaptation using a least-squares-optimal pseudoinverse under PDE constraints, demonstrated on Poisson, Helmholtz, and Burgers equations with no data required for unseen cases.

What carries the argument

Shared embedding space plus least-squares-optimal pseudoinverse for closed-form adaptation of the linear output head that satisfies PDE residual constraints.

If this is right

  • New PDE instances can be solved accurately without collecting any data or performing gradient-based retraining.
  • Prediction speed increases by 100-1000 times relative to training a fresh PINN for each equation.
  • Relative error remains 10-100 times lower than data-driven models even when only two training samples are available.
  • Combining multi-task data-driven loss with physics-informed loss yields measurable gains in accuracy and transfer performance.
  • The same backbone supports both interpolation among seen PDEs and extrapolation to unseen ones.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could reduce repeated full retraining costs in engineering workflows that solve families of related PDEs for design or optimization.
  • Similar closed-form head adaptation might transfer to other neural architectures that embed physical constraints, such as neural operators or reduced-order models.
  • Limits of the shared space could be probed by testing transfer across PDEs with increasing differences in order, nonlinearity, or dimensionality.

Load-bearing premise

The embedding space learned from a finite set of training PDEs remains general enough that pseudoinverse adaptation produces accurate solutions for completely new PDE structures without any additional data or retraining.

What would settle it

Train the model on Poisson and Helmholtz equations, then apply it without retraining or samples to a Burgers equation with altered viscosity or boundary conditions and measure whether relative error stays below the levels reported for standard PINNs.

Figures

Figures reproduced from arXiv: 2604.21761 by Abhishek Gupta, Chin Chun Ooi, Isaac Yin Chung Lai, Jian Cheng Wong, Pao-Hsiung Chiu, Yew-Soon Ong.

Figure 1
Figure 1. Figure 1: The Schematic diagram of the fast Pseudoinverse PINN framework, with the example PDE instances (Burgers’ equation). [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of baseline MLP and different Pi-PINN approaches for predicting or solving different PDE problems. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The solution of different PDE instances of the Poisson equation [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 2
Figure 2. Figure 2: A representative plot of the solutions for Poisson’s [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: The predicted results and corresponding errors for the Helmholtz [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The predicted results and corresponding errors for the nonlinear Burger’s equation (sine IC), for MLP and PiL-PINN with different PDE instances [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The predicted results and corresponding errors for the Burgers’ equation (family of IC) on new PDE instances, for MLP, MLP [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
read the original abstract

Physics-informed neural networks (PINNs) have garnered significant interest for their potential in solving partial differential equations (PDEs) that govern a wide range of physical phenomena. By incorporating physical laws into the learning process, PINN models have demonstrated the ability to learn physical outcomes reasonably well. However, current PINN approaches struggle to predict or solve new PDEs effectively when there is a lack of training examples, indicating they do not generalize well to unseen problem instances. In this paper, we present a transferable learning approach for PINNs premised on a fast Pseudoinverse PINN framework (Pi-PINN). Pi-PINN learns a transferable physics-informed representation in a shared embedding space and enables rapid solving of both known and unknown PDE instances via closed-form head adaptation using a least-squares-optimal pseudoinverse under PDE constraints. We further investigate the synergies between data-driven multi-task learning loss and physics-informed loss, providing insights into the design of more performant PINNs. We demonstrate the effectiveness of Pi-PINN on various PDE problems, including Poisson's equation, Helmholtz equation, and Burgers' equation, achieving fast and accurate physics-informed solutions without requiring any data for unseen instances. Pi-PINN can produce predictions 100-1000 times faster than a typical PINN, while producing predictions with 10-100 times lower relative error than a typical data-driven model even with only two training samples. Overall, our findings highlight the potential of transferable representations with closed-form head adaptation to enhance the efficiency and generalization of PINNs across PDE families and scientific and engineering applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces Pi-PINN, which learns a shared physics-informed representation across multiple PDE families (Poisson, Helmholtz, Burgers) in an embedding space. It then performs rapid, closed-form adaptation of a linear head via least-squares pseudoinverse under the target PDE operator and boundary conditions, claiming accurate solutions for both seen and entirely unseen PDE instances with no additional collocation data or retraining. Experiments report 100-1000x speedups over standard PINNs and 10-100x lower error than data-driven baselines even with minimal samples.

Significance. If the generalization result holds, the approach would represent a meaningful advance in efficient, transferable PINN solvers by decoupling representation learning from per-instance optimization. The exploration of synergies between multi-task data-driven losses and physics-informed losses could also inform broader multi-task scientific ML design. The absence of a universal-approximation or completeness argument for the embedding, however, leaves the strongest claims dependent on empirical validation within the tested PDE distribution.

major comments (2)
  1. [Methods] Methods (pseudoinverse head adaptation): The central claim that the closed-form pseudoinverse yields accurate solutions for unseen PDE instances without data assumes the learned embedding is sufficiently complete that its span contains the target solution; no error bound, completeness argument, or analysis of the distance of the target operator from the training distribution is supplied, making the data-free generalization assertion load-bearing yet unsupported.
  2. [Experiments] Experiments: The reported speed and error gains on the three PDEs are presented without error bars, ablation on training-PDE diversity or count, or explicit controls for hyperparameter choices in the shared embedding stage; this leaves the support for the 'no data needed for unseen instances' claim only moderately convincing and risks confounding embedding quality with adaptation success.
minor comments (2)
  1. [Abstract] Abstract: The comparison '10-100 times lower relative error than a typical data-driven model even with only two training samples' should specify the exact architecture and training regime of the data-driven baseline for reproducibility.
  2. Notation: The symbols for the shared embedding network, the pseudoinverse matrix, and the PDE residual operator should be defined once and used consistently to avoid ambiguity when describing the adaptation step.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive feedback. We address each major comment below and describe the revisions planned to improve clarity and rigor.

read point-by-point responses

  1. Referee: [Methods] Methods (pseudoinverse head adaptation): The central claim that the closed-form pseudoinverse yields accurate solutions for unseen PDE instances without data assumes the learned embedding is sufficiently complete that its span contains the target solution; no error bound, completeness argument, or analysis of the distance of the target operator from the training distribution is supplied, making the data-free generalization assertion load-bearing yet unsupported.

    Authors: We acknowledge that the manuscript is primarily empirical and does not supply a formal error bound or completeness proof for the embedding. The data-free adaptation claim rests on the empirical observation that the learned representation enables low-residual solutions for operators both inside and outside the training set. In revision we will add a dedicated limitations and assumptions subsection that (i) explicitly states the span assumption, (ii) provides an empirical analysis of embedding expressivity via residual norms and singular-value spectra on held-out operators, and (iii) discusses the distance of test operators from the training distribution. These additions will make the scope of the generalization claims transparent without altering the core empirical contribution. revision: partial

  2. Referee: [Experiments] Experiments: The reported speed and error gains on the three PDEs are presented without error bars, ablation on training-PDE diversity or count, or explicit controls for hyperparameter choices in the shared embedding stage; this leaves the support for the 'no data needed for unseen instances' claim only moderately convincing and risks confounding embedding quality with adaptation success.

    Authors: We agree that additional controls and statistical reporting are warranted. In the revised manuscript we will (i) report mean and standard deviation over at least five independent runs with different random seeds for all quantitative results, (ii) include ablations that vary both the number and the diversity of training PDE families, and (iii) add a hyperparameter sensitivity study for the shared embedding stage (learning rate, embedding dimension, loss weighting). These experiments will isolate the contribution of the closed-form adaptation from embedding quality and strengthen support for the data-free claim. revision: yes

standing simulated objections not resolved
  • A formal universal-approximation or completeness theorem for the learned embedding space; developing such a result would require substantial new theoretical analysis outside the empirical scope of the present work.

Circularity Check

0 steps flagged

No significant circularity in Pi-PINN derivation chain

full rationale

The paper's core method learns a shared embedding representation from a finite collection of training PDEs (Poisson, Helmholtz, Burgers) via combined data-driven multi-task and physics-informed losses, then applies a mathematically distinct closed-form adaptation step for new instances. This adaptation uses the fixed embedding and a least-squares pseudoinverse constrained solely by the target PDE operator, without refitting or additional data. No step reduces by construction to its inputs: the embedding training and pseudoinverse head solve are sequential and independent, the pseudoinverse is a standard linear algebra operation, and generalization claims rest on empirical demonstrations rather than self-definition, fitted-input renaming, or load-bearing self-citations. The derivation remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The approach rests on standard neural-network approximation capabilities for PDEs and introduces a new representation-learning step whose generality is asserted rather than derived from first principles.

axioms (1)
  • domain assumption Neural networks can approximate solutions to PDEs when the loss includes the PDE residual
    Core premise of all PINN methods invoked throughout the abstract.
invented entities (1)
  • Transferable physics-informed representation in shared embedding space no independent evidence
    purpose: To enable rapid closed-form adaptation to new PDE instances
    Central new construct introduced by the Pi-PINN framework; no independent falsifiable evidence provided in the abstract.

pith-pipeline@v0.9.0 · 5605 in / 1362 out tokens · 39569 ms · 2026-05-09T22:15:11.909064+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,

    M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” Journal of Computational physics, vol. 378, pp. 686–707, 2019

    work page 2019

  2. [2]

    Physics-informed machine learning,

    G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,”Nature Reviews Physics, vol. 3, no. 6, pp. 422–440, 2021

    work page 2021

  3. [3]

    Spline-pinn: Approaching pdes without data using fast, physics-informed hermite- spline cnns,

    N. Wandel, M. Weinmann, M. Neidlin, and R. Klein, “Spline-pinn: Approaching pdes without data using fast, physics-informed hermite- spline cnns,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 8, 2022, pp. 8529–8538

    work page 2022

  4. [4]

    Dpm: A novel training method for physics-informed neural networks in extrapolation,

    J. Kim, K. Lee, D. Lee, S. Y . Jhin, and N. Park, “Dpm: A novel training method for physics-informed neural networks in extrapolation,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9, 2021, pp. 8146–8154

    work page 2021

  5. [5]

    Pixel: Physics- informed cell representations for fast and accurate pde solvers,

    N. Kang, B. Lee, Y . Hong, S.-B. Yun, and E. Park, “Pixel: Physics- informed cell representations for fast and accurate pde solvers,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 7, 2023, pp. 8186–8194

    work page 2023

  6. [6]

    Dmis: Dynamic mesh-based importance sampling for training physics-informed neural networks,

    Z. Yang, Z. Qiu, and D. Fu, “Dmis: Dynamic mesh-based importance sampling for training physics-informed neural networks,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 4, 2023, pp. 5375–5383

    work page 2023

  7. [7]

    Learning in sinusoidal spaces with physics-informed neural networks,

    J. C. Wong, C. Ooi, A. Gupta, and Y .-S. Ong, “Learning in sinusoidal spaces with physics-informed neural networks,”IEEE Transactions on Artificial Intelligence, 2022

    work page 2022

  8. [8]

    Characterizing possible failure modes in physics-informed neural net- works,

    A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, and M. W. Mahoney, “Characterizing possible failure modes in physics-informed neural net- works,”Advances in Neural Information Processing Systems, vol. 34, pp. 26 548–26 560, 2021

    work page 2021

  9. [9]

    Understanding and mitigating gradient flow pathologies in physics-informed neural networks,

    S. Wang, Y . Teng, and P. Perdikaris, “Understanding and mitigating gradient flow pathologies in physics-informed neural networks,”SIAM Journal on Scientific Computing, vol. 43, no. 5, pp. A3055–A3081, 2021

    work page 2021

  10. [10]

    Scientific machine learning through physics–informed neural networks: Where we are and what’s next,

    S. Cuomo, V . S. Di Cola, F. Giampaolo, G. Rozza, M. Raissi, and F. Piccialli, “Scientific machine learning through physics–informed neural networks: Where we are and what’s next,”Journal of Scientific Computing, vol. 92, no. 3, p. 88, 2022

    work page 2022

  11. [11]

    Evolutionary optimization of physics-informed neural networks: Evo-pinn frontiers and opportunities,

    J. C. Wong, A. Gupta, C. C. Ooi, P.-H. Chiu, J. Liu, and Y .-S. Ong, “Evolutionary optimization of physics-informed neural networks: Evo-pinn frontiers and opportunities,”IEEE Computational Intelligence Magazine, vol. 21, no. 1, pp. 16–36, 2026

    work page 2026

  12. [12]

    Can transfer neuroevolution tractably solve your differential equations?

    J. C. Wong, A. Gupta, and Y .-S. Ong, “Can transfer neuroevolution tractably solve your differential equations?”IEEE Computational Intel- ligence Magazine, vol. 16, no. 2, pp. 14–30, 2021

    work page 2021

  13. [13]

    Transfer learning enhanced physics informed neural network for phase-field modeling of fracture,

    S. Goswami, C. Anitescu, S. Chakraborty, and T. Rabczuk, “Transfer learning enhanced physics informed neural network for phase-field modeling of fracture,”Theoretical and Applied Fracture Mechanics, vol. 106, p. 102447, 2020

    work page 2020

  14. [14]

    Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equa- tions,

    S. Dong and Z. Li, “Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equa- tions,”Computer Methods in Applied Mechanics and Engineering, vol. 387, p. 114129, 2021

    work page 2021

  15. [15]

    Extreme learning machine col- location for the numerical solution of elliptic pdes with sharp gradients,

    F. Calabr `o, G. Fabiani, and C. Siettos, “Extreme learning machine col- location for the numerical solution of elliptic pdes with sharp gradients,” Computer Methods in Applied Mechanics and Engineering, vol. 387, p. 114188, 2021

    work page 2021

  16. [16]

    On best approximate solutions of linear matrix equations,

    R. Penrose, “On best approximate solutions of linear matrix equations,” Mathematical Proceedings of the Cambridge Philosophical Society, vol. 52, pp. 17–19, 1956

    work page 1956

  17. [17]

    On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks,

    S. Wang, H. Wang, and P. Perdikaris, “On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks,”Computer Methods in Applied Mechanics and Engineering, vol. 384, p. 113938, 2021

    work page 2021

  18. [18]

    Evolutionary optimization of physics-informed neural networks: Advancing generalizability by the baldwin effect,

    J. C. Wong, C. C. Ooi, A. Gupta, P.-H. Chiu, J. S. Z. Low, M. H. Dao, and Y .-S. Ong, “Evolutionary optimization of physics-informed neural networks: Advancing generalizability by the baldwin effect,” IEEE Transactions on Evolutionary Computation, 2026

    work page 2026

  19. [19]

    Revisiting unreasonable effectiveness of data in deep learning era,

    C. Sun, A. Shrivastava, S. Singh, and A. Gupta, “Revisiting unreasonable effectiveness of data in deep learning era,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 843–852

    work page 2017

  20. [20]

    Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth

    T. Nguyen, M. Raghu, and S. Kornblith, “Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth,”arXiv preprint arXiv:2010.15327, 2020

    work page arXiv 2010

  21. [21]

    Nerfies: Deformable neural radiance fields,

    K. Park, U. Sinha, J. T. Barron, S. Bouaziz, D. B. Goldman, S. M. Seitz, and R. Martin-Brualla, “Nerfies: Deformable neural radiance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5865–5874

    work page 2021

  22. [22]

    L-hydra: Multi-head physics-informed neural networks

    Z. Zou and G. E. Karniadakis, “L-hydra: Multi-head physics-informed neural networks,”arXiv preprint arXiv:2301.02152, 2023

    work page arXiv 2023

  23. [23]

    One-shot transfer learn- ing of physics-informed neural networks.arXiv preprint arXiv:2110.11286, 2021

    S. Desai, M. Mattheakis, H. Joy, P. Protopapas, and S. Roberts, “One- shot transfer learning of physics-informed neural networks,”arXiv preprint arXiv:2110.11286, 2021

    work page arXiv 2021

  24. [24]

    A novel meta- learning initialization method for physics-informed neural networks,

    X. Liu, X. Zhang, W. Peng, W. Zhou, and W. Yao, “A novel meta- learning initialization method for physics-informed neural networks,” Neural Computing and Applications, vol. 34, no. 17, pp. 14 511–14 534, 2022

    work page 2022

  25. [25]

    Hypernetwork-based meta- learning for low-rank physics-informed neural networks,

    W. Cho, K. Lee, D. Rim, and N. Park, “Hypernetwork-based meta- learning for low-rank physics-informed neural networks,”Advances in Neural Information Processing Systems, vol. 36, 2024

    work page 2024

  26. [26]

    An expert's guide to training physics-informed neural networks

    S. Wang, S. Sankaran, H. Wang, and P. Perdikaris, “An expert’s guide to training physics-informed neural networks,”arXiv preprint arXiv:2308.08468, 2023

    work page arXiv 2023