Pith. sign in

REVIEW 2 major objections 3 minor 94 references

Continual-Learning Physics-Informed Neural Networks for Parameterized Partial Differential Equations

T0 review · 2 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Treating each PDE parameter value as a task learned sequentially — with a Bayesian-optimized order and sparse physics replay — lets a single network cover a whole parameter family at lower query cost and higher accuracy than…

desk verdict Genuinely useful and unusually honest ParamPINN framework paper; central claim holds, but the external-method comparison is weaker than the within-case ablations. read the letter →

arxiv 2608.04778 v1 pith:LH7Q7LLN submitted 2026-08-05 cs.LG

Xujia Chen , Xinyue Hu , Letian Chen , Yi Liu , Wenhui Fan This is my paper
classification cs.LG
keywords physics-informedneuralnetworksparameterizedpartialdifferentialequationscontinuallearningBayesianoptimizationexperiencereplayactiveparameterselectioncatastrophicforgettingdynamiclossweighting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes CL-PINN, a method for training one physics-informed neural network to solve a whole family of PDEs across a range of physical parameter values. Instead of sampling parameters uniformly or scanning a fixed grid, it treats each parameter value as a task in a continual-learning sequence: Bayesian optimization decides which parameter to add next, dynamic weighting rebalances tasks whose losses shrink at different rates, and a sparse replay buffer keeps a few physics constraints from displaced older tasks so that earlier solutions are not forgotten. The central claim is that, under the prescribed per-case resource protocols, this schedule gives more accurate and more balanced solutions than uniform, fixed-set, and grid-greedy baselines while using 26.0% to 91.9% fewer objective-loss queries than grid-greedy search. The paper also shows that a frozen base model plus a compact residual head (ACR2-finetune) improves local accuracy at unseen parameters within tens of seconds on three of four PDEs, and it is explicit that every component's benefit is equation-dependent.

What carries the argument

The carrying mechanism is the continual-learning loop over parameter tasks, which rests on a stated regularity premise: the parameter-to-solution map is continuous with bounded rate of variation, so adjacent tasks have correlated physics losses that a Gaussian-process surrogate can exploit. Inside the loop, three components do the work. Bayesian optimization fits a GP to evaluated task-level losses and selects the next parameter by upper-confidence-bound acquisition, limiting expensive loss evaluations. Dynamic task weighting — combining each task's normalized current loss with its recent relative loss decrease — redistributes optimization effort among admitted tasks. Sparse physics-constrained experience replay retains displaced tasks as a parameter value plus a small set of physical coordinates and recomputes their residual losses during later training, so memory stays bounded and earlier tasks are not forgotten. An optional parameter subnetwork separates parameter encoding from coordinate fitting, and ACR2-finetune appends a zero-initialized one-hidden-layer residual head to the frozen global model for rapid local adaptation at a requested parameter.

What would settle it

Run CL-PINN on a parameterized PDE family whose solution map is discontinuous or bifurcates inside the parameter domain (for example, across a shock-formation threshold), and measure the rank correlation between GP-predicted task losses and the actually evaluated losses at the candidate parameters; if the correlation falls to chance, the regularity premise is violated and the Bayesian selector should perform no better than uniform random task admission.

Watch

Extended reading notes

Core claim

The paper's central claim is that training a parameterized PINN is fundamentally a task-allocation problem: with a hard cap on the number of actively trained parameter tasks, the order in which tasks are admitted, the weights they receive, and the memory kept of displaced tasks determine how well the single model generalizes across the parameter domain. CL-PINN answers this with a continual-learning loop. A Gaussian-process surrogate models the task-level physics loss over the parameter space, and an upper-confidence-bound acquisition selects the next parameter to admit, which the authors report reduces complete objective-loss queries by 26.0% to 91.9% relative to grid-greedy search. Task-wise dynamic weights (a normalized current-loss term plus a recent-relative-decrease term) rebalance optimization effort among admitted tasks, and sparse physics-constrained replay stores a displaced task as its parameter value plus a small set of physical coordinates, recomputing the governing-equation residuals on the fly. Across five benchmarks, the authors report that the combined pipeline generally achieves lower and more balanced reference error than the baselines, that sparse replay beats no replay in all five cases, and that the full ACR2 pipeline reaches lower error at a fixed 500-step endpoint than reimplemented P2INN, HyperPINN, and Meta-PINN.

Load-bearing premise

The load-bearing premise is that the PDE family's solutions change smoothly with the parameters, so neighboring parameter tasks are similar enough that a Gaussian-process model of their losses can steer task selection; if the parameter domain contains an abrupt regime change or discontinuity, the authors themselves note this premise can fail and Bayesian active selection loses its justification.

Editorial extensions

If this is right

  • Objective-loss queries for parameter selection drop by 26.0% to 91.9% relative to grid-greedy search, with the largest reduction on the four-parameter Poisson–Boltzmann problem, where exhaustive grid queries would need 1,296 evaluations per search event.
  • Sparse replay keeps only about 10% of the sparsifiable physical points per displaced task yet beats no replay in all five benchmarks, reducing macro relative $L_2$ error by 55.0%, 51.7%, 68.1%, 85.7%, and 11.7%; on Schaffer-like and Kovasznay it also beats uncapped full replay, so more replay is not automatically better.
  • Adapting only a compact residual head with the base model frozen lowers the relative $L_2$ error at fixed unseen targets within 500 steps for Burgers (40.7% reduction), Kovasznay (38.9%), and Poisson–Boltzmann (78.9%), improving 14/15, 9/9, and 9/9 prespecified seed–target pairs; Allen–Cahn degrades at this budget, so the benefit is equation-dependent.
  • The physics training loss is an imperfect proxy for reference-solution error — in Burgers, mean PDE residuals at two viscosities differ by about 2.9% while relative $L_2$ errors differ by about 45.9% — so the authors rank methods by reference error rather than training loss.
  • The 2×2 ablation shows no universal winner: Bayesian selection alone performs best on Burgers, Allen–Cahn, and Kovasznay, dynamic weighting dominates on Schaffer-like, and the combination wins on Poisson–Boltzmann.
  • The paper notes that the same query-reduction principle extends to supervised active learning, where each query might be an expensive solver run or physical experiment rather than a residual evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the query-reduction result carries over to settings where each query is an expensive simulation or experiment, the same Bayesian task selection should yield even larger savings; the paper notes this extension but does not test it.
  • The Allen–Cahn trajectory — degraded at 500 steps yet improved for all nine seed–target pairs by 5,000 steps — suggests that ACR2-finetune needs a budget-adaptive stopping rule tied to local optimization difficulty rather than a fixed step count; a natural test is to stop per target when the residual head's loss saturates.
  • The sharpest test of the regularity premise would be a parameterized family with a genuine solution discontinuity or bifurcation inside the parameter domain: if the GP surrogate's task ranking falls to chance, the active-selection advantage should collapse while replay and weighting still carry the accuracy gains.
  • Because sparse replay outperformed uncapped full replay on two benchmarks, the binding constraint in sequential PINN training appears to be optimization capacity rather than memory; a testable corollary is that per-task replay density can be tuned to sit at the retention–adaptation frontier for each equation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper proposes CL-PINN, a continual-learning training framework for parameterized physics-informed neural networks. It treats each PDE parameter value as a task and combines Bayesian-optimization-based active parameter selection, task-wise dynamic loss weighting, sparse physics-constrained replay, a separate parameter subnetwork, and an optional single-parameter residual fine-tuning stage. The method is evaluated on one continuous-function benchmark and four parameterized PDEs (Burgers, Allen–Cahn, Kovasznay flow, and linearized Poisson–Boltzmann) with fixed seeds, prespecified test grids, and reference solutions generated analytically or by external solvers. The main reported findings are that Bayesian selection reduces complete objective-loss queries by 26.0–91.9% relative to grid-greedy search, sparse replay improves retention relative to no replay, the parameter subnetwork improves accuracy on some but not all problems, and ACR2-finetune reaches lower reference error at a fixed 500-step online endpoint than reimplemented P2INN, HyperPINN, and Meta-PINN baselines under the stated protocol.

Significance. If the results hold, the paper offers a practical and honestly evaluated recipe for capacity-limited parameterized PINN training, with particularly valuable evidence on query efficiency and replay. The manuscript's experimental discipline is a strength: fixed seeds, prespecified test sets, explicit disclosure of confounds, public code, and a clear separation between protocol-matched and combined-protocol comparisons. The paper does not oversell the parameter subnetwork or the external-method comparison at the level of its own caveats. The main gap is that the replay ablation is confounded for three of five cases by resampling-period differences, weakening one of the three headline contributions without additional matched experiments or revised claims.

major comments (2)
  1. [§4.4.2 / Table 14] The primary contrast in Table 14 (no replay versus fixed-capacity sparse replay) is protocol-matched only for Schaffer-like and Burgers. For Allen–Cahn, Kovasznay, and Poisson–Boltzmann the resampling period differs between AC and ACR: respectively N_resample = 2,000 vs 1,000; 1,000 vs 500; and 500 vs 250. The text acknowledges only the Allen–Cahn confound and then concludes that sparse replay outperforms no replay on all five cases and reduces macro L2 by 55.0%, 51.7%, 68.1%, 85.7%, and 11.7%. Because sparse replay is a headline contribution, the authors should either run protocol-matched replay ablations for these three cases or explicitly restrict the claim to the two matched cases and present the other three as combined-protocol results.
  2. [§4.3 / Table 11] The external-method comparison at the fixed 500-step endpoint is not a method-level comparison because offline pretraining budgets are unequal. The ACR base used by ACR2-finetune was trained with Adam plus L-BFGS to a much lower step-0 error, whereas the P2INN and HyperPINN reimplementations are Adam-only (Table S.3.5). Section 4.3.1 discloses this, but Table 11 and the opening of Section 4.3.2 present the endpoint as 'ACR2-finetune attains the lowest MSE and mean EL2 on all four PDEs,' which invites an interpretation the experiment cannot support. I recommend either adding a budget-controlled comparison (for example, equal offline Adam updates for all methods) or explicitly labeling this subsection as a base-representation/initialization-quality comparison and removing it from the method-level contribution claims.
minor comments (3)
  1. [§4.1.5 / Table 6] The table numbered 'Table 6' appears twice in the main text; the second instance, which continues the state-transition history, should be renumbered or merged with the first to avoid a citation ambiguity.
  2. [§S.3.4.1 / §4.4.3] The supplementary timing audit states that all 30/30 case–method runs completed, while Section 4.4.3 reports non-finite runs in the Burgers freeze-containing ablation cells; please clarify that the 30/30 statement refers only to the timing audit and not to all ablation configurations.
  3. [§3.1 / Eq. (11)] The symbol q is used both for the weight-update event index in Eq. (6) and for the number of residual components in Eq. (11); although the contexts are distinct, a different symbol for one of these quantities would reduce reader confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CL-PINN's predictions are checked against external reference solutions and controlled ablations; acknowledged proxy and budget limitations are confounds, not circular reductions.

full rationale

I walked the derivation chain for each load-bearing claim. The central accuracy claims are evaluated on fixed test parameters against analytical or solver-generated reference solutions that are never used for training, selection, weighting, or early stopping (S.1.3: 'Reference solutions are used only for evaluation and never for training, parameter selection, dynamic weighting, early stopping, or model selection'). The physics losses are recomputed from the governing equations and condition residuals rather than from fitted labels, so training signal is not the evaluation target in disguise. Bayesian selection is presented as a query-allocation heuristic with an explicitly stated regularity premise and with acknowledged failure modes; it does not by construction guarantee the measured query reduction or accuracy. The replay ablation compares no-replay versus sparse-replay under prespecified protocols and reports external reference error; even where resampling periods differ (Allen-Cahn), the paper explicitly labels that contrast as not a strict single-factor attribution. The external-method comparison in Section 4.3 does not use equal offline budgets, and the paper states this directly: 'The offline costs are not identical' and 'this experiment provides a numerical comparison on common targets under a fixed online budget, but it is neither a reproduction of every method's complete original protocol nor a comparison under strictly equal offline cost.' That is a fairness and interpretability limitation, not a circularity: the 500-step endpoint is a real fixed endpoint, and the claim is explicitly conditioned on the stated protocol. No load-bearing self-citation, imported uniqueness theorem, ansatz-smuggling citation, or renaming of a known result as a new derivation appears. The acknowledged imperfection of physics loss as a proxy for reference error is a stated limitation, not a circular step, because the final metrics are reference-based. Overall, the paper's core claims have independent empirical content and are not forced by construction.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claim rests on several domain assumptions, most importantly the regularity premise and the proxy quality of physics losses, plus a modest set of hand-chosen per-case hyperparameters. No new physical entities are introduced. The reproducibility artifacts (code on GitHub, fixed seeds, locked protocols) are present, but the per-case tuning means the method is best viewed as a validated recipe rather than a parameter-free algorithm.

free parameters (8)
  • Exploration coefficient kappa = 5 for all cases
    Controls UCB exploration in Bayesian acquisition; hand-set, not tuned per case.
  • Dynamic weighting coefficients (lambda_static, lambda_dynamic) = (2,-2) for Schaffer and Allen-Cahn; (1,-1) for Burgers, Kovasznay, Poisson-Boltzmann
    Hand-chosen per benchmark; strongly affect which tasks receive training weight.
  • Training/search prior f_prior = exp[1.5(log10 a + 3)] for Schaffer; rho^3 exp(-2 log10 nu) for Allen-Cahn; log10 Re for Kovasznay; 1 for Burgers and…
    Problem-specific prior weighting for dynamic weights and sometimes BO score shaping; chosen by the authors per problem.
  • Active and replay task capacities = 15/15, 9/9, 9/9, 10/10, 80/80 for the five cases
    Bound the memory budget; chosen per case.
  • Resampling period N_resample = e.g., 1000/1000/500 for Schaffer, 2000/2000/1000 for Burgers, etc.
    Interval for active updates; differs across methods and cases, creating some protocol confounds.
  • Bayesian evaluations per active update = 10, 10, 20, 15, 50 for the five cases
    Number of GP-surrogate queries per selection event; affects query counts and accuracy.
  • Parameter-branch decay and freezing controls = not reported numerically
    Regularization controls for the parameter subnetwork; chosen empirically (Section 4.4.3).
  • ACR2-finetune residual head width = 25 for Burgers, Allen-Cahn, Kovasznay; 15 for Poisson-Boltzmann
    Prespecified after a diagnostic and not selected by test error.
assumptions (6)
  • domain assumption The parameter-to-solution map is continuous with bounded rate of variation over the considered domains, so neighboring parameters have correlated task losses.
    Invoked in Sections 3.1 and 3.3 to justify the BO surrogate and the parameter subnetwork; the paper explicitly notes it may fail at abrupt regime transitions or discontinuities.
  • domain assumption The physics-informed task loss is a usable proxy for reference-solution error for task selection and weighting.
    Active selection ranks tasks by physics loss (Section 3.1); Paper S.2.2 shows similar PDE residuals can coexist with substantially different reference errors, so this assumption is partial and acknowledged.
  • standard math Minimizing residual losses yields the PDE solution; automatic differentiation computes the required derivatives.
    Standard PINN assumption used throughout Section 2.1.
  • domain assumption A Gaussian process with a stationary kernel, refit at each update, can model the task-loss field.
    Section 3.1 notes nonconvexity, multiscale peaks, and incompatible length scales may break stationarity; refitting and UCB exploration mitigate but do not remove this.
  • domain assumption Experience replay of sparse physics constraints preserves earlier-task knowledge without storing solution labels.
    Central replay mechanism in Section 3.2; validated empirically rather than proven.
  • domain assumption Bounded active-task capacity accurately models limited computational resources.
    Problem setup and motivation; memory scaling in S.2.1 illustrates the constraint.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Continual-Learning Physics-Informed Neural Networks for Parameterized Partial Differential Equations." pith.science (2026). https://pith.science/paper/LH7Q7LLN

@misc{pith2026260804778,
  author       = {Pith},
  title        = {Pith review of: Continual-Learning Physics-Informed Neural Networks for Parameterized Partial Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LH7Q7LLN}},
  note         = {Machine review of arXiv:2608.04778}
}
read the original abstract

Physics-informed neural networks (PINNs) incorporate governing equations into neural-network training and can approximate PDE solutions without requiring large observational datasets. Parameterized PINNs (ParamPINNs) further take physical parameters as inputs, allowing a single model to represent a family of PDE solutions over a parameter domain. Existing ParamPINNs, however, still face inefficient training, uneven accuracy across parameters, and overfitting to a limited set of sampled parameter tasks, which can impair generalization to unsampled parameters. To address these issues, we propose a continual-learning physics-informed neural network (CL-PINN), which treats PDE instances at different parameter values as related tasks and learns them sequentially. CL-PINN combines Bayesian-optimization-based active parameter selection, task-wise dynamic loss weighting, sparse physics-constrained replay, and an optional parameter subnetwork to improve task allocation and knowledge retention under bounded active-task capacity. It requires no observational data and is designed to solve parameterized PDEs over relatively broad parameter domains under limited computational resources. Multi-seed evaluations on five benchmarks, including one continuous function and four parameterized PDEs, show that Bayesian selection substantially reduces objective-loss queries relative to grid-greedy search, while sparse replay mitigates forgetting of earlier tasks. Under the prescribed within-case resource protocols, CL-PINN generally provides higher and more balanced solution accuracy than fixed-sampling and grid-greedy baselines. CL-PINN offers a practical route toward learning PDE solutions that generalize across physical parameters and has the potential to support reusable physics-informed surrogates for large-scale engineering parameter studies.

Figures

Figures reproduced from arXiv: 2608.04778 by the authors.

Figure 1
Figure 1. Background and motivation for CL-PINN: parameterized PINNs face inefficient training, imbalanced cross-parameter accuracy, and poor generaliza￾tion caused by overfitting to a limited set of sampled tasks. alone. Physics-informed neural networks (PINNs), introduced by Raissi et al. [17], approximate a PDE solution with a neural network and place governing equations, boundary conditions (BCs), and initial conditions (… view at source ↗
Figure 2
Figure 2. Structure of a ParamPINN. A multilayer perceptron maps physical coordi￾nates and PDE parameters to the solution, while automatic differentiation evaluates the governing residuals. 2. Problem formulation 2.1. Parameterized physics-informed neural networks (ParamPINNs) A parameterized physics-informed neural network (ParamPINN) extends a conventional PINN by fitting a family of PDE instances rather than a single insta… view at source ↗
Figure 3
Figure 3. Overall CL-PINN workflow. BO selects parameter tasks, dynamic weighting balances admitted tasks, sparse replay preserves constraints from displaced tasks, and the parameter subnetwork represents physical parameters; optional residual adaptation follows global training. date and becomes rapidly more expensive as the parameter domain grows in size or dimension. The regularity premise used here is that, over the do￾mai… view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: Parameter subnetwork and optional single-parameter adaptation. The parameter branch is subject to optional Adam weight decay and L-BFGS-stage freezing; after global training, a compact residual head can be adapted at one specified parameter. the physical parameters to …
Figure 5
Figure 5. Figure 5: ACR2-arch prediction, reference, and absolute error for the Schaffer￾like benchmark at a = 0.001, 0.5, and 1.0, representing the lower endpoint, interior, and upper endpoint of the parameter domain. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Parameter-wise MSE and EL2 of ACR2-arch over 28 Schaffer-like test parameters. Thin curves denote independent runs, the black curve their pointwise mean, and the vertical axes are logarithmic [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Parameter-state evolution of ACR2-arch in a representative Schaffer￾like run, showing newly selected, densely trained, replayed, and state￾transitioned tasks [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: ACR2-arch prediction, reference, and absolute error for Burgers at ν = 0.01, 0.1, and 1.0, showing the transition from a sharp low-viscosity near￾shock layer to smoother solutions. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Parameter-wise MSE and EL2 of ACR2-arch over 100 Burgers test parameters. Thin curves denote independent runs, the black curve their pointwise mean, and the vertical axes are logarithmic [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Parameter-state evolution of ACR2-arch in a representative Burgers run, showing newly selected, densely trained, replayed, and state-transitioned tasks. here. Adding sparse replay in ACR reduces MSE and relative L2 error by 58.0% and 51.7% relative to AC. ACR2-arch fu…
Figure 11
Figure 11. Figure 11: ACR prediction, reference, and absolute error for Allen–Cahn at (ν, ρ) = (0.001, 3), (0.01, 4), and (0.1, 5), showing distinct diffusion–reaction regimes across the two-dimensional parameter domain. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Mean ACR test errors over the Allen–Cahn parameter domain. The heatmaps show MSE and EL2 , averaged pointwise over independent runs on logarithmic color scales [PITH_FULL_IMAGE:figures/full_fig_p027_12.png]
Figure 13
Figure 13. Figure 13: Parameter selection and state evolution of ACR in a representative Allen–Cahn run. Labels give the search-event order, and markers distinguish newly selected, densely trained, and replayed tasks. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]
Figure 14
Figure 14. Figure 14: ACR2-arch prediction, analytical reference, and absolute error for Kovasznay u, v, and p at Re = 5. Component-wise color scales account for their different magnitudes [PITH_FULL_IMAGE:figures/full_fig_p030_14.png]
Figure 15
Figure 15. Figure 15: ACR2-arch prediction, analytical reference, and absolute error for Kovasznay u, v, and p at Re = 500 [PITH_FULL_IMAGE:figures/full_fig_p031_15.png]
Figure 16
Figure 16. Figure 16: Predicted Kovasznay velocity magnitude and streamlines at Re = 5 and Re = 500, showing the overall flow structure beyond the component fields [PITH_FULL_IMAGE:figures/full_fig_p032_16.png]
Figure 17
Figure 17. Figure 17: Reynolds-number selection and parameter-state evolution of ACR2- arch in a representative Kovasznay run, distinguishing newly selected, densely trained, and replayed tasks. 32 [PITH_FULL_IMAGE:figures/full_fig_p032_17.png]
Figure 18
Figure 18. Figure 18: Mean component-wise MSE and EL2 of ACR2-arch over 100 Reynolds numbers, averaged pointwise over independent runs. inant velocity component. By contrast, ACR gives the lowest v MSE and the lowest relative L2 errors for v and p, and is therefore better balanced across t…
Figure 19
Figure 19. Figure 19: ACR predictions, reference solutions, and absolute errors at four rep￾resentative parameter tuples of the linearized Poisson–Boltzmann benchmark. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_19.png]
Figure 20
Figure 20. Figure 20: Representative conditional error slices of ACR on the four￾dimensional Poisson–Boltzmann test grid. Each column fixes one (k, A) pair, with µ1 and µ2 on the horizontal and vertical axes; the upper and lower rows show MSE and EL2 , respectively, on logarithmic color sc…
Figure 21
Figure 21. Figure 21: Zero-shot versus post-adaptation EL2 after 500 ACR2-finetune steps over all strictly unseen parameters. Points below the identity line improve; orange markers denote the highest zero-shot-error quartile, and all degraded cases are retained. 42 [PITH_FULL_IMAGE:figure…
Figure 22
Figure 22. Figure 22: Field comparisons at four difficult parameters selected by the largest zero-shot error. Each panel shows the reference solution, zero-shot prediction, post￾adaptation prediction, and the absolute errors before and after adaptation. Burgers, Ko￾vasznay, and Poisson–Bol…
Figure 23
Figure 23. Figure 23: Mean target EL2 from direct prediction (step 0) to 500-step adapta￾tion under the common four-PDE protocol. Markers and intervals show the mean and population standard deviation on a logarithmic scale; Kovasznay is macro-averaged over u, v, and p. Meta-PINN uses a dif…
Figure 24
Figure 24. Figure 24: Macro relative L2 for no replay, fixed-capacity sparse replay, and uncapped full replay across five cases. The horizontal axis is logarithmic, and the three marker types denote the replay settings; uncapped full replay is a capacity reference. no replay, not that it p…
Figure 25
Figure 25. Figure 25: Process diagnostic of sparse versus no replay on Allen–Cahn. Panel (a) shows task sample states, panels (b,c) fixed-probe macro/worst relative L2, and panel (d) the final parameter grid. The dashed line marks the Adam/L-BFGS transition; this single run illustrates the…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

94 extracted references · 67 canonical work pages

  1. [1]

    W. Cho, M. Jo, H. Lim, K. Lee, D. Lee, S. Hong, N. Park, Parameterized physics-informed neural networks for parameterized pdes, in: Proceedings of the 41st International Conference on Machine Learning, Vol. 235 of Proceedings of Machine Learning Research, PMLR, 2024, pp. 8510--8533

    2024

  2. [2]

    de Avila Belbute-Peres, Y.-f

    F. de Avila Belbute-Peres, Y.-f. Chen, F. Sha, Hyperpinn: Learning parameterized differential equations with physics-informed hypernetworks, in: The symbiosis of deep learning and differential equations, 2021

    2021

  3. [3]

    Penwarden, S

    M. Penwarden, S. Zhe, A. Narayan, R. M. Kirby, A metalearning approach for physics-informed neural networks (pinns): Application to parameterized pdes, Journal of Computational Physics 477 (2023) 111912

    2023

  4. [4]

    2005 , publisher=

    Partial differential equations: modeling, analysis, computation , author=. 2005 , publisher=

    2005

  5. [5]

    2014 , publisher=

    Numerical methods for partial differential equations , author=. 2014 , publisher=

    2014

  6. [7]

    Computer Methods in Applied Mechanics and Engineering , volume=

    PhyCRNet: Physics-informed convolutional-recurrent network for solving spatiotemporal PDEs , author=. Computer Methods in Applied Mechanics and Engineering , volume=. 2022 , publisher=

    2022

  7. [8]

    Physics of Fluids , volume=

    A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries , author=. Physics of Fluids , volume=. 2021 , publisher=

    2021

  8. [10]

    Machine learning and knowledge extraction , volume=

    Deep theory of functional connections: A new method for estimating the solutions of partial differential equations , author=. Machine learning and knowledge extraction , volume=. 2020 , publisher=

    2020

Show all 94 references
  1. [13]

    Computer Methods in Applied Mechanics and Engineering , volume=

    PPINN: Parareal physics-informed neural network for time-dependent PDEs , author=. Computer Methods in Applied Mechanics and Engineering , volume=. 2020 , publisher=

    2020

  2. [14]

    International Journal of Heat and Fluid Flow , volume=

    Predicting high-fidelity multiphysics data from low-fidelity fluid flow and transport solvers using physics-informed neural networks , author=. International Journal of Heat and Fluid Flow , volume=. 2022 , publisher=

    2022

  3. [15]

    Journal of Computational Physics , volume=

    Hybrid FEM-NN models: Combining artificial neural networks with the finite element method , author=. Journal of Computational Physics , volume=. 2021 , publisher=

    2021

  4. [17]

    Journal of Computational Physics , volume=

    DeepMoD: Deep learning for model discovery in noisy data , author=. Journal of Computational Physics , volume=. 2021 , publisher=

    2021

  5. [18]

    Nature communications , volume=

    Physics-informed learning of governing equations from scarce data , author=. Nature communications , volume=. 2021 , publisher=

    2021

  6. [19]

    Neural Networks , volume=

    PDE-READ: Human-readable partial differential equation discovery using deep learning , author=. Neural Networks , volume=. 2022 , publisher=

    2022

  7. [20]

    SIAM Journal on Scientific Computing , volume=

    Physics-informed neural networks with hard constraints for inverse design , author=. SIAM Journal on Scientific Computing , volume=. 2021 , publisher=

    2021

  8. [21]

    Journal of Computational physics , volume=

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , author=. Journal of Computational physics , volume=. 2019 , publisher=

    2019

  9. [22]

    Advances in Neural Information Processing Systems , volume=

    Characterizing possible failure modes in physics-informed neural networks , author=. Advances in Neural Information Processing Systems , volume=

  10. [23]

    2023 International Joint Conference on Neural Networks (IJCNN) , pages=

    Improved training of physics-informed neural networks with model ensembles , author=. 2023 International Joint Conference on Neural Networks (IJCNN) , pages=. 2023 , organization=

    2023

  11. [24]

    Neurocomputing , volume=

    Self-adaptive loss balanced physics-informed neural networks , author=. Neurocomputing , volume=. 2022 , publisher=

    2022

  12. [25]

    SIAM Review , volume=

    DeepXDE: A deep learning library for solving differential equations , author=. SIAM Review , volume=. 2021 , publisher=

    2021

  13. [26]

    Journal of Computational Physics , volume=

    Active training of physics-informed neural networks to aggregate and interpolate parametric solutions to the Navier-Stokes equations , author=. Journal of Computational Physics , volume=. 2021 , publisher=

    2021

  14. [27]

    Extension of Physics-informed Neural Networks to Solving Parameterized PDEs , author=

  15. [28]

    Proceedings of the 41st International Conference on Machine Learning , series=

    Parameterized Physics-informed Neural Networks for Parameterized PDEs , author=. Proceedings of the 41st International Conference on Machine Learning , series=. 2024 , publisher=

    2024

  16. [29]

    Energies , volume=

    A Parametric Physics-Informed Deep Learning Method for Probabilistic Design of Thermal Protection Systems , author=. Energies , volume=. 2023 , publisher=

    2023

  17. [30]

    Combustion and Flame , volume=

    Surrogate modeling of parameterized multi-dimensional premixed combustion with physics-informed neural networks for rapid exploration of design space , author=. Combustion and Flame , volume=. 2023 , publisher=

    2023

  18. [31]

    Computer Methods in Applied Mechanics and Engineering , volume=

    Isogeometric neural networks: A new deep learning approach for solving parameterized partial differential equations , author=. Computer Methods in Applied Mechanics and Engineering , volume=. 2023 , publisher=

    2023

  19. [32]

    Neural Computing and Applications , volume=

    A novel meta-learning initialization method for physics-informed neural networks , author=. Neural Computing and Applications , volume=. 2022 , publisher=

    2022

  20. [33]

    IEEE Transactions on Energy Conversion , volume=

    Physics-informed neural networks for solving parametric magnetostatic problems , author=. IEEE Transactions on Energy Conversion , volume=. 2022 , publisher=

    2022

  21. [34]

    Computers & Mathematics with Applications , volume=

    An extended physics informed neural network for preliminary analysis of parametric optimal control problems , author=. Computers & Mathematics with Applications , volume=. 2023 , publisher=

    2023

  22. [35]

    Aerodynamic Flow Field Prediction across Geometric and Physical-Fluidic Variations Using Data-Driven and Physics Informed Deep Learning Models , author=

  23. [36]

    The symbiosis of deep learning and differential equations , year=

    HyperPINN: Learning parameterized differential equations with physics-informed hypernetworks , author=. The symbiosis of deep learning and differential equations , year=

  24. [37]

    Journal of Computational Physics , volume=

    A metalearning approach for physics-informed neural networks (PINNs): Application to parameterized PDEs , author=. Journal of Computational Physics , volume=. 2023 , publisher=

    2023

  25. [38]

    Journal of Computational Physics , volume=

    NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations , author=. Journal of Computational Physics , volume=. 2021 , publisher=

    2021

  26. [39]

    2022 , publisher=

    Lifelong machine learning , author=. 2022 , publisher=

    2022

  27. [40]

    Recent advances in optimization and modeling of contemporary problems , pages=

    Bayesian optimization , author=. Recent advances in optimization and modeling of contemporary problems , pages=. 2018 , publisher=

    2018

  28. [41]

    The European Physical Journal C , volume=

    Parameterized neural networks for high-energy physics , author=. The European Physical Journal C , volume=. 2016 , publisher=

    2016

  29. [42]

    Computer Methods in Applied Mechanics and Engineering , volume=

    Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data , author=. Computer Methods in Applied Mechanics and Engineering , volume=. 2020 , publisher=

    2020

  30. [43]

    Fernando Nogueira , title =

  31. [45]

    Proceedings of the first international conference on genetic algorithms and their applications , pages=

    Multiple objective optimization with vector evaluated genetic algorithms , author=. Proceedings of the first international conference on genetic algorithms and their applications , pages=. 2014 , organization=

    2014

  32. [46]

    Pramana , volume=

    A systematic literature review of Burgers’ equation with recent advances , author=. Pramana , volume=. 2018 , publisher=

    2018

  33. [47]

    Numerical methods for nonlinear partial differential equations , pages=

    The Allen--Cahn Equation , author=. Numerical methods for nonlinear partial differential equations , pages=. 2015 , publisher=

    2015

  34. [48]

    2024 , publisher=

    Navier--Stokes equations: theory and numerical analysis , author=. 2024 , publisher=

    2024

  35. [49]

    Mathematical Proceedings of the Cambridge Philosophical Society , volume=

    Laminar flow behind a two-dimensional grid , author=. Mathematical Proceedings of the Cambridge Philosophical Society , volume=. 1948 , organization=

    1948

  36. [50]

    International Journal for Numerical Methods in Fluids , volume=

    Exact fully 3D Navier--Stokes solutions for benchmarking , author=. International Journal for Numerical Methods in Fluids , volume=. 1994 , publisher=

    1994

  37. [51]

    2022 , doi=

    Fries, William and He, Xiaolong and Choi, Youngsoo , journal=. 2022 , doi=

    2022

  38. [52]

    , journal=

    Bonneville, Christophe and Choi, Youngsoo and Ghosh, Debojyoti and Belof, Jonathan L. , journal=. 2024 , note=

    2024

  39. [53]

    Advances in Neural Information Processing Systems , volume=

    Gradient Episodic Memory for Continual Learning , author=. Advances in Neural Information Processing Systems , volume=

  40. [54]

    Advances in Neural Information Processing Systems , volume=

    Experience Replay for Continual Learning , author=. Advances in Neural Information Processing Systems , volume=

  41. [55]

    R. M. Mattheij, S. W. Rienstra, J. T. T. Boonkkamp, Partial differential equations: modeling, analysis, computation, SIAM, 2005

    2005

  42. [56]

    W. F. Ames, Numerical methods for partial differential equations, Academic press, 2014

    2014

  43. [57]

    M. D. Ribeiro, A. Rehman, S. Ahmed, A. Dengel, Deepcfd: Efficient steady-state laminar flow approximation with deep convolutional neural networks, arXiv preprint arXiv:2004.08826 (2020)

    2020 arXiv

  44. [58]

    P. Ren, C. Rao, Y. Liu, J.-X. Wang, H. Sun, Phycrnet: Physics-informed convolutional-recurrent network for solving spatiotemporal pdes, Computer Methods in Applied Mechanics and Engineering 389 (2022) 114399

    2022

  45. [59]

    Kashefi, D

    A. Kashefi, D. Rempe, L. J. Guibas, A point-cloud deep learning framework for prediction of fluid flow fields on irregular geometries, Physics of Fluids 33 (2) (2021)

    2021

  46. [60]

    Kumar, S

    Y. Kumar, S. Chakraborty, Grade: A graph based data-driven solver for time-dependent nonlinear partial differential equations, arXiv preprint arXiv:2108.10639 (2021)

    2021 arXiv

  47. [61]

    L. Lu, P. Jin, G. E. Karniadakis, Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators, arXiv preprint arXiv:1910.03193 (2019)

    2019 arXiv

  48. [62]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, A. Anandkumar, Fourier neural operator for parametric partial differential equations, arXiv preprint arXiv:2010.08895 (2020)

    2020 arXiv

  49. [63]

    X. Meng, Z. Li, D. Zhang, G. E. Karniadakis, Ppinn: Parareal physics-informed neural network for time-dependent pdes, Computer Methods in Applied Mechanics and Engineering 370 (2020) 113250

    2020

  50. [64]

    Aliakbari, M

    M. Aliakbari, M. Mahmoudi, P. Vadasz, A. Arzani, Predicting high-fidelity multiphysics data from low-fidelity fluid flow and transport solvers using physics-informed neural networks, International Journal of Heat and Fluid Flow 96 (2022) 109002

    2022

  51. [65]

    S. K. Mitusch, S. W. Funke, M. Kuchta, Hybrid fem-nn models: Combining artificial neural networks with the finite element method, Journal of Computational Physics 446 (2021) 110651

    2021

  52. [66]

    Z. Hao, S. Liu, Y. Zhang, C. Ying, Y. Feng, H. Su, J. Zhu, Physics-informed machine learning: A survey on problems, methods and applications, arXiv preprint arXiv:2211.08064 (2022)

    2022 arXiv

  53. [67]

    G.-J. Both, S. Choudhury, P. Sens, R. Kusters, Deepmod: Deep learning for model discovery in noisy data, Journal of Computational Physics 428 (2021) 109985

    2021

  54. [68]

    Z. Chen, Y. Liu, H. Sun, Physics-informed learning of governing equations from scarce data, Nature communications 12 (1) (2021) 6136

    2021

  55. [69]

    Stephany, C

    R. Stephany, C. Earls, Pde-read: Human-readable partial differential equation discovery using deep learning, Neural Networks 154 (2022) 360--382

    2022

  56. [70]

    L. Lu, R. Pestourie, W. Yao, Z. Wang, F. Verdugo, S. G. Johnson, Physics-informed neural networks with hard constraints for inverse design, SIAM Journal on Scientific Computing 43 (6) (2021) B1105--B1132

    2021

  57. [71]

    Raissi, P

    M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational physics 378 (2019) 686--707

    2019

  58. [72]

    Krishnapriyan, A

    A. Krishnapriyan, A. Gholami, S. Zhe, R. Kirby, M. W. Mahoney, Characterizing possible failure modes in physics-informed neural networks, Advances in Neural Information Processing Systems 34 (2021) 26548--26560

    2021

  59. [73]

    Haitsiukevich, A

    K. Haitsiukevich, A. Ilin, Improved training of physics-informed neural networks with model ensembles, in: 2023 International Joint Conference on Neural Networks (IJCNN), IEEE, 2023, pp. 1--8

    2023

  60. [74]

    Leake, D

    C. Leake, D. Mortari, Deep theory of functional connections: A new method for estimating the solutions of partial differential equations, Machine learning and knowledge extraction 2 (1) (2020) 37--55

    2020

  61. [75]

    L. Lu, X. Meng, Z. Mao, G. E. Karniadakis, Deepxde: A deep learning library for solving differential equations, SIAM Review 63 (1) (2021) 208--228. https://doi.org/10.1137/19M1274067 doi:10.1137/19M1274067

    2021 doi

  62. [76]

    Xiang, W

    Z. Xiang, W. Peng, X. Liu, W. Yao, Self-adaptive loss balanced physics-informed neural networks, Neurocomputing 496 (2022) 11--34

    2022

  63. [77]

    X. Liu, X. Zhang, W. Peng, W. Zhou, W. Yao, A novel meta-learning initialization method for physics-informed neural networks, Neural Computing and Applications 34 (17) (2022) 14511--14534

    2022

  64. [78]

    W. Cho, M. Jo, H. Lim, K. Lee, D. Lee, S. Hong, N. Park, Extension of physics-informed neural networks to solving parameterized pdes, in: ICLR 2024 Workshop on AI4DifferentialEquations In Science

    2024

  65. [79]

    Gasick, X

    J. Gasick, X. Qian, Isogeometric neural networks: A new deep learning approach for solving parameterized partial differential equations, Computer Methods in Applied Mechanics and Engineering 405 (2023) 115839

    2023

  66. [80]

    N. Demo, M. Strazzullo, G. Rozza, An extended physics informed neural network for preliminary analysis of parametric optimal control problems, Computers & Mathematics with Applications 143 (2023) 383--396

    2023

  67. [81]

    Zhang, N

    R. Zhang, N. Xu, K. Zhang, L. Wang, G. Lu, A parametric physics-informed deep learning method for probabilistic design of thermal protection systems, Energies 16 (9) (2023) 3820

    2023

  68. [82]

    K. Liu, K. Luo, Y. Cheng, A. Liu, H. Li, J. Fan, S. Balachandar, Surrogate modeling of parameterized multi-dimensional premixed combustion with physics-informed neural networks for rapid exploration of design space, Combustion and Flame 258 (2023) 113094

    2023

  69. [83]

    Beltr \'a n-Pulido, I

    A. Beltr \'a n-Pulido, I. Bilionis, D. Aliprantis, Physics-informed neural networks for solving parametric magnetostatic problems, IEEE Transactions on Energy Conversion 37 (4) (2022) 2678--2689

    2022

  70. [84]

    K. M. Tangsali, Aerodynamic flow field prediction across geometric and physical-fluidic variations using data-driven and physics informed deep learning models, Ph.D. thesis (2020)

    2020

  71. [85]

    Fries, X

    W. Fries, X. He, Y. Choi, LaSDI : Parametric latent space dynamics identification, Computer Methods in Applied Mechanics and Engineering 399 (2022) 115436. https://doi.org/10.1016/j.cma.2022.115436 doi:10.1016/j.cma.2022.115436

    2022

  72. [86]

    Bonneville, Y

    C. Bonneville, Y. Choi, D. Ghosh, J. L. Belof, GPLaSDI : Gaussian process-based interpretable latent space dynamics identification through deep autoencoder, Computer Methods in Applied Mechanics and EngineeringPreprint arXiv:2308.05882 (2024)

    2024 arXiv

  73. [87]

    C. J. Arthurs, A. P. King, Active training of physics-informed neural networks to aggregate and interpolate parametric solutions to the navier-stokes equations, Journal of Computational Physics 438 (2021) 110364

    2021

  74. [88]

    Z. Chen, B. Liu, Lifelong machine learning, Springer Nature, 2022

    2022

  75. [89]

    Lopez-Paz, M

    D. Lopez-Paz, M. Ranzato, Gradient episodic memory for continual learning, in: Advances in Neural Information Processing Systems, Vol. 30, 2017

    2017

  76. [90]

    Rolnick, A

    D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, G. Wayne, Experience replay for continual learning, in: Advances in Neural Information Processing Systems, Vol. 32, 2019

    2019

  77. [91]

    P. I. Frazier, Bayesian optimization, in: Recent advances in optimization and modeling of contemporary problems, Informs, 2018, pp. 255--278

    2018

  78. [92]

    Baldi, K

    P. Baldi, K. Cranmer, T. Faucett, P. Sadowski, D. Whiteson, Parameterized neural networks for high-energy physics, The European Physical Journal C 76 (5) (2016) 1--7

    2016

  79. [93]

    L. Sun, H. Gao, S. Pan, J.-X. Wang, Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data, Computer Methods in Applied Mechanics and Engineering 361 (2020) 112732

    2020

  80. [94]

    Nogueira, https://github.com/bayesian-optimization/BayesianOptimization Bayesian Optimization : Open source constrained global optimization tool for Python (2014--)

    F. Nogueira, https://github.com/bayesian-optimization/BayesianOptimization Bayesian Optimization : Open source constrained global optimization tool for Python (2014--). ://github.com/bayesian-optimization/BayesianOptimization

    2014

  81. [95]

    J. D. Schaffer, Multiple objective optimization with vector evaluated genetic algorithms, in: Proceedings of the first international conference on genetic algorithms and their applications, Psychology Press, 2014, pp. 93--100

    2014

  82. [96]

    M. P. Bonkile, A. Awasthi, C. Lakshmi, V. Mukundan, V. Aswin, A systematic literature review of burgers’ equation with recent advances, Pramana 90 (2018) 1--21

    2018

  83. [97]

    Bartels, S

    S. Bartels, S. Bartels, The allen--cahn equation, Numerical methods for nonlinear partial differential equations (2015) 153--182

    2015

  84. [98]

    L. I. G. Kovasznay, Laminar flow behind a two-dimensional grid, in: Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 44, Cambridge University Press, 1948, pp. 58--62

    1948

  85. [99]

    Temam, Navier--Stokes equations: theory and numerical analysis, Vol

    R. Temam, Navier--Stokes equations: theory and numerical analysis, Vol. 343, American Mathematical Society, 2024

    2024

  86. [100]

    Z. Hao, J. Yao, C. Su, H. Su, Z. Wang, F. Lu, Z. Xia, Y. Zhang, S. Liu, L. Lu, et al., Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving pdes, arXiv preprint arXiv:2306.08827 (2023)

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.