Pith. sign in

REVIEW 3 major objections 5 minor 34 references

B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Bayesian PINN stabilizes forward PDE training by labeling low-variance points.

desk verdict A useful, honest comparison of Bayesian pseudo-labeling vs ensembles for PINN stabilization, but the variance-as-consensus mechanism is shakier than the title suggests. read the letter →

arxiv 2507.01714 v1 pith:HIAWEOBQ submitted 2025-07-02 cs.LG

Kevin Innerebner , Franz M. Rohrhofer , Bernhard C. Geiger This is my paper
classification cs.LG
keywords physics-informedneuralnetworksBayesianpseudolabelingpartialdifferentialequationsposteriorvariancetrainingstabilizationMCMCforwardproblems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Training physics-informed neural networks (PINNs) for forward problems often gets stuck because information from initial and boundary conditions does not propagate into the interior of the computational domain. This paper proposes B-PL-PINN, a training method that replaces the multi-network ensemble of earlier pseudo-labeling work with a single Bayesian PINN, and replaces ensemble consensus with a threshold on the Bayesian posterior variance. A collocation point becomes a pseudo-label only when it is close to an already labeled point and the sampled posterior agrees there, and only such points are active in the physics loss. On four benchmark 1D PDE systems the paper reports lower relative $L^2$ errors than the ensemble baselines on most of the eight tested settings and competitive performance against ensembles additionally optimized by LBFGS, at up to about nine times the baseline runtime.

What carries the argument

The load-bearing object is the Bayesian PINN posterior, a distribution over network parameters built from a Gaussian prior and Gaussian likelihoods on labeled data (initial conditions plus pseudo-labels) and on unlabeled physics and boundary residuals. Markov chain Monte Carlo (HMC or NUTS) draws parameter samples from this posterior; the variance of the sampled solutions at a collocation point is the consensus measure. Pseudo-labeling then expands the training domain: a point is labeled only when the variance is below $\sigma^2_{\mathrm{consens}}$, the point is within distance $\Delta$ of a labeled point, and the nearest label is consistent with the sampled mean within $\epsilon$; subsequently, only collocation and boundary points within $\Delta_{\mathrm{pde}}$ of a label contribute to the loss. This incremental, uncertainty-gated expansion is what carries information from the initial condition into the interior.

What would settle it

Run the B-PL-PINN procedure on the reaction system with $\rho=5$ while removing the distance constraint (making $\Delta$ arbitrarily large and relying on the variance threshold alone). If the error grows in low-variance regions similar to those shown in the paper, or if any collocation point with posterior variance below $\sigma^2_{\mathrm{consens}}$ has large absolute error during training, the labeling criterion is not reliable and the stabilization claim would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that the Bayesian posterior variance of a physics-informed neural network can serve as a mathematically grounded surrogate for ensemble consensus, and that combining this variance criterion with proximity to labeled points yields stable propagation of initial-condition information. Concretely, the paper samples $N$ parameter sets from the posterior via Hamiltonian Monte Carlo, computes the variance and median of the sampled solutions at candidate collocation points, and adds a pseudo-label only if the point lies within distance $\Delta$ of an existing label, the closest label agrees with the sampled mean within $\epsilon$, and the posterior variance is below $\sigma^2_{\mathrm{consens}}$. The paper reports that this procedure outperforms the ensemble baselines on most of the eight tested systems and is competitive with ensemble methods trained with both Adam and LBFGS, at a runtime below 9 times that of the ensemble baseline.

Load-bearing premise

The load-bearing premise is that low posterior variance at a collocation point near an already labeled point means the model's prediction there is accurate enough to be frozen as a pseudo-label; the paper's own ablation shows this holds for the convection system but is not sufficient for the reaction system without the added distance constraint.

Editorial extensions

If this is right

  • A single network can substitute for an ensemble in pseudo-label expansion, so the method cuts the model count while keeping accuracy on the tested benchmarks.
  • The variance-plus-distance labeling rule is the mechanism, and the ablation indicates the distance constraint is necessary because posterior variance alone is not a reliable error indicator for the reaction system.
  • Because the training domain expands according to epistemic uncertainty rather than a fixed schedule, easier-to-learn regions advance to larger times faster, as shown in the convection-system propagation behavior.
  • The method composes with other stabilization devices such as loss weighting, domain decomposition, and curriculum learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same variance-threshold rule could be plugged into cheaper uncertainty estimators, such as deep ensembles or MC dropout, but the paper does not test this.
  • Editorial inference: the failed diffusion case with the larger parameter suggests the method inherits a strong dependence on a well-fit initial condition, so a curriculum that verifies IC accuracy before expanding could improve robustness.
  • Editorial inference: a variational-inference version of the posterior could reduce the measured runtime penalty while keeping the same expansion criterion; the paper lists variational inference as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes B-PL-PINN, a training stabilization method for PINNs on forward PDE problems. It replaces the ensemble of PINNs used by Haitsiukevich and Ilin with a single Bayesian PINN, and uses the posterior variance of an MCMC ensemble as a proxy for consensus. Collocation points within a distance of existing labeled points are turned into pseudo-labels when the posterior variance is below a threshold and the closest label is consistent. The training domain (active collocation and boundary points) is expanded iteratively around these labels. Experiments on four 1D systems (reaction, diffusion, reaction-diffusion, convection) with two parameter settings each compare B-PL-PINN against vanilla PINN, ensemble variants, and Adam/LBFGS combinations. The method achieves relative L2 errors below 5% on all systems and is competitive with or better than the ensemble baselines, at 6–9 times the ensemble runtime.

Significance. If the claims hold, the paper offers a viable single-network alternative to ensemble-based domain-expanding PINN training, with uncertainty-based labeling, and provides evidence that Bayesian PINNs can be competitive with optimized ensemble training. The paper ships code, includes a clear ablation (Section V-B), and reports runtimes. However, the significance is reduced by the following: the ablation shows that posterior variance is not a reliable error proxy on some systems; the Bayesian No-PL baseline is actually better than Bayesian PL on 4 of 8 benchmarks; hyperparameters were tuned on a single problem; and results are single runs. These issues mean the contribution of Bayesian pseudo-labeling per se is not established.

major comments (3)
  1. [Section V-B and Table II] The paper's central mechanism is the pseudo-labeling rule (7c), which equates low posterior variance with accurate predictions. The ablation in Section V-B (Fig. 3) shows that for the reaction system (ρ=5) large parts of the domain have variance below σ²_consens despite substantial absolute error, both after the first iteration and after 10 iterations (e.g., x∈[0,2], t∈[0.75,1]). Moreover, Table II shows that the Bayesian No-PL variant achieves lower relative L2 error than Bayesian PL on four of the eight systems: diffusion d=5 (1.58e-2 vs 1.61e-2), reaction-diffusion d=2 (7.18e-3 vs 7.62e-3), convection β=30 (1.21e-2 vs 1.43e-2), and convection β=40 (1.17e-2 vs 3.39e-2). The latter case is a factor-of-three degradation. This directly contradicts the abstract's implication that Bayesian pseudo-labeling is the source of the gains; the distance constraint (7a) appears to be doing the work. The authors should either (i) establish conditions under which variance-based labels are reliable, (ii) report the two ablation variants as co-equal baselines and temper the 'mathematically principled' framing, or (iii) provide an explanation for why performance degrades when labels are added.
  2. [Section IV-C and Table II] Hyperparameters were tuned with Optuna on the convection system with β=30, then manually adjusted (Section IV-C, Table I), and the same values were applied to all eight benchmark systems. This creates a risk that the comparison is favorable only for a tuned configuration; no evidence is provided that the selected hyperparameters are not overfit to that system. In addition, the result for diffusion d=10 required increasing the initial Adam epochs from 4000 to 40,000 (Table II footnote), which is a post-hoc modification outside the stated protocol. All accuracy numbers in Table II are single runs without error bars or multiple seeds, so the claimed 'outperforms the ensemble' is not substantiated statistically; differences of a few 1e-3 could easily be within run-to-run variation. The authors should report mean±std over at least 5 seeds, and ideally a sensitivity analysis of the key thresholds (∆, σ²_consens, ϵ).
  3. [Section III-C] The algorithm is self-referential: pseudo-labels are added to the training set D_l, which is then used to compute the posterior (6c), whose variance determines the next batch of pseudo-labels. This can produce a confirmation loop in which the model becomes overconfident in its own incorrect predictions; the reaction-system ablation (Section V-B) provides evidence that this happens. The paper acknowledges that consensus alone is insufficient and that the distance constraint is essential, which is honest, but it undercuts the claim that the Bayesian mechanism is 'mathematically principled'. The authors should provide a theoretical analysis of when condition (7c) is a valid proxy for small error, or explicitly characterize the heuristic nature of the method.
minor comments (5)
  1. [Section II-C] The phrase 'estimate consent' should read 'estimate consensus'.
  2. [Algorithm 1 and Eq. (8)] The pseudo-label value is defined as the mean of the ensemble predictions in Eq. (8), but Algorithm 1 sets u := median(u_1, ..., u_N); this inconsistency should be resolved, and Section V-A's reference to the median operation suggests the algorithm is what was actually used.
  3. [Section IV-B] The phrase 'latin hyper-cube sampling' should be 'Latin hypercube sampling'.
  4. [Table III] The runtime entry '165m23' is missing a unit or separator; it should read '165m23s' or '165m 23s'.
  5. [Abstract and Conclusion] The term 'mathematically principled' is used without a definition; the authors should clarify what is meant, especially given the heuristic elements in the pseudo-labeling rule.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the B-PL-PINN procedure is self-training by design, but its reported predictions are evaluated against external analytic solutions and its central comparisons are not forced by construction.

full rationale

The paper's core mechanism—converting low-variance collocation points into pseudo-labels using the posterior predictive mean—is self-referential in the ordinary pseudo-labeling sense, but the paper does not present this as a derived first-principles prediction; it presents it as an algorithm and tests it against known solutions. The variance threshold σ²_consens and distance thresholds ∆, ∆pde are hyperparameters tuned with Optuna on one benchmark (convection β=30) and manually adjusted, but this is standard hyperparameter selection, not fitting the reported relative L2 errors; the reported errors are computed at randomly sampled points against analytic or numerical ground truth. The ablation (Section V-B, Fig. 3) explicitly shows that posterior variance alone is not a reliable error predictor for the reaction system, undermining any claim that the method reduces to an assumption that low variance equals accuracy. References [2] and [10] are self-citations by the authors, but they are used only to motivate the problem (loss landscape, Pareto front) and are not load-bearing for the proposed algorithm or its evaluation. No uniqueness theorem, renamed result, or fitted-input-called-prediction pattern is present. The honest non-finding is therefore appropriate.

Assumptions & free parameters 15 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the MCMC sampling being accurate, the Gaussian noise model being appropriate, the transferability of hyperparameters, and the empirical validity of variance as an uncertainty proxy. The listed free parameters are all tuned or hand-chosen, which is standard for such methods but limits the scientific weight of the comparisons.

free parameters (15)
  • distance threshold to closest labeled point (Delta) = 0.05
    Hand-chosen; controls how far from known labels a point may be pseudo-labeled.
  • distance threshold for adding collocation points (Delta_pde) = 0.1
    Hand-chosen; expands active training domain around pseudo-labels.
  • label agreement tolerance (epsilon) = 1e-3
    Hand-chosen; requires labeled anchor to match ensemble mean.
  • posterior variance threshold (sigma^2_consens) = 2e-4 (Optuna: 1.872e-4)
    Tuned on convection beta=30; triggers pseudo-label creation.
  • prior variance (sigma_p) = 5 (Optuna: 5.681)
    Tuned; Gaussian prior on network weights.
  • IC likelihood variance (sigma_ic) = 0.001 (Optuna: 1.016e-3)
    Tuned; Gaussian noise model for initial conditions.
  • pseudo-label likelihood variance (sigma_pl) = 0.005 (Optuna: 6.308e-3)
    Tuned; Gaussian noise model for pseudo-labels.
  • BC likelihood variance (sigma_bc) = 0.001 (Optuna: 1.041e-3)
    Tuned; Gaussian noise model for boundary conditions.
  • PDE residual likelihood variance (sigma_pde) = 0.01 (Optuna: 9.802e-3)
    Tuned; Gaussian noise model for physics residual.
  • number of posterior samples (N) = 100 (Optuna: 185)
    Tuned; samples drawn from MCMC per iteration.
  • burn-in samples (N_burnin) = 100 (Optuna: 46)
    Manually increased to improve sample quality.
  • leapfrog steps (N_leapfrog) = 127 (Optuna: 256)
    Manually halved to reduce runtime.
  • number of MCMC chains (N_chains) = 2 (Optuna: 1)
    Manually increased for better coverage.
  • MCMC target acceptance rate (delta_acc) = 0.6 (Optuna: 0.56)
    Tuned; controls exploration/exploitation trade-off.
  • initial ADAM epochs = 4000 (40000 for diffusion d=10)
    Chosen; used to pretrain before MCMC; increased after failure.
assumptions (5)
  • domain assumption HMC/NUTS samples converge to the true posterior of the Bayesian PINN.
    Invoked in Section III-B; the method uses MCMC samples as a model ensemble, assuming they represent the posterior.
  • domain assumption The Gaussian prior and likelihoods adequately model the believed noise and uncertainty.
    Section III-A Eqs. (6a)-(6c); all likelihoods are Gaussian with fixed variances.
  • standard math The PDE systems in Section IV-A have unique solutions that the MLP architecture can approximate.
    The paper follows the experimental setup of [4]; no proof of well-posedness or approximation capacity is given.
  • ad hoc to paper Posterior variance is a reliable proxy for prediction error in the active training region.
    Section V-B shows this holds for convection but is insufficient for reaction system without the distance constraint; the method assumes the combination works across all benchmarks.
  • ad hoc to paper The hyperparameters tuned on convection beta=30 transfer to the other seven systems.
    Section IV-C; Optuna and manual adjustment were performed on one system and then fixed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling." pith.science (2026). https://pith.science/paper/HIAWEOBQ

@misc{pith2026250701714,
  author       = {Pith},
  title        = {Pith review of: B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HIAWEOBQ}},
  note         = {Machine review of arXiv:2507.01714}
}
read the original abstract

Training physics-informed neural networks (PINNs) for forward problems often suffers from severe convergence issues, hindering the propagation of information from regions where the desired solution is well-defined. Haitsiukevich and Ilin (2023) proposed an ensemble approach that extends the active training domain of each PINN based on i) ensemble consensus and ii) vicinity to (pseudo-)labeled points, thus ensuring that the information from the initial condition successfully propagates to the interior of the computational domain. In this work, we suggest replacing the ensemble by a Bayesian PINN, and consensus by an evaluation of the PINN's posterior variance. Our experiments show that this mathematically principled approach outperforms the ensemble on a set of benchmark problems and is competitive with PINN ensembles trained with combinations of Adam and LBFGS.

Figures

Figures reproduced from arXiv: 2507.01714 by the authors.

Figure 1
Figure 1. The solution provided by our B-PL-PINN for the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The absolute error (left) and posterior variance (right) [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The ground truth (top left), absolute error (left) and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 21 canonical work pages

  1. [1]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,

    M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” J. Comput. Phys. , vol. 378, pp. 686–707, 2019. [Online]. Available: https://doi.org/10.1016/j.jcp.2018.10.045

    doi:10.1016/j.jcp.2018.10.045 2019

  2. [2]

    On the role of fixed points of dynamical systems in training physics-informed neural networks,

    F. M. Rohrhofer, S. Posch, C. G ¨oßnitzer, and B. C. Geiger, “On the role of fixed points of dynamical systems in training physics-informed neural networks,” Trans. Mach. Learn. Res. , vol. 2023, 2023. [Online]. Available: https://openreview.net/forum?id=56cTmVrg5w

    work page 2023

  3. [3]

    Mitigating propagation failures in physics-informed neural networks using retain- resample-release (R3) sampling,

    A. Daw, J. Bu, S. Wang, P. Perdikaris, and A. Karpatne, “Mitigating propagation failures in physics-informed neural networks using retain- resample-release (R3) sampling,” in Proc. Int. Conf. on Machine Learning (ICML) , Honolulu, Hawaii, USA, Jul. 2023, pp. 7264–7302. [Online]. Available: https://proceedings.mlr.press/v202/daw23a.html

    work page 2023

  4. [4]

    Improved training of physics-informed neural networks with model ensembles,

    K. Haitsiukevich and A. Ilin, “Improved training of physics-informed neural networks with model ensembles,” in Proc. Int. Joint Conf. on Neural Networks (IJCNN) , Gold Coast, Australia, Jun. 2023, pp. 1–8. [Online]. Available: https://doi.org/10.1109/IJCNN54540.2023. 10191822

    arXiv 2023

  5. [5]

    MCMC using Hamiltonian dynamics,

    R. M. Neal et al., “MCMC using Hamiltonian dynamics,” Handbook of Markov Chain Monte Carlo , vol. 2, no. 11, p. 2, 2011

    work page 2011

  6. [6]

    The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo,

    M. D. Hoffman and A. Gelman, “The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1593–1623, 2014. [Online]. Available: https://dl.acm.org/doi/10.5555/2627435.2638586

    work page arXiv 2014

  7. [7]

    Griewank and A

    A. Griewank and A. Walther, Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition. SIAM, 2008. [Online]. Available: https://doi.org/10.1137/1.9780898717761

    doi:10.1137/1.9780898717761 2008

  8. [8]

    Understanding and mitigating gradient flow pathologies in physics-informed neural networks,

    S. Wang, Y . Teng, and P. Perdikaris, “Understanding and mitigating gradient flow pathologies in physics-informed neural networks,” SIAM Journal on Scientific Computing , vol. 43, no. 5, pp. A3055–A3081,

    work page

Show all 34 references
  1. [9]

    Self-adaptive loss balanced physics-informed neural networks,

    Z. Xiang, W. Peng, X. Liu, and W. Yao, “Self-adaptive loss balanced physics-informed neural networks,” Neurocomputing, vol. 496, pp. 11– 34, 2022

    2022

  2. [10]

    Data vs. physics: The apparent Pareto front of physics-informed neural networks,

    F. M. Rohrhofer, S. Posch, C. G ¨oßnitzer, and B. C. Geiger, “Data vs. physics: The apparent Pareto front of physics-informed neural networks,” IEEE Access , vol. 11, pp. 86 252–86 261, 2023, open- access; preprint available: arXiv:2105.00862 [cs.LG] . [Online]. Available: htt...

    2023 arXiv

  3. [11]

    Physics-informed neural networks with hard constraints for inverse design,

    L. Lu, R. Pestourie, W. Yao, Z. Wang, F. Verdugo, and S. G. Johnson, “Physics-informed neural networks with hard constraints for inverse design,” SIAM Journal on Scientific Computing , vol. 43, no. 6, pp. B1105–B1132, 2021

    2021

  4. [12]

    A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks,

    S. Dong and N. Ni, “A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks,” J. Comput. Phys. , vol. 435, p. 110242, 2021. [Online]. Available: https://doi.org/10.1016/j.jcp.2021.110242

    2021

  5. [13]

    How to avoid trivial solutions in physics- informed neural networks,

    R. Leiteritz and D. Pfl ¨uger, “How to avoid trivial solutions in physics- informed neural networks,” 2021, arXiv:2112.05620

    2021 arXiv

  6. [14]

    Learning in sinusoidal spaces with physics-informed neural networks,

    J. C. Wong, C. C. Ooi, A. Gupta, and Y . Ong, “Learning in sinusoidal spaces with physics-informed neural networks,” IEEE Trans. Artif. Intell. , vol. 5, no. 3, pp. 985–1000, 2024. [Online]. Available: https://doi.org/10.1109/TAI.2022.3192362

    2024

  7. [15]

    Characterizing possible failure modes in physics-informed neural networks,

    A. S. Krishnapriyan, A. Gholami, S. Zhe, R. M. Kirby, and M. W. Mahoney, “Characterizing possible failure modes in physics-informed neural networks,” in Proc. Advances in Neural Information Processing Systems (NeurIPS) , Dec. 2021, pp. 26 548– 26 560. [Online]. Available: http...

    2021

  8. [16]

    Respecting causality for training physics-informed neural networks,

    S. Wang, S. Sankaran, and P. Perdikaris, “Respecting causality for training physics-informed neural networks,” Computer Methods in Applied Mechanics and Engineering , vol. 421, p. 116813, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0045782524000690

    2024

  9. [17]

    Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations,

    A. D. Jagtap and G. E. Karniadakis, “Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations,” in Proc. AAAI 2021 Spring Symposium on Combining Artificial Intel...

    2021

  10. [18]

    A novel sequential method to train physics informed neural networks for Allen Cahn and Cahn Hilliard equations,

    R. Mattey and S. Ghosh, “A novel sequential method to train physics informed neural networks for Allen Cahn and Cahn Hilliard equations,” Computer Methods in Applied Mechanics and Engineering, vol. 390, p. 114474, 2022. [Online]. Available: https: //www.sciencedirect.com/scien...

    2022

  11. [19]

    PSO-PINN: physics-informed neural networks trained with particle swarm optimization,

    C. Davi and U. M. Braga-Neto, “PSO-PINN: physics-informed neural networks trained with particle swarm optimization,” 2022, arXiv:2202.01943. [Online]. Available: https://arxiv.org/abs/2202. 01943

    2022 arXiv

  12. [20]

    B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data,

    L. Yang, X. Meng, and G. E. Karniadakis, “B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data,” J. Comput. Phys. , vol. 425, p. 109913,

  13. [21]

    Prior choice affects ability of Bayesian neural networks to identify unknowns,

    D. Silvestro and T. Andermann, “Prior choice affects ability of Bayesian neural networks to identify unknowns,” 2020, arXiv:2005.04987. [Online]. Available: https://arxiv.org/abs/2005.04987

    2020 arXiv

  14. [22]

    Available: https://doi.org/10.1016/j.jcp.2020.109913

    [Online]. Available: https://doi.org/10.1016/j.jcp.2020.109913

    2020

  15. [23]

    On the limited memory BFGS method for large scale optimization,

    D. C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization,” Math. Program., vol. 45, no. 1-3, pp. 503–528,

  16. [24]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. Int. Conf. on Learning Representations (ICLR), San Diego, CA, USA, May 2015. [Online]. Available: http://arxiv.org/abs/1412.6980

    2015 arXiv

  17. [25]

    Challenges in training PINNs: A loss landscape perspective,

    P. Rathore, W. Lei, Z. Frangella, L. Lu, and M. Udell, “Challenges in training PINNs: A loss landscape perspective,” in Proc. Int. Conf. on Machine Learning (ICML) , Vienna, Austria, Jul. 2024. [Online]. Available: https://openreview.net/forum?id=mJGiFr8jLa

    2024

  18. [26]

    Fixing asymptotic uncertainty of Bayesian neural networks with infinite ReLU features,

    A. Kristiadi, M. Hein, and P. Hennig, “Fixing asymptotic uncertainty of Bayesian neural networks with infinite ReLU features,” 2020, arXiv:2010.02709. [Online]. Available: https://arxiv.org/abs/2010. 02709

    2020 arXiv

  19. [27]

    Optuna: A next-generation hyperparameter optimization framework,

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proc. ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining (KDD), Anchorage, AK, USA, Aug. 2019, pp. 2623–2631. [Online]. Available: https://doi.or...

    2019

  20. [28]

    Tensorflow distributions,

    J. V . Dillon et al., “Tensorflow distributions,” 2017. [Online]. Available: http://arxiv.org/abs/1711.10604

    2017 arXiv

  21. [29]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke et al. , “Pytorch: An imperative style, high-performance deep learning library,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada, Dec. 2019, pp. 8024–8035. [Online]. Available: https://proceedings.neurips.cc/paper/ 2019/has...

    2019

  22. [30]

    TensorFlow: Large-scale machine learning on heterogeneous systems,

    M. Abadi et al. , “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org. [Online]. Available: https://www.tensorflow.org/

    2015

  23. [31]

    On last-layer algorithms for classification: Decoupling representation from uncertainty estimation,

    N. Brosse, C. Riquelme, A. Martin, S. Gelly, and E. Moulines, “On last-layer algorithms for classification: Decoupling representation from uncertainty estimation,” 2020, arXiv:2001.08049. [Online]. Available: https://arxiv.org/abs/2001.08049

    2020 arXiv

  24. [33]

    The relevance of Bayesian layer positioning to model uncertainty in deep Bayesian active learning,

    J. Zeng, A. Lesnikowski, and J. M. Alvarez, “The relevance of Bayesian layer positioning to model uncertainty in deep Bayesian active learning,” in Workshop on Bayesian Deep Learning @ NIPS, Montr ´eal, Canada, Dec. 2018. [Online]. Available: https: //bayesiandeeplearning.org/...

    2018

  25. [1989]

    Available: https://doi.org/10.1007/BF01589116

    [Online]. Available: https://doi.org/10.1007/BF01589116

    doi

  26. [2021]

    Available: https://doi.org/10.1137/20M1318043

    [Online]. Available: https://doi.org/10.1137/20M1318043

    doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.