{"id":"6aa249d9-c58e-4fe3-bf29-af4b3159f7f8","arxiv_id":"2507.01714","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":15,"one_line_summary":"A Bayesian PINN with posterior-variance-based pseudo labeling stabilizes PINN training and outperforms the ensemble baseline on six of eight benchmark PDE problems.","lead":"This paper replaces an ensemble of physics-informed neural networks with one Bayesian network that uses the spread of its predictions to decide where to add new training points. The resulting method stabilizes training for forward PDE problems and often beats the ensemble baseline, but the evaluation rests on single runs and tuning choices.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Posterior-variance pseudo-labeling may inject incorrect labels; the paper's own ablation shows low variance in high-error regions for the reaction system.","rationale":"The reader's weakest_assumption identifies essentially the same load-bearing concern: low posterior variance near labeled points is assumed to imply accurate predictions, but the paper's own ablation shows counterexamples (reaction system). This is the key assumption on which the pseudo-labeling mechanism rests. If it fails, wrong labels are injected, and the method's benefit over plain Bayesian training becomes questionable. The concrete test would settle whether the labels are actually correct in practice. The verdict remains CONDITIONAL as the reader already stated, because the empirical claim is plausible but not statistically robust. My analysis does not change that verdict, so UNCHANGED is appropriate.","tokens_in":13252,"tokens_out":5643,"duration_ms":63829,"concrete_test":"Instrument the released code to log every newly created pseudo-label (x,t,u), along with the ensemble variance and distance to the nearest labeled point, during training on the reaction system ρ=5 and convection β=40. After training, compare each pseudo-label value u to the exact solution u(x,t) and compute the fraction of added pseudo-labels whose relative error exceeds 10% (or absolute error > 0.05). Repeat for all eight benchmarks. If the fraction of bad pseudo-labels is non-negligible (e.g., >10%) in systems where PL still helps, the variance-criterion assumption is empirically false and the success must be attributed to other factors; if the fraction is near zero, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central mechanism is Eq. (7c): a low posterior variance at a candidate collocation point is treated as evidence that the ensemble prediction is accurate enough to be fixed as a pseudo-label. The paper's own ablation (Section V-B, Fig. 3) contradicts this premise: for the reaction system with ρ=5, after the initial iteration the posterior variance is below σ²_consens over most of the spatial domain even where the absolute error is large (e.g., x∈[2,4] at early t), and after 10 iterations large-error regions with low variance persist (e.g., x∈[0,2], t∈[0.75,1]). The algorithm still succeeds because the distance constraint (7a) restricts labeling to points within Δ of a trusted label, but this is an ad-hoc geometric heuristic, not a Bayesian guarantee. If the variance criterion periodically admits bad labels, the method trains on wrong targets. This concern is supported by Table II: Bayesian PL is worse than Bayesian No-PL on 3 of 8 systems (notably convection β=40: 3.39e-2 vs 1.17e-2), indicating pseudo-labeling can be detrimental. The headline claim that B-PL-PINN beats ensembles holds, but the responsible mechanism--Bayesian pseudo-labeling--is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes B-PL-PINN, a training stabilization method for PINNs on forward PDE problems. It replaces the ensemble of PINNs used by Haitsiukevich and Ilin with a single Bayesian PINN, and uses the posterior variance of an MCMC ensemble as a proxy for consensus. Collocation points within a distance of existing labeled points are turned into pseudo-labels when the posterior variance is below a threshold and the closest label is consistent. The training domain (active collocation and boundary points) is expanded iteratively around these labels. Experiments on four 1D systems (reaction, diffusion, reaction-diffusion, convection) with two parameter settings each compare B-PL-PINN against vanilla PINN, ensemble variants, and Adam/LBFGS combinations. The method achieves relative L2 errors below 5% on all systems and is competitive with or better than the ensemble baselines, at 6–9 times the ensemble runtime.","tokens_in":13612,"tokens_out":6269,"duration_ms":61576,"significance":"If the claims hold, the paper offers a viable single-network alternative to ensemble-based domain-expanding PINN training, with uncertainty-based labeling, and provides evidence that Bayesian PINNs can be competitive with optimized ensemble training. The paper ships code, includes a clear ablation (Section V-B), and reports runtimes. However, the significance is reduced by the following: the ablation shows that posterior variance is not a reliable error proxy on some systems; the Bayesian No-PL baseline is actually better than Bayesian PL on 4 of 8 benchmarks; hyperparameters were tuned on a single problem; and results are single runs. These issues mean the contribution of Bayesian pseudo-labeling per se is not established.","major_comments":[{"comment":"The paper's central mechanism is the pseudo-labeling rule (7c), which equates low posterior variance with accurate predictions. The ablation in Section V-B (Fig. 3) shows that for the reaction system (ρ=5) large parts of the domain have variance below σ²_consens despite substantial absolute error, both after the first iteration and after 10 iterations (e.g., x∈[0,2], t∈[0.75,1]). Moreover, Table II shows that the Bayesian No-PL variant achieves lower relative L2 error than Bayesian PL on four of the eight systems: diffusion d=5 (1.58e-2 vs 1.61e-2), reaction-diffusion d=2 (7.18e-3 vs 7.62e-3), convection β=30 (1.21e-2 vs 1.43e-2), and convection β=40 (1.17e-2 vs 3.39e-2). The latter case is a factor-of-three degradation. This directly contradicts the abstract's implication that Bayesian pseudo-labeling is the source of the gains; the distance constraint (7a) appears to be doing the work. The authors should either (i) establish conditions under which variance-based labels are reliable, (ii) report the two ablation variants as co-equal baselines and temper the 'mathematically principled' framing, or (iii) provide an explanation for why performance degrades when labels are added.","section":"Section V-B and Table II"},{"comment":"Hyperparameters were tuned with Optuna on the convection system with β=30, then manually adjusted (Section IV-C, Table I), and the same values were applied to all eight benchmark systems. This creates a risk that the comparison is favorable only for a tuned configuration; no evidence is provided that the selected hyperparameters are not overfit to that system. In addition, the result for diffusion d=10 required increasing the initial Adam epochs from 4000 to 40,000 (Table II footnote), which is a post-hoc modification outside the stated protocol. All accuracy numbers in Table II are single runs without error bars or multiple seeds, so the claimed 'outperforms the ensemble' is not substantiated statistically; differences of a few 1e-3 could easily be within run-to-run variation. The authors should report mean±std over at least 5 seeds, and ideally a sensitivity analysis of the key thresholds (∆, σ²_consens, ϵ).","section":"Section IV-C and Table II"},{"comment":"The algorithm is self-referential: pseudo-labels are added to the training set D_l, which is then used to compute the posterior (6c), whose variance determines the next batch of pseudo-labels. This can produce a confirmation loop in which the model becomes overconfident in its own incorrect predictions; the reaction-system ablation (Section V-B) provides evidence that this happens. The paper acknowledges that consensus alone is insufficient and that the distance constraint is essential, which is honest, but it undercuts the claim that the Bayesian mechanism is 'mathematically principled'. The authors should provide a theoretical analysis of when condition (7c) is a valid proxy for small error, or explicitly characterize the heuristic nature of the method.","section":"Section III-C"}],"minor_comments":[{"comment":"The phrase 'estimate consent' should read 'estimate consensus'.","section":"Section II-C"},{"comment":"The pseudo-label value is defined as the mean of the ensemble predictions in Eq. (8), but Algorithm 1 sets u := median(u_1, ..., u_N); this inconsistency should be resolved, and Section V-A's reference to the median operation suggests the algorithm is what was actually used.","section":"Algorithm 1 and Eq. (8)"},{"comment":"The phrase 'latin hyper-cube sampling' should be 'Latin hypercube sampling'.","section":"Section IV-B"},{"comment":"The runtime entry '165m23' is missing a unit or separator; it should read '165m23s' or '165m 23s'.","section":"Table III"},{"comment":"The term 'mathematically principled' is used without a definition; the authors should clarify what is meant, especially given the heuristic elements in the pseudo-labeling rule.","section":"Abstract and Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper's own ablation and the No-PL comparison are the key issues: the data do not currently support the claim that Bayesian pseudo-labeling is the responsible mechanism, and the single-run, tuned-on-one-system results make the headline 'outperforms ensemble' fragile. I would ask the authors to reposition the contribution as a domain-expansion method with optional pseudo-labels, or to provide substantially stronger evidence for the pseudo-labeling component (e.g., multi-seed results and a system-by-system analysis of when labels help or hurt)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Kevin, you should know this paper is a decent incremental contribution, but the central mechanism is not as clean as advertised. The authors replace the ensemble in Haitsiukevich and Ilin's pseudo-labeling scheme with a Bayesian PINN, using posterior variance as the consensus signal. That combination is new, and they test it on eight 1D PDE benchmarks against the ensemble baseline, with and without LBFGS, plus an ablation and a runtime table. Code is public. I think the strongest part is the honesty of the ablation: Figure 3 shows that for the reaction system, low posterior variance persists in high-error regions, so variance alone is not a reliable label-quality signal. They acknowledge this and note that the distance-to-labeled-points constraint is essential. That's a real contribution, but it cuts against the 'mathematically principled' phrasing in the abstract.\n\nThe soft spots are the usual ones. All reported errors are single runs with no error bars, so the six-of-eight wins over the ensemble could be noise. Hyperparameters were tuned on one problem (convection β=30) and then manually adjusted, which is common but worth flagging. One benchmark required a post-hoc increase in initial Adam epochs, which they mark with an asterisk; fine, but it suggests some fragility. More importantly, pseudo-labeling is not always helpful: the Bayesian No-PL variant beats Bayesian PL on convection β=40 (1.17e-2 vs 3.39e-2) and is comparable on diffusion. So the benefit of the Bayesian pseudo-labels themselves is not established; the gain seems to come mostly from the active-training-domain expansion, which is the same as the ensemble method.\n\nI would not call this a takedown. The paper is clearly written, the comparison is fair, and the runtime analysis (5-9x more expensive) is honest. But the advertised advantage of replacing the ensemble with a Bayesian net is partially undermined by its own results. I'd send it to a serious venue with a request for repeated runs and error bars, and a revised abstract that does not overclaim the principle.\n\nWho is this for? Researchers working on PINN training stabilization, specifically those considering ensemble or Bayesian approaches. They'll get a useful data point and a good ablation. I would not cite it as establishing the superiority of Bayesian pseudo-labeling, but I'd cite it as a fair empirical comparison. For peer review: yes, it deserves referee time, but it needs revision before acceptance.","headline":"A useful, honest comparison of Bayesian pseudo-labeling vs ensembles for PINN stabilization, but the variance-as-consensus mechanism is shakier than the title suggests.","tokens_in":14154,"tokens_out":1945,"would_cite":false,"duration_ms":21909,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Bayesian PINN stabilizes forward PDE training by labeling low-variance points.","keywords":["physics-informed neural networks","Bayesian pseudo labeling","partial differential equations","Bayesian neural networks","posterior variance","training stabilization","MCMC","forward problems"],"falsifier":"Run the B-PL-PINN procedure on the reaction system with $\\rho=5$ while removing the distance constraint (making $\\Delta$ arbitrarily large and relying on the variance threshold alone). If the error grows in low-variance regions similar to those shown in the paper, or if any collocation point with posterior variance below $\\sigma^2_{\\mathrm{consens}}$ has large absolute error during training, the labeling criterion is not reliable and the stabilization claim would be falsified.","tokens_in":13020,"feed_emoji":"🧮","tokens_out":9386,"duration_ms":92884,"temperature":0.7,"pith_summary":"Training physics-informed neural networks (PINNs) for forward problems often gets stuck because information from initial and boundary conditions does not propagate into the interior of the computational domain. This paper proposes B-PL-PINN, a training method that replaces the multi-network ensemble of earlier pseudo-labeling work with a single Bayesian PINN, and replaces ensemble consensus with a threshold on the Bayesian posterior variance. A collocation point becomes a pseudo-label only when it is close to an already labeled point and the sampled posterior agrees there, and only such points are active in the physics loss. On four benchmark 1D PDE systems the paper reports lower relative $L^2$ errors than the ensemble baselines on most of the eight tested settings and competitive performance against ensembles additionally optimized by LBFGS, at up to about nine times the baseline runtime.","feed_headline":"Uncertainty replaces ensembles in physics-informed neural training","feed_subtitle":"A single Bayesian PINN labels low-variance points to carry initial-condition information across PDE benchmarks.","key_machinery":"The load-bearing object is the Bayesian PINN posterior, a distribution over network parameters built from a Gaussian prior and Gaussian likelihoods on labeled data (initial conditions plus pseudo-labels) and on unlabeled physics and boundary residuals. Markov chain Monte Carlo (HMC or NUTS) draws parameter samples from this posterior; the variance of the sampled solutions at a collocation point is the consensus measure. Pseudo-labeling then expands the training domain: a point is labeled only when the variance is below $\\sigma^2_{\\mathrm{consens}}$, the point is within distance $\\Delta$ of a labeled point, and the nearest label is consistent with the sampled mean within $\\epsilon$; subsequently, only collocation and boundary points within $\\Delta_{\\mathrm{pde}}$ of a label contribute to the loss. This incremental, uncertainty-gated expansion is what carries information from the initial condition into the interior.","core_discovery":"The central claim is that the Bayesian posterior variance of a physics-informed neural network can serve as a mathematically grounded surrogate for ensemble consensus, and that combining this variance criterion with proximity to labeled points yields stable propagation of initial-condition information. Concretely, the paper samples $N$ parameter sets from the posterior via Hamiltonian Monte Carlo, computes the variance and median of the sampled solutions at candidate collocation points, and adds a pseudo-label only if the point lies within distance $\\Delta$ of an existing label, the closest label agrees with the sampled mean within $\\epsilon$, and the posterior variance is below $\\sigma^2_{\\mathrm{consens}}$. The paper reports that this procedure outperforms the ensemble baselines on most of the eight tested systems and is competitive with ensemble methods trained with both Adam and LBFGS, at a runtime below 9 times that of the ensemble baseline.","pith_inferences":["Editorial inference: the same variance-threshold rule could be plugged into cheaper uncertainty estimators, such as deep ensembles or MC dropout, but the paper does not test this.","Editorial inference: the failed diffusion case with the larger parameter suggests the method inherits a strong dependence on a well-fit initial condition, so a curriculum that verifies IC accuracy before expanding could improve robustness.","Editorial inference: a variational-inference version of the posterior could reduce the measured runtime penalty while keeping the same expansion criterion; the paper lists variational inference as future work."],"forward_implications":["A single network can substitute for an ensemble in pseudo-label expansion, so the method cuts the model count while keeping accuracy on the tested benchmarks.","The variance-plus-distance labeling rule is the mechanism, and the ablation indicates the distance constraint is necessary because posterior variance alone is not a reliable error indicator for the reaction system.","Because the training domain expands according to epistemic uncertainty rather than a fixed schedule, easier-to-learn regions advance to larger times faster, as shown in the convection-system propagation behavior.","The method composes with other stabilization devices such as loss weighting, domain decomposition, and curriculum learning."],"supporting_citations":[{"why":"Defines the PINN loss and training setup that the proposed method modifies.","marker":"[1]"},{"why":"Documents the propagation-failure problem that motivates pseudo-label training-domain expansion.","marker":"[3]"},{"why":"Supplies the ensemble pseudo-labeling algorithm and baselines replaced by the Bayesian approach.","marker":"[4]"},{"why":"Provides the Hamiltonian Monte Carlo sampler used to draw posterior parameter samples.","marker":"[5]"},{"why":"Provides the NUTS sampler considered as an alternative to HMC.","marker":"[6]"},{"why":"Introduces Bayesian PINNs and MCMC-based posterior sampling, the foundation of the method.","marker":"[20]"},{"why":"Motivates the Adam-plus-LBFGS ensemble baseline that the method is compared against.","marker":"[25]"},{"why":"Backs the ablation discussion of Bayesian neural networks underestimating uncertainty far from training points.","marker":"[26]"}],"fun_headline_variants":["Bayesian pseudo labels stabilize PINN training","Posterior variance replaces ensembles in PINNs","Single Bayesian PINN outperforms ensembles on PDEs","Variance-guided labeling stabilizes physics-informed nets","Bayesian PINN uses uncertainty to propagate initial data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that low posterior variance at a collocation point near an already labeled point means the model's prediction there is accurate enough to be frozen as a pseudo-label; the paper's own ablation shows this holds for the convection system but is not sufficient for the reaction system without the added distance constraint.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian pseudo labels stabilize PINN training","Posterior variance replaces ensembles in PINNs","Single Bayesian PINN outperforms ensembles on PDEs","Variance-guided labeling stabilizes physics-informed nets","Bayesian PINN uses uncertainty to propagate initial data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1219,"prompt_tokens":852,"completion_tokens":367,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":296}},"tokens_in":468,"tokens_out":367,"duration_ms":4409,"temperature":1.0,"reasoning_tokens":296,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:44:46.580580+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the B-PL-PINN procedure on the reaction system with $\\rho=5$ while removing the distance constraint (making $\\Delta$ arbitrarily large and relying on the variance threshold alone). If the error grows in low-variance regions similar to those shown in the paper, or if any collocation point with posterior variance below $\\sigma^2_{\\mathrm{consens}}$ has large absolute error during training, the labeling criterion is not reliable and the stabilization claim would be falsified.","supporting_citations":[{"cited_title":"Mitigating propagation failures in physics-informed neural networks using retain- resample-release (R3) sampling,","cited_arxiv_id":null,"evidence_quote":"Documents the propagation-failure problem that motivates pseudo-label training-domain expansion."},{"cited_title":"MCMC using Hamiltonian dynamics,","cited_arxiv_id":null,"evidence_quote":"Provides the Hamiltonian Monte Carlo sampler used to draw posterior parameter samples."},{"cited_title":"The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo,","cited_arxiv_id":null,"evidence_quote":"Provides the NUTS sampler considered as an alternative to HMC."},{"cited_title":"B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data,","cited_arxiv_id":null,"evidence_quote":"Introduces Bayesian PINNs and MCMC-based posterior sampling, the foundation of the method."},{"cited_title":"Challenges in training PINNs: A loss landscape perspective,","cited_arxiv_id":null,"evidence_quote":"Motivates the Adam-plus-LBFGS ensemble baseline that the method is compared against."},{"cited_title":"An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence","cited_arxiv_id":"2010.02709","evidence_quote":"Backs the ablation discussion of Bayesian neural networks underestimating uncertainty far from training points."}],"review_version":1}