{"id":"f7e1874b-4bca-43b8-b197-63bbf056d1c5","arxiv_id":"2411.17858","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An interleaved adaptive Gaussian process surrogate that optimizes both evaluation locations and tolerances can reduce the computational cost of posterior sampling in Bayesian inverse problems.","lead":"This paper proposes an adaptive strategy for Bayesian inverse problems where a Gaussian process surrogate is trained while posterior samples are drawn, jointly choosing where to evaluate the forward model and how accurately. The authors report significant computational savings compared with position-adaptive or static training designs on test problems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The efficiency claim rests on simulated Gaussian discretization noise; real FE errors violate the model stated in Remark 1, so the claimed computational savings may not transfer.","rationale":"The single condition that must hold for the central efficiency claim to be credible is that the tolerance τ used by the algorithm faithfully represents the statistical behavior of real evaluation errors. The paper's own Remark 1 states that the Gaussian independence assumption is 'for sure not satisfied' for FE discretization errors, and Section 5.1 confirms that all experiments only simulate such errors by adding Gaussian noise to analytic forward models. Since the acquisition function and tolerance optimization are derived from the GP predictive variance under that noise model, the demonstrated gains could be an artifact of the idealized setting rather than a property of the method for actual PDE solves. This is an external-validity concern rather than an internal contradiction: the derivations are largely self-consistent, and the reader's conditional verdict already accounts for this limitation. I considered other potential issues, such as the factor-2 slip in the proof of Theorem 1 and the reuse of stale posterior samples in the sliding-window sampler, but neither is as directly load-bearing for the abstract's empirical claim as the missing real-solver validation. The proposed FE benchmark is the one check that would settle whether the concern lands.","tokens_in":17568,"tokens_out":10632,"duration_ms":99168,"concrete_test":"Replace the Gaussian-noise emulation in Experiment 5.4 with an actual adaptive finite-element solver for the Poisson equation with point sources, imposing τ through an a posteriori error estimator and mesh refinement. Measure wall-clock time or total degrees of freedom for each strategy, and evaluate posterior quality against a reference solution computed on a very fine mesh using a sample-based Hellinger distance. If AGP-const and AGP-geom do not outperform posAGP and LHSGP for equal measured budget, the abstract's efficiency claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim is an empirical statement about computational effort: jointly optimizing evaluation positions and tolerances 'significantly' reduces cost versus position-adaptive and static designs. The evidence is generated under the Section 3.1 assumption that each evaluation error is independent N(0, τ_i^2). Remark 1 concedes this is false for finite-element discretization errors, and Section 5.1 implements the experiments by adding Gaussian noise to analytic forward models; no actual discretization is solved. Consequently, the work model W(τ)=τ^{-l/r} in Eq. (15) is assigned, not measured, and the tolerance-optimization objectives in Eqs. (18) and (21) are driven by a predictive variance Γ(p) computed from the assumed noise variances. For real FE solves the error is deterministic, typically biased and spatially correlated; treating τ as a standard deviation misstates the likelihood and can misdirect the choice of tolerances. The numerical comparison therefore establishes a reduction within the Gaussian-noise idealization, but not yet the claimed reduction for the intended application.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a fully adaptive Gaussian-process surrogate strategy for sampling Bayesian posteriors. Starting from an initial design, Algorithm 1 interleaves MCMC sampling from the current surrogate-based posterior with greedy design updates: candidate evaluation points are selected as local maximizers of the acquisition function (20), and evaluation tolerances are re-optimized in (21) under a work-model constraint. Two error models are derived, the KL-divergence model (13) based on Theorem 1 and Lemma 1, and the posterior-weighted L2 model (14). Numerical experiments on a 2D synthetic example, a 3D diffusion example, and a 4D Poisson example compare AGP-const and AGP-geom with position-only-adaptive (posAGP) and static LHSGP designs, reporting improved convergence and fewer training points. The abstract's central claim is that joint optimization of positions and tolerances significantly reduces computational effort.","tokens_in":17752,"tokens_out":7908,"duration_ms":72598,"significance":"Strengths: the fully stochastic likelihood (6) is a principled treatment of surrogate uncertainty; Theorem 1 and Lemma 1 provide explicit, analytically derived error bounds; Appendix A gives closed-form derivatives, making the acquisition and tolerance optimization concrete and reproducible; the experimental matrix across dimensions, cost models, and error metrics is extensive. If the observed gains survive contact with genuine adaptively-refined PDE solvers, the method would be a practical improvement over position-only-adaptive surrogates. However, the current numerical evidence is produced under the Gaussian-noise idealization of Section 3.1, not under the FE discretization errors the method is motivated by, so the significance for the intended application is not yet established.","major_comments":[{"comment":"The central numerical claim in the abstract, that jointly optimizing evaluation positions and tolerances yields a significant reduction of computational effort, is supported only under the idealized error model of Section 3.1, in which evaluation errors are independent N(0, tau_i^2) draws. Section 5.1 implements this by adding Gaussian noise to analytic forward models; no actual adaptive FE solver is run. Because FE discretization errors are deterministic, generally biased, and spatially correlated (as Remark 1 concedes), the tolerance-optimization steps in (18) and (21) use the wrong likelihood for the intended application. A concrete test would be to run Algorithm 1 with an actual adaptive FE solve, or with a deterministic error model having known, correlated error, and to report whether the observed reduction over posAGP and LHSGP persists; without such a test the abstract's claim is overstated.","section":"5.1; Remark 1"},{"comment":"Theorem 1 assumes pi_D(y_m) <= alpha pi(y_m) for some finite alpha, but the error model sets log alpha = 0 'optimistically'. Since L_D(p) is the convolution of the measurement error with the GP predictive density, the evidence ratio pi_D(y_m)/pi(y_m) is not generally bounded by 1, especially when the surrogate mean is biased. Dropping log alpha systematically underestimates e^KL_D(p) and can change the selected positions and tolerances in (20) and (21). The authors should either justify alpha = 1 from the setting, estimate the ratio, or report the sensitivity of the resulting designs to the value of log alpha.","section":"4.1.1, Eq. (13)"},{"comment":"The work model W(tau) = tau^{-l/r} is assumed and not measured, and the reported savings are expressed in units of this assumed model rather than in wall-clock time or actual solver effort. Table 1 and Table 2 report numbers of training points, but the adaptive strategies also spend computational effort on multi-start optimization for (20), SLSQP for (21), and MCMC sampling between iterations; this overhead is not included in the comparison. To substantiate the claimed reduction in computational effort, the paper should report the overhead of the design optimization and, ideally, total runtime for each strategy.","section":"5.2-5.4, Eq. (15)"},{"comment":"The acquisition function and the tolerance optimization in (19) replace the posterior pi(.|y_m,D_j) by the currently available pi(.|y_m,D_{j-1}) and freeze the predictive mean at the previous iterate. These substitutions are acknowledged, but they are not covered by Theorem 1, so the resulting error model is an ad-hoc heuristic whose failure modes are not assessed. The numerical section should include a sensitivity check, for example comparing the designs produced by the heuristic against a small number of random design perturbations, or against an oracle that evaluates (8) with the exact posterior, to show that the design decisions are robust to this approximation.","section":"4.3.1, Eq. (18)"}],"minor_comments":[{"comment":"The sentence 'Gaussian process Regression (GPR) is an approximation technique that can efficiently learn large classes of functions [23], providing a .' contains an incomplete final phrase; the missing object should be supplied.","section":"3.1"},{"comment":"The notation r_j is used before it is defined, and the final design is written as D_j = ((p_i,tau_i))_{i=1,...,s_j+1}, which does not clearly account for candidates excluded by tau_i = infinity; please clarify the indexing.","section":"Algorithm 1, step 3"},{"comment":"The convergence plots do not label their axes; please state explicitly whether the horizontal axis is iteration number or accumulated work and what the vertical metric is (e.g., E^KL or E^{L2}).","section":"Figures 5, 6, 10"},{"comment":"Reference [30] is cited only as 'In Proceedings Of The Conference Algoritmy, 2024'; please provide complete bibliographic information.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's headline result is validated only on simulated Gaussian discretization noise; for a numerical-analysis venue this is a substantial gap. I would encourage requiring at least one experiment with a genuine adaptive FE solver, or a clear re-scoping of the abstract to the Gaussian-noise model, together with an honest treatment of alpha in Eq. (13)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid, honest extension of the authors' earlier adaptive surrogate-sampling idea. What is genuinely new: a KL-divergence error model with a derived bound (Theorem 1, Lemma 1), two error metrics, a complete statement of Algorithm 1, and systematic experiments on 2D, 3D, and 4D inverse problems. The derivations in Section 4 check out, and the authors are upfront in Remark 1 that the independent Gaussian evaluation-error assumption is false for finite-element discretization errors. Within its own idealized setup, the comparison against position-adaptive and static designs is useful and the tolerance-optimization results are internally consistent.\n\nThe soft spots are real but not fatal. The abstract claims a \"significant reduction of the computational effort\" for the intended application, yet every experiment in Section 5.1 replaces discretization error with zero-mean Gaussian noise added to analytic forward models. No actual FE solve is performed. For real FE errors, which are deterministic, biased, and spatially correlated, the likelihood in (6) and the tolerance-optimization objectives in (18) and (21) are formally mis-specified. These experiments therefore establish an efficiency gain under the Gaussian-noise idealization, not yet for real discretized PDEs. The stress-test note is correct on this point, but it is also fair to say the authors flag the limitation themselves; the problem is that the abstract overstates the transfer. A second, smaller issue is that alpha is set to 1 in (13) even though Theorem 1 only assumes a finite alpha, which may be larger; the authors call this optimistic, and it weakens the bound's sharpness but not the algorithm's practical behavior. No code or data is provided, which makes independent benchmarking harder and should be addressed in any revised version.\n\nWho is this for? Researchers working on surrogate-based Bayesian inversion, adaptive experimental design, and UQ for expensive forward models. It deserves a serious referee: the theoretical error bound is a real contribution, the algorithm is clearly specified, and the experiments are extensive for the idealized setting. I would send it to peer review, with the recommendation that the authors either add one real FE-based experiment that respects the work model, or temper the abstract and conclusions to make the Gaussian-noise idealization explicit.","headline":"Careful extension of adaptive GP surrogates with a real KL error bound, but the headline savings claim is only demonstrated under simulated Gaussian solver noise, not real FE solves.","tokens_in":18268,"tokens_out":1736,"would_cite":true,"duration_ms":17629,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","60G15","65C60","65N21"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that interleaving posterior sampling with joint optimization of evaluation positions and tolerances substantially reduces the cost of surrogate-based Bayesian posterior sampling.","keywords":["Bayesian inverse problems","Gaussian process regression","adaptive design of experiments","posterior sampling","surrogate models","evaluation tolerance","Markov chain Monte Carlo","uncertainty quantification"],"falsifier":"Run the same algorithm on a genuine adaptive finite-element problem with a corner singularity, where discretization error is controlled by mesh refinement and is not independent Gaussian noise, and compare posterior error at the same budget; if the tolerance-optimizing variant does not beat position-only adaptation, the central efficiency claim fails.","tokens_in":17349,"feed_emoji":"🎯","tokens_out":8231,"duration_ms":71574,"temperature":0.7,"pith_summary":"This paper argues that, when a Gaussian-process surrogate stands in for an expensive forward model in a Bayesian inverse problem, the surrogate should be built and the posterior sampled as one interleaved process, not as separate phases. Its central claim is that greedily choosing new evaluation points by their sensitivity to posterior error and simultaneously re-optimizing how accurately old and new points are evaluated, all under a fixed work budget, substantially reduces the computational effort needed to sample the posterior compared with adapting positions only or using a static space-filling design. The efficiency gain is demonstrated on three forward models (synthetic, diffusion, and Poisson) in 2, 3, and 4 parameter dimensions, with discretization error emulated by Gaussian noise. If the claim holds, practitioners with expensive PDE solvers could obtain faithful uncertainty estimates with fewer solver runs by spending accuracy only where the posterior places mass.","feed_headline":"Adaptive GP surrogates cut Bayesian posterior sampling cost","feed_subtitle":"Choosing where and how precisely to evaluate a forward model cuts the number of runs needed for inverse problems.","key_machinery":"The load-bearing object is the surrogate-based posterior $\\pi(p\\mid y^m,D)$ of Eq. (7), in which the Gaussian-process predictive distribution $y_D(p)\\sim N(\\bar y(p),\\Gamma(p))$ enters the likelihood through Eq. (6); including the predictive variance prevents the surrogate from overconfidently narrowing the posterior. Around this object the paper builds two computable error models, the GP-averaged KL upper bound (13) and the posterior-weighted $L^2$ error (14), and a work model $W(\\tau)=\\tau^{-l/r}$ for adaptive finite-element solves. The greedy mechanism is Algorithm 1: draw or update MCMC samples from the current surrogate posterior, use those samples to evaluate the acquisition function (20) for candidate positions, solve the tolerance optimization problem (21) under the remaining budget, and re-evaluate the forward model only at changed designs. The sliding-window recycling of samples is what makes the optimization problems cheap enough to solve at every iteration.","core_discovery":"The paper's central discovery is a working formulation of this interleaved idea: it converts the unknown error between the true posterior and the surrogate-based posterior into computable error models, namely an upper bound on the Kullback-Leibler divergence (Theorem 1, Eq. (13)) and a posterior-weighted $L^2$ error model (Eq. (14)), and then uses these to drive a sequential greedy design. At each iteration, MCMC samples from the current surrogate-based posterior weight the acquisition function (20) that scores candidate evaluation points, and the same samples discretize the tolerance optimization problem (21) that reallocates the remaining budget among old and new points. The paper shows numerically that this joint adaptation beats position-only adaptation, that tolerance optimization is most valuable when more accurate solves are cheap relative to the work model $W(\\tau)=\\tau^{-l/r}$, and that the fully adaptive designs end with far fewer training points, so both forward evaluations and surrogate evaluations become cheaper.","pith_inferences":["Because the experiments emulate discretization error as independent zero-mean Gaussian noise with variance $\\tau^2$, the reported efficiency gains are best read as an upper bound for real adaptive PDE solvers, whose discretization errors are correlated and not Gaussian; the acquisition rule may misallocate budget in that regime.","Tolerance re-optimization assumes evaluations can be stored and continued from previous grids; for black-box solvers that cannot resume a solve, the method loses much of its advantage even if the error model held.","A natural testable extension is to treat the reported tolerance $\\tau_i$ as a tunable hyperparameter or to model correlated error components, and to compare the greedy acquisition (20) against a fully Bayesian design under the same budget.","The bound (13) sets $\\alpha=1$ optimistically once the surrogate is reasonable, so in early iterations the true bound may be larger and practical convergence could depend on the initial design more than the asymptotic analysis suggests."],"forward_implications":["Under the paper's error model, tolerance optimization plus position adaptation converges to the target posterior with fewer forward-model evaluations than position-only or static designs on the same budget.","Tolerance optimization pays off most when the work model exponent $l/r$ is small, i.e. when increasing evaluation accuracy is comparatively cheap; for expensive accurate solves, position adaptation nearly suffices.","Fully adaptive designs end with substantially fewer training points, making the surrogate itself cheaper to evaluate during sampling.","Using the stochastic likelihood (6) instead of plugging in the GP mean guards against overconfident posteriors in early iterations.","Reusing samples in a sliding window reduces sampling cost when the posterior changes little between iterations."],"supporting_citations":[{"why":"the interleaved surrogate-training-with-sampling strategy this paper extends and evaluates in detail","marker":"[30]"},{"why":"supplies the joint position-and-tolerance design formulation and the work model $W(\\tau)=\\tau^{-l/r}$","marker":"[25]"},{"why":"provides the Gaussian-process regression formulas and predictive-variance machinery that define the surrogate and error indicators","marker":"[23]"},{"why":"justifies sampling positions from the approximate posterior by convergence results for posterior-weighted experimental design","marker":"[15]"},{"why":"introduces the goal-oriented, posterior-weighted design-of-experiments viewpoint used in the error models","marker":"[27]"},{"why":"the affine-invariant ensemble sampler used to draw the MCMC samples that drive acquisition and tolerance selection","marker":"[13]"}],"fun_headline_variants":["Jointly adapt training points and tolerances to slash posterior sampling cost","Adaptive GP surrogates pick where and how precisely to sample","Greedy GP training with error bounds speeds Bayesian inference","Posterior-aware design cuts inverse problem sampling cost","Adapting both position and accuracy of GP surrogates pays off"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything hinges on the assumption that a forward-model evaluation with tolerance $\\tau$ returns the true value plus independent zero-mean Gaussian noise with standard deviation $\\tau$; real discretization errors from adaptive solvers are not independent or Gaussian, and the numerical experiments substitute Gaussian noise for actual solver error.","fun_headline_variants_meta":{"raw":{"variants":["Jointly adapt training points and tolerances to slash posterior sampling cost","Adaptive GP surrogates pick where and how precisely to sample","Greedy GP training with error bounds speeds Bayesian inference","Posterior-aware design cuts inverse problem sampling cost","Adapting both position and accuracy of GP surrogates pays off"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00046,"raw_usage":{"total_tokens":2291,"prompt_tokens":921,"completion_tokens":1370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":1285}},"tokens_in":537,"tokens_out":1370,"duration_ms":8706,"temperature":1.0,"reasoning_tokens":1285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:45:49.786892+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same algorithm on a genuine adaptive finite-element problem with a corner singularity, where discretization error is controlled by mesh refinement and is not independent Gaussian noise, and compare posterior error at the same budget; if the tolerance-optimizing variant does not beat position-only adaptation, the central efficiency claim fails.","supporting_citations":[{"cited_title":"Villani, J","cited_arxiv_id":null,"evidence_quote":"the interleaved surrogate-training-with-sampling strategy this paper extends and evaluates in detail"},{"cited_title":"Semler and M","cited_arxiv_id":null,"evidence_quote":"supplies the joint position-and-tolerance design formulation and the work model $W(\\tau)=\\tau^{-l/r}$"},{"cited_title":"Rasmussen and C.K.I","cited_arxiv_id":null,"evidence_quote":"provides the Gaussian-process regression formulas and predictive-variance machinery that define the surrogate and error indicators"},{"cited_title":"Helin, A.M","cited_arxiv_id":null,"evidence_quote":"justifies sampling positions from the approximate posterior by convergence results for posterior-weighted experimental design"},{"cited_title":"Sinsbeck and W","cited_arxiv_id":null,"evidence_quote":"introduces the goal-oriented, posterior-weighted design-of-experiments viewpoint used in the error models"},{"cited_title":"Goodman and J","cited_arxiv_id":null,"evidence_quote":"the affine-invariant ensemble sampler used to draw the MCMC samples that drive acquisition and tolerance selection"}],"review_version":1}