{"id":"f5e6284e-6cd6-4e95-84d9-fcfe2af68328","arxiv_id":"2411.17387","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"LOCBO calibrates the GP likelihood with localized online conformal prediction and then denoises it, claiming utility lower bounds that are not actually established for the expected-improvement setting.","lead":"This paper combines localized online conformal prediction with Bayesian optimization to make the surrogate model's uncertainty estimates robust when the Gaussian process prior is wrong. The authors report improved optimization in synthetic and wireless-network tasks, but the theoretical guarantee that is the paper's main advertised contribution rests on assumptions that are not satisfied by the experimental setup.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proof of Theorem 1 hinges on an invalid 'C=0 WLOG' step; with the nonnegative expected-improvement utility used in the experiments, inequality (54) reverses, so the utility certificate is unproven for the implemented algorithm.","rationale":"The reader's weakest_assumption was Assumption 4, which is indeed serious: no bound on ϵ is derived, and for misspecified GPs the posterior CDF ∫_{−∞}^{y} p(f|D∪{(x,y)}) df can be arbitrarily close to zero, making the certificate vacuous. However, the single most load-bearing defect is the C=0 'WLOG' step in the proof of Theorem 1. This step is load-bearing because the theorem is the paper's advertised central contribution, and the proof's algebraic chain collapses for the very utility used in all reported experiments. The theorem cannot be salvaged by reinterpreting C=0 as a normalization: because inequality (37) involves both u and aLOCBO, shifting u by a constant changes the inequality unless αϵ=2, which is not generally true. This is an internal inconsistency in the proof as stated, not an external condition that one could verify separately. The concrete test isolates the failure at Eq. (54) with elementary numerics and settles that the proof does not apply to nonnegative utilities such as expected improvement. Both flaws support the reader's REJECT verdict; the C=0 issue is the more decisive because it is a mathematical error within the theorem's own proof.","tokens_in":20813,"tokens_out":10459,"duration_ms":99896,"concrete_test":"Run the following analytic check: fix D_t=∅, GP posterior p(f|D_t)=N(0,1), utility u(f)=max(0,f) (the EI utility used in Section V), and α=0.2. In Appendix E choose f' as an α/2-quantile of the calibrated likelihood (≈−1.28) and evaluate both sides of inequality (54): the left side equals ∫_0^∞ f φ(f) df ≈ 0.399, while the right side equals u(f')·Pr[f≤f'] = 0. The inequality (54) is therefore reversed; verify that this reversal persists for any nonnegative utility, demonstrating that C=0 is essential and not a harmless normalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix E's proof of Theorem 1 requires 'setting, without loss of generality, C=0 in Assumption 3' (Section IV-E). This is not WLOG: Assumption 3 only bounds utility above, and subtracting C from u does not preserve the asserted inequality (37) because aLOCBO contains the same constant in the denominator. Consequently the proof's chain (53)-(59) is valid only for non-positive utilities u≤0. For the expected-improvement utility used in all experiments (Section V-A), u(f)=max(0,f−μ) is nonnegative and the key step (54) fails: with D empty, x fixed, posterior N(0,1), and f' the α/2-quantile (≈−1.28 for α=0.2), the right-hand side of (54) is u(x,f',D)·Pr[f≤f']=0, while the left-hand side aLOCBO(x|D)=∫_0^∞ f φ(f) df≈0.399>0. Thus Theorem 1's central certificate—that LOCBO's estimated utility lower-bounds the true utility of its iterates—is not established for the algorithm that is actually implemented and benchmarked. The unquantified Assumption 4 compounds the gap, but the C=0 step is an internal contradiction in the stated theorem.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LOCBO, a Bayesian optimization algorithm that calibrates the GP likelihood with localized online conformal prediction and then denoises it to obtain a calibrated posterior over the objective function. The main theoretical claims are Lemma 2, a long-run coverage guarantee for the unobserved objective under a noise-symmetry condition, and Theorem 1, a probabilistic certificate that the estimated acquisition utility lower-bounds the true utility of LOCBO's iterates. The method is evaluated against BO, CBO, OCBO, and an OCBO-L variant on a synthetic Ackley function and a UAV radio-resource-management problem, with code made publicly available.","tokens_in":21107,"tokens_out":9819,"duration_ms":98348,"significance":"If Theorem 1 were valid, the paper would make a useful contribution: it would extend online-CP calibration to noisy observations with localized, input-dependent coverage and connect the calibrated posterior to a performance certificate for the acquisition function. The empirical study covers relevant baselines and includes a real-world engineering problem, and the availability of code is a strength. However, the central utility certificate is not established for the implemented algorithm, and the proof contains a nonlocal error; the contribution as stated is therefore not yet supported.","major_comments":[{"comment":"The theorem's phrase 'setting, without loss of generality, C=0' is not a valid normalization. Assumption 3 only bounds u above; subtracting C from u changes the acquisition function aLOCBO in Eq. (30), so a bound proved for u-C does not transfer to the original u. Inequality (54) drops the integral over f>f'; this is an upper bound only when u is non-positive. For the expected-improvement utility used in Section V-A, u(f)=max(0,f-τ) is nonnegative, so the dropped term is nonnegative and inequality (54) reverses. Concretely, take f' below the threshold τ; then u(x,f',Dt)=0, the right-hand side of (54) is 0, while aLOCBO(x|Dt) is strictly positive for a nondegenerate posterior. Moreover, EI is unbounded above as f grows, so it does not satisfy the boundedness part of Assumption 3 even before any shift. Thus Theorem 1 is unproven for the algorithm actually implemented and benchmarked.","section":"Section IV-E / Appendix E, Eqs. (53)-(54)"},{"comment":"The step 'which implies f' not in Γ by construction (28)' is not justified. Equation (28) fixes the total mass outside ΓLOCBO to be α, but it does not control how that mass is split between the left and right tails. The left endpoint L of the interval can have CDF below α/2 while still belonging to Γ, so ∫_{-∞}^{f'} pLOCBO(y|x,Dt) dy < α/2 does not imply f' is outside the interval. This breaks the reduction to Lemma 2 even for the non-positive-utility version of the proof.","section":"Appendix E, Eqs. (60)-(61)"},{"comment":"Assumption 4 is load-bearing for Theorem 1 because ϵ appears in the denominator of the certificate, but the paper gives no argument that a misspecified GP posterior satisfies a uniform lower bound of size ϵ>0 over all histories and all pairs (x,y). For a GP posterior conditioned on an observation y at x, the CDF at y can be arbitrarily close to zero with sufficiently large observation noise or a conflicting prior, so the assumption may hold only with ϵ=0, in which case the theorem's inequality is vacuous. The assumption should be derived from concrete GP and noise conditions, or replaced by a quantitative statement with a lower bound on ϵ.","section":"Assumption 4, Eq. (36)"}],"minor_comments":[{"comment":"The notation in Eq. (36) is unclear: the integrand p(y'|Dt∪{(x,y)}) uses y' as both the integration variable and the conditioning observation; it should be written as p(f(x)=y'|Dt∪{(x,y)}) or clarified with a different symbol.","section":"Assumption 4, Eq. (36)"},{"comment":"Lemma 1 contains an additive kernel-norm term κ in the miscoverage bound, and the text says this radius vanishes 'in the absence of localization (κ=0)'. But in the experiments, localization is controlled by the RBF length scale l, with κ a fixed scaling parameter, while Lemma 1's κ is the kernel upper bound from Assumption 1. Setting κ=0 in Lemma 1 is not the same as the non-localized limit l=∞ used in Section V, so the connection between the theory and the localization sweep in Figure 6 is not made precise.","section":"Lemma 1 / Section V-B"},{"comment":"The closed-form expression for pLOCBOα(f(x)|Dt) in Eq. (45) is typeset as one unbroken equation and is very difficult to verify; please split it into labeled components and define all symbols before the display.","section":"Lemma 3, Eq. (45)"},{"comment":"The experimental comparison reports means and 70% confidence intervals over only 5 or 7 trials; no significance tests are reported, so it is unclear whether the observed improvements of LOCBO over OCBO-L and OCBO are stable across seeds.","section":"Section V-A"}],"recommendation":"reject","confidential_remarks":"To the editor: the main theorem's proof error is not a minor fix. The 'C=0 without loss of generality' step changes the acquisition function, and the natural repair of restricting the theorem to non-positive utilities excludes the expected-improvement utility used in all experiments. The proof also contains a second gap in the step from Eq. (60) to Eq. (61), and Assumption 4 remains unquantified. These issues jointly undermine the central theoretical claim of the paper. The empirical comparison and the application of localized online CP to BO may be publishable in a revised form, but the stated guarantees are not currently supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: LOCBO is a sensible combination of CBO's denoising, OCBO's online CP, and localized online CP, and the experimental work is solid; but the theoretical centerpiece, Theorem 1, is not established as stated. The proof's 'set C=0 without loss of generality' is not WLOG. Assumption 3 only bounds the utility above; shifting u by a constant changes the acquisition function, and for the expected-improvement utility used in all experiments, u is nonnegative, so the inequality at (54) runs the wrong way. The stress-test note's numerical example is correct: with an empty data set and standard normal posterior, the right-hand side is zero while aLOCBO is roughly 0.4. The proof only works for non-positive utilities, which no one uses. That's a load-bearing flaw because the abstract and intro sell Theorem 1 as a performance certificate.\n\nWhat the paper does well: the combination is genuinely new as far as I know, the denoising step is worked out in closed form, the code is public, and the experiments cover synthetic and real-world problems with honest comparisons. The localization effect is interesting and the empirical gains over CBO/OCBO are credible.\n\nSoft spots, in order of severity. First, Theorem 1 as above. Second, Lemma 1's bound includes the additive κ term, so for the localized kernels actually used (κ=2–5) the long-run coverage guarantee is vacuous; the paper acknowledges this, but the table in Figure 2 calls the guarantee 'exact, deterministic, localized' which oversells it. Third, the experiments use learning-rate decay w≈0.05 or 0.005, while the theory requires t^{-1/2}; the theoretical results do not cover the evaluated configuration. Fourth, Assumption 4 is never quantified or justified for misspecified GPs; if ϵ is tiny, the certificate is empty.\n\nNet: the algorithm may be useful, and the empirical study is worth something, but the advertised guarantees don't hold up. The right outcome is to send it to review and require a major revision: either repair Theorem 1 under a correct assumption (e.g., bounded nonnegative utility with a different argument) or drop the certificate and present the method heuristically with the empirical evidence. I'd not cite the theorem as-is, but I'd track the code.","headline":"Useful algorithm and honest experiments, but the advertised utility certificate (Theorem 1) is unproven: the 'C=0 WLOG' step fails for the nonnegative expected-improvement utility actually used.","tokens_in":21657,"tokens_out":3314,"would_cite":false,"duration_ms":32445,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LOCBO recalibrates Bayesian-optimization surrogates with localized online conformal prediction, giving a probabilistic certificate that estimated utility is a lower bound on true utility at queried points, even under misspecified models…","keywords":["Bayesian optimization","conformal prediction","online calibration","Gaussian process","model misspecification","black-box optimization","uncertainty quantification"],"falsifier":"Run LOCBO on a known objective with a deliberately misspecified GP (for example, wrong lengthscale or mean) under heavy-tailed or skewed noise, and measure the empirical frequency over T of the event u(xt, f(xt), Dt−1) ≥ 2 aLOCBO(xt|Dt−1)/(αϵ), along with the empirical coverage of f by the intervals. If, for large T, that frequency is less than the claimed lower bound 1 − (1/bξ)(α + β/√T + κ), or if the coverage of f is worse than the Lemma 2 bound, the central claim is falsified. A direct check of Assumption 4 is to compute the GP posterior c.d.f. at y for all queried pairs and see whether its minimum is bounded away from zero.","tokens_in":142,"feed_emoji":"🎯","tokens_out":4449,"duration_ms":63182,"temperature":0.7,"pith_summary":"The paper aims to make Bayesian optimization (BO) reliable when the Gaussian-process surrogate is wrong about the objective or the noise. It proposes LOCBO, which uses localized online conformal prediction to recalibrate the GP likelihood at each query, then \"denoises\" the calibrated likelihood into a posterior over the objective function. The central promise is a certificate: for the points LOCBO actually queries, the utility it estimates is, with high probability, a lower bound on the true utility, for any target miscoverage level and under only mild symmetry of the noise. This matters because misspecified surrogates are the norm in real black-box optimization, and existing conformal BO methods either need hard-to-get likelihood ratios or assume noiseless observations.","feed_headline":"LOCBO certifies utility in misspecified Bayesian optimization","feed_subtitle":"Localized conformal prediction recalibrates GP surrogates, giving provable optimization guarantees under noisy, misspecified objectives.","key_machinery":"The load-bearing object is the localized online conformal prediction threshold λt(x) = gt(x) + ct, with gt drawn from an RKHS defined by a kernel; this threshold produces input-dependent intervals for the noisy observation. The update rules adjust the constant and the function online from the observed miscoverage events, and the interval is converted into a calibrated likelihood that is flat inside the set and proportional to the GP likelihood outside it. That calibrated likelihood is then marginalized against the GP posterior in closed form, producing the calibrated posterior used in the acquisition function. The localization kernel's length scale controls how much calibration specializes to each region of the input space, and setting it to infinity recovers standard online conformal prediction.","core_discovery":"LOCBO replaces the GP likelihood with a likelihood calibrated by localized online conformal prediction, producing an input-dependent prediction interval at each step, and then integrates the GP posterior over that interval to obtain a calibrated posterior for the unobserved objective f. The paper's Theorem 1 states that, under assumptions on kernel regularity, noise symmetry, bounded increasing utility, and a uniform positive lower bound on the GP posterior cumulative distribution, the average probability that the true utility of the queried points is at least 2 aLOCBO/(αϵ) is at least 1 − (1/bξ)(α + β/√T + κ). Lemma 2 establishes that the intervals cover f itself in long-run average, with the miscoverage rate inflated by 1/bξ relative to the noisy-observation guarantee. The guarantees hold for the unobserved objective, not just the noisy observations, and recover earlier noiseless results when localization is switched off.","pith_inferences":["A natural extension the paper does not pursue is to learn the localization length scale online from observed miscoverage events, which could remove the current sensitivity to that hyperparameter and make the method adaptive to local smoothness.","The certificate's dependence on 1/bξ suggests the method should degrade gracefully as noise becomes imbalanced; a testable prediction is that replacing symmetric noise with skewed noise of equal variance inflates the measured miscoverage rate of the intervals by roughly the predicted factor.","The same localized calibration plus denoising recipe could be applied to other surrogate-based sequential decision problems, such as active learning with misspecified priors, wherever the posterior can be evaluated cheaply.","The closed-form expression of the calibrated posterior means the algorithm's per-iteration cost stays comparable to standard GP inference plus one kernel online gradient update, so the robustness gain does not require inner Monte Carlo loops at each query."],"forward_implications":["If Theorem 1 holds, the acquisition function computed from the calibrated posterior is a meaningful guide: a point with large estimated utility cannot be, with high frequency, a point with small true utility.","The guarantee is localized, so calibration effort concentrates near promising regions, which the synthetic and UAV-network experiments show cuts simple regret roughly in half versus non-localized calibration under heteroscedastic noise.","With localization switched off, LOCBO recovers the noiseless OCBO guarantee, making the new result a strict extension to noisy and misspecified observation models.","The choice of miscoverage level α interacts with the GP conservativeness ϵ: setting α proportional to ϵ keeps the multiplicative gap between estimated and true utility stable, giving a practical design rule for α."],"supporting_citations":[{"why":"Supplies the localized online conformal prediction method and the long-run coverage guarantee used as Lemma 1.","marker":"[13]"},{"why":"Provides the OCBO baseline and the online-CP-based BO framework that LOCBO extends to noisy observations.","marker":"[12]"},{"why":"Introduces the likelihood-calibration and denoising idea that LOCBO adapts, and serves as the CBO baseline.","marker":"[11]"},{"why":"Gives the online conformal prediction update rule that underlies the threshold update in LOCBO.","marker":"[25]"},{"why":"Provides the inequality relating noise symmetry bξ to objective miscoverage, which is the bridge for Lemma 2.","marker":"[40]"}],"fun_headline_variants":["LOCBO guarantees robust BO under model misspecification","Localized conformal prediction recalibrates GP surrogates","Provable BO robustness via localized online conformal prediction","LOCBO certifies robust BO with localized conformal prediction","Online conformal calibration makes BO robust to misspecification"],"cache_read_input_tokens":23680,"weakest_assumption_plain":"The proof of the utility certificate requires Assumption 4: the GP posterior must assign probability at least some fixed ε>0 to the event that the objective lies below any observed value y, for every input, every history, and every observation, and the paper never quantifies ε or shows misspecified GPs satisfy it.","fun_headline_variants_meta":{"raw":{"variants":["LOCBO guarantees robust BO under model misspecification","Localized conformal prediction recalibrates GP surrogates","Provable BO robustness via localized online conformal prediction","LOCBO certifies robust BO with localized conformal prediction","Online conformal calibration makes BO robust to misspecification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001193,"raw_usage":{"total_tokens":4919,"prompt_tokens":942,"completion_tokens":3977,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":3897}},"tokens_in":558,"tokens_out":3977,"duration_ms":26763,"temperature":1.0,"reasoning_tokens":3897,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:10:49.994329+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LOCBO on a known objective with a deliberately misspecified GP (for example, wrong lengthscale or mean) under heavy-tailed or skewed noise, and measure the empirical frequency over T of the event u(xt, f(xt), Dt−1) ≥ 2 aLOCBO(xt|Dt−1)/(αϵ), along with the empirical coverage of f by the intervals. If, for large T, that frequency is less than the claimed lower bound 1 − (1/bξ)(α + β/√T + κ), or if the coverage of f is worse than the Lemma 2 bound, the central claim is falsified. A direct check of Assumption 4 is to compute the GP posterior c.d.f. at y for all queried pairs and see whether its minimum is bounded away from zero.","supporting_citations":[{"cited_title":"Localized adaptive risk control,","cited_arxiv_id":null,"evidence_quote":"Supplies the localized online conformal prediction method and the long-run coverage guarantee used as Lemma 1."},{"cited_title":"Online calibrated and conformal prediction improves Bayesian optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the OCBO baseline and the online-CP-based BO framework that LOCBO extends to noisy observations."},{"cited_title":"Bayesian optimization with conformal prediction sets,","cited_arxiv_id":null,"evidence_quote":"Introduces the likelihood-calibration and denoising idea that LOCBO adapts, and serves as the CBO baseline."},{"cited_title":"Adaptive conformal inference under distribu- tion shift,","cited_arxiv_id":null,"evidence_quote":"Gives the online conformal prediction update rule that underlies the threshold update in LOCBO."},{"cited_title":"From Conformal Predictions to Confidence Regions","cited_arxiv_id":"2405.18601","evidence_quote":"Provides the inequality relating noise symmetry bξ to objective miscoverage, which is the bridge for Lemma 2."}],"review_version":1}