{"id":"c275669d-5b58-4588-9df1-6aa37ce38346","arxiv_id":"2506.14619","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FuRBO redefines trust regions using surrogate-ranked 'inspector' points and outperforms SCBO and other baselines on constrained BBOB and physics-inspired benchmarks, especially when feasible regions are rare.","lead":"The paper introduces FuRBO, a Bayesian optimization algorithm that reshapes the search region using predicted feasibility, to handle expensive optimization problems with hard-to-find feasible solutions. A smart generalist might care because it claims faster and more reliable optimization in high-dimensional engineering and AutoML settings where most candidate designs are infeasible.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified","rationale":"The reader's verdict is CONDITIONAL with high confidence. My read agrees with the core assessment (empirically plausible claim, honest limitations, addressable presentation issues) but does not find a load-bearing technical concern that would move the verdict. The inspector-ranking assumption is indeed a heuristic, and the paper's own ablation (Section E.3) shows sensitivity to the inspector percentage P, yet that sensitivity does not contradict the central claim since P=10% was used consistently and comparisons were made against SCBO under the same budget. The strongest independent support is the monotone trend in Tables 1-3: FuRBO's advantage grows with dimensionality and constraint severity, which is exactly the regime the central claim targets. The weakest point is that the moderate-constraint 40D results (Table 3, 9+floor(3D/4)) show FuRBO finding feasible solutions where SCBO reports n/a; this is dramatic and is exactly where a hidden implementation difference could matter, but there is no stated internal inconsistency that would make me doubt it. The budget phrasing inconsistency (10D in Section 3 vs 30D in experiments) is real but cosmetic; the reported total budgets, batch sizes, initial designs, and repetition counts are consistent across the experiment sections. I therefore recommend the verdict stay as CONDITIONAL, with the condition being the requested verification of the boundary case, not a change in the manuscript's substantive claims. Agreement with the reader is partial because I do not share the weighting that makes the weakest_assumption load-bearing; I see it as a plausible heuristic limitation, not a threat to the central claim.","tokens_in":20368,"tokens_out":1586,"duration_ms":15481,"concrete_test":"Verify the reproducibility of the headline result by running the released GitHub code on the 10D BBOB-constrained suite for a representative hard setting (e.g., bent_cigar with 24 constraints) and checking that the final loss and standard error match Table 2 to within the reported precision. A successful rerun would confirm that the trend driving the central claim is not an artifact of the experimental pipeline or a reporting error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"No significant objection identified. I read the central claim as: FuRBO's inspector-guided trust region lets it match or beat SCBO and other baselines on constrained BO, with the largest gains when feasibility is rare. The paper supports this with full-suite BBOB-constrained experiments at 2D/10D/40D, representative-function baselines, ablations, and additional benchmarks up to 60D. The strongest evidence is the systematic trend in Tables 1-3 (larger FuRBO advantage as constraint count and dimension increase) and code being publicly available. The main logical risk is that the empirical trend is specific to the BBOB-constrained construction, which couples the number of constraints to the number of active constraints; the paper's own limitation section concedes that both methods fail in the most extreme 40D settings, so the superiority claim is honestly scoped to rare-but-locatable feasibility. The budget inconsistency flagged by the reader (Section 3 says 10D while experiments use 30D; Batched paragraph in Section 5.1 says 30D) is a presentational defect, not a logical flaw: for a batch-parallel method the relevant count per model update is the batch size, and the paper states that too. No internally inconsistent derivation, hidden circularity, or unsupported mechanism claim was found. The central claim does not require that the inspectors locate a globally promising feasible region; only that this heuristic helps relative to SCBO, which the experimental distribution supports.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FuRBO, a trust-region Bayesian optimization algorithm for constrained black-box problems. FuRBO extends SCBO by constructing the trust region from a set of 'inspector' points sampled uniformly in a ball around the current best solution, ranking them with surrogate models of the objective and constraints, and taking the smallest hyperrectangle containing the top P% of inspectors. The algorithm is evaluated on the BBOB-constrained COCO benchmark (3 instances in 2, 10, and 40 dimensions), on three representative functions against cEI, COBYLA, CMA-ES, and random search, and on additional benchmarks (Keane bump, spring, welded beam, pressure vessel, speed reducer, rover trajectory) against SCBO. The central claim is that FuRBO ties or outperforms state-of-the-art alternatives, with the largest gains when feasibility is rare and hard to locate.","tokens_in":20625,"tokens_out":6249,"duration_ms":56438,"significance":"If the empirical claims hold, FuRBO provides a practical improvement for constrained BO in settings where feasible regions are narrow. The paper's strengths include a publicly available implementation, statistical significance testing (Wilcoxon rank-sum) across replications, a systematic ablation study of four hyperparameters, and an honest limitations section that acknowledges failure in extremely constrained high-dimensional problems. The trend in Tables 1-3—growing advantage over SCBO as dimension and constraint count increase—is consistent with the proposed mechanism. However, the overstated evaluation scope ('full suite' vs 3 instances) and the internal budget inconsistency need correction before the claims can be taken at face value.","major_comments":[{"comment":"The problem definition in Section 3 states a total evaluation budget of 10D function evaluations, while every experimental setup in Section 5.1 uses a total evaluation budget of 30D (e.g., the Baselines Setup paragraph and the constrained BBOB paragraph). This internal inconsistency affects the framing of the method as a 'low-budget' optimizer and must be reconciled by either changing the problem definition or explicitly justifying the experimental budget.","section":"Section 3 and Section 5.1"},{"comment":"The abstract and Section 5.1 claim that FuRBO is tested on 'the full BBOB-constrained COCO benchmark suite,' but the experimental protocol uses 3 instances (and 10 repetitions) rather than the 15 instances that constitute the full suite, in dimensions 2, 10, and 40 only. This overstates the evaluation scope; the authors should either run the full 15-instance suite or revise the wording to accurately describe the subset used.","section":"Abstract and Section 5.1"},{"comment":"The ablation study on the inspector percentage P in Appendix E.3 states that FuRBO's performance deteriorates as P increases, with the best results at P=1%, yet the main experiments fix P=10% without a stated reason. Given that the paper claims superiority partly through this feasibility-driven mechanism, the sensitivity of the algorithm to P should be addressed, either by justifying the chosen value across a wider set of problems or by tuning P per problem. As written, the choice appears to favor the reported results.","section":"Appendix E.3"}],"minor_comments":[{"comment":"The sentence 'For simpler problems (leftmost columns with 1–3 constraints)' is inaccurate because the leftmost columns in Table 2 correspond to 1, 3, and 9 constraints, not 1–3.","section":"Section 5.2.1"},{"comment":"The phrase 'configurations with 17 or more constraints' does not match the actual constraint counts in the rightmost columns (e.g., 16, 24, 54 constraints in 10D); please correct the threshold to align with the table.","section":"Section 5.2.1"},{"comment":"The caption contains a typo: 'fro constrained optimization' should read 'for constrained optimization'.","section":"Appendix A, Table 4 caption"},{"comment":"The ranking metric r used in Algorithm 1 (lines 2 and 5) is only described in prose in Section 4.2; providing a formal numbered definition (including the normalization of constraint violations and the concatenation of feasible and infeasible samples) would significantly improve clarity.","section":"Algorithm 1 and Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a reasonable candidate for publication after revision. The central algorithmic idea is sound and the empirical study is extensive, but the overstated 'full suite' claim and the 10D vs 30D budget inconsistency are issues that could erode reader trust if left unaddressed. The hyperparameter sensitivity in Appendix E.3 also deserves a more careful discussion, as the choice of P=10% appears inconsistent with the ablation trend."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, read this one if you care about constrained BO in the rare-feasibility regime. The new thing is FuRBO's trust region construction: instead of centering the TR on the best point like SCBO, it samples inspectors in a ball around the current best, ranks them with the surrogate models, and sets the TR as the smallest box containing the top P%. That is a genuine mechanism change, and the empirical story is consistent: it matches SCBO on lightly constrained problems and beats it as constraint count and dimension go up. The paper is honest about where it fails, and the code is public.\n\nWhat's solid: the algorithm is clearly described, the ablations cover the new hyperparameters, and the Wilcoxon tests are standard. The additional benchmarks (Keane, spring, welded beam, pressure vessel, speed reducer, 60D rover) show the effect is not a BBOB artifact. The limitation section is candid: both methods fail in the most extreme 40D settings.\n\nWhere it's soft, in proportion: the abstract says full BBOB-constrained COCO benchmark suite, but the experiments use 3 instances per configuration, not the full 15. That's an overstatement, not a fatal flaw. More annoying: Section 3 says the budget is 10D, but every experiment uses 30D. The reader flagged this, and the stress-tester says it's presentational. I think it's a bit more than presentational—it weakens the tight-budget framing. If the method is about finding feasibility when evaluations are scarce, 30D is a generous budget; the 10D number in the problem definition seems like a leftover. They should reconcile it. Also, the new hyperparameters (P%, R, tau_s, tau_f) are tuned on one function, bent_cigar, and the paper admits P% is sensitive. That's minor; every TR method has such knobs.\n\nThe weakest conceptual step: the inspectors are drawn uniformly in a ball around the current best. If the feasible region is disconnected from that ball, or the surrogate ranking is wrong early on, the TR can be misplaced. The paper acknowledges this in the limitation for heavily constrained high-D cases. I don't see a load-bearing flaw: the central claim only needs the heuristic to help relative to SCBO, and the data supports that.\n\nVerdict: this deserves a serious referee. I'd recommend accept with minor revisions—fix the suite/instance wording, reconcile the budget numbers, and clarify the early-TR failure mode. If I were editing, I'd send it out.","headline":"FuRBO is a solid incremental improvement over SCBO for rare-feasibility constrained BO; the inspector-based trust region is the real contribution, and the experiments support it despite some presentational overreach.","tokens_in":21139,"tokens_out":3144,"would_cite":true,"duration_ms":26922,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FuRBO, a feasibility-driven trust-region Bayesian optimization method, matches or beats state-of-the-art constrained optimizers on benchmark suites and finds feasible solutions where competitors fail, especially when feasible regions are…","keywords":["Bayesian optimization","constrained black-box optimization","trust region","inspector sampling","Gaussian process surrogate","Thompson sampling","BBOB-constrained benchmark","rare feasibility"],"falsifier":"On a synthetic problem whose feasible region is a narrow curved corridor lying entirely outside the initial ball of radius $R$ around the current best, run FuRBO and a global random-search baseline for the same budget; if FuRBO returns no feasible point while random search does, the claim that inspector-guided trust regions accelerate feasibility discovery is refuted.","tokens_in":20175,"feed_emoji":"🎯","tokens_out":11072,"duration_ms":100273,"temperature":0.7,"pith_summary":"The paper introduces FuRBO, a Bayesian optimization method for black-box problems with expensive, unknown constraints, aimed at settings where merely finding a feasible point is hard. Rather than centering the trust region only on the best evaluated point, as the SCBO algorithm does, FuRBO samples a set of 'inspector' points uniformly in a ball around the current best, ranks them with Gaussian-process surrogate models of the objective and constraints, and builds its trust region as the smallest hyperrectangle containing the top-ranked inspectors. Thompson sampling then proposes new candidates inside that region, and the region's radius doubles or halves with progress, letting the search refocus rapidly. The paper reports that FuRBO ties SCBO on easy problems and outperforms it as constraints become numerous and hard to satisfy, and on physics-inspired benchmarks it is comparable or better, with its clearest wins where feasible regions are rare.","feed_headline":"Feasibility-first optimizer wins when valid points are rare","feed_subtitle":"FuRBO ranks probe points by predicted feasibility to set its trust region, beating SCBO on hard benchmark problems.","key_machinery":"The carrying mechanism is the inspector-defined trust region: at each iteration, a population of inspector points is sampled uniformly inside a ball of radius $R$ centered at the current best candidate, ranked by surrogate predictions of objective value and constraint violation, and the top $P\\%$ of inspectors are enclosed by the smallest axis-aligned hyperrectangle, which becomes the trust region for Thompson sampling. The ranking metric orders feasible samples by objective value and infeasible samples by maximum normalized constraint violation, so the trust region points toward the part of the landscape the surrogates judge most likely to contain high-quality feasible solutions. The radius $R$ is doubled on success and halved on failure, letting the search refocus aggressively.","core_discovery":"The central claim is that the trust region itself can be made feasibility-driven: instead of using the best feasible point only as a center, FuRBO samples inspector points uniformly in a ball of radius $R$ around the current best, ranks them using the GP surrogate models of objective and constraints, and defines the next trust region as the axis-aligned hyperrectangle containing the top $P\\%$ of inspectors. Thompson sampling then selects new candidates within this region. The paper argues that this procedure actively reuses constraint-model information to hunt for feasible territory, making the trust region shift and resize substantially between iterations. Empirically, the authors show that FuRBO either ties or outperforms SCBO, constrained EI, COBYLA, CMA-ES, and random search across the BBOB-constrained benchmark suite and physics-inspired problems, with the largest margins when constraints are numerous, active, or hard to satisfy, and in high dimensions where SCBO stops finding feasible solutions.","pith_inferences":["The paper does not explore this, but the inspector-ranking idea could be extended to a covariance-shaped trust region fit to the top inspectors, which would likely help when the feasible set is a narrow rotated corridor, a limitation the paper itself acknowledges.","A testable prediction of the mechanism is that FuRBO's advantage over SCBO grows with the fraction of the budget spent before the first feasible point is found; a benchmark that varies only that fraction would isolate the trust-region effect.","Because the extra cost comes from surrogate evaluations of inspectors, FuRBO is most attractive when each true evaluation is expensive; on cheap problems the overhead may erase the gain, a tradeoff the paper notes only in passing."],"forward_implications":["On the full BBOB-constrained suite in 10D, FuRBO converges faster and reaches lower final regret than SCBO on problems with 17 or more constraints, while performing comparably on problems with one to three constraints.","In 40D, FuRBO finds feasible solutions under a moderate constraint load ($9+\\lfloor 3D/4\\rfloor$ constraints) on nearly all benchmark functions, whereas SCBO systematically fails within the same budget.","On the 60D rover trajectory problem with 15 hard constraints, FuRBO reaches a higher reward than SCBO, and on the 30D Keane bump it slightly outperforms SCBO.","On low-dimensional, lightly constrained engineering problems such as spring, welded beam, and pressure vessel design, FuRBO matches SCBO without a notable advantage, indicating its benefit is concentrated in hard-feasibility settings.","Larger inspector percentages and very small initial radii degrade FuRBO's performance, so its gains depend on the choice of these hyperparameters."],"supporting_citations":[{"why":"Introduces SCBO, the trust-region constrained Bayesian optimization framework that FuRBO builds on and its primary baseline.","marker":"[Eriksson and Poloczek, 2021]"},{"why":"Provides the scalable trust-region machinery that SCBO and FuRBO extend, including the modified rover trajectory problem used in the 60D comparison.","marker":"[Eriksson et al., 2019]"},{"why":"Defines constrained expected improvement (cEI), one of the baseline algorithms compared against FuRBO.","marker":"[Schonlau et al., 1998]"},{"why":"Provides the benchmark platform and the BBOB-constrained testbed on which FuRBO and SCBO are compared.","marker":"[Hansen et al., 2021]"},{"why":"Defines the full BBOB-constrained COCO test suite with its constraint sets and severity levels.","marker":"[Dufossé et al., 2022]"},{"why":"Supplies the Thompson sampling acquisition used to propose candidate points inside the trust region.","marker":"[Thompson, 1933]"},{"why":"Source of the physics-inspired engineering benchmarks (spring, welded beam, pressure vessel, speed reducer) used in the appendix comparisons.","marker":"[Lemonge et al., 2010]"},{"why":"Provides the original rover trajectory planning problem, modified with 15 hard constraints for the 60D comparison.","marker":"[Wang et al., 2018]"}],"fun_headline_variants":["Feasibility-driven trust region guides BO past rare valid points","FuRBO pivots trust region to find scarce feasible spots","When valid points are rare, let the trust region hunt feasibility","BO that shifts its search window toward feasible territory","Trust region follows feasibility probes to beat hard-constraint benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the top-ranked inspector points, sampled in a ball around the current best and scored by Gaussian-process surrogates, outline a hyperrectangle that actually contains a promising feasible region.","fun_headline_variants_meta":{"raw":{"variants":["Feasibility-driven trust region guides BO past rare valid points","FuRBO pivots trust region to find scarce feasible spots","When valid points are rare, let the trust region hunt feasibility","BO that shifts its search window toward feasible territory","Trust region follows feasibility probes to beat hard-constraint benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000451,"raw_usage":{"total_tokens":2275,"prompt_tokens":951,"completion_tokens":1324,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":1241}},"tokens_in":567,"tokens_out":1324,"duration_ms":10388,"temperature":1.0,"reasoning_tokens":1241,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:49:45.165357+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a synthetic problem whose feasible region is a narrow curved corridor lying entirely outside the initial ball of radius $R$ around the current best, run FuRBO and a global random-search baseline for the same budget; if FuRBO returns no feasible point while random search does, the claim that inspector-guided trust regions accelerate feasibility discovery is refuted.","supporting_citations":[],"review_version":2}