{"id":"4b1f1206-ffca-4c3f-ab82-dd00637a541c","arxiv_id":"2501.13332","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CLBO, which combines a multi-output GP trained on bootstrap subsets with shared length-scales and a full-data GP, consistently reaches the best or near-best solutions on the tested benchmarks.","lead":"This paper introduces CLBO, a Bayesian optimization algorithm that trains several Gaussian process models on overlapping data subsets and forces them to share a length-scale parameter. The authors report that CLBO reaches better or comparable final solutions on five numerical and three engineering benchmark problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq.(20) ties length-scales across bootstrap subsets; co-training theory (Eq.2, [32]) concerns agreement of predictions on unlabeled points. This mapping is not derived, so the paper's central 'agreement on unlabeled information' mechanism is unsupported and may just be ensemble diversity.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the mapping between Eq.(20)'s shared length-scale and the co-training agreement theory is asserted, not derived. My stress-test confirms this is the central theoretical vulnerability. The paper presents consistent empirical improvements, but those improvements do not by themselves validate the specific mechanism claimed, because the ablation in Section 4.5 only varies the number of MFGP predictions and does not isolate the agreement constraint from simple ensembling or hyperparameter tying. The conditional verdict is therefore appropriate: the empirical comparison is suggestive but the theoretical basis for the central claim is unsupported. No stronger verdict is warranted because the empirical results could still be valid and useful even if the co-training analogy fails; the paper's contribution would then be reduced to an ensemble-of-GPs heuristic rather than a co-learning method. I also note the paper drops some baselines in the engineering tests and reports no significance testing, but those are secondary to the theoretical gap. My concrete test would settle whether the agreement constraint is actually responsible for the reported gains, or whether the gains come from bootstrap diversity or the full-data SOGP.","tokens_in":17536,"tokens_out":2718,"duration_ms":25660,"concrete_test":"Run a controlled ablation on Michalewicz-5D and Rastrigin-5D comparing three versions of CLBO's MFGP: (a) tied length-scale as in Eq.(20); (b) independent length-scales per subset (Eq.(19) for each subset, no agreement); (c) an explicit agreement penalty on predictions at random unlabeled points, e.g., add λΣ_{i<j}||f_i(x_u)-f_j(x_u)||^2 to the MFGP marginal likelihood. If (a) does not outperform (b) in surrogate accuracy near the optimum or in final regret, the paper's attribution of gains to agreement on unlabeled information is not supported. Also re-derive Eq.(2)'s application: state the hypothesis class F and loss L for MFGP and show Eq.(20) shrinks Rademacher complexity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that CLBO exploits model agreement on unlabeled information to reduce hypothesis-space complexity and improve surrogate accuracy (Abstract, Section I, Section IV). The only 'agreement' mechanism in the method is Eq.(20): l^2_{h,1}=...=l^2_{h,m}, a single tied length-scale used in the multi-output GP (MFGP) built on bootstrap subsets. This is a hyperparameter-sharing assumption, not an agreement constraint on predictions at unlabeled inputs. The cited theory (Eq.(2), refs [32],[33]) bounds Rademacher complexity when multiple views agree on unlabeled data; Eq.(20) imposes no such agreement on unlabeled points, and the paper gives no derivation showing MFGP's tied length-scale reduces the relevant hypothesis-space complexity. Moreover, the bootstrap subsets are all labeled with the same objective, so there are no truly unlabeled data in the training procedure. Thus the theoretical justification for CLBO's improvement is missing; the observed gains could be due to multi-model ensembling or the SOGP complement, not to co-training-style agreement. Since the abstract and Section IV repeatedly attribute CLBO's success to this agreement mechanism, the central claim rests on an unverified analogy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CLBO, a Bayesian optimization framework that replaces the single Gaussian-process surrogate with (i) a multi-output GP (MFGP) built on bootstrap subsamples of the training data, with all sub-models forced to share one length-scale hyperparameter, and (ii) a standard single-output GP (SOGP) trained on the full sample set. EI is applied to each sub-model to generate multiple candidate points per iteration, and the training sets are exchanged according to a hand-designed rule. The authors motivate the shared length-scale as an \"agreement constraint on curve bumpiness,\" invoke co-training/Rademacher-complexity theory to argue that this agreement reduces the sample complexity of the hypothesis space, and present experiments on five numerical benchmarks and three engineering benchmarks against sequential, batch, and multi-surrogate BO baselines.","tokens_in":17858,"tokens_out":3197,"duration_ms":31884,"significance":"If the mechanism claimed by the paper were established, CLBO would be a useful template for combining model diversity with agreement-based regularization in BO, and the engineering demonstrations on robot pushing, bipedal walking, and airfoil design are potentially valuable. The algorithm itself is clearly specified in Algorithms 1-4, the experimental setup is mostly reproducible from the text, and the comparison with a deliberately constructed MSBO baseline is a good idea for isolating the effect of the multi-model structure. However, the theoretical mechanism asserted in the abstract and Section IV is not derived: the only implemented constraint, the shared length-scale in Eq. (20), is a hyperparameter tie rather than an agreement constraint on unlabeled predictions, and the paper gives no argument connecting it to the Rademacher-complexity bound in Eq. (2). The empirical evaluation also lacks statistical significance tests and omits several baselines from the engineering sections. The central claim is therefore currently supported only by the raw experimental curves, which are suggestive but not conclusive.","major_comments":[{"comment":"The central theoretical motivation is not supported by the implemented constraint. Eq. (20) forces all MFGP sub-models to share a single length-scale l_h^2, which is a hyperparameter-sharing constraint. The cited co-training/Rademacher theory in Eq. (2) and refs. [32],[33] concerns agreement of hypotheses on unlabeled inputs, i.e., a constraint on predictions at previously unseen points. The paper does not identify any unlabeled data used in training, does not define the hypothesis space F or loss class L in Eq. (2) for MFGP, and does not show that Eq. (20) reduces the relevant Rademacher complexity. This is load-bearing because the abstract and Sections I and IV repeatedly attribute CLBO's success to \"agreement on unlabeled information.\" The paper should either derive a Rademacher-complexity reduction for the tied-length-scale MFGP hypothesis class or substantially soften the theoretical attribution and re-frame the contribution as an empirical ensemble heuristic.","section":"Section III-B, Eq. (20)"},{"comment":"The experimental comparisons are presented only as convergence plots and boxplots over 20 or 10 runs, with no significance tests or effect-size measures. Several claims such as \"CLBO always achieves the best or second best optimal solutions\" and \"the variation of final solutions of CLBO is the smallest\" would be much stronger with pairwise significance tests (e.g., Mann-Whitney U with a multiple-comparison correction) or at least a table of medians, interquartile ranges, and p-values. Without these, the reader cannot judge whether the visible differences, particularly on Hartman6 and the engineering benchmarks, are real or due to run-to-run variability.","section":"Section IV-C and IV-D, Figures 2-6 and 9-11"},{"comment":"Three of the baselines used in the numerical tests—BBO-LP, BO-MCMC, and MSBO—are excluded from the engineering benchmark comparisons, with only a brief explanation. Since the paper's broad conclusion is that CLBO outperforms both sequential and batch BO algorithms, the absence of batch BBO-LP and of the multi-surrogate MSBO baseline from the engineering results weakens that conclusion. The authors should either provide the engineering results for these baselines or explicitly report the failure mode that prevented the comparison, rather than only stating that code modification was inconvenient for BBO-LP.","section":"Section IV-D"},{"comment":"The sensitivity analysis does not actually isolate the effect of the agreement constraint. The paper states that \"the effectiveness of the agreement constraint in MFGP will be verified in Section IV,\" but Section IV-E varies only the number of MFGP sub-models and whether SOGP is included; it does not compare MFGP with tied length-scales against an otherwise identical model with independent per-subset length-scales (which is essentially the MSBO construction). Without that ablation, the improved accuracy cannot be attributed to the agreement constraint rather than to the multi-output structure or to sharing information through the joint covariance matrix in Eq. (15).","section":"Section IV-E, Figure 12"}],"minor_comments":[{"comment":"The EI expression has a notational error: the integrand should be max(fmin - Y(x), 0), not (fmin - Y(x), 0).","section":"Section II-A, Eq. (7)"},{"comment":"The equation references in the text are inconsistent: the text says \"As shown in Eq.(11), the condition z > 0\" but Eq. (11) is the z < -3 approximation, and the z > 0 statement appears in Eq. (12). Please renumber or fix the cross-references.","section":"Section II-A.2, Eqs. (10)-(12)"},{"comment":"The sentence defining the symbols in Eq. (1) appears to invert the roles: it says \"where, yhat^(i) and yhat denote the overall prediction and the individual model prediction,\" but the equation and the following text indicate that yhat is the overall prediction and yhat^(i) are the individual predictions.","section":"Section II-A, Eq. (1)"},{"comment":"Line 6 uses the notation \"arg maxP EI(x, xnew)\" without defining the optimization variable or the domain; it should state explicitly that the maximization is over x and that PEI is being maximized as a function of x.","section":"Algorithm 3"},{"comment":"The phrase \"agreement on unlabeled information\" is used loosely throughout; since no unlabeled data are ever introduced in the training procedure, this phrase should be defined operationally or replaced by a more precise term such as \"shared hyperparameter constraint\" unless the authors actually use unlabeled input locations in the MFGP training objective.","section":"Abstract and Section I"}],"recommendation":"major_revision","confidential_remarks":"The manuscript does not provide a code repository or data for the engineering benchmarks, which would help reproducibility. The citation pattern relies heavily on the authors' own previous work for benchmark settings, which is reasonable but should be clearly flagged. The biggest risk to the paper is the gap between the theoretical narrative and the implemented mechanism; if the authors can close that gap or reframe the contribution, the work may be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: CLBO is a reasonable engineering contribution—bootstrap subsets, a multi-output GP with shared lengthscales, plus a full-data GP, queried with parallel EI. The writing is clear and the algorithm is specified well enough to reimplement. The empirical story is consistent: on five numerical and three engineering benchmarks, CLBO lands at or near the best final solution, and the sensitivity analysis (MFGP vs MFGP+SOGP) gives some insight into when each component helps. That is real value.\n\nThe paper's central theoretical claim, however, does not hold up. The whole motivation leans on co-training/Rademacher theory, where agreement on unlabeled data reduces the hypothesis-space complexity (Eq.2, ref [32]). What the method actually enforces is Eq.20: tied lengthscales across bootstrap subsets in a multi-output GP. That is a hyperparameter-sharing inductive bias, not an agreement constraint on predictions at unlabeled points. The paper says the agreement is on 'curve bumpiness,' but there is no derivation connecting tied lengthscales to a reduced Rademacher complexity, and no unlabeled data in the usual co-training sense—all subsets are labeled with the same objective. So the stated reason for the improvement is unsupported. The observed gains could easily come from ensembling/diversity, or from the full-data SOGP complementing the subset models. The abstract and Section IV repeatedly attribute success to this agreement mechanism; I see that as an overreach rather than a fatal flaw, because the empirical comparison is the actual evidence.\n\nOther soft spots: no significance tests (only 20 runs numerical, 10 engineering), and the engineering section drops BBO-LP, BO-MCMC, and MSBO, so the claim of superiority on engineering problems is based on fewer comparisons. The authors give practical reasons for those exclusions, so I'd call it minor-to-moderate. No code is shipped, which limits reproducibility, though the MTGP toolbox is referenced.\n\nWhat's genuinely new is modest: MTGP applied to bootstrap subsets of one objective with tied lengthscale is not in the cited literature, but it's a combination of existing pieces rather than a new principle. The paper's own sensitivity analysis is the most useful part, since it shows where the agreement constraint helps and where it doesn't.\n\nBottom line: this deserves a serious referee. The algorithm is clearly described and the experiments are broad enough that a competent referee can check whether the gains survive proper statistical testing and whether the theory can be either derived or quietly dropped. I'd suggest the authors either prove a bound for the tied-lengthscale MFGP or reframe the paper as an empirical ensemble method, because the co-training framing as written won't survive scrutiny.\n\nRecommendation: send to review, but flag the theoretical overreach as the main revision point.","headline":"A clearly specified ensemble-BO variant with consistent but modest empirical gains; the co-training theory is asserted, not derived, and the 'agreement' is just tied lengthscales.","tokens_in":18320,"tokens_out":2355,"would_cite":false,"duration_ms":21760,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Co-learning Bayesian optimization replaces the lone Gaussian process surrogate with an agreement-constrained GP ensemble, claiming this sharpens predictions near the optimum and fixes the over-exploitation and over-exploration failures of…","keywords":["Bayesian optimization","Gaussian process","multi-output Gaussian process","co-training","model agreement","expected improvement","sample efficiency","black-box optimization"],"falsifier":"Run CLBO's exact pipeline on the five numerical benchmarks with the shared length-scale agreement constraint removed, letting each bootstrap subset tune its own smoothness parameter, and compare prediction error near the global optimum and final regret; if the constrained version is not better, the agreement mechanism is not carrying the claimed improvement.","tokens_in":17384,"feed_emoji":"🎯","tokens_out":9882,"duration_ms":84081,"temperature":0.7,"pith_summary":"The paper proposes CLBO, a Bayesian optimization algorithm that replaces the usual single Gaussian process (GP) surrogate with a small ensemble: several GPs trained on bootstrap subsets of the data, plus one GP trained on all the data. The claim is that these models complement each other and improve surrogate accuracy exactly in the region where the true optimum lies, where expected-improvement-based BO is known to over-exploit or over-explore and stall. To keep the subset models individually accurate, the paper imposes an agreement constraint: the subset models must share one length-scale hyperparameter, meaning they agree on the bumpiness of the predicted curve. Citing Rademacher-complexity bounds for co-training, the paper argues this agreement on unlabeled information lowers the number of samples needed for a target surrogate accuracy. On five numerical benchmarks and three engineering problems, CLBO is reported to reach the best or second-best final solutions and to converge faster than sequential and batch BO baselines while leaving the acquisition function unchanged.","feed_headline":"Agreement-trained GP ensembles find better optima in fewer calls","feed_subtitle":"A shared-curvature multi-output GP plus a full-data GP sharpens surrogates where optima hide.","key_machinery":"The load-bearing object is the multi-form GP (MFGP), a multi-output Gaussian process in which each output is a GP trained on one bootstrap subset of the data, and the outputs are coupled by a correlation matrix $\\rho_{ij}$. The agreement constraint is the shared length-scale condition $l^2_{h,1} = \\cdots = l^2_{h,m} \\equiv l^2_h$: all subsets must model the same curve bumpiness. This is the unlabeled-information agreement that the paper connects to co-training theory; the correlation coefficients and noise terms remain free hyperparameters, which preserves diversity. A second component, the single-output GP (SOGP) trained on the full data set, supplies a globally reasonable averaged prediction. The final ensemble combines MFGP and SOGP predictions, and each cycle selects one new point per model by maximizing expected improvement, with a minimum-distance rule and pseudo-EI as repair when sampled points are too close together.","core_discovery":"The central discovery claimed is that an agreement-constrained multi-output GP (called MFGP) built from bootstrap subsets, combined with a single-output GP (SOGP) trained on the full data, yields a surrogate ensemble whose overall prediction error near optima is lower than that of a standard GP, and that this improvement transfers into better optimization outcomes under an unchanged expected-improvement acquisition rule. The mechanism rests on two identities: the bias-variance decomposition $E = E_{\\mathrm{ind}} - \\mathrm{Diversity}$, which says combining diverse models helps only if individual errors stay low, and the Rademacher-complexity bound $E(L(Y,f(X))) \\le \\hat{E}_n(L(Y,f(X))) + R_n(\\tilde{L}\\circ F) + \\sqrt{8\\log(2/\\delta)/n}$, which says agreement on unlabeled information can shrink the hypothesis space. The agreement constraint is implemented as $l^2_{h,1} = \\cdots = l^2_{h,m} \\equiv l^2_h$, forcing all bootstrap-trained sub-models to share one length-scale hyperparameter, i.e., the same curve bumpiness. On Michalewicz, Rastrigin, Ackley, Hartman6, and Trid functions and on robot pushing, bipedal-walker control, and airfoil design, CLBO is reported to achieve the best or second-best solutions within budget, with lower variance across runs than the baselines.","pith_inferences":["Because the acquisition function is untouched, the same agreement-constrained surrogate ensemble could be paired with UCB, entropy search, or knowledge-gradient acquisition rules; the paper only mentions non-EI acquisition functions as future work.","Stronger agreement constraints, such as sharing gradient information or penalizing predictive disagreement at unlabeled points, might sharpen the co-training benefit further, though at the cost of diversity; the paper does not test these variants.","The same co-learning construction could apply to non-GP surrogates, since the bias-variance and Rademacher-complexity arguments are not GP-specific, but the paper reports no such experiments.","The structure suggests a generic recipe for batch BO: run any single-point acquisition on an agreement-constrained ensemble and query one point per member; this is a generalization the paper does not state."],"forward_implications":["Improving surrogate accuracy near optima is a viable route to fix EI's over-exploitation and over-exploration, so the suboptimality of BO can be attacked at the surrogate level rather than by redesigning the acquisition function.","Because each model contributes one query per cycle, CLBO obtains a batch of samples without modifying EI, and in the paper's comparisons it is the only batch method whose convergence beats the sequential baselines.","The agreement constraint reduces the sample complexity of the effective hypothesis space, so with the same number of subset samples the constrained individual models are expected to be more accurate than unconstrained subset models.","Combining a full-data GP with a few agreement-constrained subset GPs balances diversity and individual accuracy; using too many agreement-constrained models reduces diversity and worsens performance, as the sensitivity analysis shows."],"supporting_citations":[{"why":"Supplies the proof that model agreement on unlabeled information reduces hypothesis-space complexity, the theoretical basis for CLBO's agreement constraint.","marker":"[32]"},{"why":"Gives the bias-variance decomposition used to justify combining diverse individual models while keeping their errors low.","marker":"[30]"},{"why":"MTGP: multi-task GP whose correlated outputs and shared hyperparameters inspire the MFGP construction.","marker":"[38]"},{"why":"MSEGO baseline that combines multiple surrogates without an agreement constraint, isolating CLBO's contribution.","marker":"[26]"},{"why":"EGO baseline and source of the expected-improvement acquisition used by CLBO and most baselines.","marker":"[7]"},{"why":"Pseudo-EI, used as a repair step when the minimum-distance rule would otherwise block a query.","marker":"[51]"},{"why":"Provides the airfoil benchmark settings and the calibrated-EI perspective on the over-exploitation and over-exploration problem.","marker":"[2]"},{"why":"Source of the five numerical test functions used for the main comparisons.","marker":"[55]"},{"why":"XFOIL tool used to evaluate lift-to-drag ratio in the airfoil design benchmark.","marker":"[60]"}],"fun_headline_variants":["GP ensembles share curvature to dodge bad optima","Co-learn GPs agree on shape, find better peaks","Multiple GPs with shared bumpiness beat single surrogates","Agreement-constrained GP ensemble boosts BO efficiency","Diverse GPs that agree on curvature find optima faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that forcing bootstrap-trained GPs to share one smoothness parameter is the same kind of agreement on unlabeled points that co-training theory shows reduces the required number of samples.","fun_headline_variants_meta":{"raw":{"variants":["GP ensembles share curvature to dodge bad optima","Co-learn GPs agree on shape, find better peaks","Multiple GPs with shared bumpiness beat single surrogates","Agreement-constrained GP ensemble boosts BO efficiency","Diverse GPs that agree on curvature find optima faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000422,"raw_usage":{"total_tokens":2241,"prompt_tokens":1094,"completion_tokens":1147,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":710,"completion_tokens_details":{"reasoning_tokens":1067}},"tokens_in":710,"tokens_out":1147,"duration_ms":12516,"temperature":1.0,"reasoning_tokens":1067,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:14:26.883208+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CLBO's exact pipeline on the five numerical benchmarks with the shared length-scale agreement constraint removed, letting each bootstrap subset tune its own smoothness parameter, and compare prediction error near the global optimum and final regret; if the constrained version is not better, the agreement mechanism is not carrying the claimed improvement.","supporting_citations":[{"cited_title":"The value of agreement a new boosting algorithm,","cited_arxiv_id":null,"evidence_quote":"Supplies the proof that model agreement on unlabeled information reduces hypothesis-space complexity, the theoretical basis for CLBO's agreement constraint."},{"cited_title":"Neural network ensembles, cross validation, and active learning,","cited_arxiv_id":null,"evidence_quote":"Gives the bias-variance decomposition used to justify combining diverse individual models while keeping their errors low."},{"cited_title":"Multi-task gaussian process prediction,","cited_arxiv_id":null,"evidence_quote":"MTGP: multi-task GP whose correlated outputs and shared hyperparameters inspire the MFGP construction."},{"cited_title":"Efficient global optimization algorithm assisted by multiple surrogate techniques,","cited_arxiv_id":null,"evidence_quote":"MSEGO baseline that combines multiple surrogates without an agreement constraint, isolating CLBO's contribution."},{"cited_title":"Efficient global optimization of expensive black-box functions,","cited_arxiv_id":null,"evidence_quote":"EGO baseline and source of the expected-improvement acquisition used by CLBO and most baselines."},{"cited_title":"Pseudo expected improvement criterion for parallel ego algo- rithm,","cited_arxiv_id":null,"evidence_quote":"Pseudo-EI, used as a repair step when the minimum-distance rule would otherwise block a query."},{"cited_title":"Calibrated and recalibrated expected improvements for bayesian optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the airfoil benchmark settings and the calibrated-EI perspective on the over-exploitation and over-exploration problem."},{"cited_title":"Virtual library of simulation experiments: Test functions and datasets,","cited_arxiv_id":null,"evidence_quote":"Source of the five numerical test functions used for the main comparisons."},{"cited_title":"Xfoil: An analysis and design system for low reynolds number airfoils,","cited_arxiv_id":null,"evidence_quote":"XFOIL tool used to evaluate lift-to-drag ratio in the airfoil design benchmark."}],"review_version":1}