FigBO: A Generalized Acquisition Function Framework with Look-Ahead Capability for Bayesian Optimization

Hui Chen; Longbing Cao; Xuhui Fan; Zhangkai Wu

REVIEW 2 major objections 5 minor 36 references

FigBO: A Generalized Acquisition Function Framework with Look-Ahead Capability for Bayesian Optimization

T0 review · 2 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A decaying global-information term adds look-ahead to myopic acquisition functions without changing EI's convergence rate.

desk verdict The plug-and-play idea is reasonable, but the main no-harm convergence claim rests on an invalid limit argument that fails exactly in the smooth-1D regime; the paper is worth refereeing but needs substantial theory revision. read the letter →

arxiv 2504.20307 v1 pith:O3IDK2JL submitted 2025-04-28 cs.LG

Hui Chen , Xuhui Fan , Zhangkai Wu , Longbing Cao This is my paper · ORCID

classification cs.LG

keywords Bayesianoptimizationacquisitionfunctionlook-aheadglobalinformationgainexpectedimprovementGaussianprocessregretboundMonteCarloapproximation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a cheap additive term can give myopic Bayesian optimization acquisition functions a look-ahead capability. Concretely, it proposes choosing queries by maximizing α(x) + λΓ(x), where α is any myopic acquisition function and Γ measures how much a candidate point would reduce the Gaussian process's predictive variance over the whole input space, with λ = η/n decaying to zero. For expected improvement, the paper proves that this modified policy has the same asymptotic regret rate as vanilla EI, so the extra exploration does not hurt convergence. If true, practitioners could upgrade almost any existing acquisition function without replacing their surrogate model or paying the full cost of non-myopic methods. The paper also reports faster convergence on GP prior samples, synthetic functions, and MLP hyperparameter tuning tasks.

What carries the argument

The central object is the global information-gain function $\Gamma(x)$, the integrated reduction in posterior variance over the whole input space that would result from adding candidate point $x$ to the data. It is approximated by Monte Carlo as $\Gamma(x) \approx \frac{1}{L}\sum_{l=1}^L k_{n,l}(x)^\top (K_{n,l}+\sigma_\epsilon^2 I)^{-1} k_{n,l}(x)$, and it carries the look-ahead: maximizing $\alpha(x)+\lambda\Gamma(x)$ with $\lambda=\eta/n$ spends early iterations reducing global uncertainty, then reverts to the base acquisition function. The proof machinery is the regret analysis for the vanilla EI convergence rate, and the computational machinery is a rank-one update of the inverse Gram matrix that keeps the per-candidate cost at $O(n^2)$.

What would settle it

Recompute the proof with the exact integral in place of the Monte Carlo approximation and check whether $C' = \frac{\tau(R/\sigma)}{\tau(-R/\sigma)}\max_{x\in\mathcal{X}}\lambda\Gamma(x)$ divided by $L_n(EI_n)$ tends to 0 as $n\to\infty$ under $\lambda=\eta/n$; if it does not, the asymptotic equivalence in Corollary 1 fails. A direct experiment: run FigBO and vanilla EI on a one-dimensional squared-exponential GP sample for many iterations and compare the tail slope of log regret; a worse slope for FigBO would contradict the claimed rate.

Watch

Extended reading notes

Core claim

The paper's central claim is that the query rule $x_{n+1} \in \arg\max_{x\in\mathcal{X}} [\alpha(x) + \lambda\Gamma(x)]$, with $\lambda = \eta/n$ and $\Gamma$ the global posterior-variance reduction integral, endows any myopic acquisition function with look-ahead capability. For expected improvement, Theorem 1 bounds the expected regret of the combined strategy by the regret of vanilla EI plus a constant $C'$, and Corollary 1 states the two are asymptotically equivalent, giving the same rate $O(n^{-(\nu\wedge 1)/d}(\log n)^\beta)$. Empirically, the paper reports that FigBO-EI, FigBO-UCB, and FigBO-PI converge faster and reach lower log regret than their base functions and than entropy-search baselines on GP prior samples, Branin, Levy, Hartmann, and four MLP classification tasks.

Load-bearing premise

The argument depends on the fixed-sample Monte Carlo estimate of the global uncertainty integral being a faithful stand-in for the exact integral, and on the decaying weight $\lambda=\eta/n$ shrinking fast enough that the added term becomes negligible next to EI's regret; for smooth kernels in one dimension the required decay condition can fail.

Editorial extensions

If this is right

Any myopic acquisition function can be upgraded by adding the decaying term $\lambda\Gamma(x)$, and in the EI case the combined policy keeps the same asymptotic convergence rate as vanilla EI.
Early in the run, the global-information term dominates and steers queries toward high-uncertainty regions; later $\lambda \to 0$, so the policy reverts to the base acquisition function.
The extra computational cost is mild: a rank-one update keeps the per-candidate evaluation of $\Gamma$ at $O(n^2)$, far cheaper than entropy-search methods.
The reported experiments show faster convergence and lower log regret than the base myopic functions and non-myopic baselines on GP prior samples, synthetic benchmarks, and MLP classification tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A batched version could compute $\Gamma$ over a whole candidate set rather than one point; the paper does not explore this.
Since the 12D experiments show a smaller advantage, dimension-aware Monte Carlo sampling that grows $L$ with dimension is a natural extension the paper does not test.
The asymptotic equivalence is proved for EI only; the UCB and PI speedups are empirical, and extending the proof to those base functions is an open question.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper proposes FigBO: add a decaying global-variance-reduction term Γ(x) to any myopic acquisition function, with weight λ=η/n. The idea is simple and useful, and the experiments suggest it helps early convergence without hurting later performance. The Γ term itself is essentially the integrated mean squared error / integrated variance reduction criterion from spatial statistics and active learning, so the novelty is the plug-and-play combination with a decay schedule—modest, but real. I like the Sherman-Morrison trick to keep the added cost low.

The soft spot is the theory, and it's load-bearing. Corollary 1 claims asymptotic equivalence between FigBO-EI and vanilla EI, i.e., no harm to the convergence rate. The proof in Appendix B (Eq. B9) is not valid: it bounds the ratio by 1 + C′/(Bull bound + C′), then lets C′ → 0 because λ → 0. But the denominator also goes to zero because the Bull bound is O(n^{-(ν∧1)/d}(log n)^β). The limit is 0/0 unless C′/L_n(EI_n) → 0, which requires (ν∧1)/d < 1. That condition is never stated, and it fails for Matérn ν > 1 or RBF kernels in one dimension—where the added term is the same order as the EI rate and can change the leading constant. So the headline 'asymptotically equivalent' is unsupported in exactly the smooth-kernel low-D setting that matters most. This is a genuine flaw, not a nitpick.

There's also the MC approximation gap: the theory is for exact Γ(x), but the algorithm uses L=100 uniform samples, and no error analysis connects the two. That may be minor if L is large enough, but it's unaddressed. Empirically, the results look plausible—FigBO often converges faster—but there's no released code and no significance tests, only standard errors. For a paper claiming 'significantly faster', I'd want paired comparisons or confidence intervals.

Who is this for? Practitioners who want a cheap way to boost a myopic acquisition function. The empirical phenomenon may survive even if the theory is cut back to a conditional statement. The paper deserves a serious referee, but the B9 issue should be flagged as a major revision: either state the (ν∧1)/d < 1 condition and restrict the claim, or prove a weaker no-harm result that doesn't drop a vanishing denominator. I'd send it to peer review, not desk reject, but the authors need to fix the math before publication.

Referee Report

2 major / 5 minor

Summary. The paper proposes FigBO, a framework that augments any myopic acquisition function with a look-ahead term: the next query is selected as argmax_x [α(x) + λΓ(x)], where Γ(x) is the integrated reduction of the GP posterior variance over the input space when x is added, λ = η/n is a decaying weight, and the integral is approximated by L uniform Monte Carlo samples. The theoretical section focuses on the case α = expected improvement (EI). Theorem 1 gives an upper bound on the expected regret of the resulting strategy EI_{Γ,n} in terms of Bull's EI bound plus an additive term C' = O(η/n). Corollary 1 claims that as n → ∞, the loss of EI_{Γ,n} is asymptotically equivalent to that of vanilla EI, yielding the rate O(n^{-(ν∧1)/d}(log n)^β). Empirically, the authors test FigBO on GP prior samples, synthetic functions (Branin, Levy, Hartmann), and MLP hyperparameter-tuning tasks, reporting improved average log regret or accuracy relative to myopic and non-myopic baselines, and they include ablations over the MC sample count L and the decay hyperparameter η.

Significance. If the theoretical claim held, FigBO would be an attractive and conceptually simple plug-and-play upgrade to most myopic acquisition functions, with a proof that the extra exploration term does not degrade the asymptotic convergence rate while providing empirical speedups. The idea of adding a decaying, information-based exploration bonus is plausible, and the empirical results, while not accompanied by formal significance tests, are consistent across a range of benchmarks and show a clear advantage over the myopic baselines. The method is considerably cheaper than non-myopic alternatives such as PES. However, the central 'no-harm' guarantee rests on a limit argument that is invalid as written, and the theory applies only to an exact-Γ version while the implementation uses a Monte Carlo approximation with no error control. These issues make the main theoretical contribution unsupported in its current form and need to be fixed before the manuscript can be recommended for acceptance.

major comments (2)

[Appendix B, Eq. (B9); Corollary 1] The proof of Corollary 1 is not valid as written. The ratio in (B9) is bounded by 1 + C'/(Bull_bound + C'), where C' = (τ(R/σ)/τ(−R/σ)) max_x λΓ(x) and Bull_bound = (τ(R/σ)/τ(−R/σ))[2Rm^{-1} + (R+σ)C m^{-(ν∧1)/d}(log m)^β]. The argument lets only the numerator C' → 0, but the denominator also tends to 0 because Bull_bound → 0 as n → ∞. Consequently the limit is of the form 0/0 and the conclusion '= 1' does not follow. A valid comparison requires C'/Bull_bound → 0, i.e., (η/n)/n^{-(ν∧1)/d}(log n)^β → 0, which is equivalent to (ν∧1)/d < 1 (up to logarithmic factors). This condition is never stated. For Matérn kernels with ν > 1 or the RBF kernel in dimension d = 1, (ν∧1)/d = 1 and the Γ term is of the same order as Bull's EI bound, so the leading constant of the regret can be affected. As stated, Corollary 1 is unsupported.
[Section 3.2, Eq. (9) and Algorithm 1] The theoretical results (Theorem 1 and Corollary 1) are stated for the exact global-information integral Γ(x), but the deployed algorithm uses the Monte Carlo approximation (9) with L = 100 uniform samples. No bound on the approximation error is given, and the analysis does not show that the error is small enough for the theoretical guarantee to carry over to the implemented acquisition function. This is a load-bearing gap between theory and practice: even if the exact-Γ analysis were correct, it would not justify the empirical behavior of the algorithm that actually selects points using the MC estimate. The authors should either prove a uniform error bound for the MC approximation (e.g., under regularity conditions on the kernel) or explicitly restrict the theoretical claims to the exact version and discuss the approximation as an additional heuristic assumption.

minor comments (5)

[Section 5.5] In the first sentence of Section 5.5, 'n low-dimensional tasks' appears to be a typo for 'in low-dimensional tasks'.
[Eqs. (7)–(9)] The notation in Eqs. (7)–(9) is confusing: the integration variable in (7) is also denoted x, while in (9) the candidate point is x and the MC samples are indexed by l. This makes it hard to see that k_{n,l}(x) is the cross-covariance between the l-th MC point and the n+1 points including the candidate. Please clarify the notation, e.g., by renaming the integration variable and defining k_{n,l}(x) explicitly.
[Introduction and Related Work] The paper claims that FigBO 'seamlessly integrates with most myopic acquisition functions,' but the theoretical analysis covers only EI. The empirical section shows that the framework also helps UCB and PI, but there is no theoretical justification for these cases. The authors should state this scope limitation more clearly.
[Section 5] The empirical evaluation reports average results with standard errors over 20 or 50 repetitions, but no statistical significance tests (e.g., paired t-tests or Wilcoxon tests) are provided. Adding such tests would strengthen the claim that FigBO's improvements are not due to random variation. In addition, the code is not released for review, which limits reproducibility of the results.
[Section 5.5] Figure 6 shows that the performance of FigBO in the 6D Hartmann task improves substantially when L increases from 5 to 500. This is an important practical consideration, since the default L=100 may be insufficient for higher-dimensional problems; the paper would benefit from a concrete guidance on choosing L as a function of dimension.

Circularity Check

1 steps flagged · score 6.0 of 10

The theoretical 'no-harm' convergence claim reduces to the construction λ=η/n; Appendix B's limit is indeterminate, but the empirical evaluation is self-contained.

self definitional [Algorithm 1 (Eq. 11); Theorem 1 (Eq. 13); Appendix B (Eq. B9)]
"'xnew← arg max_{x∈X} {α(x) + λΓ(x)}, where λ = η/n' (Algorithm 1); 'C′ = τ(R/σ)/τ(−R/σ) max_{x∈X} λΓ(x)' (Theorem 1); 'As the number of iterations n approaches infinity, the value of our coefficient λ approaches zero... =1' (Appendix B)."

The claimed asymptotic no-harm guarantee is built into the definition of FigBO: EIΓ,n is defined as EIn plus λΓ, and λ is set to η/n so the added term decays. Theorem 1's bound is therefore the EI bound plus max λΓ, and Corollary 1's 'asymptotic equivalence' is obtained by letting that added term vanish, not by analyzing which points FigBO actually selects or how Γ changes the acquisition dynamics. Moreover, Eq. B9 is an indeterminate 0/0 limit unless C′/Bull → 0, a condition never stated; so the derivation reduces to the choice λ = η/n rather than demonstrating an independent rate. The empirical speedups remain independent evidence, which is why the circularity is partial.

full rationale

FigBO's empirical claims rest on self-contained benchmark comparisons and do not depend on a fitted parameter being renamed as a prediction. The theoretical claim, however, is different: the central result that FigBO preserves EI's convergence rate is essentially a consequence of the construction. The acquisition is defined as α + λΓ (Eq. 11), and the coefficient is defined to decay as η/n, so the extra term is forced to vanish in the limit. Theorem 1 and Corollary 1 then state that the regret bound differs from Bull's EI bound only by max λΓ, and that the ratio tends to 1 because λ tends to 0. That is a tautological no-harm statement rather than a dynamical analysis of FigBO's selected points. The proof in Appendix B is also formally invalid as written: the denominator in Eq. B9 tends to zero alongside the numerator, so the limit is 0/0 unless the extra term decays faster than Bull's bound, a condition not proved or stated. This is partly a correctness problem and partly a circularity problem: the theoretical support for 'look-ahead does not hurt' reduces to the definition of λ. No load-bearing self-citation chain is present; Bull's EI bound is external and legitimate. For these reasons the paper is not wholly circular, but its headline theoretical guarantee is substantially by construction, warranting a score of 6.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The method rests on standard GP posterior equations, on Bull's EI convergence lemmas, and on two hand-chosen hyperparameters (η and L) plus a uniform Monte Carlo approximation of Γ. The decay schedule λ = η/n is central to the asymptotic claim and is an ad hoc design choice. No invented physical or mathematical entities are introduced.

free parameters (3)

η (decay-rate hyperparameter) = N/10 across all experiments; sensitivity explored in Section 5.6 with η ∈ {5, 10, 20, 50}
Controls how quickly the global-information term λ = η/n decays; chosen by hand, not derived from theory.
L (number of Monte Carlo samples) = 100 in all main experiments; varied in {5, 20, 100, 500} in Section 5.5
Controls the accuracy of the Γ integral approximation and per-iteration cost; set empirically.
M (initial design size) = not specified in the experimental text
Algorithm 1 takes M as input, but the experiments do not state which value was used, so replication requires guessing.

assumptions (6)

standard math The GP posterior mean and variance formulas (Eqs. 1-2) and the Gaussian mutual information formula (Eq. 5) are correct.
Standard results from GP regression and information theory, used as background.
standard math Bull's convergence-rate lemmas for expected improvement (Lemma 7 and 8 in [19], restated as Lemma 1-2 here) are valid and apply in this setting.
The proof of Theorem 1 imports these bounds verbatim without re-deriving or extending them.
domain assumption The objective lies in an RKHS ball with symmetric positive definite kernel, and the loss definition (Eq. 12) with u independent of R captures the relevant performance.
Section 4 adopts the assumptions of [19]; the theoretical result only holds for this function class.
ad hoc to paper The global-information integral Γ can be replaced by an L-sample uniform Monte Carlo average (Eq. 9) with negligible error.
No convergence guarantee is provided for this approximation; L = 100 is fixed empirically.
ad hoc to paper The decay schedule λ = η/n makes the added term vanish fast enough relative to the EI regret that L_n(EI_{Γ,n}) ~ L_n(EI_n).
Corollary 1 relies on this, but it requires the unstated condition (ν∧1)/d < 1; for faster-decaying EI regret in low dimension, the 1/n term can dominate.
domain assumption GP hyperparameters are locally smooth, so a single hyperparameter set can be reused when evaluating candidate points and rank-1 updates remain valid.
Section 3.2 asserts 'local smoothness of Gaussian process hyperparameters' to justify computational efficiency; no quantitative bound is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FigBO: A Generalized Acquisition Function Framework with Look-Ahead Capability for Bayesian Optimization." pith.science (2026). https://pith.science/paper/O3IDK2JL

@misc{pith2026250420307,
  author       = {Pith},
  title        = {Pith review of: FigBO: A Generalized Acquisition Function Framework with Look-Ahead Capability for Bayesian Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O3IDK2JL}},
  note         = {Machine review of arXiv:2504.20307}
}

read the original abstract

Bayesian optimization is a powerful technique for optimizing expensive-to-evaluate black-box functions, consisting of two main components: a surrogate model and an acquisition function. In recent years, myopic acquisition functions have been widely adopted for their simplicity and effectiveness. However, their lack of look-ahead capability limits their performance. To address this limitation, we propose FigBO, a generalized acquisition function that incorporates the future impact of candidate points on global information gain. FigBO is a plug-and-play method that can integrate seamlessly with most existing myopic acquisition functions. Theoretically, we analyze the regret bound and convergence rate of FigBO when combined with the myopic base acquisition function expected improvement (EI), comparing them to those of standard EI. Empirically, extensive experimental results across diverse tasks demonstrate that FigBO achieves state-of-the-art performance and significantly faster convergence compared to existing methods.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 27 canonical work pages

[1]

Cambridge University Press, Cambridge, UK (2023)

Garnett, R.: Bayesian Optimization. Cambridge University Press, Cambridge, UK (2023)

work page 2023
[2]

Machine Learning, 1–30 (2022) 17

Hayashi, S., Honda, J., Kashima, H.: Bayesian optimization with partially specified queries. Machine Learning, 1–30 (2022) 17

work page 2022
[3]

Machine Learning 113(5), 2701–2731 (2024)

Candelieri, A., Ponti, A., Archetti, F.: Fair and green hyperparameter optimiza- tion via multi-objective and multiple information source Bayesian optimization. Machine Learning 113(5), 2701–2731 (2024)

work page 2024
[4]

Advances in Neural Information Processing Systems 24 (2011)

Bergstra, J., Bardenet, R., Bengio, Y., K´ egl, B.: Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems 24 (2011)

work page 2011
[5]

arXiv preprint arXiv:2204.11051 (2022)

Hvarfner, C., Stoll, D., Souza, A., Lindauer, M., Hutter, F., Nardi, L.: πBO: Augmenting acquisition functions with user beliefs for Bayesian optimization. arXiv preprint arXiv:2204.11051 (2022)

arXiv 2022
[6]

In: International Con- ference on Automated Machine Learning, pp

ˇSehi´ c, K., Gramfort, A., Salmon, J., Nardi, L.: Lassobench: A high-dimensional hyperparameter optimization benchmark suite for lasso. In: International Con- ference on Automated Machine Learning, pp. 2–1 (2022). PMLR

work page 2022
[7]

Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels

Ru, B., Wan, X., Dong, X., Osborne, M.: Interpretable neural architecture search via Bayesian optimisation with weisfeiler-Lehman kernels. arXiv preprint arXiv:2006.07556 (2020)

work page Pith review arXiv 2020
[8]

IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9), 5503–5515 (2021)

Wang, L., Xie, S., Li, T., Fonseca, R., Tian, Y.: Sample-efficient neural architec- ture search by learning actions for monte carlo tree search. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9), 5503–5515 (2021)

work page 2021

Show all 36 references

[9]

arXiv preprint arXiv:1602.04450 (2016)

Berkenkamp, F., Krause, A., Schoellig, A.P.: Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics.” arxiv. arXiv preprint arXiv:1602.04450 (2016)

2016 arXiv
[10]

In: Learning and Intelligent Optimiza- tion: 8th International Conference, Lion 8, Gainesville, FL, USA, February 16-21,

Calandra, R., Gopalan, N., Seyfarth, A., Peters, J., Deisenroth, M.P.: Bayesian gait optimization for bipedal locomotion. In: Learning and Intelligent Optimiza- tion: 8th International Conference, Lion 8, Gainesville, FL, USA, February 16-21,
[11]

43347–43368 (2023)

Ziomek, J.K., Ammar, H.B.: Are random decompositions all we need in high dimensional Bayesian optimisation? In: International Conference on Machine Learning, pp. 43347–43368 (2023). PMLR

2023
[12]

In: Uncertainty in Artificial Intelligence, pp

Eriksson, D., Jankowiak, M.: High-dimensional bayesian optimization with sparse axis-aligned subspaces. In: Uncertainty in Artificial Intelligence, pp. 493–503 (2021). PMLR

2021
[13]

Advances in Neural Information Processing Systems 36, 20577–20612 (2023)

Ament, S., Daulton, S., Eriksson, D., Balandat, M., Bakshy, E.: Unexpected improvements to expected improvement for Bayesian optimization. Advances in Neural Information Processing Systems 36, 20577–20612 (2023)

2023
[14]

Journal of Global Optimization 13, 455–492 (1998) 18

Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13, 455–492 (1998) 18

1998
[15]

IEEE Transactions on Information Theory 58(5), 3250–3265 (2012)

Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.W.: Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory 58(5), 3250–3265 (2012)

2012
[16]

Biometrika 25(3-4), 285–294 (1933)

Thompson, W.R.: On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3-4), 285–294 (1933)

1933
[17]

Journal of Machine Learning Research 13(6) (2012)

Hennig, P., Schuler, C.J.: Entropy search for information-efficient global opti- mization. Journal of Machine Learning Research 13(6) (2012)

2012
[18]

Advances in Neural Information Processing Systems 35, 11494–11506 (2022)

Hvarfner, C., Hutter, F., Nardi, L.: Joint entropy search for maximally-informed Bayesian optimization. Advances in Neural Information Processing Systems 35, 11494–11506 (2022)

2022
[19]

Journal of Machine Learning Research 12(10) (2011)

Bull, A.D.: Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research 12(10) (2011)

2011
[20]

arXiv preprint arXiv:1012.2599 (2010)

Brochu, E., Cora, V.M., De Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 (2010)

2010 arXiv
[21]

arXiv preprint arXiv:2311.14645 (2023)

Hvarfner, C., Hutter, F., Nardi, L.: A general framework for user-guided Bayesian optimization. arXiv preprint arXiv:2311.14645 (2023)

2023 arXiv
[22]

Advances in Neural Information Processing Systems 25 (2012)

Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems 25 (2012)

2012
[23]

In: Learning and Intelligent Optimization: 5th International Conference, LION 5, Rome, Italy, January 17-21, 2011

Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Learning and Intelligent Optimization: 5th International Conference, LION 5, Rome, Italy, January 17-21, 2011. Selected Papers 5, pp. 507–523 (2011). Springer

2011
[24]

MIT Press, Cambridge, MA (2006)

Williams, C.K.I., Rasmussen, C.E.: Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA (2006)

2006
[25]

Machine Learning 45, 5–32 (2001)

Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)

2001
[26]

In: International Conference on Machine Learning, pp

Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: International Conference on Machine Learning, pp. 1613–1622 (2015). PMLR

2015
[27]

Journal of Global Optimization 21, 345–383 (2001)

Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization 21, 345–383 (2001)

2001
[28]

Advances in Neural 19 Information Processing Systems 27 (2014)

Hern´ andez-Lobato, J.M., Hoffman, M.W., Ghahramani, Z.: Predictive entropy search for efficient global optimization of black-box functions. Advances in Neural 19 Information Processing Systems 27 (2014)

2014
[29]

In: International Conference on Machine Learning, pp

Wang, Z., Jegelka, S.: Max-value entropy search for efficient Bayesian optimiza- tion. In: International Conference on Machine Learning, pp. 3627–3635 (2017). PMLR

2017
[30]

SIAM Journal on Control and Optimization 47(5), 2410–2439 (2008)

Frazier, P.I., Powell, W.B., Dayanik, S.: A knowledge-gradient policy for sequen- tial information collection. SIAM Journal on Control and Optimization 47(5), 2410–2439 (2008)

2008
[31]

arXiv preprint arXiv:2402.02229 (2024)

Hvarfner, C., Hellsten, E.O., Nardi, L.: Vanilla Bayesian optimization performs great in high dimension. arXiv preprint arXiv:2402.02229 (2024)

2024 arXiv
[32]

Operations Research 43(4), 684–691 (1995)

Ko, C.-W., Lee, J., Queyranne, M.: An exact algorithm for maximum entropy sampling. Operations Research 43(4), 684–691 (1995)

1995
[33]

The Annals of Mathematical Statistics 21(1), 124–127 (1950)

Sherman, J., Morrison, W.J.: Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. The Annals of Mathematical Statistics 21(1), 124–127 (1950)

1950
[34]

arXiv preprint arXiv:2109.06716 (2021)

Eggensperger, K., M¨ uller, P., Mallik, N., Feurer, M., Sass, R., Klein, A., Awad, N., Lindauer, M., Hutter, F.: HPOBench: A collection of reproducible multi-fidelity benchmark problems for HPO. arXiv preprint arXiv:2109.06716 (2021)

2021 arXiv
[35]

Advances in Neural Information Processing systems 20 (2007) 20

Rahimi, A., Recht, B.: Random features for large-scale kernel machines. Advances in Neural Information Processing systems 20 (2007) 20

2007
[2014]

274–290 (2014)

Revised Selected Papers 8, pp. 274–290 (2014). Springer

2014

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Cambridge University Press, Cambridge, UK (2023)

Garnett, R.: Bayesian Optimization. Cambridge University Press, Cambridge, UK (2023)

work page 2023

[2] [2]

Machine Learning, 1–30 (2022) 17

Hayashi, S., Honda, J., Kashima, H.: Bayesian optimization with partially specified queries. Machine Learning, 1–30 (2022) 17

work page 2022

[3] [3]

Machine Learning 113(5), 2701–2731 (2024)

Candelieri, A., Ponti, A., Archetti, F.: Fair and green hyperparameter optimiza- tion via multi-objective and multiple information source Bayesian optimization. Machine Learning 113(5), 2701–2731 (2024)

work page 2024

[4] [4]

Advances in Neural Information Processing Systems 24 (2011)

Bergstra, J., Bardenet, R., Bengio, Y., K´ egl, B.: Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems 24 (2011)

work page 2011

[5] [5]

arXiv preprint arXiv:2204.11051 (2022)

Hvarfner, C., Stoll, D., Souza, A., Lindauer, M., Hutter, F., Nardi, L.: πBO: Augmenting acquisition functions with user beliefs for Bayesian optimization. arXiv preprint arXiv:2204.11051 (2022)

arXiv 2022

[6] [6]

In: International Con- ference on Automated Machine Learning, pp

ˇSehi´ c, K., Gramfort, A., Salmon, J., Nardi, L.: Lassobench: A high-dimensional hyperparameter optimization benchmark suite for lasso. In: International Con- ference on Automated Machine Learning, pp. 2–1 (2022). PMLR

work page 2022

[7] [7]

Interpretable Neural Architecture Search via Bayesian Optimisation with Weisfeiler-Lehman Kernels

Ru, B., Wan, X., Dong, X., Osborne, M.: Interpretable neural architecture search via Bayesian optimisation with weisfeiler-Lehman kernels. arXiv preprint arXiv:2006.07556 (2020)

work page Pith review arXiv 2020

[8] [8]

IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9), 5503–5515 (2021)

Wang, L., Xie, S., Li, T., Fonseca, R., Tian, Y.: Sample-efficient neural architec- ture search by learning actions for monte carlo tree search. IEEE Transactions on Pattern Analysis and Machine Intelligence 44(9), 5503–5515 (2021)

work page 2021