Pith. sign in

REVIEW 3 major objections 6 minor 85 references

Adaptive Parameter Optimization in Gaussian Processes: A Comprehensive Study of Uncertainty Quantification and Dimensional Scaling

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adaptive Gaussian-process search beats fixed strategies, with the largest gains in high-dimensional noisy problems.

desk verdict A plausible adaptive-GP-UCB heuristic, but the central 'always outperform' claim rests on a one-sentence proof sketch and on empirical results that exist only as prose. read the letter →

arxiv 2507.15138 v1 pith:CAN53XNS submitted 2025-07-20 math.OC

Nishant Gadde This is my paper · ORCID
classification math.OC
keywords GaussianprocessoptimizationBayesianadaptiveexploration-exploitationtrade-offuncertaintyquantificationregretboundshigh-dimensionalnoisyblack-boxupperconfidencebound
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that Gaussian-process optimization improves when the exploration-exploitation parameter and an uncertainty-penalty weight are updated online instead of fixed. The proposed acquisition function combines the posterior mean, an adaptive exploration term, and a penalty that grows with posterior variance and local curvature. The paper derives a sublinear cumulative-regret bound for this scheme and reports that adaptive strategies beat fixed-parameter UCB in every tested configuration, with the biggest margins at high dimension and high noise. If correct, this gives a practical route to removing manual parameter tuning in expensive black-box optimization.

What carries the argument

The carrying mechanism is the uncertainty-aware acquisition function of Eq. (15): $\alpha_t(x) = \mu_t(x) + \kappa_t \sigma_t(x) - \lambda_t U_t(x)$, where $U_t(x) = \sigma_t^2(x) C_t(x)$ and $C_t(x) = \sum_{i=1}^{d} \max(|\lambda_i|, \epsilon)$ uses the eigenvalues of the Hessian of the posterior mean to estimate local curvature. Adaptive update rules move $\kappa_t$ up when recent prediction error exceeds its moving average and move $\lambda_t$ up when integrated posterior variance exceeds its moving average. These rules are what the regret analysis extends; the maximum information gain $\gamma_T$ enters through $\beta_T$ and through kernel-dependent bounds on $\gamma_T$ for linear, squared-exponential, and Mat\'ern kernels.

What would settle it

Run the algorithm on the Ackley function in $d=20$ with $\sigma_n = 0.05$, record $\lambda_t$ and $C_t$ at each of 100 evaluations, and compare observed regret with Theorem 1's bound; if $\lambda_t$ or $C_t$ exceeds the assumed $\lambda_{\max}$ or $C_{\max}$ while regret stays sublinear, the boundedness assumption rather than the adaptive rule is carrying the theoretical guarantee.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that uncertainty quantification can serve as a control signal: raise exploration when recent prediction errors exceed their moving average, and raise the penalty on uncertain regions when global integrated variance exceeds its moving average. Formally, for the acquisition function $\alpha_t(x) = \mu_t(x) + \kappa_t \sigma_t(x) - \lambda_t \sigma_t^2(x) C_t(x)$, where $C_t(x)$ is the sum of absolute Hessian eigenvalues of the posterior mean, the paper derives a cumulative-regret bound $R_T \le \sqrt{C_1 T \beta_T \gamma_T} + C_2$ holding with probability at least $1-\delta$, under boundedness assumptions on $\kappa_t$, $\lambda_t$, and $C_t$. Empirically, the paper reports consistent outperformance of fixed-parameter GP-UCB across dimensions $d \in \{2,5,10,20\}$ and noise levels $\sigma_n \in \{0.001, 0.005, 0.01, 0.05\}$, with advantages growing from roughly 12-15% in easy settings to 31-37% in hard ones.

Load-bearing premise

The regret guarantee presumes that the adaptive parameters and the curvature-based complexity factor stay inside fixed bounds, but the paper does not prove its update rules enforce those bounds.

Editorial extensions

If this is right

  • Practitioners could drop manual tuning of $\kappa$ in GP-UCB: the adaptive rule reproduces or beats the best fixed $\kappa$ in every tested configuration.
  • The advantage of adaptivity grows with both dimension and noise, so the method is most valuable exactly where Bayesian optimization is hardest.
  • The sublinear regret bound transfers to linear, squared-exponential, and Mat\'ern kernels, so convergence to the global optimum is preserved despite parameter adaptivity.
  • Simple regret converges at $O(\sqrt{\gamma_T / T})$ up to logarithmic factors, matching known rates for fixed-parameter GP optimization.
  • Lower variance across random initializations means fewer repeated runs are needed to obtain a reliable solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper compares against fixed-$\kappa$ UCB, EI, random search, CMA-ES, and BOHAMIANN, but it never ablates its own components; an experiment with only the $\kappa$ update active and $\lambda = 0$ would reveal which mechanism drives the reported gains.
  • Editorial inference: the Hessian-based complexity factor requires finite-difference estimates after only 10-100 evaluations in $d=20$; a cheaper proxy such as local nearest-neighbor variance or a GP gradient norm could test whether the complexity term is essential or merely suppresses exploration.
  • Editorial inference: the proof assumes bounds on $\lambda_t$ and $C_t$ that the update rules do not explicitly enforce; if those quantities drift, the empirical success may rest on a mechanism other than the stated regret theorem.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an adaptive extension of GP-UCB in which the exploration parameter κ_t and an uncertainty-penalty coefficient λ_t are updated online from prediction errors and integrated posterior variance, and an extra term λ_t U_t(x) is subtracted from the acquisition function. It states a regret bound (Theorem 1), a simple-regret bound (Theorem 2), and reports a large empirical comparison claiming that adaptive strategies always outperform fixed-parameter baselines, especially in high-dimensional and noisy settings. The theoretical section gives only a one-sentence proof sketch for Theorem 1, and the empirical section reports only prose summaries and figure captions, without tables of raw results, error bars, code, or reproducibility data.

Significance. If the regret bound and the empirical superiority claim were fully established, the adaptive update heuristics could be a practically useful contribution to Bayesian optimization. The paper does provide a clear algorithmic description (Algorithm 1), a broad literature review, and a plausible mechanism for adapting exploration. However, the central theoretical result is not derived, and the central empirical claim is not verifiable from the manuscript as written. The paper therefore does not currently meet the evidentiary standard for its headline conclusions.

major comments (3)
  1. [Section 3.2.1, Theorem 1 and Assumptions 3-4]
  2. [Section 5, Results and Analysis] The paper's central empirical claim—that adaptive strategies 'always outperform' fixed ones and do so by large margins in difficult settings—is reported only through prose and figure captions (e.g., '23% higher', '37% in 20 dimensions', '31% at σ_n=0.05'). No tables of mean values, standard deviations, standard errors, or per-trial data are provided, and the figures themselves are not included in the manuscript text. This makes the claim non-verifiable. The robustness analysis in Section 5.0.7 mentions a 40% lower coefficient of variation but again gives no supporting numbers. For a 'comprehensive empirical study', the absence of raw data or a reproducibility artifact is a serious deficiency.
  3. [Sections 3.1.3 and 4.4, parameter choices and baselines] The update-rate constants β=0.1, γ=0.05, and η=0.1 are described as set 'based on preliminary experiments' (Section 3.1.3) and the experiments in Section 5 are run on the same test functions and noise settings used for that informal tuning. The fixed-κ baselines are drawn from a small hand-picked set {0.1, 0.5, 1.0, 2.0} rather than a systematic sweep or a theoretically motivated schedule. Consequently, the reported 'always outperform' conclusion may reflect tuning advantage rather than a general property of adaptive strategies, and the paper provides no cross-validation or sensitivity analysis to rule this out.
minor comments (6)
  1. [Abstract] The abstract contains the word 'strategery' and the phrase 'show conclusively', which overstates the evidence presented in the manuscript.
  2. [Section 2.2] The phrase 'usefully useful' appears in the discussion of GP assets; this should be corrected.
  3. [Algorithm 1, line 5] There is a time-index inconsistency: Algorithm 1 computes α_{t-1}(x) = μ_{t-1}(x) + κ_t σ_{t-1}(x) - λ_t U_{t-1}(x), while Eq. (15) defines α_t(x) with matching time subscripts. This should be harmonized.
  4. [Section 6.2.4] Reference [17] is cited for meta-learning and transfer-learning approaches, but the cited paper (Feurer et al., hyperparameter optimization via spectral approach) is not a transfer- or meta-learning method; a more appropriate citation is needed.
  5. [General] No code or data availability statement is provided, which further hinders reproducibility of the empirical claims. A statement with a repository link or supplemental material should be added if a revised version is produced.
  6. [Section 4.2] The spelling 'Matérn' is inconsistent in several places, and the section numbering in Section 5 uses third-level subsections such as 5.0.1, which should be renumbered.

Circularity Check

2 steps flagged · score 4.0 of 10

Theorem 1's regret proof reduces to its own boundedness assumption, and the empirical superiority claim relies on adaptation constants tuned on the same benchmarks.

  1. other [Section 3.2.1, Assumption 3 and proof of Theorem 1 (Eq. 24)]
    "The adaptive parameters κt and λt remain bounded: κmin ≤ κt ≤ κmax and 0 ≤ λt ≤ λmax for all t. ... The proof follows from extending the analysis of Srinivas to account for our adaptive parameter update rules and the uncertainty penalty term. The key insight is that our update rules ensure that κt and λt remain bounded, allowing us to leverage existing regret bounds while benefiting from the adaptive nature of our approach."

    Theorem 1's proof never uses the functional form of the adaptive updates (Eqs. 18-21). The only bridge from the algorithm to the regret bound is the claim that the update rules keep κt and λt bounded, which is exactly Assumption 3. Eq. (20) multiplies λt by 1 + γ(It−Īt)/Īt with no cap, so global boundedness is not a consequence of the algorithm. The regret bound therefore reduces to its own hypothesis: assume the parameters stay in a box, then restate a fixed-parameter regret bound. No property of adaptivity is actually derived.

  2. fitted input called prediction [Sections 3.1.3/4.2 (learning-rate selection) and Section 5.0.1 (reported superiority)]
    "In our implementation, we set β = 0.1 and γ = 0.05 based on preliminary experiments, which provide a good balance between adaptivity and stability. ... The results demonstrate that our adaptive approach consistently outperforms all baseline methods in terms of both convergence speed and final solution quality."

    The adaptation constants β, γ, and η were selected by 'preliminary experiments' on the same family of test functions, dimensionalities, and noise levels later used for the reported comparison, and the fixed baselines are limited to κ ∈ {0.1, 0.5, 1.0, 2.0}. The headline claim that adaptive strategies 'always outperform fixed ones' is therefore an in-sample evaluation of a tuned configuration, not an out-of-sample prediction of an untuned adaptive strategy. The fitted constants are inputs to the experiment, and the 'always outperform' result is closely tied to those inputs rather than being an independent finding.

full rationale

The strongest circularity is in the theoretical section: Theorem 1 is presented as a regret assurance for the adaptive algorithm, but the proof sketch identifies the boundedness of κt and λt as the key insight, which is precisely Assumption 3 and is never derived from the update equations. Eq. (20) has no cap and can push λt past any prescribed λmax, so the theorem's conclusion depends on an unproved hypothesis, making the 'assurance' a conditional restatement rather than a derivation. Separately, the empirical claim that adaptive strategies always outperform fixed ones is weakened because the adaptive method's free constants were tuned on the same benchmark problems and the fixed baselines are a small hand-picked set; the reported superiority is partly an artifact of that tuning. No load-bearing self-citation is present: the citations to Srinivas and others are external, and the paper does not rely on an author-specific uniqueness theorem. The omitted proof of Theorem 1, missing data/code tables, and nonexistent section cross-references are serious correctness and verifiability defects, but they are not themselves circularity. Overall, the circularity is partial, not total, so a score of 4 is appropriate.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities. The main burden is carried by hand-tuned constants (beta, gamma, eta, initial values) and unproved regularity assumptions about bounded adaptive parameters and complexity factors. The regret proof and the universal empirical claim both lean on these rather than on a self-contained derivation.

free parameters (5)
  • beta (kappa learning rate) = 0.1
    Chosen 'based on preliminary experiments' (Sections 3.1.3 and 4.2), i.e., tuned on test problems before the reported comparisons.
  • gamma (lambda learning rate) = 0.05
    Chosen 'based on preliminary experiments' (Sections 3.1.3 and 4.2), with no described search procedure.
  • eta (moving average smoothing) = 0.1
    Set by hand in Section 3.1.3 as a 'reasonable compromise'; affects how quickly kappa and lambda respond.
  • initial kappa_1 = not reported
    Required by Algorithm 1 but its default value is not stated, making exact replication impossible.
  • initial lambda_1 = varied in {0.001, 0.01, 0.1}
    An experimental input studied in Section 4.1.1; the paper gives no principle for choosing it outside the sweep.
assumptions (5)
  • domain assumption The objective function has bounded RKHS norm and the observation noise is sub-Gaussian.
    Assumptions 1 and 2 of Theorem 1 in Section 3.2.1; standard GP-UCB conditions, but not verified for any of the benchmark or real-world problems.
  • ad hoc to paper The adaptive parameters kappa_t and lambda_t remain bounded, and the complexity factor C_t(x) is uniformly bounded.
    Assumptions 3 and 4 of Theorem 1. The paper does not prove that Eqs. (18)-(21) preserve these bounds, and lambda_t has no cap in Eq. (20). The theorem depends on this.
  • ad hoc to paper The existing GP-UCB regret analysis of Srinivas extends to the adaptive update rules and uncertainty penalty term.
    The only proof of Theorem 1 is the sentence 'The proof follows from extending the analysis of Srinivas to account for our adaptive parameter update rules and the uncertainty penalty term'; no formal extension is supplied.
  • domain assumption The GP surrogate is correctly specified enough for its predictive variance to be a reliable uncertainty signal.
    The adaptive update rules and uncertainty penalty only help if sigma_t(x) reflects true predictive uncertainty; no misspecification analysis is given.
  • domain assumption The two-stage acquisition optimizer (Sobol global search with 1000 samples plus L-BFGS-B refinement) returns the true argmax of the acquisition function.
    The regret theory assumes exact selection x_t = argmax; the paper notes the optimization is only approximate, and the effect of the approximation on regret is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Parameter Optimization in Gaussian Processes: A Comprehensive Study of Uncertainty Quantification and Dimensional Scaling." pith.science (2026). https://pith.science/paper/CAN53XNS

@misc{pith2026250715138,
  author       = {Pith},
  title        = {Pith review of: Adaptive Parameter Optimization in Gaussian Processes: A Comprehensive Study of Uncertainty Quantification and Dimensional Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CAN53XNS}},
  note         = {Machine review of arXiv:2507.15138}
}
read the original abstract

Gaussian Process (GP) models have also become extremely useful for optimization under uncertainty algorithms, especially where the objective functions are costly to compute. Yet, the more classical methods usually adopt strategies that, in certain circumstances, might be effective but not flexible to be applied to a wide range of problem terrains. This study aims to adapt parameter optimization in GP models and especially how uncertainty quantification can assist in the learning process. We investigate the effect of adaptive kappa parameters that govern the exploration-exploitation trade-off and the interplay between dimensionality, penalty on uncertainty, and noise levels to influence optimization results. Uncertainty quantification is built directly into our comprehensive theoretical framework and gives us new algorithms to dynamically tune exploration-exploitation trade-offs according to the uncertainty trend observed in nature. We rigorously empirically test various strategies, parametrizing our tests by dimensionality, noise, penalty terms, and evaluate the performance of any given strategery in a wide variety of test settings, and show conclusively that adaptive strategies always outperform fixed ones, but in difficult settings, where the dimensions are large and the noise is severe, the advantage is enormous. We build theoretical assurances of convergence under different settings as well as furnish a sensible direction on the application of adaptive GP-based optimization even in very complicated conditions. The results of our work will help in the development of more efficient and robust methods of optimization of realistic problems in which there are only a few functions available for evaluation, and when quantifying the uncertainty, there is a need to know more about the uncertainty.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 75 canonical work pages

  1. [1]

    Alvarez, Lorenzo Rosasco, and Neil D

    Mauricio A. Alvarez, Lorenzo Rosasco, and Neil D. Lawrence. Kernels for vector-valued functions: A review. Foundations and Trends in Machine Learning , 4(3):195–266, 2012

    2012

  2. [2]

    Bayesian optimization of composite functions

    Raul Astudillo and Peter Frazier. Bayesian optimization of composite functions. Inter- national Conference on Machine Learning , pages 354–363, 2019

    2019

  3. [3]

    Using confidence bounds for exploitation-exploration trade-offs

    Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3:397–422, 2002

    2002

  4. [4]

    Schoellig, and Andreas Krause

    Felix Berkenkamp, Angela P. Schoellig, and Andreas Krause. No-regret Bayesian optimization with unknown hyperparameters. Journal of Machine Learning Research , 20(50):1–24, 2019

    2019

  5. [5]

    Adversarially robust optimization with Gaussian processes

    Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, and Volkan Cevher. Adversarially robust optimization with Gaussian processes. Advances in Neural Information Processing Systems, 31, 2018

    2018

  6. [6]

    Cora, and Nando De Freitas

    Eric Brochu, Vlad M. Cora, and Nando De Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 , 2010

    arXiv 2010

  7. [7]

    Adam D. Bull. Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, 12:2879–2904, 2011

    work page 2011

  8. [8]

    Rates of convergence for sparse variational Gaussian process regression

    David Burt, Carl Edward Rasmussen, and Mark Van Der Wilk. Rates of convergence for sparse variational Gaussian process regression. Proceedings of the 36th International Conference on Machine Learning , pages 862–871, 2019

    work page 2019

Show all 85 references
  1. [9]

    Gaussian process optimization with adaptive sketching: Scalable and no regret

    Daniele Calandriello, Luigi Carratino, Alessandro Lazaric, Michal Valko, and Lorenzo Rosasco. Gaussian process optimization with adaptive sketching: Scalable and no regret. Proceedings of the 32nd Conference on Learning Theory , pages 533–557, 2019

    2019

  2. [10]

    Kernelized bandits: An algorithm for function optimization

    Sayak Ray Chowdhury and Aditya Gopalan. Kernelized bandits: An algorithm for function optimization. arXiv preprint arXiv:1706.06290 , 2017

    2017 arXiv

  3. [11]

    Constantine, Eric Dow, and Qiqi Wang

    Paul G. Constantine, Eric Dow, and Qiqi Wang. Active subspaces: Emerging ideas for dimension reduction in parameter studies. SIAM Journal on Scientific Computing , 36(4):A1500–A1524, 2014

    2014

  4. [12]

    Lawrence

    Andreas Damianou and Neil D. Lawrence. Deep Gaussian processes.Artificial Intelligence and Statistics, pages 207–215, 2013

    2013

  5. [13]

    Additive Gaussian processes

    David Duvenaud, Hannes Nickisch, and Carl Edward Rasmussen. Additive Gaussian processes. Advances in Neural Information Processing Systems , 24, 2011

    2011

  6. [14]

    Duvenaud

    David K. Duvenaud. Automatic model construction with Gaussian processes. PhD Thesis, University of Cambridge , 2014. 44

    2014

  7. [15]

    Kernel cookbook

    David Duvenaud. Kernel cookbook. URL: https://www.cs.toronto.edu/ duve- naud/cookbook/, 2014

    2014

  8. [16]

    Turner, and Matthias Poloczek

    David Eriksson, Michael Pearce, Jacob Gardner, Ryan D. Turner, and Matthias Poloczek. Scalable global optimization via local Bayesian optimization. Advances in Neural Information Processing Systems, 32, 2019

    2019

  9. [17]

    Hyperparameter optimization: A spectral approach

    Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. Hyperparameter optimization: A spectral approach. arXiv preprint arXiv:1706.00764, 2019

    2019 arXiv

  10. [18]

    Priors in Bayesian deep learning: A review

    Vincent Fortuin. Priors in Bayesian deep learning: A review. International Statistical Review, 91(1):142–174, 2023

    2023

  11. [19]

    Peter I. Frazier. A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811 , 2018

    2018 arXiv

  12. [20]

    Mathematical foundations of Gaussian processes

    Theo Galy-Fajou, Manfred Opper, and Cedric Archambeau. Mathematical foundations of Gaussian processes. arXiv preprint arXiv:2201.12045 , 2022

    2022 arXiv

  13. [21]

    Weinberger, David Bindel, and Andrew G

    Jacob Gardner, Geoff Pleiss, Kilian Q. Weinberger, David Bindel, and Andrew G. Wilson. GPyTorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration. Advances in Neural Information Processing Systems , 31, 2018

    2018

  14. [22]

    Bayesian Optimization

    Roman Garnett. Bayesian Optimization. Cambridge University Press , 2023

    2023

  15. [23]

    Marc G. Genton. Classes of kernels for machine learning: a statistics perspective. Journal of Machine Learning Research, 2:299–312, 2001

    2001

  16. [24]

    Batch Bayesian optimization via local penalization

    Javier Gonzalez, Zhenwen Dai, Philipp Hennig, and Neil Lawrence. Batch Bayesian optimization via local penalization. Artificial Intelligence and Statistics , pages 648–657, 2016

    2016

  17. [25]

    Robert B. Gramacy. Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences. Chapman and Hall/CRC , 2020

    2020

  18. [26]

    Completely derandomized self-adaptation in evolution strategies

    Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation , 9(2):159–195, 2001

    2001

  19. [27]

    M¨ uller, and Petros Koumoutsakos

    Nikolaus Hansen, Sibylle D. M¨ uller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation , 11(1):1–18, 2003

    2003

  20. [28]

    Osborne, and Hans P

    Philipp Hennig, Michael A. Osborne, and Hans P. Kersting. Probabilistic numerics: Computation as machine learning. Cambridge University Press , 2022

    2022

  21. [29]

    Lawrence

    James Hensman, Nicolo Fusi, and Neil D. Lawrence. Gaussian processes for big data. Uncertainty in Artificial Intelligence , pages 282–290, 2013. 45

    2013

  22. [30]

    Hoffman, and Zoubin Ghahramani

    Jos´ e Miguel Hern´ andez-Lobato, Matthew W. Hoffman, and Zoubin Ghahramani. Predic- tive entropy search for efficient global optimization of black-box functions. Advances in Neural Information Processing Systems , 27, 2014

    2014

  23. [31]

    Hoffman, Eric Brochu, and Nando de Freitas

    Matthew W. Hoffman, Eric Brochu, and Nando de Freitas. Portfolio allocation for Bayesian optimization. Uncertainty in Artificial Intelligence , pages 327–336, 2011

    2011

  24. [32]

    Hoffman, David M

    Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. Stochastic variational inference. Journal of Machine Learning Research , 14(1):1303–1347, 2013

    2013

  25. [33]

    Jones, Matthias Schonlau, and William J

    Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization , 13(4):455–492, 1998

    1998

  26. [34]

    Sriperumbudur

    Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic, and Bharath K. Sriperumbudur. Gaussian processes and kernel methods: A review on connections and equivalences. arXiv preprint arXiv:1807.02582 , 2018

    2018 arXiv

  27. [35]

    Oliva, Jeff Schneider, and Barnabas Poczos

    Kirthevasan Kandasamy, Gautam Dasarathy, Junier B. Oliva, Jeff Schneider, and Barnabas Poczos. Multi-fidelity Bayesian optimisation with continuous approximations. Proceedings of the 34th International Conference on Machine Learning , pages 1799–1808, 2017

    2017

  28. [36]

    Parallelised Bayesian optimisation via Thompson sampling

    Kirthevasan Kandasamy, Akshay Krishnamurthy, Jeff Schneider, and Barnab´ as P´ oczos. Parallelised Bayesian optimisation via Thompson sampling. International Conference on Artificial Intelligence and Statistics , pages 133–142, 2018

    2018

  29. [37]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  30. [38]

    Adaptive Gaussian process bandits with heteroscedastic noise

    Johannes Kirschner, Mojmir Mutny, Nicole Hiller, Rasmus Ischebeck, and Andreas Krause. Adaptive Gaussian process bandits with heteroscedastic noise. International Conference on Machine Learning , pages 3458–3467, 2019

    2019

  31. [39]

    Fast Bayesian optimization of machine learning hyperparameters on large datasets

    Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, and Frank Hutter. Fast Bayesian optimization of machine learning hyperparameters on large datasets. Artificial Intelligence and Statistics , pages 528–536, 2017

    2017

  32. [40]

    Accurate uncertainties for deep learning using calibrated regression

    Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. International Conference on Machine Learning, pages 2796–2804, 2018

    2018

  33. [41]

    Harold J. Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering , 86(1):97–106, 1964

    1964

  34. [42]

    Le, Alex J

    Quoc V. Le, Alex J. Smola, and St´ ephane Canu. Heteroscedastic Gaussian process regression. Proceedings of the 22nd International Conference on Machine Learning, pages 489–496, 2005. 46

    2005

  35. [43]

    Re-examining linear embeddings for high-dimensional Bayesian optimization

    Benjamin Letham, Brian Karrer, Guilherme Ottoni, and Eytan Bakshy. Re-examining linear embeddings for high-dimensional Bayesian optimization. Advances in Neural Information Processing Systems, 33:1546–1558, 2020

    2020

  36. [44]

    Hyperband: A novel bandit-based approach to hyperparameter optimization

    Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research, 18(1):6765–6816, 2018

    2018

  37. [45]

    Gaussian process regression with het- eroscedastic residuals and exact marginal likelihood

    Haitao Liu, Jianfei Cai, and Yew-Soon Ong. Gaussian process regression with het- eroscedastic residuals and exact marginal likelihood. International Joint Conference on Artificial Intelligence, pages 2339–2345, 2018

    2018

  38. [46]

    Gaussian Processes for Machine Learning

    Haitao Liu, Yew-Soon Ong, Xiaobo Shen, and Jianfei Cai. Gaussian Processes for Machine Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence , 42(11):2816–2833, 2020

    2020

  39. [47]

    David J. C. MacKay. Introduction to Gaussian processes. NATO ASI Series F Computer and Systems Sciences , 168:133–166, 1998

    1998

  40. [48]

    Practical Bayesian optimiza- tion in the presence of outliers

    Ruben Martinez-Cantin, Kevin Tee, and Michael McCourt. Practical Bayesian optimiza- tion in the presence of outliers. International Conference on Artificial Intelligence and Statistics, pages 1722–1731, 2018

    2018

  41. [49]

    Alexander G. de G. Matthews. Sparse Gaussian process methods for higher-dimensional pattern recognition problems. PhD Thesis, University of Cambridge , 2016

    2016

  42. [50]

    The application of Bayesian methods for seeking the extremum

    Jonas Mockus. The application of Bayesian methods for seeking the extremum. Towards Global Optimization, 2(117-129):2, 1978

    1978

  43. [51]

    Iain Murray and Ryan P. Adams. Slice sampling covariance hyperparameters of latent Gaussian models. Advances in Neural Information Processing Systems , 23, 2010

    2010

  44. [52]

    Recent Advances in Bayesian Optimization

    Vu Nguyen, Sunil Gupta, Santu Rana, Cheng Li, and Svetha Venkatesh. Recent Advances in Bayesian Optimization. ACM Computing Surveys , 2022

    2022

  45. [53]

    Gaussian Process Approximation & Uncertainty Quantification for Scientific Machine Learning

    Marcus Noack. Gaussian Process Approximation & Uncertainty Quantification for Scientific Machine Learning. Lawrence Berkeley National Laboratory Presentation, 2023

    2023

  46. [54]

    Oakley and Anthony O’Hagan

    Jeremy E. Oakley and Anthony O’Hagan. Probabilistic sensitivity analysis of complex models: a Bayesian approach. Journal of the Royal Statistical Society: Series B , 66(3):751–769, 2004

    2004

  47. [55]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in Neural Information Processing Systems , 20, 2008

    2008

  48. [56]

    Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006. 47

    2006

  49. [57]

    High-dimensional Bayesian optimization using low-dimensional feature spaces

    Paul Rolland, Jonathan Scarlett, Ilija Bogunovic, and Volkan Cevher. High-dimensional Bayesian optimization using low-dimensional feature spaces. Machine Learning , 107(8):1531–1558, 2018

    2018

  50. [58]

    Learning to optimize via posterior sampling

    Daniel Russo and Benjamin Van Roy. Learning to optimize via posterior sampling. Mathematics of Operations Research, 39(4):1221–1243, 2014

    2014

  51. [59]

    A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions

    Eric Schulz, Maarten Speekenbrink, and Andreas Krause. A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions. Journal of Mathematical Psychology, 85:1–16, 2018

    2018

  52. [60]

    Gaussian processes for machine learning

    Matthias Seeger. Gaussian processes for machine learning. International Journal of Neural Systems, 14(02):69–106, 2004

    2004

  53. [61]

    Adams, and Nando De Freitas

    Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando De Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE , 104(1):148–175, 2015

    2015

  54. [62]

    Sparse Gaussian processes using pseudo-inputs

    Edward Snelson and Zoubin Ghahramani. Sparse Gaussian processes using pseudo-inputs. Advances in Neural Information Processing Systems , 18, 2006

    2006

  55. [63]

    Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems , 25, 2012

    2012

  56. [64]

    Scalable Bayesian opti- mization using deep neural networks

    Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable Bayesian opti- mization using deep neural networks. International Conference on Machine Learning , pages 2171–2180, 2015

    2015

  57. [65]

    Bayesian optimization with robust Bayesian neural networks

    Jost Tobias Springenberg, Aaron Klein, Stefan Falkner, and Frank Hutter. Bayesian optimization with robust Bayesian neural networks. Advances in Neural Information Processing Systems, 29, 2016

    2016

  58. [66]

    Kakade, and Matthias Seeger

    Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias Seeger. Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design. Proceedings of the 27th International Conference on Machine Learning , pages 1015–1022, 2010

    2010

  59. [67]

    Michael L. Stein. Interpolation of spatial data: some theory for kriging. Springer Science & Business Media , 1999

    1999

  60. [68]

    Stagewise safe Bayesian optimization with Gaussian processes

    Yanan Sui, Joel Burdick, and Yisong Yue. Stagewise safe Bayesian optimization with Gaussian processes. International Conference on Machine Learning , pages 4781–4789, 2018

    2018

  61. [69]

    Kevin Swersky, Jasper Snoek, and Ryan P. Adams. Multi-task Bayesian optimization. Advances in Neural Information Processing Systems , 26, 2013. 48

    2013

  62. [70]

    Teckentrup

    Aretha L. Teckentrup. Convergence of Gaussian process regression with estimated hyper-parameters and applications in Bayesian inverse problems. SIAM/ASA Journal on Uncertainty Quantification , 8(4):1310–1337, 2020

    2020

  63. [71]

    Variational learning of inducing variables in sparse Gaussian processes

    Michalis Titsias. Variational learning of inducing variables in sparse Gaussian processes. Artificial Intelligence and Statistics , pages 567–574, 2009

    2009

  64. [72]

    Bayesian optimization is robust to noise

    Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, and Isabelle Guyon. Bayesian optimization is robust to noise. International Conference on Artificial Intelligence and Statistics , pages 1660–1668, 2021

    2021

  65. [73]

    Information-theoretic regret bounds for Gaussian process optimization in the Bandit setting

    Sattar Vakili, Kia Khezeli, and Victor Picheny. Information-theoretic regret bounds for Gaussian process optimization in the Bandit setting. IEEE Transactions on Information Theory, 67(7):4383–4399, 2021

    2021

  66. [74]

    Uncertainty quantification with Gaussian processes

    Joachim Van der Herten. Uncertainty quantification with Gaussian processes. PhD Thesis, Ghent University , 2020

    2020

  67. [75]

    Bayesian optimization in a billion dimensions via random embeddings

    Ziyu Wang, Frank Hutter, Masrour Zoghi, David Matheson, and Nando de Feitas. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55:361–387, 2016

    2016

  68. [76]

    Optimization as Estimation with Gaussian Processes in Bandit Settings

    Zi Wang and Bolei Zhou. Optimization as Estimation with Gaussian Processes in Bandit Settings. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pages 1022–1031, 2016

    2016

  69. [77]

    Max-value entropy search for efficient Bayesian optimiza- tion

    Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient Bayesian optimiza- tion. International Conference on Machine Learning , pages 3627–3635, 2017

    2017

  70. [78]

    Batched large-scale Bayesian optimization in high-dimensional spaces

    Zi Wang, Clement Gehring, Pushmeet Kohli, and Stefanie Jegelka. Batched large-scale Bayesian optimization in high-dimensional spaces. International Conference on Artificial Intelligence and Statistics , pages 745–754, 2018

    2018

  71. [79]

    Convergence of Gaussian process regression with misspecified model assumptions

    Yifan Wang, Alireza Karbalayghareh, and James Sharpnack. Convergence of Gaussian process regression with misspecified model assumptions. arXiv preprint arXiv:2104.09778, 2021

    2021 arXiv

  72. [80]

    Deep Bayesian Gaussian processes for uncertainty estimation in electronic health records

    Xiao Wang, David Sontag, and Fei Wang. Deep Bayesian Gaussian processes for uncertainty estimation in electronic health records. Nature Scientific Reports, 11(1):1–12, 2021

    2021

  73. [81]

    Christopher K. I. Williams and Carl Edward Rasmussen. Gaussian processes for regres- sion. Advances in Neural Information Processing Systems , pages 514–520, 1996

    1996

  74. [82]

    Christopher K. I. Williams. Introduction to Gaussian Processes. NATO ASI Series F Computer and Systems Sciences , 168:229–268, 2000

    2000

  75. [83]

    Gaussian process kernels for pattern discovery and extrapolation

    Andrew Gordon Wilson and Ryan Prescott Adams. Gaussian process kernels for pattern discovery and extrapolation. International Conference on Machine Learning , pages 1067–1075, 2013. 49

    2013

  76. [84]

    Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P. Xing. Deep kernel learning. Artificial Intelligence and Statistics , pages 370–378, 2016

    2016

  77. [85]

    Frazier, and Andrew Gordon Wilson

    Jian Wu, Saul Toscano-Palmerin, Peter I. Frazier, and Andrew Gordon Wilson. Practical multi-fidelity Bayesian optimization for hyperparameter tuning. Uncertainty in Artificial Intelligence, pages 788–798, 2019. 50

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.