REVIEW 8 cited by
Vanilla Bayesian Optimization Performs Great in High Dimensions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Vanilla Bayesian Optimization Performs Great in High Dimensions
read the original abstract
High-dimensional problems have long been considered the Achilles' heel of Bayesian optimization algorithms. Spurred by the curse of dimensionality, a large collection of algorithms aim to make it more performant in this setting, commonly by imposing various simplifying assumptions on the objective. In this paper, we identify the degeneracies that make vanilla Bayesian optimization poorly suited to high-dimensional tasks, and further show how existing algorithms address these degeneracies through the lens of lowering the model complexity. Moreover, we propose an enhancement to the prior assumptions that are typical to vanilla Bayesian optimization algorithms, which reduces the complexity to manageable levels without imposing structural restrictions on the objective. Our modification - a simple scaling of the Gaussian process lengthscale prior with the dimensionality - reveals that standard Bayesian optimization works drastically better than previously thought in high dimensions, clearly outperforming existing state-of-the-art algorithms on multiple commonly considered real-world high-dimensional tasks.
Forward citations
Cited by 8 Pith papers
-
Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
Conditional diffusion models trained with BO-aware strategies approximate the optimum distribution, enabling a Diffusion-based Mode Seeking acquisition function with a sub-optimality guarantee that outperforms baselin...
-
Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights
AB-SID-iVAR enables Gaussian process active learning for self-induced Boltzmann distributions by closed-form approximation of the target, with high-probability error vanishing guarantees and empirical gains on PES and...
-
Constraining the lives and times of exoplanets through evolutionary Bayesian retrievals
Evolutionary Bayesian retrievals using the PROTEUS model can infer initial volatile inventories and interior conditions of exoplanets from synthetic observables, with best success for terrestrial-mass planets.
-
Constrained Bayesian Optimisation with Multiple Information Sources
A multi-source extension of constrained Max-value Entropy Search for Bayesian optimization incorporates auxiliary data sources to improve early exploration and performance under constraints even with weak correlations.
-
Generative Refinement for Low-Budget Black-Box Optimization
SPARROW is a black-box optimization method that treats a fixed generative sampler as a structured proposal operator and applies rank-based selection over evaluated candidates to achieve low-budget optimization with as...
-
Joint Parameter and State-Space Bayesian Optimization: Using Process Expertise to Accelerate Manufacturing Optimization
POGPN-JPSS integrates partially observable Gaussian process networks with joint parameter and state-space modeling to leverage expert-derived low-dimensional features from high-dimensional intermediate observations fo...
-
Information-Based Exploration via Random Features for Reinforcement Learning
Random-feature Gaussian-process information gain is turned into a closed-form exploration bonus for PPO that matches RND/VIME/#Explo on 12 control, navigation, and sparse-locomotion tasks, with error bounds on the app...
-
Modernizing HEBO: a robust Bayesian optimization baseline for practical heteroskedastic and non-stationary problems
tidyHEBO modernizes HEBO in BoTorch and shows competitive-to-better, more robust sequential optimization on scientific and HPO benchmarks under default hyperparameters.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.