REVIEW 4 cited by
Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
This paper presents the results and insights from the black-box optimization (BBO) challenge at NeurIPS 2020 which ran from July-October, 2020. The challenge emphasized the importance of evaluating derivative-free optimizers for tuning the hyperparameters of machine learning models. This was the first black-box optimization challenge with a machine learning emphasis. It was based on tuning (validation set) performance of standard machine learning models on real datasets. This competition has widespread impact as black-box optimization (e.g., Bayesian optimization) is relevant for hyperparameter tuning in almost every machine learning project as well as many applications outside of machine learning. The final leaderboard was determined using the optimization performance on held-out (hidden) objective functions, where the optimizers ran without human intervention. Baselines were set using the default settings of several open-source black-box optimization packages as well as random search.
Forward citations
Cited by 4 Pith papers
-
Are encoders able to learn landmarkers for warm-starting of Hyperparameter Optimization?
Encoders trained to reproduce landmarker distances capture that structure, but the resulting representations do not significantly improve warm-starting of Bayesian hyperparameter optimization.
-
Predicting Extubation Failure in Intensive Care: The Development of a Novel, End-to-End Actionable and Interpretable Prediction System
A fusion of LSTM and TCN models built from sampling-frequency feature subsets achieved only modest extubation-failure prediction (AUC-ROC about 0.6) on 4,701 MIMIC-IV patients.
-
Modernizing HEBO: a robust Bayesian optimization baseline for practical heteroskedastic and non-stationary problems
tidyHEBO modernizes HEBO in BoTorch and shows competitive-to-better, more robust sequential optimization on scientific and HPO benchmarks under default hyperparameters.
-
Machine Learning for Complex Instrument Design and Optimization
A review chapter summarizing machine-learning pipelines for complex instrument operations and design, with no new experimental or theoretical contribution.
Discussion (0). Continue with ORCID to comment.