Using Bayesian optimization over Gaussian-process surrogates, data mixtures for LLM training can be found much faster than with linear or exponential regression baselines, including across model sizes.
Combining hyperband and bayesian optimization
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
stat.ML 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
ADMIRE-BayesOpt: Accelerated Data MIxture RE-weighting for Language Models with Bayesian Optimization
Using Bayesian optimization over Gaussian-process surrogates, data mixtures for LLM training can be found much faster than with linear or exponential regression baselines, including across model sizes.