Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Injecting user priors mid-run speeds up hyperparameter search without sacrificing convergence.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:07 UTC pith:IYRVC6KE

load-bearing objection DynaBO is a sensible dynamic-prior BO method with an honest-but-conditional acceleration theorem, and the abstract overclaims what the experiments and theory support. the 3 major comments →

arxiv 2511.02570 v3 pith:IYRVC6KE submitted 2025-11-04 cs.LG

Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

Lukas Fehring , Marcel Wever , Maximilian Splieth\"over , Leona Hennig , Henning Wachsmuth , Marius Lindauer This is my paper
classification cs.LG
keywords Bayesian optimizationhyperparameter optimizationuser priorsinteractive optimizationacquisition functionsconvergence guaranteesprior rejectionhuman-in-the-loop
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces DynaBO, a Bayesian optimization procedure whose acquisition function is multiplied by a sum of user-provided prior distributions that decay individually with age, so a practitioner can add beliefs about where the optimum lies while a run is in progress. The authors' central claim is that this does not cost convergence: under mild assumptions, the sequence of queried points still reaches a global optimum with probability one, misleading priors cause only finitely much extra exploration, and informative priors can lower the regret bound to depend only on information gain near the optimum. They add a surrogate-based safeguard that compares the expected lower confidence bound inside the prior's suggested region with that near the current incumbent and rejects priors below a threshold. A sympathetic reader would care because it targets a real bottleneck — experts distrust black-box tuning and want to steer it — and it promises to make BO steerable at essentially no asymptotic risk. The main prize, if correct, is a practical workflow where humans and optimizers collaborate over time rather than at initialization only.

Core claim

Formally, DynaBO replaces the standard acquisition function α with α_dyna(λ)=α(λ) · Σ_m π_m(λ)^{β/(t−t_m)}, where each prior π_m is a distribution over configurations supplied at time t_m and fades to a constant 1 as t grows; summing keeps earlier hints active while letting their influence decay individually. The paper shows almost-sure convergence to a global optimum, proves that the asymptotic simple regret equals that of unguided GP-UCB regardless of prior quality, and derives an O(√(T β_T γ_T(U_ϵ))) + δ T B regret bound when a prior concentrates mass in an ε-neighborhood of the optimum. A rejection mechanism estimates whether a suggested region beats the incumbent region, using LCB value

What carries the argument

The load-bearing object is the dynamically weighted acquisition function, formed by multiplying the base acquisition function by a sum of time-decaying priors: α_dyna(λ)=α(λ)·Σ_m π_m(λ)^{β/(t−t_m)}. Each prior contributes as a multiplicative preference concentrated on a user-specified region, and its exponent β/(t−t_m) makes its influence vanish as the run continues. A second mechanism, the prior safeguard, estimates the expected lower confidence bound of configurations drawn from the prior versus those drawn around the current incumbent, and rejects any prior whose advantage falls below a threshold τ. Together they carry the argument: the decay term is what preserves vanilla convergence, th

Load-bearing premise

The acceleration guarantee rests entirely on an unproven assumption stated in the appendix: that when a prior concentrates most of its mass in a neighborhood of the optimum, DynaBO actually samples there at least 1−δ of the time; if a surrogate misestimates the region, the promised speedup does not follow.

What would settle it

Take a low-dimensional synthetic GP with a known optimum, place a highly informative prior tightly around the optimum, run DynaBO for a finite horizon, and count how often the selected points fall inside that neighborhood. If the empirical frequency is well below 1−δ while the prior's mass in the neighborhood is at least 1−δ, the sampling-concentration premise behind the acceleration theorem fails; likewise, if informative priors produced no faster regret decay than unguided GP-UCB on such functions, the main acceleration claim would be contradicted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A user can add or correct a search hint mid-run, and the procedure will eventually recover the behavior of unguided Bayesian optimization once all priors have decayed.
  • A wrong prior does not permanently derail the run: its damage is confined to finitely many iterations before the asymptotic regret matches that of the base algorithm.
  • An informative prior that concentrates near the optimum yields a tighter regret bound in which the maximum information gain is computed only inside the neighborhood of the optimum, a strictly smaller quantity than the global information gain.
  • The rejection threshold τ gives an operator a single knob to trade willingness to follow user advice against protection from bad advice.
  • The construction is agnostic to the base acquisition function, so the same dynamic-prior mechanism can be layered onto EI, UCB, or other acquisition rules used in practice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The main open step, which the paper leaves as an assumption rather than a theorem, is bridging from prior mass concentration to sampling concentration; if a proof of that link existed, the acceleration guarantee would become unconditional and would likely transfer to related prior-steered algorithms.
  • Because the safeguard is a surrogate-based estimate, its reliability inherits the surrogate's calibration; an adaptive threshold that tracks the empirical acceptance-rejection outcomes could reduce sensitivity to τ in future iterations.
  • A natural testable extension is multi-fidelity settings: since priors decay with iteration count rather than wall-clock cost, the same mechanism could guide cheap and expensive evaluations, and the safeguard's comparison could be re-run at each fidelity.
  • The dynamic-prior mechanism could also serve as a generic interface for other sources of knowledge, such as automated suggestions or earlier runs on related tasks, since any distribution over configurations can be injected at any time.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DynaBO, an extension of Bayesian optimization (BO) for hyperparameter optimization (HPO) that allows users to supply multiple priors at runtime, rather than only at initialization. The acquisition function is multiplied by a sum of time-decaying prior densities, and a surrogate-based safeguard can detect and reject misleading priors. The authors prove almost-sure convergence and robustness to misleading priors, and claim an acceleration bound when priors are informative. They evaluate against πBO, vanilla BO, and probabilistic-circuit baselines on PD1, lcbench, and xgboost benchmarks using synthetic expert, advanced, local, and adversarial priors.

Significance. If the claims are established, DynaBO addresses a real gap: most prior-guided BO methods accept user knowledge only at initialization, while practitioners often want to steer optimization online. The dynamic stacking of decaying priors is a simple, acquisition-function-agnostic mechanism, and the safeguard is practically motivated. Strengths include a standard-extension convergence/robustness argument, a broad empirical study across several benchmarks and surrogate types, and public code links. However, the acceleration theorem is conditional on an unproven sampling-concentration premise, and the empirical conclusions in the abstract exceed what Section 6.4 reports. The method itself appears plausible and the experiments are useful, but the paper's advertised guarantees need to be re-aligned with what is actually proven.

major comments (3)
  1. [Appendix A.3 / Theorem 5.3] The acceleration theorem assumes the very effect it claims to establish. Equation (10) defines prior mass concentration qt(U_epsilon(lambda*)) >= 1 - delta, and the text then states: 'We assume that this prior mass concentration induces a corresponding sampling concentration of DynaBO: P(lambda_t in U_epsilon(lambda*)) >= 1 - delta.' This sampling-concentration assumption is the engine of the proof: it bounds E[|I_out|] <= delta T and restricts the information-gain term to gamma_T(U_epsilon). It is never derived from the prior, the acquisition function, or the safeguard. A point outside U_epsilon can maximize alpha_dyna even when the prior places almost all mass inside U_epsilon, because GP-UCB's alpha is not bounded relative to pi; a point with high posterior mean or high uncertainty can dominate the product. Consequently, bound (11) is a conditional statement about the algorithm's beha
  2. [Section 6.4 vs. Abstract] The abstract claims DynaBO 'consistently outperforms state-of-the-art competitors across all benchmarks and for all prior kinds.' The body reports several exceptions: on widernet and xformer, 'DynaBO is predominated by piBO until the second prior is provided'; on resnet, local priors reduce performance for both piBO and DynaBO; and adversarial priors generally cause performance degradation for both methods, with the rejection mechanism giving a significant boost on all scenarios except transformer lm1b. Thus the evidence in Figure 5 does not support 'consistently outperforms ... across all benchmarks and for all prior kinds' as stated. Please temper the abstract and conclusion to what the results actually show, e.g., 'outperforms competitors on most benchmarks and in aggregate,' and consider adding significance tests or error bars in a way that supports the chosen wording.
  3. [Section 5 / Section 6.2] The theoretical guarantees are explicitly stated only for UCB as the acquisition function ('Note that we assume a finite prior set, and the utilization of UCB as an acquisition function'), while the experimental evaluation uses EI ('Although our theoretical analysis focuses on LCB, we conduct our experimental evaluation using the common EI acquisition function'). This disconnect means that the theoretical results do not directly apply to the empirical implementation that substantiates the main claims. If the intended contribution is that DynaBO works with any acquisition function, the proofs need to be extended or a transfer argument supplied; otherwise, the paper should state more prominently that the guarantees are for UCB only and that the EI experiments are heuristic evidence.
minor comments (4)
  1. [Section 4.1] The phrase 'multiplying the sum of the priors' is ambiguous; the formula alpha_dyna = alpha * sum_m pi^(m)^(beta/(t-t^(m))) is a product of the acquisition function with a sum, not a product of sums. Please clarify the wording.
  2. [Section 4.2 / Appendix B.3 / C.4] Equation (3) uses an acquisition function xi, Appendix B.3 reintroduces the criterion with alpha_hatf, and Appendix C.4 implements it with LCB. The notation is inconsistent; unify the symbol for the rejection acquisition function.
  3. [Figures] Several figure legends contain typos: duplicated 'BO' in 'Vanilla BO BO' (Figures 5, 6), 'Advaserial' instead of 'Adversarial' (Figures 10-13), and axis labels such as '=-1' that omit the tau symbol (Figure 7). Please correct.
  4. [Appendix C.0] A few typos in the appendix: 'implemenmtaion' (Appendix C), 'configuraiton' (Appendix B.1/B.2). A light proofread would improve clarity.

Circularity Check

1 steps flagged

Theorem 5.3's acceleration guarantee assumes the sampling concentration it is meant to derive; the a.s.-convergence and robustness results are independent.

specific steps
  1. other [Theorem 5.3 (Section 5) and Appendix A.3, Eqs. (10)-(16)]
    "Suppose there exists a prior π(m) such that the probability that λt is in a neighborhood Uϵ(λ∗) ... satisfies P(λt ∈U ϵ(λ∗))≥1−δ ... Then, the expected cumulative regret E[RT ] ... satisfies E[RT ] =O(p T βU CBT γT (Uϵ)) +δT B. ... We assume that this prior mass concentration induces a corresponding sampling concentration of DynaBO: P(λt ∈U ϵ(λ∗))≥1−δ,∀t∈ {1, . . . , T}."

    The theorem's hypothesis is already the desired concentration of DynaBO's queries, not a consequence of informative priors. Eq. (10) defines prior mass concentration qt(Uϵ)≥1−δ, but the proof then simply assumes it 'induces' P(λt∈Uϵ)≥1−δ, and uses that assumption to bound E[|Iout|]≤δT and restrict the information gain to γT(Uϵ). No argument connects prior mass concentration to the maximizer of αdyna(λ,t)=α(λ,t)·Σ_m π^(m)(λ)^{β/(t−t^(m))}; a point outside Uϵ with high UCB value can dominate the product. Thus the advertised O(√(T β_T^UCB γ_T(Uϵ)))+δTB bound is the conclusion inserted as a premise, and the abstract's claim of proving 'accelerated convergence when informative priors are provided' is not supported by the derivation.

full rationale

Theorems 5.1 and Corollary 5.2 are not circular: given Assumptions A1-A3, the prior weighting converges uniformly to M, so DynaBO's chosen points are asymptotically exact maximizers of GP-UCB, and the standard Srinivas et al. regret analysis applies; misleading priors cause only finite additional exploration. The self-citations to πBO/Srinivas et al. are background and do not load-bear in a way that makes the derivation equivalent to its inputs. The circular/conditional content is localized in Theorem 5.3/Appendix A.3. The acceleration theorem is a valid conditional statement if one grants sampling concentration, but it does not derive that concentration from prior mass; it assumes it. Because this assumed concentration is precisely the speedup phenomenon advertised in the abstract, the theoretical support for the acceleration claim is partially question-begging. I do not count the Section 6.5 choice of τ=−0.15 on the same PD1 benchmarks as circularity: it is a benchmark-tuning/overfitting concern about external validity, not a reduction of a prediction to a fitted input. Overall, the central convergence and robustness contributions remain independent, so a moderate score of 4 is appropriate.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method is mostly a design extension of πBO. The main added epistemic burden is the unproven concentration premise behind Theorem 5.3, plus hand-set constants β, τ, and the candidate-sampling decay rate. No new physical or mathematical entities are postulated.

free parameters (4)
  • β (prior decay scaling) = N/10 (N = number of trials)
    Controls how quickly prior influence decays in π^(β/(t−t_m)); inherited from the πBO heuristic and affects all theoretical and empirical results.
  • τ (prior rejection threshold) = -0.15 in main experiments
    Selected after a sensitivity analysis on the same PD1 benchmarks used for the main results (§6.5); governs the accept/reject tradeoff of the safeguard.
  • φ (candidate-sampling decay constant) = -0.126 (Appendix B.2)
    Sets the weight ω_m = e^{φ(t−t_m)} for how many acquisition-optimizer candidates are drawn near each prior; hand-chosen and not justified by theory or data.
  • Decay shape = linear in t−t_m
    Appendix D.7 ablates logarithmic/quadratic/cubic and higher-power decays; linear is chosen as a tradeoff for the benchmark suite, not derived from first principles.
axioms (5)
  • domain assumption Objective f is a sample from a GP with bounded RKHS norm on compact Λ (Assumption A1).
    Standard GP-UCB regularity from Srinivas et al. 2012; used to bound the acquisition function and regret in Theorem 5.1.
  • domain assumption User priors π^(m) are finite, continuous, strictly positive on a compact domain (Assumption A2).
    Guarantees log-priors are bounded and the decay terms are well-defined; assumed for all theorems.
  • ad hoc to paper Prior influence vanishes asymptotically: π^(m)^(β/(t−t^(m))) → 1 (Assumption A3).
    This is enforced by the decay design rather than an external fact, but the paper lists it as an assumption for the convergence proof.
  • ad hoc to paper Prior mass concentration induces sampling concentration P(λ_t ∈ U_ϵ(λ*)) ≥ 1−δ (Theorem 5.3 / Appendix A.3).
    This is the load-bearing unproven premise of the acceleration theorem; it assumes the algorithm samples near the optimum instead of deriving it from the prior.
  • standard math GP-UCB approximate-maximizer regret bound of Srinivas et al. 2012.
    Used in the Theorem 5.1 proof to convert asymptotic exact maximization of α into vanishing average regret.

pith-pipeline@v1.3.0-alltime-deepseek · 27777 in / 15625 out tokens · 157138 ms · 2026-08-04T00:07:28.825824+00:00 · methodology

0 comments
read the original abstract

Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO). However, most existing HPO methods only incorporate expert knowledge during initialization, limiting practitioners' ability to influence the optimization process as new insights emerge. This limits the applicability of BO in iterative machine learning development workflows. We propose DynaBO, a BO framework that enables continuous user control of the optimization process. Over time, DynaBO leverages provided user priors by augmenting the acquisition function with decaying, prior-weighted preferences while preserving asymptotic convergence guarantees. To enhance robustness, we introduce a surrogate-model-based safeguard that detects and, possibly, rejects misleading priors. We prove theoretical results on near-certain convergence, robustness to deceptive priors, and accelerated convergence when informative priors are provided. Extensive experiments across various HPO benchmarks show that DynaBO consistently outperforms state-of-the-art competitors across all benchmarks and for all prior kinds. Our results demonstrate that DynaBO enables reliable and efficient collaborative BO, bridging automated and manually controlled model development.

Figures

Figures reproduced from arXiv: 2511.02570 by Henning Wachsmuth, Leona Hennig, Lukas Fehring, Marcel Wever, Marius Lindauer, Maximilian Splieth\"over.

Figure 1
Figure 1. Figure 1: Overview of the proposed dynamic Bayesian optimiza￾tion (DynaBO) method. Provided a dataset, a performance mea￾sure, a configuration space, and an optional initial prior, the loop iteratively selects new hyperparameter configurations. At each step, a candidate configuration is evaluated, and it is assessed. The process continues until an optimized configuration is identified. The framework allows users to … view at source ↗
Figure 2
Figure 2. Figure 2: Acquisition function impact of priors π 1 , π2 , π3 , pro￾vided at t = 10, 20, and 30, with π(λ) = 0.5. In constrast to the work of Hvarfner et al. (2022), we sup￾pose a finite sequence of user-specified priors {π (m)}M m=1 provided at times {t (m)}M m=1, t (1) < . . . < t(M) ≤ T. We define a dynamically-adapted acquisition function αdyna : Λ → R by multiplying the sum of the priors: α dyna fˆ (λ) := αfˆ(λ… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of DynaBO with PCs with vanilla BO as a reference. The shaded areas indicate standard error. The results are averaged over different PD1 scenarios. Informative priors include Expert, Advanced, and Local Priors. Expert Priors bias the optimization process toward clus￾ters spanning significantly-better regions of the configura￾tion space, that is, ℓλˆ ≥ ℓci . A cluster c + is sampled with probabil… view at source ↗
Figure 5
Figure 5. Figure 5: Mean regret for lcbench, xgboost, and PD1 using Expert, Advanced, Local, and Adversarial priors. Priors are provided at vertical lines. The shaded areas visualize the standard error. For lcbench and xgboost, the plots average all datasets. The results indicate DynaBO outperforming πBO and remaining competitive to vanilla BO for adversarial priors. 6.4. Comparison to Vanilla BO and πBO In [PITH_FULL_IMAGE:… view at source ↗
Figure 6
Figure 6. Figure 6: Anytime regret for PD1 averaged over 30 seeds, and scenarios comparing vanilla BO, πBO, DynaBO-accept all priors, and DynaBO with validation (DynaBO-validation). Expert Advanced Local Adversarial Prior Kind 0.00 0.05 0.10 0.15 Final Regret =-1 =-0.5 =-0.25 =-0.2 =-0.15 =-0.1 =-0.05 =0 =0.25 =0.5 Expert Advanced Local Adversarial Prior Kind 0.00 0.05 0.10 0.15 Final Regret =-1 =-0.5 =-0.25 =-0.2 =-0.15 =-0.… view at source ↗
Figure 7
Figure 7. Figure 7: Sensitivity analysis of different thresholds τ . Setting τ = −∞ accepts all, while setting τ = ∞ rejects all priors. The box plots contain the merged results from all scenarios. 6.5. Sensitivity Analysis of the Prior Rejection Criterion Our prior rejection scheme utilizes a threshold τ , encoding the minimum estimated average improvement over the cur￾rent incumbent needed to accept a prior. This improvemen… view at source ↗
Figure 8
Figure 8. Figure 8: Evaluation results across scenarios (rows) for two surrogate models (columns). 22 [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Sensitivity analysis of different thresholds τ . τ = −∞ accepts all, and τ = ∞ rejects all priors. overall contains the merged results from all scenarios. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of vanilla-BO, DynaBO, and different variants of probabilistic circuits on PD1 using Expert, Advanced, Local, and Adversarial priors. The plots show the mean regret over time with standard errors visualized as shaded areas. Priors are provided at the dashed vertical lines. Discussion of the Overall Results Our results show that both DynaBO and vanilla-BO outperform PCs. The difference becomes e… view at source ↗
Figure 11
Figure 11. Figure 11: Mean regret for and PD1 using Expert, Advanced, Local, and Adversarial priors, with Gaussian processes as surrogate models. Priors are provided at vertical lines. The shaded areas visualize the standard error. The results indicate DynaBO outperforming πBO and remaining competitive with vanilla BO for adversarial priors. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Mean regret for PD1 using Expert, Advanced, Local, and Adversarial priors, with random forests as surrogate models. Priors are provided at vertical lines. The shaded areas visualize the standard error. The results indicate DynaBO outperforming πBO and remaining competitive with vanilla BO for adversarial priors. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Mean regret for PD1 using Expert, Advanced, Local, and Adversarial priors, with random forests as surrogate models. The shaded areas visualize the standard error. The results indicate DynaBO outperforming πBO and remaining competitive with vanilla BO for adversarial priors. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Mean regret for PD1 using Expert, Advanced, Local, and Adversarial priors, with random forests as surrogate models. The shaded areas visualize the standard error. The results indicate DynaBO outperforming πBO and remaining competitive with vanilla BO for adversarial priors. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Investigation of helpful priors followed by adversarial priors. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch

    cs.LG 2026-07 conditional novelty 6.0

    An LLM agent that fully controls a reconfigurable Bayesian-optimization backend preserves standard BO reliability, outperforms LLM-only optimizers, and exploits natural-language priors and mid-run problem reformulation.

Reference graph

Works this paper leans on

68 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Adachi, B

    M. Adachi, B. Planden, D. A. Howey, M. A. Osborne, S. Orbell, N. Ares, K. Muandet, and S. L. Chau. Looping in the human: Collaborative and explainable bayesian optimization. In S. Dasgupta, S. Mandt, and Y. Li (eds.), Proceedings of the 27th International Conference on Artificial Intelligence and Statistics ( AISTATS '24) . Proceedings of Machine Learning...

    2024

  2. [2]

    R. Agrawal. Sample mean based index policies by o (log n) regret for the multi-armed bandit problem. Advances in Applied Probability, 1995

    1995

  3. [3]

    Bansal, D

    A. Bansal, D. Stoll, M. Janowski, A. Zela, and F. Hutter. JAHS -bench-201: A foundation for research on joint architecture and hyperparameter search. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Proceedings of the 36th International Conference on Advances in Neural Information Processing Systems ( N eur IPS '22) . Curran As...

    2022

  4. [4]

    Bergstra and Y

    J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13: 0 281--305, 2012

    2012

  5. [5]

    Bergstra, R

    J. Bergstra, R. Bardenet, Y. Bengio, and B. K \'e gl. Algorithms for hyper-parameter optimization. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Weinberger (eds.), Proceedings of the 25th International Conference on Advances in Neural Information Processing Systems ( N eur IPS '11) , pp.\ 2546--2554. Curran Associates, 2011

    2011

  6. [6]

    Bischl, M

    B. Bischl, M. Binder, M. Lang, T. Pielok, J. Richter, S. Coors, J. Thomas, T. Ullmann, M. Becker, A. - L. Boulesteix, D. Deng, and M. Lindauer. Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, pp.\ e1484, 2023

    2023

  7. [7]

    Bojar, R

    O. Bojar, R. Chatterjee, C. Federmann, B. Haddow, M. Huck, C. Hokamp, P. Koehn, V. Logacheva, C. Monz, M. Negri, M. Post, C. Scarton, L. Specia, and M. Turchi". Findings of the 2015 workshop on statistical machine translation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, 2015

    2015

  8. [8]

    Bouthillier and G

    X. Bouthillier and G. Varoquaux. Survey of machine-learning experimental methods at NeurIPS2019 and ICLR2020 . Research report [hal-02447823], Inria Saclay Ile de France , 2020

    2020

  9. [9]

    Chang, M

    C. Chang, M. Azvar, C. Okwudire, and R. A. Kontar. Llinbo: Trustworthy llm-in-the-loop bayesian optimization. arXiv:2505.14756 [cs.LG], 2025

    arXiv 2025

  10. [10]

    Chen and C

    T. Chen and C. Guestrin. XGBoost : A scalable tree boosting system. In B. Krishnapuram, M. Shah, A. Smola, C. Aggarwal, D. Shen, and R. Rastogi (eds.), Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining ( KDD '16) , pp.\ 785--794. ACM Press, 2016

    2016

  11. [11]

    Cowen-Rivers, W

    A. Cowen-Rivers, W. Lyu, R. Tutunov, Z. Wang, A. Grosnit, R. Griffiths, A. Maraval, H. Jianye, J. Wang, J. Peters, and H. Ammar. HEBO : Pushing the limits of sample-efficient hyper-parameter optimisation. Journal of Artificial Intelligence Research, 74: 0 1269--1349, 2022

    2022

  12. [12]

    J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database . In Proceedings of the International Conference on Computer Vision and Pattern Recognition ( CVPR '09) , pp.\ 248--255. ieeecs, IEEE, 2009

    2009

  13. [13]

    Eggensperger, P

    K. Eggensperger, P. M \"u ller, N. Mallik, M. Feurer, R. Sass, A. Klein, N. Awad, M. Lindauer, and F. Hutter. HPOBench : A collection of reproducible multi-fidelity benchmark problems for HPO . In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. Curran Associates, 2021

    2021

  14. [14]

    Falkner, A

    S. Falkner, A. Klein, and F. Hutter. BOHB : Robust and efficient H yperparameter O ptimization at scale. In J. Dy and A. Krause (eds.), Proceedings of the 35th International Conference on Machine Learning ( ICML '18) , volume 80, pp.\ 1437--1446. Proceedings of Machine Learning Research, 2018

    2018

  15. [15]

    Feurer, J

    M. Feurer, J. Springenberg, and F. Hutter. Initializing B ayesian H yperparameter O ptimization via meta-learning. In B. Bonet and S. Koenig (eds.), Proceedings of the Twenty-ninth AAAI Conference on Artificial Intelligence ( AAAI '15) , pp.\ 1128--1135. AAAI Press, 2015

    2015

  16. [16]

    Feurer, B

    M. Feurer, B. Letham, and E. Bakshy. Scalable meta-learning for bayesian optimization using ranking-weighted gaussian process ensembles. In R. Garnett, F. Hutter, J. Vanschoren, P. Brazdil, R. Caruana, C. Giraud-Carrier, I. Guyon, and B. Kégl (eds.), ICML workshop on Automated Machine Learning ( Auto ML workshop 2018) , 2018

    2018

  17. [17]

    Feurer, B

    M. Feurer, B. Letham, F. Hutter, and E. Bakshy. Practical transfer learning for bayesian optimization. arXiv:1802.02219v4 [stat.ML] , 2022

    Pith/arXiv arXiv 2022

  18. [18]

    J. Gower. A general coefficient of similarity and some of its properties. Biometrics, pp.\ 857--871, 1971

    1971

  19. [19]

    Guyon, M

    I. Guyon, M. Lindauer, M. van der Schaar, F. Hutter, and R. Garnett (eds.). Proceedings of the First International Conference on Automated Machine Learning, 2022. Proceedings of Machine Learning Research

    2022

  20. [20]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the International Conference on Computer Vision and Pattern Recognition ( CVPR '16) , pp.\ 770--778. Computer Vision Foundation and IEEE Computer Society, IEEE, 2016

    2016

  21. [21]

    Hutter, H

    F. Hutter, H. Hoos, and K. Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In C. Coello (ed.), Proceedings of the Fifth International Conference on Learning and Intelligent Optimization ( LION '11) , volume 6683 of Lecture Notes in Computer Science, pp.\ 507--523. Springer, 2011

    2011

  22. [22]

    Hutter, H

    F. Hutter, H. Hoos, and K. Leyton-Brown. An efficient approach for assessing hyperparameter importance. In E. Xing and T. Jebara (eds.), Proceedings of the 31th International Conference on Machine Learning, ( ICML '14) , pp.\ 754--762. Omnipress, 2014

    2014

  23. [23]

    Hutter, L

    F. Hutter, L. Kotthoff, and J. Vanschoren (eds.). Automated Machine Learning: Methods, Systems, Challenges. Springer, 2019. Available for free at http://automl.org/book

    2019

  24. [24]

    Hvarfner, D

    C. Hvarfner, D. Stoll, A. Souza, L. Nardi, M. Lindauer, and F. Hutter. BO : A ugmenting A cquisition F unctions with U ser B eliefs for B ayesian O ptimization. In The Tenth International Conference on Learning Representations ( ICLR '22) . ICLR, 2022. Published online: iclr.cc

    2022

  25. [25]

    Hvarfner, F

    C. Hvarfner, F. Hutter, and L. Nardi. A general framework for user-guided bayesian optimization. In The Twelfth International Conference on Learning Representations ( ICLR '24) iclr24 . Published online: iclr.cc

  26. [26]

    Proceedings of the International Conference on Learning Representations ( ICLR '24) , 2024. ICLR. Published online: iclr.cc

    2024

  27. [27]

    Jones, M

    D. Jones, M. Schonlau, and W. Welch. Efficient global optimization of expensive black box functions. Journal of Global Optimization, 13: 0 455--492, 1998

    1998

  28. [28]

    Hasebrook, F

    N Kannengie er, N. Hasebrook, F. Morsbach, M. Z \"o ller, J. K. H. Franke, and M. Lindauer F. Hutter A. Sunyaev. Practitioner motives to use different hyperparameter optimization methods. ACM Transactions on Computer‐Human Interaction, 37 0 (4), 2025. doi:10.1145/3745771

    work page doi:10.1145/3745771 2025

  29. [29]

    Krizhevsky

    A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  30. [30]

    L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar. Hyperband: Bandit-based configuration evaluation for H yperparameter O ptimization. In The Fifth International Conference on Learning Representations ( ICLR '17) . ICLR, 2017. Published online: iclr.cc

    2017

  31. [31]

    Lindauer, K

    M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, D. Deng, C. Benjamins, T. Ruhkopf, R. Sass, and F. Hutter. SMAC3 : A versatile bayesian optimization package for H yperparameter O ptimization. Journal of Machine Learning Research, 23 0 (54): 0 1--9, 2022

    2022

  32. [32]

    Lindauer, F

    M. Lindauer, F. Karl, A. Klier, J. Moosbauer, A. Tornede, A. M \" u ller, F. Hutter, M. Feurer, and B. Bischl. Position: A call to action for a human-centered automl paradigm. In icml24

  33. [33]

    Makarova, H

    A. Makarova, H. Shen, V. Perrone, A. Klein, J.B. Faddoul, A. Krause, M. Seeger, and C. Archambeau. Automatic termination for hyperparameter optimization. In automlconf22

  34. [34]

    Mallik, C

    N. Mallik, C. Hvarfner, E. Bergman, D. Stoll, M. Janowski, M. Lindauer, L. Nardi, and F. Hutter. PriorBand : Practical hyperparameter optimization in the age of deep learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Proceedings of the 37th International Conference on Advances in Neural Information Processing Systems ...

    2023

  35. [35]

    Mo c kus

    J. Mo c kus. On bayesian methods for seeking the extremum. In G. Marchuk (ed.), Optimization Techniques IFIP Technical Conference 1974, pp.\ 400--404. Springer, Springer, 1975

    1974

  36. [36]

    Moosbauer, J

    J. Moosbauer, J. Herbinger, G. Casalicchio, M. Lindauer, and B. Bischl. Explaining hyperparameter optimization via partial dependence plots. In M. Ranzato, A. Beygelzimer, K. Nguyen, P. Liang, J. Vaughan, and Y. Dauphin (eds.), Proceedings of the 35th International Conference on Advances in Neural Information Processing Systems ( N eur IPS '21) , pp.\ 228...

    2021

  37. [37]

    M \"u ller, M

    S. M \"u ller, M. Feurer, N. Hollmann, and F. Hutter. PFNs4BO: In-Context Learning for Bayesian Optimization . In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning ( ICML '23) , volume 202 of Proceedings of Machine Learning Research. PMLR, 2023

    2023

  38. [38]

    R. Niwa, S. Yoshida, Y. Koyama, and Y. Ushiku. Cooperative design optimization through natural language interaction. arXiv:2508.16077 [cs.HC], 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Papenmeier, N

    L. Papenmeier, N. Cheng, S. Becker, and L. Nardi. Exploring exploration in bayesian optimization. arXiv:2502.08208 [cs.LG], 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Perrone, H

    V. Perrone, H. Shen, M. Seeger, C. Archambeau, and R. Jenatton. Learning search spaces for bayesian optimization: Another view of hyperparameter transfer learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alche Buc, E. Fox, and R. Garnett (eds.), Proceedings of the 33rd International Conference on Advances in Neural Information Processing System...

    2019

  41. [41]

    Pfisterer, L

    F. Pfisterer, L. Schneider, J. Moosbauer, M. Binder, and B. Bischl. YAHPO Gym -- an efficient multi-objective multi-fidelity benchmark for hyperparameter optimization. In automlconf22

  42. [42]

    Pineda Arango, F

    S. Pineda Arango, F. Ferreira, Kadra A., Hutter F., and Grabocka J. Quick-tune: Quickly learning which pretrained model to finetune and how. In The Twelfth International Conference on Learning Representations ( ICLR '24) iclr24 . Published online: iclr.cc

  43. [43]

    Rakotoarison, S

    H. Rakotoarison, S. Adriaensen, N. Mallik, S. Garibov, E. Bergman, and F. Hutter. In-context freeze-thaw bayesian optimization for hyperparameter optimization. In icml24

  44. [44]

    Ramachandran, S

    A. Ramachandran, S. Gupta, S. Rana, C. Li, and S. Venkatesh. Incorporating expert prior in Bayesian optimisation via space warping. Knowledge-Based Systems, 195, 2020

    2020

  45. [45]

    Salakhutdinov, Z

    R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp (eds.). Proceedings of the 41st International Conference on Machine Learning ( ICML '24) , volume 251 of Proceedings of Machine Learning Research, 2024. PMLR

    2024

  46. [46]

    R. Sass, E. Bergman, A. Biedenkapp, F. Hutter, and M. Lindauer. Deepcave: An interactive analysis tool for automated machine learning. In M. Mutny, I. Bogunovic, W. Neiswanger, S. Ermon, Y. Yue, and A. Krause (eds.), ICML Adaptive Experimental Design and Active Learning in the Real World (ReALML Workshop 2022) , 2022

    2022

  47. [47]

    Segel, H

    S. Segel, H. Graf, A. Tornede, B. Bischl, and M. Lindauer. Symbolic explanations for hyperparameter optimization. In A. Faust, C. White, F. Hutter, R. Garnett, and J. Gardner (eds.), Proceedings of the Second International Conference on Automated Machine Learning. Proceedings of Machine Learning Research, 2023

    2023

  48. [48]

    J. Seng, F. Ventola, Z. Yu, and K. Kersting. Hyperparameter optimization via interacting with probabilistic circuits. In R. Garnett, C. Doerr, J. van Rijn, and L. Akoglu (eds.), Proceedings of the Third International Conference on Automated Machine Learning. Proceedings of Machine Learning Research, 2025

    2025

  49. [49]

    Shahriari, K

    B. Shahriari, K. Swersky, Z. Wang, R. Adams, and N. de Freitas. Taking the human out of the loop: A review of B ayesian optimization. Proceedings of the IEEE , 104 0 (1): 0 148--175, 2016

    2016

  50. [50]

    Snoek, H

    J. Snoek, H. Larochelle, and R. Adams. Practical B ayesian optimization of machine learning algorithms. In P. Bartlett, F. Pereira, C. Burges, L. Bottou, and K. Weinberger (eds.), Proceedings of the 26th International Conference on Advances in Neural Information Processing Systems ( N eur IPS '12) , pp.\ 2960--2968. Curran Associates, 2012

    2012

  51. [51]

    Souza, L

    A. Souza, L. Nardi, L. Oliveira, K. Olukotun, M. Lindauer, and F. Hutter. Bayesian optimization with a prior for the optimum. In N. Oliver, F. Pérez-Cruz, S. Kramer, J. Read, and J. A. Lozano (eds.), Machine Learning and Knowledge Discovery in Databases. Research Track, volume 12975 of Lecture Notes in Artificial Intelligence, pp.\ 265–296. Springer-Verla...

    2021

  52. [52]

    Souza, L

    A. Souza, L. Nardi, L. Oliveira, K. Olukotun, M. Lindauer, and F. Hutter. Bayesian optimization with a prior for the optimum. In N. Oliver, F. Pérez-Cruz, S. Kramer, J. Read, and J. Lozano (eds.), Machine Learning and Knowledge Discovery in Databases ( ECML / PKDD '21) , volume 12975 of Lecture Notes in Computer Science, pp.\ 265--296. Springer, 2021 b

    2021

  53. [53]

    Srinivas, A

    N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Trans. Inf. Theory , 58 0 (5): 0 3250--3265, 2012

    2012

  54. [54]

    Studer, T

    S. Studer, T. Binh Bui, C. Drescher, A. Hanuschkin, L. Winkler, S. Peters, and K. M \" u ller. Towards CRISP-ML(Q): A machine learning process model with quality assurance methodology. Mach. Learn. Knowl. Extr., 2021

    2021

  55. [55]

    Swersky, J

    K. Swersky, J. Snoek, and R. Adams. Multi-task Bayesian optimization. In C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger (eds.), Proceedings of the 27th International Conference on Advances in Neural Information Processing Systems ( N eur IPS '13) , pp.\ 2004--2012. Curran Associates, 2013

    2004

  56. [56]

    Tornede, A

    T. Tornede, A. Tornede, J. Hanselle, F. Mohr, M. Wever, and E. H \" u llermeier. Towards green automated machine learning: Status quo and future directions. Journal of Artificial Intelligence Research, 77: 0 427--457, 2023

    2023

  57. [57]

    Turner, D

    R. Turner, D. Eriksson, M. McCourt, J. Kiili, E. Laaksonen, Z. Xu, and I. Guyon. Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the Black-Box Optimization Challenge 2020 . In H. Escalante and K. Hofmann (eds.), Proceedings of the Neural Information Processing Systems Track Competition and Demonst...

    2020

  58. [58]

    Van der Blom, A

    K. Van der Blom, A. Serban, H. Hoos, and J. Visser. Automl adoption in ml software. In 8th ICML Workshop on Automated Machine Learning (AutoML), 2021

    2021

  59. [59]

    van Rijn and F

    J. van Rijn and F. Hutter. Hyperparameter importance across datasets. In Y. Guo and F. Farooq (eds.), Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining ( KDD '18) , pp.\ 2367--2376. ACM Press, 2018

    2018

  60. [60]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Proceedings of the 31st International Conference on Advances in Neural Information Processing Systems ( N eur IPS '17) . Curran ...

    2017

  61. [61]

    Q. Wang, Y. Ming, Z. Jin, Q. Shen, D. Liu, M. Smith, K. Veeramachaneni, and H. Qu. Atmseer: Increasing transparency and controllability in automated machine learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI'19) , pp.\ 1–12. ACM Press, 2019

    2019

  62. [62]

    Z. Wang, G. Dahl, K. Swersky, C. Lee, Z. Mariet, Z. Nado, J. Gilmer, J. Snoek, and Z. Ghahramani. Pre-trained G aussian processes for B ayesian optimization. J. Mach. Learn. Res., 2024

    2024

  63. [63]

    W. Xu, M. Adachi, C. N. Jones, and M. A. Osborne. Principled bayesian optimisation in collaboration with human experts. arXiv:2410.10452 [cs.LG], 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Z \" o ller, W

    M. Z \" o ller, W. Titov, T. Schlegel, and M. Huber. Xautoml: A visual analytics tool for understanding and validating automated machine learning. ACM Trans. Interact. Intell. Syst. , 13 0 (4): 0 28:1--28:39, 2023

    2023

  65. [65]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  66. [66]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  67. [67]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  68. [68]

    Journal of Data Semantics

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    2004