Pith. sign in

REVIEW 4 major objections 5 minor 59 references

Direct Regret Optimization in Bayesian Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DRO trains a transformer on simulated GP rollouts to directly minimize final simple regret in Bayesian optimization.

desk verdict A novel combination of GP-ensemble rollouts and decision transformers for BO, but the central claim of direct regret minimization is not yet supported by the evidence. read the letter →

arxiv 2507.06529 v1 pith:ZIHAMA5S submitted 2025-07-09 cs.LG

Fengxue Zhang , Yuxin Chen This is my paper · ORCID
classification cs.LG
keywords Bayesianoptimizationdecisiontransformersimpleregretnon-myopicacquisitionGaussianprocessensembleregionofinterestofflinereinforcementlearningdensetrainingsparse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that Bayesian optimization can be built around a learned query policy instead of hand-crafted acquisition functions. It proposes Direct Regret Optimization (DRO), which trains a decision transformer offline on many simulated BO runs generated by an ensemble of Gaussian processes, restricted to adaptive regions of interest and stopped early when expected improvement is small. The transformer is trained to output the next query point that minimizes the final simple regret after a multi-step horizon, and at deployment it is conditioned on a target return-to-go of 1.0. Results on the Ackley function up to 20 dimensions, three hyperparameter-tuning tasks, and a LunarLander controller are reported to show that DRO reaches lower simple regret than standard GP-BO, TuRBO, PFNs4BO, and SCoreBO, with ablations indicating ROI filtering and ensemble size drive the gain.

What carries the argument

The central object is the decision transformer, a sequence model that maps histories of (state, action, return-to-go) triples to the next action. State features include each ensemble GP's hyperparameters, the best value found, the iteration number, and coordinates of the best points; actions are the query points chosen by a conventional acquisition during simulation; the return-to-go is the simulated simple regret relative to that GP's estimated optimum. Simulations are generated by an ensemble of Gaussian processes with varied hyperparameters, constrained to the adaptive Region of Interest $\hat{X}_{m,t} = \{x : \mathrm{UCB}_{m,t}(x) \ge \max_{x'} \mathrm{LCB}_{m,t}(x')\}$, and truncated by a Bayesian early-stop rule based on maximum expected improvement falling below $\delta = 10^{-4}$. At inference, the transformer is conditioned on a target return of 1.0 (zero regret) and directly proposes the next real query.

What would settle it

Run DRO on a benchmark whose true optimum sits in a narrow attraction basin that RBF GPs with lengthscales in [0.1, 10] are unlikely to cover, e.g., a multimodal function with a very sharp global peak; if the policy's final simple regret is no better than a myopic EI baseline in that setting, the simulation-to-real transfer assumption is falsified. A second check is to condition the transformer on target returns other than 1.0 and see whether its regret tracks the target; a flat response would show the return signal is not doing the claimed work.

Watch

Extended reading notes

Core claim

DRO claims that a decision transformer can learn a non-myopic BO policy by distilling simulated trajectories rather than by optimizing a myopic acquisition at query time. The training signal is the return-to-go computed from each simulating GP's own estimate of the optimum; the policy is evaluated by the regret it achieves on the real objective. The paper asserts this policy consistently achieves lower final simple regret than the baselines and that the adaptive region-of-interest constraint and the diversity of the GP ensemble are the components that make the simulations informative. A second claimed component is the dense-training--sparse-learning split: abundant offline simulated rollouts train the policy, while the small number of real evaluations only updates the GPs and refines the policy online.

Load-bearing premise

The load-bearing premise is that training against approximate GP models, where success is measured by each model's own guess of the optimum, transfers to the real objective when the model is asked to produce a high target return. If the GP ensemble is systematically wrong about where the optimum is, or if the target return of 1.0 is not on the same scale as the real rewards, the learned policy could be optimizing a phantom objective.

Editorial extensions

If this is right

  • Users can drop the per-problem choice of acquisition function and its hyperparameters, since the transformer embodies the acquisition policy.
  • Constraining simulated rollouts to the adaptive ROI is necessary for the method's performance; removing it degrades results substantially.
  • The ensemble size matters: more GPs (M=5 or M=10) yield more diverse training trajectories and better final objective values than M=1.
  • The dense-training/sparse-learning split lets the policy be trained largely offline on cheap simulations, with only a few real evaluations used to correct the GP ensemble online.
  • The same simulated-trajectory distillation recipe transfers, in principle, to constrained BO and multi-fidelity BO by changing what the simulator models and what the action represents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper defers a full theoretical analysis; its stated regret guarantee covers an ROI-constrained base acquisition, not the learned transformer policy itself, so the empirical transfer result is the load-bearing evidence.
  • A natural extension the authors do not test is varying the target return-to-go at inference to trace an exploration-exploitation frontier; if regret does not respond to the target, the return conditioning is not the active mechanism.
  • If the approach generalizes, the same distillation recipe could be applied with a high-fidelity simulator in place of GP rollouts, which would remove the GP-ensemble misspecification concern in domains where simulators exist.
  • The paper's robustness argument is most exposed when the GP ensemble's kernel family is wrong in a structured way, so a deliberate misspecification experiment (e.g., periodic or discontinuous objectives) would sharpen the claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Direct Regret Optimization (DRO), a Bayesian optimization method that trains a decision transformer offline on simulated rollouts from an ensemble of Gaussian processes (GPs) with varying hyperparameters. Simulated trajectories are restricted to an adaptive Region of Interest (ROI), generated by rotating conventional acquisition functions (EI, UCB, PI, MES), and terminated by a Bayesian early stopping criterion. At inference, the transformer proposes the next query conditioned on a target return-to-go, and real evaluations update the GP ensemble. The authors claim that DRO jointly learns the surrogate and a non-myopic acquisition policy and explicitly minimizes final simple regret, supporting this with experiments on Ackley functions (2D–20D), three hyperparameter tuning tasks, LunarLander, and ablations on ROI filtering and ensemble size.

Significance. The intended contribution is timely: replacing hand-crafted myopic acquisition functions with an end-to-end learned, return-conditioned policy would address a recognized weakness of standard BO, and the concrete design of ensemble-based ROI-constrained rollouts plus dense offline training is a plausible recipe. The paper has clear strengths: it evaluates on external benchmarks, includes ablations for ROI filtering and ensemble size, and describes the simulated-data pipeline in enough detail to be reproduced with effort. If the central mechanism were verified, DRO would be a meaningful practical contribution. However, the paper currently overstates its central claim: no multi-step planning step is present in the training pipeline, the return-to-go target is not calibrated to real objective scale, and the empirical evidence consists of shaded mean curves without significance tests or numerical tables. These gaps are load-bearing for the abstract's claim that DRO 'explicitly targets minimizing the multi-step regret.'

major comments (4)
  1. [§4.5, Algorithm 1, Appendix D.4] The central claim that DRO learns a non-myopic policy that directly minimizes final simple regret is not supported by the described training procedure. In every simulated rollout, each action is obtained by maximizing a one-step acquisition function (EI, UCB, PI, or MES) over the current GP posterior within the ROI, and the decision transformer is trained to predict those actions conditioned on a return-to-go. This is return-conditioned imitation of myopic demonstrators: there is no multi-step planner, no rollout-based policy improvement, and no return-maximization step in Algorithm 1. As described, the transformer's action distribution is restricted to the support of the myopic base acquisitions, so any claimed non-myopic advantage must be empirically isolated. Please add an ablation that varies or removes the return conditioning—for example, compare DRO against an unconditional imitation baseline, a version with shuffled return-to-go values, and a true multi-step lookahead policy—to demonstrate that the return signal is doing causal work.
  2. [Appendix D.1, §4.5] The return-to-go used for training is defined relative to an estimate ilde f^*_m described as 'an optimistic estimate of the true optimum from GP_m's perspective or a normalized target,' while inference conditions on a fixed target return-to-go of 1.0. The paper never specifies how ilde f^*_m is computed, how the return-to-go is normalized across GP models and objective scales, or how the value 1.0 corresponds to zero simple regret on the real objective. Since the real objective's scale is unknown and the ensemble GPs can disagree about the optimum, the training signal is not demonstrably aligned with the final simple regret that the paper claims to minimize. Please specify the exact return-to-go formula, the normalization procedure, and the calibration of the inference target, and ablate over target values.
  3. [§5.1, §6] The empirical claim of 'consistently outperforms BO baselines' rests on mean curves with ±1 standard error shading, but the paper provides no tables of final means and variances and no significance tests. In several panels, such as Adam Iris and Adam Breast Cancer in Figure 3, the shaded bands overlap substantially, so the qualitative claim is not supported for those tasks. Please report per-benchmark final simple-regret (or best-objective) means and standard errors, the number of trials, and pairwise significance tests across seeds (e.g., Wilcoxon signed-rank tests or bootstrap confidence intervals), and state explicitly which benchmarks support the 'consistent outperformance' claim.
  4. [§4.7] The paper explicitly states that 'full theoretical analysis of DRO is future work,' and the two propositions provided concern only ROI preservation for base acquisitions and convergence of expected improvement under converging simple regret. Neither proposition addresses the regret of the learned decision-transformer policy. In light of the abstract's claim that DRO 'explicitly targets minimizing the multi-step regret,' this is a gap between the stated objective and the provided support. Either derive a formal statement of the DRO training objective and a regret or consistency argument for the learned policy, or rephrase the abstract, §4.5, and §6.1 to present DRO as a heuristic distillation method rather than a directly regret-minimizing method.
minor comments (5)
  1. [Figure 3] The y-axis tick labels in Figure 3 appear garbled (e.g., '10 1', '2 × 10 1'); please render the exponent notation with standard formatted tick labels.
  2. [References] Reference [4] contains editorial annotations such as 'Often cited tutorial; relevant for discussing integrated acquisition functions'; such annotations should be removed from the reference list.
  3. [Algorithm 1 vs. Appendix D.4] Algorithm 1 refers to 'ROI parameters (e.g., β_t)', while Appendix D.4 states that the UCB/LCB constraint in the ROI definition uses κ = 6.0; please unify the notation and clarify which parameter controls the ROI.
  4. [§5.1] For the HPO tasks, the paper reports simple regret but does not state how the unknown global maximum f^* is estimated for XGBoost, Adam, or the other tasks; this affects the interpretation of the y-axis in Figure 3.
  5. [Algorithm 1 and §5.1] Algorithm 1 retrains or fine-tunes the decision transformer at every real iteration, but the paper does not report the wall-clock or computational cost of this training relative to the cost of real evaluations; please add this information, since BO is motivated by expensive function evaluations.

Circularity Check

2 steps flagged · score 3.0 of 10

Training regret is defined against each GP's own optimistic optimum, and the ROI-regret justification imports a self-cited lemma; empirical benchmark comparisons remain external.

  1. self definitional [Appendix D.1 (Decision Transformer), Return-to-Go Calculation; see also Section 4.5 and Algorithm 1 step 13]
    "For a simulated trajectory of length Lm,k ending with best point x(m,k) best sim,Lm,k, the return-to-go for step τ could be f(x(m,k) best sim,Lm,k ) − f˜∗ m, where f˜∗ m is an optimistic estimate of the true optimum from GPm’s perspective or a normalized target."

    The signal that the paper calls 'simulated simple regret' is computed relative to f˜∗ m, an estimate produced by the same GPm that generated the rollout, not relative to the true f∗ defined in Section 3. The decision transformer is therefore trained to reduce the gap to each GP's own optimistic optimum, so the claim that DRO 'directly minimizes final simple regret' is true only for this self-referential, model-defined quantity. The inference-time target return of 1.0 (D.1) is not calibrated to the real f∗, so the link from the learned return-to-go to actual simple regret is assumed rather than derived.

  2. self citation load bearing [Appendix B, Proposition 1 (Regret Guarantee with ROI-Constrained Base Acquisition)]
    "such that the true optimum x∗ is contained in Xˆt with high probability, i.e., P[x∗ ∈ Xˆt] ≥ 1 − δt for some small δt > 0 (akin to Lemma 1 in Zhang et al. (2024) [46])."

    The regret-preservation argument for ROI filtering, which the paper uses to justify a core design choice, imports its key probabilistic containment lemma from the authors' own prior work [46] without proving it here or citing an independent external source. Proposition 1's bound is therefore conditional on a self-cited result: if that lemma is not accepted, the theoretical justification for restricting rollouts to the ROI does not stand on its own within this paper. This is load-bearing for the theoretical framing, though not for the external benchmark comparisons.

full rationale

The central empirical evaluation is not circular: DRO is compared against BO, TuRBO, PFNs4BO, and SCoreBO on Ackley, HPO, and LunarLander benchmarks, so the reported gains in simple regret are externally tested rather than forced by construction. However, two elements create partial circularity. First, the training return-to-go is defined in Appendix D.1 using each GP's own optimistic optimum f˜∗ m, while Section 3 defines simple regret using the true f∗; the paper's claim of 'direct regret minimization' therefore equivocates between a model-defined simulated regret and the real objective. Second, the theoretical justification for the ROI mechanism in Proposition 1 leans on Lemma 1 from the authors' prior work [46] without independent verification, making the theoretical soundness argument partly self-referential. The additional concern that all training actions come from myopic acquisition functions is a correctness and support gap rather than a circularity, and it does not by itself raise the circularity score further.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The method rests on several domain assumptions about sim-to-real transfer, the reliability of the ROI criterion, and the learnability of the decision transformer from offline trajectories. There are many hand-chosen free parameters, most notably the fixed inference target return, the early stop threshold, and the ROI exploration parameter. No genuinely new physical or conceptual entities are introduced.

free parameters (8)
  • Ensemble size M = 10
    Number of GPs in the ensemble; ablation shows performance improves with M up to 10, chosen as default.
  • Bayesian early stop threshold δ = 1e-4
    Rollout terminated when max EI within ROI falls below this; hand-chosen.
  • ROI UCB/LCB exploration parameter κ = 6.0
    Used to define the UCB >= max LCB ROI; hand-chosen.
  • Simulation UCB κ = 2.0
    Exploration parameter for the UCB acquisition during rollouts.
  • EI/PI trade-off ξ = 0.01
    Trade-off parameter for EI and PI acquisitions during rollouts.
  • Inference target return-to-go = 1.0
    Fixed high target used to condition the decision transformer at inference; not derived from data and not normalized to the objective scale.
  • Number of rollouts per GP K = unspecified
    Algorithm 1 requires K, but the paper does not report its value or sensitivity.
  • Transformer training epochs = 100
    Retrained for 100 epochs at each real iteration; chosen, not analyzed.
assumptions (5)
  • domain assumption The true optimum x* is contained in the identified ROI with high probability, based on the UCB >= max LCB criterion (Lemma 1 of Zhang et al. [46]).
    Invoked in Proposition 1 and in Section 4.2 to justify restricting rollouts to the ROI. If the optimum is excluded, simulated trajectories miss the relevant region.
  • domain assumption Simulated trajectories from the GP ensemble are representative enough of the real objective that a policy trained on them reduces real simple regret.
    Core sim-to-real transfer assumption stated in Sections 4.5-4.6; acknowledged but not quantified.
  • standard math The GP posterior variance does not collapse prematurely for unexplored suboptimal points.
    Condition for Proposition 2, following Nguyen et al. [32]; used to motivate the early stop criterion.
  • standard math Standard smoothness and kernel conditions for GP regret bounds hold.
    Assumed at the start of Proposition 1 in Appendix B.
  • domain assumption The decision transformer can learn a policy from offline trajectories that generalizes to online states (offline RL learnability).
    The entire method depends on this; no convergence or generalization guarantee is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Direct Regret Optimization in Bayesian Optimization." pith.science (2026). https://pith.science/paper/ZIHAMA5S

@misc{pith2026250706529,
  author       = {Pith},
  title        = {Pith review of: Direct Regret Optimization in Bayesian Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZIHAMA5S}},
  note         = {Machine review of arXiv:2507.06529}
}
read the original abstract

Bayesian optimization (BO) is a powerful paradigm for optimizing expensive black-box functions. Traditional BO methods typically rely on separate hand-crafted acquisition functions and surrogate models for the underlying function, and often operate in a myopic manner. In this paper, we propose a novel direct regret optimization approach that jointly learns the optimal model and non-myopic acquisition by distilling from a set of candidate models and acquisitions, and explicitly targets minimizing the multi-step regret. Our framework leverages an ensemble of Gaussian Processes (GPs) with varying hyperparameters to generate simulated BO trajectories, each guided by an acquisition function chosen from a pool of conventional choices, until a Bayesian early stop criterion is met. These simulated trajectories, capturing multi-step exploration strategies, are used to train an end-to-end decision transformer that directly learns to select next query points aimed at improving the ultimate objective. We further adopt a dense training--sparse learning paradigm: The decision transformer is trained offline with abundant simulated data sampled from ensemble GPs and acquisitions, while a limited number of real evaluations refine the GPs online. Experimental results on synthetic and real-world benchmarks suggest that our method consistently outperforms BO baselines, achieving lower simple regret and demonstrating more robust exploration in high-dimensional or noisy settings.

Figures

Figures reproduced from arXiv: 2507.06529 by the authors.

Figure 1
Figure 1. The DRO framework. Gray arrows indicates processes involving simulated data, while green arrows correspond to real data. The dense training (gray) and sparse learning (green) steps are depicted in hollow ar￾rows. Direct Regret Optimization. Motivated by these challenges, we propose Di￾rect Regret Optimization (DRO)1 , a novel approach that jointly learns an opti￾mal decision-making model and a non￾myopic acquisition… view at source ↗
Figure 2
Figure 2. Performance on Ackley Function across dimensions (2D, 5D, 10D, 20D - Best Objective [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Performance on HPO Tasks (Simple Regret). Lower values are better. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance on LunarLander and Ablation studies for DRO on Ackley 10D. Higher values [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Performance comparison of different acquisition function strategies used within DRO’s [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 52 canonical work pages

  1. [1]

    Unexpected improvements to expected improvement for bayesian optimization

    Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Unexpected improvements to expected improvement for bayesian optimization. Advances in Neural Information Processing Systems, 36:20577–20612, 2023

    2023

  2. [2]

    Botorch: A framework for efficient monte-carlo bayesian opti- mization

    Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian opti- mization. Advances in neural information processing systems, 33:21524–21538, 2020

    work page 2020

  3. [3]

    Random search for hyper-parameter optimization

    James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. In Journal of Machine Learning Research, volume 13, pages 281–305, 2012

    work page 2012

  4. [4]

    Cora, and Nando De Freitas

    Eric Brochu, Vlad M. Cora, and Nando De Freitas. A tutorial on bayesian optimization of ex- pensive cost functions, with application to active user modeling and hierarchical reinforcement learning. Technical Report TR-2010-23, Dept. of Computer Science, University of British Columbia (UBC), 2010. Often cited tutorial; relevant for discussing integrated ac...

    work page 2010

  5. [5]

    Bayesian opti- mization for policy search on robots

    Roberto Calandra, Andre Seyfarth, Jan Peters, and Marc Peter Deisenroth. Bayesian opti- mization for policy search on robots. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 270–277. IEEE, 2016

    work page 2016

  6. [6]

    Decision transformer: Reinforcement learning via sequence modeling

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems, 34:15084–15097, 2021

    2021

  7. [7]

    Towards learn- ing universal hyperparameter optimizers with transformers

    Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Richard Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc’Aurelio Ranzato, et al. Towards learn- ing universal hyperparameter optimizers with transformers. Advances in Neural Information Processing Systems, 35:32053–32068, 2022

    work page 2022

  8. [8]

    Meta-learning acquisition functions for bayesian optimization

    Zhongxiang Chen, Yunchuan Zhang, Chen Shi, Lina Zhao, and James T Kwok. Meta-learning acquisition functions for bayesian optimization. In International Conference on Learning Rep- resentations (ICLR 2022), 2022. Explicitly addresses meta-learning acquisition functions for BO

    work page 2022

Show all 59 references
  1. [9]

    Reinforcement learning based multi-step look-ahead bayesian optimization

    Mujin Cheon, Haeun Byeon, and Jay Hyung Lee. Reinforcement learning based multi-step look-ahead bayesian optimization. In Proc. 13th IFAC Symposium on Dynamics and Control of Process Systems (DYCOPS), pages 100–105, 2022

    2022

  2. [10]

    Lee, Dong-Yeun Koh, and Calvin Tsay

    Mujin Cheon, Jay H. Lee, Dong-Yeun Koh, and Calvin Tsay. EARL-BO: Reinforcement learning for multi-step lookahead, high-dimensional bayesian optimization. arXiv preprint arXiv:2411.00171, 2024

    2024 arXiv

  3. [11]

    Combining multi- objective bayesian optimization with reinforcement learning for tinyml

    Mark Deutel, Georgios Kontes, Christopher Mutschler, and J ¨urgen Teich. Combining multi- objective bayesian optimization with reinforcement learning for tinyml. ACM Trans. Evolu- tionary Learning and Optimization, 2025. to appear

    2025

  4. [12]

    Scal- able global optimization via local bayesian optimization

    David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scal- able global optimization via local bayesian optimization. Advances in neural information pro- cessing systems, 32, 2019

    2019

  5. [13]

    Scalable constrained bayesian optimization

    David Eriksson and Matthias Poloczek. Scalable constrained bayesian optimization. In Inter- national conference on artificial intelligence and statistics, pages 730–738. PMLR, 2021

    2021

  6. [14]

    Mvf–multivariate test functions library in c for uncon- strained global optimization

    P Adorio Ernesto and UP Diliman. Mvf–multivariate test functions library in c for uncon- strained global optimization. University of the Philippines Diliman, Quezon City, 2005

    2005

  7. [15]

    Integrating parameter uncertainty into bayesian optimization

    Benedikt Philipp Vinzent Flick and Patrick Van Der Smagt. Integrating parameter uncertainty into bayesian optimization. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020) , volume 108 of Proceedings of Machine Learn- in...

    2020

  8. [16]

    Rezende, S

    Marta Garnelo, Johannes Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami, and Yee Whye Teh. Conditional neural processes. InICML 2018 Workshop on Theoret- ical Foundations and Applications of Deep Generative Models, 2018. Introduced Conditional Neural P...

    2018

  9. [17]

    Reinforced few-shot acquisition function learning for bayesian optimization

    Bing-Jing Hsieh, Ping-Chun Hsieh, and Xi Liu. Reinforced few-shot acquisition function learning for bayesian optimization. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, 2021

    2021

  10. [18]

    RLBOF: Reinforcement learning from bayesian optimization feedback

    Hailong Huang, Xiubo Liang, Quanwei Zhang, Hongzhi Wang, and Xiangdong Li. RLBOF: Reinforcement learning from bayesian optimization feedback. In Proc. International Joint Conference on Neural Networks (IJCNN). IEEE, 2024

    2024

  11. [19]

    Self-correcting bayesian opti- mization through bayesian active learning

    Carl Hvarfner, Erik Hellsten, Frank Hutter, and Luigi Nardi. Self-correcting bayesian opti- mization through bayesian active learning. Advances in Neural Information Processing Sys- tems, 36:79173–79199, 2023

    2023

  12. [20]

    Jones, Matthias Schonlau, and William J

    Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492, 1998

    1998

  13. [21]

    Oliva, Jeff Schneider, and Barnab ´as P´oczos

    Kirthevasan Kandasamy, Gautam Dasarathy, Junier B. Oliva, Jeff Schneider, and Barnab ´as P´oczos. Gaussian process based approaches for multi-fidelity optimization. In Maria Flo- rina Balcan and Kilian Q. Weinberger, editors, Proceedings of the 33rd International Con- ference ...

    2016

  14. [22]

    Multi- fidelity bayesian optimisation with continuous approximations

    Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, and Barnab ´as P ´oczos. Multi- fidelity bayesian optimisation with continuous approximations. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning (ICML 2017), ...

    2017

  15. [23]

    Predicting the output from a complex computer code when fast approximations are available.Biometrika, 87(1):1–13, 2000

    Marc C Kennedy and Anthony O’Hagan. Predicting the output from a complex computer code when fast approximations are available.Biometrika, 87(1):1–13, 2000. Key paper on modeling discrepancy between fidelities using GPs (auto-regressive model)

    2000

  16. [24]

    Attentive neural processes

    Hyunjik Kim, Andriy Mnih, Johannes Schwarz, Marta Garnelo, Ali Eslami, Yee Whye Teh, and Dan Rosenbaum. Attentive neural processes. In International Conference on Learning Representations (ICLR 2019), 2019. Introduced Attentive Neural Processes (ANPs)

    2019

  17. [25]

    Lu, Charlotte Loh, Jamie Smith, Jasper Snoek, and Marin Solja ˇci´c

    Samuel Kim, Peter Y . Lu, Charlotte Loh, Jamie Smith, Jasper Snoek, and Marin Solja ˇci´c. Deep learning for bayesian optimization of scientific problems with high-dimensional struc- ture, 2021. arXiv preprint arXiv:2104.11667

    2021 arXiv

  18. [26]

    Gaussian process bandit optimization with few batches

    Zihan Li and Jonathan Scarlett. Gaussian process bandit optimization with few batches. In International Conference on Artificial Intelligence and Statistics, pages 92–107. PMLR, 2022

    2022

  19. [27]

    Robust bayesian optimization with reinforcement learned acquisition functions

    Zijing Liu, Xiyao Qu, Xuejun Liu, and Hongqiang Lyu. Robust bayesian optimization with reinforcement learned acquisition functions. arXiv preprint arXiv:2210.00476, 2022

    2022 arXiv

  20. [28]

    End-to-end meta-bayesian optimisation with transformer neural processes

    Alexandre Maraval, Matthieu Zimmer, Antoine Grosnit, and Haitham Bou Ammar. End-to-end meta-bayesian optimisation with transformer neural processes. Advances in Neural Informa- tion Processing Systems, 36:11246–11260, 2023

    2023

  21. [29]

    Pfns4bo: In-context learning for bayesian optimization

    Samuel M ¨uller, Matthias Feurer, Noah Hollmann, and Frank Hutter. Pfns4bo: In-context learning for bayesian optimization. In International Conference on Machine Learning, pages 25444–25470. PMLR, 2023

    2023

  22. [30]

    Transformers can do bayesian inference

    Samuel M ¨uller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do bayesian inference. arXiv preprint arXiv:2112.10510, 2021

    2021 arXiv

  23. [31]

    Statistical foundations of prior-data fitted networks

    Thomas Nagler. Statistical foundations of prior-data fitted networks. In International Confer- ence on Machine Learning, pages 25660–25676. PMLR, 2023. 9

    2023

  24. [32]

    Regret for expected improvement over the best-observed value and stopping condition

    Vu Nguyen, Sunil Gupta, Santu Rana, Cheng Li, and Svetha Venkatesh. Regret for expected improvement over the best-observed value and stopping condition. In Asian conference on machine learning, pages 279–294. PMLR, 2017

    2017

  25. [33]

    Yahpo gym-an efficient multi-objective multi-fidelity benchmark for hyperparameter optimization

    Florian Pfisterer, Lennart Schneider, Julia Moosbauer, Martin Binder, and Bernd Bischl. Yahpo gym-an efficient multi-objective multi-fidelity benchmark for hyperparameter optimization. In International Conference on Automated Machine Learning, pages 3–1. PMLR, 2022

    2022

  26. [34]

    Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, 2006. The standard reference for Gaussian Processes, relevant for surrogate modeling and hyperparameter estimation (MLE/MAP)

    2006

  27. [35]

    Adams, and Nando de Freitas

    Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE , 104(1):148–175, 2016

    2016

  28. [36]

    Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical bayesian optimization of ma- chine learning algorithms. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25 (NIPS 2012) , pages 2951–

    2012

  29. [37]

    Kakade, and Matthias W

    Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias W. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. of the 27th International Conference on Machine Learning (ICML), pages 1015–1022, 2010

    2010

  30. [38]

    Amortized bayesian optimization over discrete spaces

    Kevin Swersky, Jasper Snoek, and Ryan P Adams. Amortized bayesian optimization over discrete spaces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5940–5947, 2020. Example of using learned models (related to NPs) for amortized/meta- BO

    2020

  31. [39]

    Multi-fidelity active learning with max-value entropy search

    Shion Takeno, Carl Hvarfner, Thomas G ¨artner, C ´edric Archambeau, and Philipp Hennig. Multi-fidelity active learning with max-value entropy search. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Sys- tems...

    2020

  32. [40]

    Gym- nasium: A standard interface for reinforcement learning environments

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goul ˜ao, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gym- nasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032, 2024

    2024 arXiv

  33. [41]

    Sample-efficient opti- mization in the latent space of deep generative models via weighted retraining

    Alexander Tripp, Erik Daxberger, and Jos ´e Miguel Hern´andez-Lobato. Sample-efficient opti- mization in the latent space of deep generative models via weighted retraining. In Advances in Neural Information Processing Systems (NeurIPS), volume 33. Curran Associates, Inc., 2020

    2020

  34. [42]

    Scalable gaussian process- based bayesian optimization ensembles

    Martin Wistuba, Nicolas Schilling, and Lars Schmidt-Thieme. Scalable gaussian process- based bayesian optimization ensembles. In Proceedings of the 2018 SIAM International Con- ference on Data Mining (SDM 2018), pages 513–521. SIAM, 2018. Proposes using ensembles of GPs with d...

    2018

  35. [43]

    Constrained efficient global optimization of expensive black-box functions

    Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, and Colin Jones. Constrained efficient global optimization of expensive black-box functions. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceed- ings of th...

    2023

  36. [44]

    Robust multi-fidelity bayesian opti- mization with deep kernel and partition

    Fengxue Zhang, Thomas Desautels, and Yuxin Chen. Robust multi-fidelity bayesian opti- mization with deep kernel and partition. In The 28th International Conference on Artificial Intelligence and Statistics, 2025. 10

    2025

  37. [45]

    Bowden, Alexander Ladd, Yisong Yue, Thomas De- sautels, and Yuxin Chen

    Fengxue Zhang, Jialin Song, James C. Bowden, Alexander Ladd, Yisong Yue, Thomas De- sautels, and Yuxin Chen. Learning regions of interest for bayesian optimization with adaptive level-set estimation. InProceedings of the 40th International Conference on Machine Learning (ICML)...

    2023

  38. [46]

    Finding interior optimum of black-box con- strained objective with bayesian optimization

    Fengxue Zhang, Zejie Zhu, and Yuxin Chen. Finding interior optimum of black-box con- strained objective with bayesian optimization. In NeurIPS 2024 Workshop on Bayesian Decision-making and Uncertainty, 2024. 11 A Detailed Discussion on Literature A.1 Reinforcement Learning in ...

    2024

  39. [49]

    Learning Regions of Interest

    provided early evidence that an RL-driven policy can outperform the greedy one-step Expected Improvement strategy by optimizing over a multi-step horizon. Their approach treated BO as a sequential decision process and showed that a learned policy can yield better long-term out...

    2024

  40. [50]

    This allows for capturing model uncertainty for all unknown functions

    Ensemble GP Models for Objective and Constraints: Independent ensembles of Gaus- sian Process (GP) models would be maintained for the objective function and for each of the M constraints. This allows for capturing model uncertainty for all unknown functions. All GPs would be c...

  41. [51]

    This can 16 be achieved by adapting ROI identification strategies from constrained Bayesian optimiza- tion (CBO) literature, such as the approach in Zhang et al

    Constrained Region of Interest for Simulations: During the simulation (rollout gener- ation) for a specific set of sampled models (one objective GP and one GP for each con- straint from their respective ensembles), a constrained ROI must be identified. This can 16 be achieved ...

  42. [52]

    Constraint-Aware Rollout Generation: Simulated trajectories are generated by itera- tively selecting points within the dynamically identified constrained ROI. The acquisition function used within the simulation must be constraint-aware (e.g., by multiplying a stan- dard acquis...

  43. [53]

    Augmented State Representation: The state sτ for the decision transformer must include information about the constraints, such as hyperparameters of the constraint GPs, historical constraint observations, and potentially features describing the estimated feasible region

  44. [54]

    This conceptual extension enables DRO to learn a non-myopic policy that navigates the trade-offs inherent in constrained optimization

    Training Objective: The decision transformer is trained to predict actions that minimize the final simple regret with respect to the true constrained optimum, using return-to-go signals calculated based on the bestfeasible objective value found in simulated trajectories. This ...

  45. [55]

    Each GP(f ) m is trained on data observed at fidelity f

    Fidelity-Specific GP Ensembles: Independent ensembles of GPs are maintained for each fidelity level, i.e., {GP(f ) m }Mf m=1 for each f ∈ F. Each GP(f ) m is trained on data observed at fidelity f. Relationships between fidelities (e.g., via auto-regressive models or deep kern...

  46. [56]

    For instance, drawing inspiration from methods like those in Zhang et al

    Fidelity-Aware and Cost-Aware ROI for Simulations: When generating rollouts, the ROI identification can be made fidelity-aware. For instance, drawing inspiration from methods like those in Zhang et al. [44] that identify robust partitions or regions of agree- ment across fidel...

  47. [57]

    Joint Action Space (Point and Fidelity): The decision transformer’s action becomes a pair (xt, ft), selecting both the point to evaluate and the fidelity level at which to perform the evaluation

  48. [58]

    • Current cumulative cost and remaining budget (if applicable)

    Augmented State Representation: The state sτ must include fidelity-related information: • Historical data, including the fidelity level and cost of each past evaluation. • Current cumulative cost and remaining budget (if applicable). • Hyperparameters or summary statistics fro...

  49. [59]

    The Bayesian early stopping criterion might be adapted to consider the cost-benefit of continuing a roll- out

    Cost-Sensitive Rollout Simulation and Policy Learning: Simulated trajectories now in- volve selecting fidelities for simulated queries, incurring simulated costs. The Bayesian early stopping criterion might be adapted to consider the cost-benefit of continuing a roll- out. The...

  50. [2017]

    Extends MFBO work, particularly relevant for continuous fidelity levels

  51. [2959]

    Seminal paper on applying BO to hyperparameter tuning; discusses practical aspects including handling GP hyperparameters (e.g., MCMC integration mentioned)

    Curran Associates, Inc., 2012. Seminal paper on applying BO to hyperparameter tuning; discusses practical aspects including handling GP hyperparameters (e.g., MCMC integration mentioned)

    2012

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.