Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Simplifying Bayesian Optimization Via In-Context Direct Optimum Sampling

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pretraining a generative model to sample the optimum directly from the observed data makes Bayesian optimization a single forward pass, with the trained distribution approximating Thompson sampling under the pretraining prior.

desk verdict Genuinely new way to run BO by direct in-context optimum sampling, with real speedups; the Thompson-sampling equivalence is only proven for the training context distribution, not the closed loop. read the letter →

arxiv 2505.23913 v1 pith:PM6OIRGH submitted 2025-05-29 cs.LG stat.ML

Gustavo Sutter Pessurno de Carvalho , Mohammed Abdulrahman , Hao Wang , Sriram Ganapathi Subramanian , Marc St-Aubin , Sharon O'Sullivan , Lawrence Wan , Luis Ricardez-Sandoval
show 2 more authors
Pascal Poupart Agustinus Kristiadi
This is my paper
classification cs.LGstat.ML
keywords Bayesianoptimizationin-contextlearningThompsonsamplingnormalizingflowsamortizedinferencedirectoptimumGaussianprocesspriorbatched
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Bayesian optimization normally alternates two expensive steps: a surrogate is refit to collected data, and an acquisition function is maximized to choose the next query. This paper proposes to bypass both: pretrain a deep generative model $p_\theta(x^* \mid D)$ that takes the observed dataset $D$ as context and directly emits a sample of the optimum $x^*$. The central claim is that minimizing the negative log-likelihood over pairs $(x^*, D)$ sampled from a prior over functions makes $p_\theta(x^* \mid D)$ approximate the Thompson sampling posterior under that prior. If true, the method is a principled, zero-shot Bayesian optimization procedure whose test-time cost is a single forward pass, and the experiments support this by matching GP and PFN baselines in final performance while being 10 to 1000 times faster in wall-clock time.

What carries the argument

The load-bearing object is the conditional generative model $p_\theta(x^* \mid D)$: an encoder $E$ that compresses a variable-size dataset $D$ into a fixed context vector $c$, followed by a normalizing-flow head $G$ that maps a Gaussian latent vector $z$ and $c$ to a point $x^*$. Pretraining minimizes the negative log-likelihood $L(\theta) = -\mathbb{E}_{x^*, D}[\log p_\theta(x^* \mid D)]$ over pairs sampled from $p(f, x^*, D) = p(x^* \mid f)\, p(D \mid f)\, p(f)$. The proof of Theorem 1 rewrites this objective as the expected KL divergence between $p_\theta(x^* \mid D)$ and $p(x^* \mid D)$, so the identity carrying the argument is the equivalence between minimizing cross-entropy and minimizing KL, making the generative model a direct approximation of Thompson sampling without any explicit acquisition function definition.

What would settle it

For a simple GP with a known posterior over the optimum, draw datasets $D$ from the prior, compute the exact $p(x^* \mid D)$ by quadrature or MCMC, and compare it to FIBO's samples; if the two distributions differ by more than sampling error, Theorem 1's claim that $p_\theta(x^* \mid D)$ approximates Thompson sampling is false. A second observation would be to run FIBO on a function class far outside the GP prior and check whether final utility stops improving when the model cannot absorb new data.

Watch

Extended reading notes

Core claim

The paper's discovery is a reduction: instead of separately modeling the surrogate posterior $p(f \mid D)$ and maximizing an acquisition function, one can train a conditional generative model on the joint distribution $p(f, x^*, D)$ and use it to sample $p_\theta(x^* \mid D)$ directly. Theorem 1 states that the training objective equals the expected KL divergence between $p_\theta(x^* \mid D)$ and the true posterior over optima $p(x^* \mid D)$, hence the trained model approximates Thompson sampling with respect to the pretraining prior $p(f)$. The practical result is that the optimization loop becomes a single forward pass through an encoder and a normalizing flow, with no surrogate refitting and no inner optimization at test time; across well-specified functions, synthetic benchmarks, and chemistry tasks, the method's GAP is statistically indistinguishable from GP-EI and PFNs4BO while wall-clock time drops by more than 35 times on average and up to three orders of magnitude for large batches.

Load-bearing premise

The load-bearing premise is that the prior over functions used to generate pretraining data, a Gaussian process with RBF kernel and fixed hyperpriors, with rejection sampling to spread optima uniformly, actually contains the objective functions FIBO will face at test time, since the model is frozen after pretraining and can neither refit nor adjust its beliefs.

Editorial extensions

If this is right

  • No surrogate refitting and no acquisition maximization are needed at test time; the next suggestion is one forward pass through the model.
  • Batch suggestions of size $q$ are obtained by sampling $q$ latent vectors with the same context, making batched Thompson sampling nearly free and removing the sequential-simulation or Monte Carlo optimization overhead of other batch methods.
  • Wall-clock time per iteration stays nearly constant as the batch size grows, in contrast to GP-EI and PFNs4BO, whose acquisition maximization cost grows with the batch.
  • The statistical analysis in the paper shows no significant GAP difference between FIBO and GP-EI, PFNs4BO, or LLA on the chemistry tasks, so the speed gain does not come with a measured loss in optimization quality.
  • Because the model is trained once per input dimension on the chosen prior, applying FIBO to a new task requires no per-task fitting, only feeding the observed data as context.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because sampling $q$ points is just $q$ forward passes from the same context, the paper's construction implies that batched Thompson sampling, normally a costly sequential or Monte Carlo procedure, becomes essentially free; this is a direct scalability consequence the authors state but do not develop into a separate protocol.
  • The model is pretrained separately for $d=3$ and $d=4$ and evaluated only up to four dimensions; a testable extension is whether the same data-generation recipe and objective transfer to higher dimensions or to a single model that conditions on dimension as part of the context.
  • One could feed FIBO's samples into an acquisition-function optimizer as restart points or proposal candidates; since Theorem 1 only guarantees the marginal distribution over the optimum, any downstream utility built on these samples inherits the same approximation error.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper introduces FIBO, a method that replaces surrogate fitting and acquisition function optimization in Bayesian optimization with a pretrained generative model p_theta(x*|D) that directly samples a candidate optimum from the posterior over the optimum given the observed context D. The model is trained by minimizing the negative log-likelihood in Eq. (1) on (x*, D) pairs generated from a Gaussian-process prior approximated with random Fourier features (Algorithm 1 and Appendix C). The paper claims in Theorem 1 that this training procedure makes FIBO equivalent to Thompson sampling, and it reports experiments on the pretraining prior, standard synthetic benchmarks, and eight Olympus chemistry tasks, showing GAP comparable to GP-EI, PFNs4BO, and LLA while being 10x-1000x faster in wall-clock time, with the speed advantage growing with batch size. The contribution is framed as simplifying the BO pipeline to a single forward pass.

Significance. If the empirical results hold, FIBO provides a practically useful speedup for batched Bayesian optimization and removes the need for acquisition-function tuning, which is a genuine engineering contribution. The evaluation is unusually broad for a methods paper: multiple batch sizes (q=10,20,50), several baselines, standard synthetic functions, real-world chemistry emulators, and pairwise hypothesis tests. The wall-clock gains are large and consistent, and the GAP values are close to the baselines on most tasks. However, the theoretical claim that FIBO is 'equivalent to Thompson sampling' is not established by the current proof, which reduces to the training objective; the practical utility of the method does not depend on that specific theoretical framing, but the framing as written overstates what is proven.

major comments (3)
  1. [Section 3.3, Theorem 1 and Appendix B] Eq. (7) shows that L(theta) = E_D[KL(p(x*|D) || p_theta(x*|D))] + const, which is the training objective being minimized. This is a definitional equivalence, not a proof that the trained model approximates Thompson sampling. Moreover, the expectation is over p(D) induced by Algorithm 1, where the query inputs are i.i.d. uniform on X; in the BO loop (Algorithm 2), the contexts D_t are produced by FIBO itself and hence are not drawn from p(D). The theorem therefore does not cover the algorithm as deployed. Please either restrict the theoretical claim to a property of the training objective and add an on-policy analysis of the closed-loop context distribution, or provide additional assumptions under which the KL guarantee transfers to the BO loop.
  2. [Appendix C, 'GP prior'] The rejection sampling used to enforce a uniform marginal distribution over x* changes the effective joint distribution from the p(f,x*,D) in Eq. (2) used in Theorem 1. The theorem is stated for the chosen prior p(f), but the training data are not sampled from that exact distribution. Please state Theorem 1 for the actual training distribution or justify that the rejection step does not affect the conclusion; otherwise the theoretical claim and the pretraining procedure are inconsistent.
  3. [Section 5.1] The 'well-specified' experiments sample functions from the GP prior, but the BO loop still generates contexts from FIBO's own proposals rather than i.i.d. uniform inputs. Hence these experiments do not test the model on the distribution where Theorem 1 provides a guarantee. A quantitative diagnostic of the on-policy context distribution—for example, a density comparison between uniform and FIBO-generated contexts, or a calibration/expected-KL estimate on BO-loop data—would be needed to support the claim that FIBO approximates Thompson sampling in the setting where it is actually used.
minor comments (6)
  1. [References] References [1] and [2] are identical; please merge them and fix the in-text citations.
  2. [Appendix C] The sentence 'This last step is necessary to unsure that FIBO is not biased' should read 'ensure'.
  3. [Section 5.3] The baseline name 'PFNs4B0' appears with a zero; it should be 'PFNs4BO'.
  4. [Table 2] The q=50 GAP comparison between FIBO and PFNs4BO yields p=0.05, which is borderline; the text's claim of 'no significant difference' would benefit from a multiple-comparison correction or a caveat.
  5. [Algorithm 1 vs Appendix A] Algorithm 1 uses 'x* = gradient_ascent(f)' while Appendix A describes multi-restart second-order optimization; these descriptions should be aligned.
  6. [Section 5, GAP definition] In the GAP definition, the symbols y_i, y_0, and y* are not defined explicitly; please add a sentence clarifying each term.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 1's TS-equivalence is the training objective restated: Eq. (7) shows the NLL loss equals the expected KL to the TS posterior, so the central theoretical claim reduces to the loss definition by construction.

  1. self definitional [Section 3.3 (Theorem 1) and Appendix B (Eq. 7)]
    "L(θ) = ED∼p(D)[KL(p(x ∗|D)||pθ(x∗|D))] demonstrating that by minimizing L(θ) in (1) we are minimizing the expected KL divergence between the true Thompson sampling distribution and the one modelled by the deep generative model."

    Theorem 1's conclusion—that FIBO approximates Thompson sampling—is the training target restated. Eq. (1) defines L(θ) as the negative log-likelihood of samples (x*, D) drawn from p(f, x*, D), where x* is the optimum whose posterior p(x*|D) is exactly the Thompson sampling distribution. Eq. (7) then rewrites L(θ) as E_D KL(p(x*|D) || p_θ(x*|D)) plus a constant. Minimizing NLL is by definition minimizing expected KL to the target posterior; no independent identification links the trained model to Thompson sampling beyond what was optimized. Moreover, the proof only concerns datasets drawn from p(D) under Algorithm 1's i.i.d. uniform query inputs, and does not cover the closed-loop contexts in Algorithm 2, where inputs are FIBO's own samples.

full rationale

The paper's central theoretical claim is that FIBO is equivalent to Thompson sampling, but the proof of Theorem 1 reduces to a definitional identity: the negative log-likelihood loss in Eq. (1) is rewritten in Eq. (7) as the expected KL divergence between the true posterior over the optimum and the model distribution. Since that posterior is the Thompson sampling distribution and the loss is constructed from samples of it, the theorem's content is the training objective itself rather than an independent derivation. The theorem also does not cover the actual deployment in Algorithm 2, where contexts are generated by FIBO's own suggestions rather than drawn i.i.d. uniformly as in Algorithm 1; that is a separate correctness gap, but it reinforces that the theorem adds little beyond the loss. The empirical contributions—wall-clock speedups and benchmark GAP comparisons against external baselines—are self-contained and not circular; no load-bearing self-citation was found. Because the central theoretical claim reduces by construction to the definition of the loss, while the empirical results remain independent, a score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the choice of the GP prior, the correctness of the global optimum computation during pretraining, and the capacity of the normalizing flow. No new physical entities are introduced.

free parameters (4)
  • Length-scale hyperprior = l_i ~ U(0.01, 5)
    Chosen by hand to define the GP prior smoothness; affects the posterior over optima and the pretraining data distribution.
  • Output-scale hyperprior = gamma^2 ~ U(1, 2)
    Chosen by hand for the GP prior amplitude; affects the range of function values and the location of optima.
  • Context dataset size range = N_min to N_max (values not specified in main text)
    Variable context sizes during pretraining; the exact bounds are not given, affecting the conditional distribution p(x*|D) seen by the model.
  • Rejection sampling to enforce uniform optima = Not quantified
    Ad hoc procedure to force the marginal distribution over optima to be uniform, altering the prior and the posterior approximation.
assumptions (5)
  • domain assumption The GP prior with RBF kernel, approximated via random Fourier features with the stated hyperpriors, adequately represents the objective functions encountered at test time.
    The method's posterior p(x*|D) is defined by this prior; performance degrades if the target function is outside the prior class. See Section 3.2 and Appendix C.
  • domain assumption L-BFGS-B with multiple restarts finds the global optimum of the sampled GP functions used for pretraining.
    The training targets x* are computed this way; local optima would corrupt the training labels. See Appendix A.
  • domain assumption The normalizing flow can accurately model the posterior over the optimum in 3-4 dimensions.
    The method relies on effective density estimation; the authors limit experiments to low dimensions. See Section 3.1 and Limitations.
  • standard math Given f, the optimum x* and the dataset D are conditionally independent.
    Used in the factorization in Eq. (2). It is a valid decomposition for the data-generating process.
  • standard math Minimizing the NLL in Eq. (1) is equivalent to minimizing expected KL divergence between p(x*|D) and p_theta(x*|D).
    This is the proof of Theorem 1 in Appendix B; it follows from the cross-entropy identity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Simplifying Bayesian Optimization Via In-Context Direct Optimum Sampling." pith.science (2026). https://pith.science/paper/PM6OIRGH

@misc{pith2026250523913,
  author       = {Pith},
  title        = {Pith review of: Simplifying Bayesian Optimization Via In-Context Direct Optimum Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PM6OIRGH}},
  note         = {Machine review of arXiv:2505.23913}
}
read the original abstract

The optimization of expensive black-box functions is ubiquitous in science and engineering. A common solution to this problem is Bayesian optimization (BO), which is generally comprised of two components: (i) a surrogate model and (ii) an acquisition function, which generally require expensive re-training and optimization steps at each iteration, respectively. Although recent work enabled in-context surrogate models that do not require re-training, virtually all existing BO methods still require acquisition function maximization to select the next observation, which introduces many knobs to tune, such as Monte Carlo samplers and multi-start optimizers. In this work, we propose a completely in-context, zero-shot solution for BO that does not require surrogate fitting or acquisition function optimization. This is done by using a pre-trained deep generative model to directly sample from the posterior over the optimum point. We show that this process is equivalent to Thompson sampling and demonstrate the capabilities and cost-effectiveness of our foundation model on a suite of real-world benchmarks. We achieve an efficiency gain of more than 35x in terms of wall-clock time when compared with Gaussian process-based BO, enabling efficient parallel and distributed BO, e.g., for high-throughput optimization.

Figures

Figures reproduced from arXiv: 2505.23913 by the authors.

Figure 1
Figure 1. Performing BO in-context with our method enables batched BO with a large batch size (almost) for free. Many problems in chemistry [14, 15], biology [37, 38] and computer science [9, 40] rely on optimizing an ex￾pensive black-box function. Often, input domains are pro￾hibitively large, and objective function evaluation needs laboratory experiments or computation-intensive simula￾tion [43]. This necessitates specific … view at source ↗
Figure 2
Figure 2. Optimization loops of different black-box optimizers: traditional BO, PFNs4BO [ [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 4
Figure 4. Illustration of a Bayesian optimization loop using FIBO. The model receives a dataset as [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Factorization of the data-generating distribution. To generate the data for pretraining FIBO, we need to sample pairs (x∗, D)’s from the joint distribution p(f, x∗, D). Consider a factor￾ization of such a distribution to the graphical model in [PITH_FULL_IMAGE:figures…
Figure 5
Figure 5. Figure 5: Time comparison on 4D Ackley function with q = 10 Since the model is pretrained, no surrogate fitting is per￾formed at test time. In addition, because we directly sam￾ple from the posterior over the optimal point there is no ex￾plicit acquisition function maximization.…
Figure 6
Figure 6. Figure 6: Comparison of Bayesian optimization methods on function sampled from FIBO’s pretrain [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Bayesian optimization methods on standard synthetic functions across [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Comparison of Bayesian optimization methods on various chemistry tasks from Olympus [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models

    cs.CL 2026-07 conditional novelty 7.0 of 10

    LLM probability estimates violate the law of total probability across partitions, and subgroup-aggregated estimates often beat direct population-level estimates (the macro fallacy).

Reference graph

Works this paper leans on

49 extracted references · 43 canonical work pages · cited by 1 Pith paper

  1. [2]

    Unexpected improvements to expected improvement for Bayesian optimization

    Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Unexpected improvements to expected improvement for Bayesian optimization. InNeurIPS, 2023

    work page 2023

  2. [3]

    Using confidence bounds for exploitation-exploration trade-offs.JMLR, 3(null), 2003

    Peter Auer. Using confidence bounds for exploitation-exploration trade-offs.JMLR, 3(null), 2003

    work page 2003

  3. [4]

    Botorch: A framework for efficient monte-carlo bayesian optimization

    Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. InNeurIPS, 2020

    work page 2020

  4. [5]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...

    work page 2020

  5. [6]

    Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu

    Richard H. Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization.SIAM Journal on Scientific Computing, 16(5), 1995

    work page 1995

  6. [7]

    Learning phrase representations using RNN encoder–decoder for statistical machine translation

    Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder–decoder for statistical machine translation. InEMNLP, 2014

    work page 2014

  7. [8]

    Neural spline flows

    Conor Durkan, Artur Bekasov, Iain Murray, and George Papamakarios. Neural spline flows. InNeurIPS, 2019

    2019

  8. [9]

    Auto-sklearn 2.0: Hands-free automl via meta-learning.JMLR, 23(261), 2022

    Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. Auto-sklearn 2.0: Hands-free automl via meta-learning.JMLR, 23(261), 2022

    work page 2022

Show all 49 references
  1. [10]

    Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and S. M. Ali Eslami. Conditional neural processes. InICML, 2018

    2018

  2. [11]

    Cambridge University Press, 2023

    Roman Garnett.Bayesian Optimization. Cambridge University Press, 2023

    2023

  3. [12]

    Springer, 2010

    David Ginsbourger, Rodolphe Le Riche, and Laurent Carraro.Kriging Is Well-Suited to Parallelize Optimization. Springer, 2010

    2010

  4. [13]

    Text-guided molecule generation with diffusion language model.Proceedings of the AAAI Conference on Artificial Intelligence, 38(1), 2024

    Haisong Gong, Qiang Liu, Shu Wu, and Liang Wang. Text-guided molecule generation with diffusion language model.Proceedings of the AAAI Conference on Artificial Intelligence, 38(1), 2024

    2024

  5. [14]

    From alchemist to ai chemist

    Rebecca L Greenaway, Kim E Jelfs, Alan C Spivey, and Sophia N Yaliraki. From alchemist to ai chemist. Nature Reviews Chemistry, 7(8), 2023

    2023

  6. [15]

    Constrained Bayesian optimization for automatic chemical design using variational autoencoders.Chem

    Ryan-Rhys Griffiths and José Miguel Hernández-Lobato. Constrained Bayesian optimization for automatic chemical design using variational autoencoders.Chem. Sci., 11, 2020

    2020

  7. [16]

    Hickman, Loïc M

    Florian Häse, Matteo Aldeghi, Riley J. Hickman, Loïc M. Roch, Melodie Christensen, Elena Liles, Jason E. Hein, and Alán Aspuru-Guzik. Olympus: a benchmarking framework for noisy optimization and experiment planning.Machine Learning: Science and Technology, 2(3), 2021

    2021

  8. [17]

    Hoffman, and Zoubin Ghahramani

    José Miguel Henrández-Lobato, Matthew W. Hoffman, and Zoubin Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. InNeurIPS, 2014

    2014

  9. [18]

    Pyzer-Knapp, and Alán Aspuru-Guzik

    José Miguel Hernández-Lobato, James Requeima, Edward O. Pyzer-Knapp, and Alán Aspuru-Guzik. Parallel and distributed Thompson sampling for large-scale accelerated exploration of chemical space. In ICML, 2017

    2017

  10. [19]

    BINOCULARS for efficient, nonmyopic sequential experimental design

    Shali Jiang, Henry Chai, Javier Gonzalez, and Roman Garnett. BINOCULARS for efficient, nonmyopic sequential experimental design. InICML, 2020

    2020

  11. [20]

    Jones, Matthias Schonlau, and William J

    Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient global optimization of expensive black-box functions.Journal of Global Optimization, 13, 1998

    1998

  12. [21]

    Diffusion models for black-box optimization

    Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, and Aditya Grover. Diffusion models for black-box optimization. InICML, 2023. 10

    2023

  13. [22]

    Promises and pitfalls of the linearized Laplace in Bayesian optimization

    Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, and Vincent Fortuin. Promises and pitfalls of the linearized Laplace in Bayesian optimization. InFifth Symposium on Advances in Approximate Bayesian Inference, 2023

    2023

  14. [23]

    A sober look at LLMs for material discovery: Are they actually good for Bayesian optimization over molecules? InICML, 2024

    Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta, Pascal Poupart, Alán Aspuru-Guzik, and Geoff Pleiss. A sober look at LLMs for material discovery: Are they actually good for Bayesian optimization over molecules? InICML, 2024

    2024

  15. [24]

    How useful is intermittent, asynchronous expert feedback for Bayesian optimization? InSixth Symposium on Advances in Approximate Bayesian Inference-Non Archival Track, 2024

    Agustinus Kristiadi, Felix Strieth-Kalthoff, Sriram Ganapathi Subramanian, Vincent Fortuin, Pascal Poupart, and Geoff Pleiss. How useful is intermittent, asynchronous expert feedback for Bayesian optimization? InSixth Symposium on Advances in Approximate Bayesian Inference-Non...

    2024

  16. [25]

    Model inversion networks for model-based optimization

    Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. InNeurIPS, 2020

    2020

  17. [26]

    Yucen Lily Li, Tim G. J. Rudner, and Andrew Gordon Wilson. A study of Bayesian neural network surrogates for Bayesian optimization. InICLR, 2024

    2024

  18. [27]

    Amortized in-context Bayesian posterior estimation, 2025

    Sarthak Mittal, Niels Leif Bracher, Guillaume Lajoie, Priyank Jaini, and Marcus Brubaker. Amortized in-context Bayesian posterior estimation, 2025

    2025

  19. [28]

    J. Moˇckus. On Bayesian methods for seeking the extremum. InOptimization Techniques IFIP Technical Conference Novosibirsk, July 1–7, 1974, 1975

    1974

  20. [29]

    Transformers can do Bayesian inference

    Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do Bayesian inference. InICLR, 2022

    2022

  21. [30]

    PFNs4BO: In-context learning for Bayesian optimization

    Samuel Müller, Matthias Feurer, Noah Hollmann, and Frank Hutter. PFNs4BO: In-context learning for Bayesian optimization. InICML, 2023

    2023

  22. [31]

    Normalizing flows for probabilistic modeling and inference.Journal of Machine Learning Research, 22(57), 2021

    George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshmi- narayanan. Normalizing flows for probabilistic modeling and inference.Journal of Machine Learning Research, 22(57), 2021

    2021

  23. [32]

    Radev, Ulf K

    Stefan T. Radev, Ulf K. Mertens, Andreas V oss, Lynton Ardizzone, and Ullrich Köthe. BayesFlow: Learning complex stochastic models with invertible neural networks.IEEE transactions on neural networks and learning systems, 33(4):1452–1466, 2020

    2020

  24. [33]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. InNeurIPS, 2007

    2007

  25. [34]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InICML, 2021

    2021

  26. [35]

    Carl Edward Rasmussen and Christopher K. I. Williams.Gaussian Processes for Machine Learning. The MIT Press, 2006

    2006

  27. [36]

    Arik Reuter, Tim G. J. Rudner, Vincent Fortuin, and David Rügamer. Can transformers learn full Bayesian inference in context?arXiv, 2025

    2025

  28. [37]

    Navigating the protein fitness landscape with Gaussian processes.Proceedings of the National Academy of Sciences, 110(3), 2013

    Philip A Romero, Andreas Krause, and Frances H Arnold. Navigating the protein fitness landscape with Gaussian processes.Proceedings of the National Academy of Sciences, 110(3), 2013

    2013

  29. [38]

    Application of Bayesian approaches in drug development: starting a virtuous cycle.Nature Reviews Drug Discovery, 22(3), 2023

    Stephen J Ruberg, Francois Beckers, Rob Hemmings, Peter Honig, Telba Irony, Lisa LaVange, Grazyna Lieberman, James Mayne, and Richard Moscicki. Application of Bayesian approaches in drug development: starting a virtuous cycle.Nature Reviews Drug Discovery, 22(3), 2023

    2023

  30. [39]

    Adams, and Nando de Freitas

    Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. Taking the human out of the loop: A review of Bayesian optimization.Proceedings of the IEEE, 2016

    2016

  31. [40]

    Practical Bayesian optimization of machine learning algorithms

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical Bayesian optimization of machine learning algorithms. InNeurIPS, 2012

    2012

  32. [41]

    normflows: A Pytorch package for normalizing flows.Journal of Open Source Software, 8(86), 2023

    Vincent Stimper, David Liu, Andrew Campbell, Vincent Berenz, Lukas Ryll, Bernhard Schölkopf, and José Miguel Hernández-Lobato. normflows: A Pytorch package for normalizing flows.Journal of Open Source Software, 8(86), 2023

    2023

  33. [42]

    On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika, 25(3-4), 1933

    William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika, 25(3-4), 1933. 11

    1933

  34. [43]

    Schmid, Sterling G

    Gary Tom, Stefan P. Schmid, Sterling G. Baird, Yang Cao, Kourosh Darvish, Han Hao, Stanley Lo, Sergio Pablo-García, Ella M. Rajaonson, Marta Skreta, Naruki Yoshikawa, Samantha Corapi, Gun Deniz Akkoc, Felix Strieth-Kalthoff, Martin Seifrid, and Alán Aspuru-Guzik. Self-driving ...

    2024

  35. [44]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems, 2017

    2017

  36. [45]

    Maximizing acquisition functions for Bayesian optimization

    James Wilson, Frank Hutter, and Marc Deisenroth. Maximizing acquisition functions for Bayesian optimization. InNeurIPS, 2018

    2018

  37. [46]

    Diff-BBO: Diffusion-based inverse modeling for black-box Optimization

    Dongxia Wu, Nikki Lijing Kuang, Ruijia Niu, Yian Ma, and Rose Yu. Diff-BBO: Diffusion-based inverse modeling for black-box Optimization. InNeurIPS 2024 Workshop on Bayesian Decision-making and Uncertainty, 2024

    2024

  38. [47]

    Omnigen: Unified image generation.arXiv preprint arXiv:2409.11340, 2024

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation.arXiv preprint arXiv:2409.11340, 2024

    2024 arXiv

  39. [48]

    An explanation of in-context learning as implicit Bayesian inference

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning as implicit Bayesian inference. InICLR, 2022

    2022

  40. [49]

    Posterior inference with diffusion models for high-dimensional black-box optimization, 2025

    Taeyoung Yun, Kiyoung Om, Jaewoo Lee, Sujin Yun, and Jinkyoo Park. Posterior inference with diffusion models for high-dimensional black-box optimization, 2025

    2025

  41. [50]

    Deep sets

    Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. InNeurIPS, 2017. 12 Appendix A Data Generation Details As described in Section 3.2 we are interested in sampling pairs of the form (x∗,D) where x∗ = arg m...

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.