pith. sign in

arxiv: 2305.06709 · v4 · submitted 2023-05-11 · 💻 cs.LG · cs.MS· stat.ML

NUBO: A Transparent Python Package for Bayesian Optimization

Mike Diessner , Kevin J. Wilson , Richard D. Whalley This is my paper

Pith reviewed 2026-05-24 08:40 UTC · model grok-4.3

classification 💻 cs.LG cs.MSstat.ML
keywords Bayesian optimizationPython packageblack-box optimizationGaussian processesacquisition functionsconstrained optimizationmixed variablesopen source software
0
0 comments X

The pith

NUBO is a Python package that lets users write their own Bayesian optimization loops with modular, tested components for black-box functions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents NUBO as a framework for optimizing expensive-to-evaluate black-box functions such as physical experiments and computer simulators. It uses Gaussian process surrogate models and acquisition functions but centers on transparency through clean code and documentation plus a modular design that lets users assemble the optimization loop themselves. The package handles sequential single-point, parallel multi-point, and asynchronous runs over bounded, constrained, or mixed discrete-continuous input spaces while restricting itself to a compact set of extensively validated algorithms. This structure aims to give researchers from any discipline access to Bayesian optimization without needing deep Python expertise. If the approach holds, practitioners could adapt the method directly to their own simulators or experiments rather than relying on opaque black-box tools.

Core claim

NUBO supplies a set of building blocks for Bayesian optimization so that users can construct and control the full sequential or parallel procedure themselves, including support for constraints and mixed variable types, while the library stays small by including only algorithms that have been extensively tested and validated.

What carries the argument

Modular building blocks that users combine to write a custom optimization loop around Gaussian process surrogates and acquisition functions.

If this is right

  • Researchers can adapt the optimization procedure to the exact constraints and variable types of their own problem instead of using a fixed pipeline.
  • The same code base supports both single-point sequential runs and parallel or asynchronous evaluations without separate packages.
  • Users avoid an overwhelming choice of algorithms because only validated options are included.
  • The open-source release under BSD 3-Clause allows inspection and modification of every step.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The modular design could be extended by users to incorporate new acquisition functions or surrogate models while preserving the transparency guarantee.
  • Asynchronous support may prove especially useful for experiments where evaluation times vary unpredictably.
  • Clean references in the code could lower the barrier for reproducing or citing specific algorithmic choices in published work.

Load-bearing premise

That restricting the package to a small number of extensively tested algorithms will deliver both reliable performance and simplicity for users across disciplines.

What would settle it

A controlled test in which domain scientists with no prior Python optimization experience attempt to optimize a real simulator or experiment using only the NUBO documentation and report whether they reach a valid optimum without external code assistance.

Figures

Figures reproduced from arXiv: 2305.06709 by Kevin J. Wilson, Mike Diessner, Richard D. Whalley.

Figure 1
Figure 1. Figure 1: Comparison of different Python packages for Bayesian optimization. A) Sequential single-point optimization on the 2D Levy function; B) Sequential single-point optimization on the 6D Hartmann function; C) Parallel multi-point optimization with a batch size of four on the 2D Levy function; D) Parallel multi-point optimization with a batch size of four on the 6D Hartmann function. Besides implementations in P… view at source ↗
Figure 2
Figure 2. Figure 2: Bayesian optimization applied to a 1-dimensional function with one local and one global maximum. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: NUBO flowchart. Overview of the recommended algorithms for specific problems. Start in yellow, decisions in blue, and recommended algorithm in green. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Latin hypercube sampling compared to random sampling. [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Results of the Bayesian optimization algorithm implemented with [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
read the original abstract

NUBO, short for Newcastle University Bayesian Optimisation, is a Bayesian optimization framework for the optimization of expensive-to-evaluate black-box functions, such as physical experiments and computer simulators. Bayesian optimization is a costefficient optimization strategy that uses surrogate modelling via Gaussian processes to represent an objective function and acquisition functions to guide the selection of candidate points to approximate the global optimum of the objective function. NUBO itself focuses on transparency and user experience to make Bayesian optimization easily accessible to researchers from all disciplines. Clean and understandable code, precise references, and thorough documentation ensure transparency, while user experience is ensured by a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimization algorithms. NUBO allows users to tailor Bayesian optimization to their specific problem by writing the optimization loop themselves using the provided building blocks. It supports sequential single-point, parallel multi-point, and asynchronous optimization of bounded, constrained, and/or mixed (discrete and continuous) parameter input spaces. Only algorithms and methods that are extensively tested and validated to perform well are included in NUBO. This ensures that the package remains compact and does not overwhelm the user with an unnecessarily large number of options. The package is written in Python but does not require expert knowledge of Python to optimize your simulators and experiments. NUBO is distributed as open-source software under the BSD 3-Clause license.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript presents NUBO, a Python package for Bayesian optimization of expensive black-box functions. It emphasizes transparency through clean code and documentation, modularity allowing users to construct custom optimization loops, and support for sequential single-point, parallel multi-point, and asynchronous optimization over bounded, constrained, and mixed (discrete/continuous) input spaces. The design rationale includes restricting the included algorithms to those that are 'extensively tested and validated to perform well' in order to keep the package compact.

Significance. If the transparency, modularity, and selection of reliable methods are delivered as described, NUBO could serve as a practical, accessible tool for researchers in machine learning and experimental sciences who need Bayesian optimization without requiring deep expertise in the underlying methods. The open-source BSD license and focus on user-written loops are positive features for reproducibility and customization.

major comments (1)
  1. [Abstract] Abstract: The central design claim that 'Only algorithms and methods that are extensively tested and validated to perform well are included in NUBO' is presented without any supporting citations, benchmark results, or references to validation studies for the chosen acquisition functions, GP variants, or other components. This premise directly supports both the compactness justification and the assertion of reliable performance, yet no evidence is supplied in the manuscript.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive report and positive assessment of NUBO. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central design claim that 'Only algorithms and methods that are extensively tested and validated to perform well are included in NUBO' is presented without any supporting citations, benchmark results, or references to validation studies for the chosen acquisition functions, GP variants, or other components. This premise directly supports both the compactness justification and the assertion of reliable performance, yet no evidence is supplied in the manuscript.

    Authors: We agree that the design claim would benefit from explicit supporting references. The included methods (e.g., expected improvement, upper confidence bound, standard Gaussian process regression) were selected because they are supported by extensive prior validation in the Bayesian optimization literature. In the revised manuscript we will add citations to key validation studies and review papers (such as Jones et al. 1998 for EI, Srinivas et al. 2010 for UCB, and Rasmussen & Williams 2006 for GPs) that demonstrate reliable performance of these components. This addition will substantiate the compactness rationale without altering the package scope or introducing new methods. revision: yes

Circularity Check

0 steps flagged

No circularity: software description paper with no derivations

full rationale

This is a software package announcement paper. It contains no equations, no predictions of quantities, no fitted parameters, and no derivation chain. The design statement that only 'extensively tested' algorithms are included is a selection criterion, not a mathematical claim that reduces to its own inputs by construction. No self-citation load-bearing steps or ansatz smuggling occur. The paper is self-contained as a description of code and features.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper introduces no mathematical content, free parameters, axioms, or invented entities; it is a description of a software implementation of existing methods.

pith-pipeline@v0.9.0 · 5784 in / 1048 out tokens · 20761 ms · 2026-05-24T08:40:58.664821+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages · 2 internal anchors

  1. [1]

    A Tutorial on Bayesian Optimization

    Peter I Frazier. A tutorial on bayesian optimization.arXiv preprint arXiv:1807.02811, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  2. [2]

    Adam: A Method for Stochastic Optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  3. [3]

    Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization.ACM Transactions on Mathematical Software (TOMS), 23(4):550–560, 1997

    Ciyou Zhu, Richard H Byrd, Peihuang Lu, and Jorge Nocedal. Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization.ACM Transactions on Mathematical Software (TOMS), 23(4):550–560, 1997

    work page 1997

  4. [4]

    Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.Journal of Global Optimization, 11(4):341–359, 1997

    Rainer Storn and Kenneth Price. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.Journal of Global Optimization, 11(4):341–359, 1997

    work page 1997

  5. [5]

    A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise.Journal of Basic Engineering, 86(1):97–106, 1964

    Harold J Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise.Journal of Basic Engineering, 86(1):97–106, 1964. 21

    work page 1964

  6. [6]

    Single-step bayesian search method for an extremum of functions of a single variable

    AG Žilinskas. Single-step bayesian search method for an extremum of functions of a single variable. Cybernetics, 11(1):160–166, 1975

    work page 1975

  7. [7]

    On bayesian methods for seeking the extremum

    Jonas Močkus. On bayesian methods for seeking the extremum. InOptimization Techniques IFIP Technical Conference: Novosibirsk, July 1–7, 1974, pages 400–404. Springer-Verlag, 1975

    work page 1974

  8. [8]

    Springer-Verlag, Dordrecht, 1st edition, 1989

    Jonas Močkus.Bayesian Approach to Global Optimization, volume 37 ofMathematics and Its Applications. Springer-Verlag, Dordrecht, 1st edition, 1989

    work page 1989

  9. [9]

    Efficient global optimization of expensive black-box functions.Journal of Global Optimization, 13(4):455, 1998

    Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions.Journal of Global Optimization, 13(4):455, 1998

    work page 1998

  10. [10]

    Practical bayesian optimization of machine learning algorithms

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25, 2012

    work page 2012

  11. [11]

    Taking the human out of the loop: A review of bayesian optimization.Proceedings of the IEEE, 104(1):148–175, 2015

    Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. Taking the human out of the loop: A review of bayesian optimization.Proceedings of the IEEE, 104(1):148–175, 2015

    work page 2015

  12. [12]

    Investigating bayesian optimization for expensive-to-evaluate black box functions: Application in fluid dynamics.Frontiers in Applied Mathematics and Statistics, 2022

    Mike Diessner, Joseph O’Connor, Andrew Wynn, Sylvain Laizet, Yu Guan, Kevin Wilson, and Richard D Whalley. Investigating bayesian optimization for expensive-to-evaluate black box functions: Application in fluid dynamics.Frontiers in Applied Mathematics and Statistics, 2022

    work page 2022

  13. [13]

    Optimisation and analysis of streamwise-varying wall-normal blowing in a turbulent boundary layer

    Joseph O’Connor, Mike Diessner, Kevin Wilson, Richard D Whalley, Andrew Wynn, and Sylvain Laizet. Optimisation and analysis of streamwise-varying wall-normal blowing in a turbulent boundary layer. Flow, Turbulence and Combustion, pages 1–29, 2023

    work page 2023

  14. [14]

    Reducing the skin-friction drag of a turbulent boundary-layer flow with low-amplitude wall-normal blowing within a bayesian optimization framework

    O A Mahfoze, A Moody, A Wynn, R D Whalley, and S Laizet. Reducing the skin-friction drag of a turbulent boundary-layer flow with low-amplitude wall-normal blowing within a bayesian optimization framework. Physical Review Fluids, 4(9):094601, 2019

    work page 2019

  15. [15]

    Bayesian optimization for chemical products and functional materials

    Ke Wang and Alexander W Dowling. Bayesian optimization for chemical products and functional materials. Current Opinion in Chemical Engineering, 36:100728, 2022

    work page 2022

  16. [16]

    Hyperparameter optimization for machine learning models based on bayesian optimization.Journal of Electronic Science and Technology, 17(1):26–40, 2019

    Jia Wu, Xiu-Yun Chen, Hao Zhang, Li-Dong Xiong, Hang Lei, and Si-Hao Deng. Hyperparameter optimization for machine learning models based on bayesian optimization.Journal of Electronic Science and Technology, 17(1):26–40, 2019

    work page 2019

  17. [17]

    Bananas: Bayesian optimization with neural architectures for neural architecture search

    Colin White, Willie Neiswanger, and Yash Savani. Bananas: Bayesian optimization with neural architectures for neural architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021

    work page 2021

  18. [18]

    The Journal of Open Source Software, 2(19):431, 2017

    José Jiménez and Josep Ginebra.pyGPGO: Bayesian optimization forPython. The Journal of Open Source Software, 2(19):431, 2017. Python package version 0.5.0

    work page 2017

  19. [19]

    bayes_opt: Open Source Constrained Global Optimization Tool forPython, 2014

    Fernando Nogueira. bayes_opt: Open Source Constrained Global Optimization Tool forPython, 2014. Python package version 1.4.3

    work page 2014

  20. [20]

    Spearmint Software, 2014

    Havard University, University of Toronto, Université de Sherbrooke, Socpra Sciences et Génie S.E.C. Spearmint Software, 2014. Python package version 0.1

    work page 2014

  21. [21]

    Python package version 2.0.0

    Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, René Sass, and Frank Hutter.SMAC3: A versatile bayesian optimization package for hyperparameter optimization.Journal of Machine Learning Research, 23(54):1–9, 2022. Python package version 2.0.0

    work page 2022

  22. [22]

    Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy.BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization

    Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy.BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. In Advances in Neural Information Processing Systems 33, 2020. Python package version 0.8.4

    work page 2020

  23. [23]

    GPyOpt: A Bayesian Optimization Framework inPython, 2016

    The GPyOpt authors. GPyOpt: A Bayesian Optimization Framework inPython, 2016. Python pack- age version 1.2.6

    work page 2016

  24. [24]

    Virtual library of simulation experiments: Test functions and datasets, n.d

    Sonja Surjanovic and Derek Bingham. Virtual library of simulation experiments: Test functions and datasets, n.d. Accessed: 2023-05-09

    work page 2023

  25. [25]

    R package ver- sion 1.2.1

    Yachen Yan.rBayesianOptimization: Bayesian Optimization of Hyperparameters, 2024. R package ver- sion 1.2.1

    work page 2024

  26. [26]

    ParBayesianOptimization: Parallel Bayesian Optimization of Hyperparameters, 2022

    Samuel Wilson. ParBayesianOptimization: Parallel Bayesian Optimization of Hyperparameters, 2022. R package version 1.2.6

    work page 2022

  27. [27]

    Maximizing acquisition functions for bayesian optimization

    James Wilson, Frank Hutter, and Marc Deisenroth. Maximizing acquisition functions for bayesian optimization. Advances in Neural Information Processing Systems, 31, 2018. 22

    work page 2018

  28. [28]

    Gramacy.Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences

    Robert B. Gramacy.Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC, Boca Raton, Florida, 1st edition, 2020.http://bobby.gramacy.com/ surrogates/

    work page 2020

  29. [29]

    A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.Technometrics, 21(2):239–245, 1979

    Michael D McKay, Richard J Beckman, and William J Conover. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.Technometrics, 21(2):239–245, 1979

    work page 1979

  30. [30]

    The MIT Press Cambridge, MA, 2006

    Carl Edward Rasmussen and Christopher K I Williams.Gaussian Processes for Machine Learning. The MIT Press Cambridge, MA, 2006

    work page 2006

  31. [31]

    Springer-Verlag, New York, 1st edition, 1996

    Radford M Neal.Bayesian Learning for Neural Networks, volume 118 ofLecture Notes in Statistics. Springer-Verlag, New York, 1st edition, 1996

    work page 1996

  32. [32]

    Gaussian process optimization in the bandit setting: No regret and experimental design

    Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seegre. Gaussian process optimization in the bandit setting: No regret and experimental design. InProceedings of the International Conference on Machine Learning, 2010, 2010

    work page 2010

  33. [33]

    Algorithm 733: Tomp–Fortran modules for optimal control calculations.ACM Transactions on Mathematical Software (TOMS), 20(3):262–281, 1994

    Dieter Kraft. Algorithm 733: Tomp–Fortran modules for optimal control calculations.ACM Transactions on Mathematical Software (TOMS), 20(3):262–281, 1994

    work page 1994

  34. [34]

    Advances in Neural Information Processing Systems, 32, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.PyTorch: An imperative style, high- performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019. Python package version 2.0.0

    work page 2019

  35. [35]

    GPyTorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration

    Jacob R Gardner, Geoff Pleiss, David Bindel, Kilian Q Weinberger, and Andrew Gordon Wilson. GPyTorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. InAdvances in Neural Information Processing Systems, 2018. Python package version 1.10

    work page 2018

  36. [36]

    Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J

    Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Mo...

    work page 2020

  37. [37]

    Harris, K

    Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matfthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Shep...

    work page 2020

  38. [38]

    Python package index - pypi, n.d

    Python Software Foundation. Python package index - pypi, n.d. Accessed: 2023-05-09

    work page 2023

  39. [39]

    pip: Package Installer forPython, 2023

    The pip developers. pip: Package Installer forPython, 2023. Python package version 22.3.1

    work page 2023

  40. [40]

    J. D. Hunter.Matplotlib: A 2d graphics environment.Computing in Science & Engineering, 9(3):90–95,

    work page

  41. [41]

    Python package version 3.9.0. 23

    work page