REVIEW 4 major objections 8 minor 39 references
Scalable Batch Bayesian Optimization Via Subspace Acquisition Functions
T0 review · 4 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper claims that batch Bayesian optimization scales to large batches by maximizing expected improvement inside random axis-aligned subspaces, and that the resulting points stay competitive with sequential EI and seven earlier batch…
desk verdict A simple subspace-sampling batch BO heuristic that genuinely helps at moderate batch sizes, but the large-q 'speedup' claim leans on an iteration axis that conflates parallelism with efficiency. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Expected SubSpace Improvement (ESSI) function, $$\mathrm{ESSI}(y) = (f_{\min} - \mu(z))\Phi\!\left(\frac{f_{\min} - \mu(z)}{\$\sigma$(z)}\right) + \$\sigma$(z)\varphi\!\left(\frac{f_{\min} - \mu(z)}{\$\sigma$(z)}\right),$$ where $z$ is obtained from the current best point $x_{\min}$ by overwriting the coordinates of the chosen axis-aligned subspace with $y$, and $\Phi$ and $\varphi$ are the standard normal cumulative distribution and density functions. This is exactly the expected-improvement formula, but confined to a slice of the design space through the current best point. The machinery works by solving $q$ such slice problems independently, one per randomly selected subspace, so a batch arises from diversity of subspaces rather than from penalizing or hallucinating points; the paper notes that $s=d$ recovers standard EI and $s=1$ recovers the expected coordinate improvement criterion. The random subspace selection, with dimension $s$ drawn uniformly from $1$ to $d$, is what keeps the method parameter-free and gives each acquisition subproblem an average dimension of $(d+1)/2$.
What would settle it
Run ESSI and sequential EI on rotated versions of separable benchmark problems, such as rotated Rosenbrock or rotated Ackley in 10 and 30 dimensions, using the same 512 additional evaluations and 30 repeats as the paper, and compare simple regret; if ESSI does not match or beat sequential EI on the majority of these rotated problems, or if its advantage disappears specifically when the optimum requires simultaneous movement of many coordinates, the central claim fails. A simpler check: on a problem whose minimum lies far from $x_{\min}$ along a direction not aligned with any single coordinate, ESSI's batch points all share $x_{\min}$ in most coordinates, so they should cluster near $x_{\min}$ and stall unless the subspaces eventually cover the needed direction.
Extended reading notes
Core claim
The central claim is that the bottleneck of batch Bayesian optimization is the acquisition side, not the search geometry: rather than optimizing one joint acquisition function over a $d\times q$-dimensional space, one can give each batch point its own low-dimensional improvement problem. For an axis-aligned subspace $Y$ of dimension $s$, the ESSI function is the usual expected-improvement expression evaluated at the point $z$ obtained by taking the current best solution $x_{\min}$ and replacing its $s$ subspace coordinates by the candidate coordinates $y$. Maximizing ESSI for $q$ randomly chosen subspaces yields $q$ different query points, each differing from $x_{\min}$ in only a few coordinates, and these points can be evaluated in parallel. The authors argue this is why the method scales: the average subspace dimension is $(d+1)/2$, so each inner acquisition problem is cheaper than the original $d$-dimensional EI problem, and the $q$ inner problems are independent and can be solved concurrently.
Load-bearing premise
The load-bearing premise is that improving expected improvement inside a randomly chosen slice of coordinates through the current best point yields points that improve the objective over the whole space, with no theoretical guarantee and the most risk on rotated or tightly coupled functions.
Editorial extensions
If this is right
- If the claim holds, batch Bayesian optimization no longer degrades as the batch reaches 128: ESSI reports better simple regret than seven batch EI baselines on 46-56 of the 58 CEC2017 problems at $q=128$, while beating sequential EI on 24 and matching it on 18.
- Wall-clock speedup becomes nearly linear when expensive evaluations dominate, because $q$ evaluations run in parallel and acquisition optimization time itself drops; the paper reports 32-73x speedups in acquisition optimization time on 10-dimensional problems at $q=128$.
- High-dimensional problems benefit more from the subspace decomposition, since each ESSI acquisition problem has dimension averaging $(d+1)/2$ rather than $d$; the authors observe larger advantages over competitors on 30-dimensional than on 10-dimensional problems.
- The method does not need artificial penalization, fake objective values, multi-objective solvers, or Monte Carlo batch integration, so it can be dropped into an existing EI-based Bayesian optimization loop with essentially no tuning beyond the batch size itself.
Reading between the lines
- A testable implication the paper does not address: because each batch point differs from $x_{\min}$ only in the coordinates of one random subspace, ESSI behaves like a randomized coordinate-descent explorer, so on landscapes whose optima require simultaneous coordinated changes of many variables, such as strongly rotated valleys, its advantage over full-dimensional EI should shrink or reverse.
- The subspace mechanism is acquisition-agnostic: the same draw-random-axis-aligned-subspaces-and-pin-the-rest construction could be applied to probability of improvement, lower confidence bound, or knowledge-gradient acquisition, producing batch variants by the same argument.
- One could isolate the source of the gains by comparing ESSI's random subspaces with fixed, length-scale-informed, or greedily selected subspaces; if random selection is already near-optimal, the decomposition itself is what matters, whereas sensitivity to subspace choice would point to tuning opportunities.
- Because the paper reports under-linear parallel speedups in acquisition optimization (about 1.6-2.7x on 30-dimensional problems at small to medium batch sizes), implementation overhead can dominate when objective evaluations are cheap; on many-core machines this suggests task scheduling or granularity changes are a natural next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ESSI (Expected SubSpace Improvement), a batch Bayesian optimization method. At each iteration, q axis-aligned subspaces are selected randomly; in each subspace, a candidate is found by maximizing an EI-like acquisition function in which all coordinates outside the subspace are pinned to the current best solution x_min. The resulting q points are evaluated in parallel. The method is tested on 58 CEC2017 benchmark problems (d=10 and d=30) with 30 runs, compared against sequential EI with q from 2 to 128 and against seven batch EI baselines at q=4, 16, and 128, with Wilcoxon signed-rank tests. Results are reported in Tables 1-4 and convergence curves in Figures 4, 6, 7, and 8.
Significance. If the claims were fully supported, the method would be a valuable contribution to batch BO: it is simple, has essentially no additional hyperparameters beyond the batch size, scales to q=128, and beats the included batch EI baselines on a majority of benchmark problems. The empirical program is extensive (58 problems, 30 runs, three batch sizes, statistical tests), and the paper provides a public Matlab implementation. However, the central claim of 'significant speedup' over sequential EI is overstated: the main evidence for it is plotted on an iteration-count axis, which conflates parallelism with per-evaluation sample efficiency. On the per-evaluation axis (Table 1), ESSI at q=128 is comparable to sequential EI rather than better. The subspace heuristic itself is reasonable but has no theoretical grounding, and the paper does not discuss when it could fail. The method's genuine strength is its competitive and often superior performance against other batch EI methods at the same batch size.
major comments (4)
- [Section 4.2, Figure 4, Table 1] The claim of 'significant speedup' over sequential EI is established on an iteration-count axis that conflates parallelism with per-evaluation efficiency. In Figure 4, the horizontal axis is 'number of iterations'; ESSI with q=128 performs 512 evaluations in only 4 iterations, whereas the q=1 EI baseline performs 4 evaluations in 4 iterations. A lower regret at iteration 4 therefore reflects the 128 evaluations already consumed, not improved sample efficiency. Table 1, which fixes the total number of evaluations at 512, is the correct per-evaluation comparison: at q=128, ESSI is significantly better than EI on 24/58 problems, significantly worse on 16/58, and similar on 18/58. The abstract's 'speedups the convergence significantly' and the Section 4.2 statement that 'the convergence speed of the proposed ESSI approach can be improved as we increase the batch size from 2 to 128' are not supported for large q on a per-evaluation basis. Please re-plot the sequential-EI comparison on an evaluation-count axis (or wall-clock time with q parallel workers) and revise the claims to separate parallel wall-clock speedup from sample efficiency.
- [Section 3.1, Eq. (9)] The ESSI acquisition function fixes all non-subspace coordinates at the current best x_min. Consequently, maximizing ESSI in a subspace is a heuristic that is guaranteed to improve only the lower-dimensional subspace, not necessarily the full-dimensional objective. The paper does not discuss conditions under which this heuristic could fail, such as strongly non-separable or rotated objectives where no axis-aligned subspace contains an improving direction from x_min. Since the CEC2017 test suite is composed of rotated and composition functions, the empirical success is nontrivial, but the absence of any analysis of this assumption is a gap. Please add a paragraph explaining the heuristic, its limitations, and the intuition (e.g., coordinate-descent-like behavior, random subspace coverage over iterations) for why it works in practice.
- [Section 4.3, q-EI implementation] The q-EI baseline is computed with only 1000 Monte Carlo samples for batch sizes up to q=128. In dimension d=30 with q=128, a 128-dimensional integral is approximated by 1000 samples; this estimator has high variance and is known to be biased for large q, which can disadvantage q-EI in the comparison. Please justify the MC sample count by reporting estimator variance (e.g., across different MC seeds) or by using a more accurate q-EI approximation (e.g., the analytical formula for small q and the Fq-EI decomposition for large q), and show that the ranking of methods is not sensitive to this choice.
- [Section 4.2, Figure 5] The acquisition optimization time comparison in Figure 5 measures the wall-clock time for ESSI with parallel parfor across q acquisition tasks versus sequential EI with one task per iteration. The text phrase 'the speedup of our proposed ESSI' attributes the reduction to the method itself, but a large part of the reduction comes from parallelizing q independent acquisition optimizations; any batch method with independent acquisition subproblems could obtain a similar speedup on the same hardware. The claim 'our proposed ESSI is able to reduce the acquisition optimization time significantly' should be rephrased to describe the parallel implementation, and the text should clarify that the linear speedup for expensive objective evaluation is a property of batch evaluation in general rather than specific to ESSI.
minor comments (8)
- [Section 4.3] The text says 'We also use Wilson signed rank test' where it should be 'Wilcoxon signed rank test'; the same typo appears in the captions of Tables 2-4. Section 4.2 uses the correct spelling.
- [Section 3.3 / Algorithm 2] Section 3.3 suggests eliminating duplicate subspaces by checking after each selection, but Algorithm 2 (Step 4-8) does not include such a check. Please either implement the rejection step in the algorithm or explicitly state that duplicate subspaces are allowed and quantify how often they occur.
- [Section 4.1] Minor wording: 'the number of acquisition samples is set 512' should be 'set to 512', and 'the maximal number of generation' should be 'generations'.
- [Abstract and Section 5] The abstract uses 'speedups the convergence'; this should be 'speeds up the convergence'. Also, Section 5's claim of 'no additional parameter' should be qualified, since the batch size q is itself a parameter, as are the GA population size and generation limit used for acquisition optimization.
- [Section 4.2, Figure 5] In Figure 5, the numeric values on the bars are not labeled with units in the bar chart itself; the y-axis is already labeled 'time in seconds', but the reader must infer that the numeric labels are seconds. Please make the unit explicit in the figure or caption.
- [Section 3.1, Eq. (9)] The definition of z in Eq. (9) is informal; writing it as a vector with the subspace coordinates inserted at their original positions would remove ambiguity about the ordering of coordinates in the Gaussian process input.
- [Section 4, general] The convergence curves in Figures 4, 6, 7, and 8 show medians and quartiles, but the corresponding simple regret tables report only the average of 30 runs. Reporting the median (or both) in the tables would make the tables and figures consistent.
- [References] Reference [33] is to the authors' own ECI work, which is disclosed in the text as the s=1 special case of ESSI; this is appropriate, but it should be mentioned explicitly that ECI is used as a self-comparison baseline in Section 3.1.
Circularity Check
No significant circularity: ESSI is an independently defined acquisition rule validated on external benchmarks.
full rationale
The derivation chain is self-contained. ESSI in Eq. (9) is defined as the standard closed-form EI of Eq. (8) evaluated at the point z obtained by replacing the coordinates of the current best solution xmin with the subspace coordinates y. This is a definition, not a fitted or derived prediction: the batch points are then produced by maximizing these q independent ESSI functions (Algorithm 2), and the central claim is tested against the external CEC 2017 benchmark suite with fixed evaluation budgets and statistical tests. There is no parameter fitted to the benchmark results and no quantity is renamed as a prediction. The only self-citation, ECI [33], is explicitly disclosed as the s=1 special case of ESSI and is not used to justify the method; the other self-citations ([15], [32]) name the authors' earlier methods only as comparators or reviews. The heuristic nature of the subspace-pinning assumption in Eq. (9) is a correctness risk, not a circularity, and the iteration-count versus evaluation-count framing in Figure 4 is a presentation issue, not a circular reduction.
Assumptions & free parameters
free parameters (6)
- batch size q =
2, 4, 8, 16, 32, 64, 128
- GA population size =
10d
- GA generation limit =
100
- initial sample count =
10d
- q-EI Monte Carlo samples =
1000
- MSMR length-scale count =
2q
assumptions (6)
- domain assumption The objective function is a realization of a Gaussian process with constant mean and squared exponential kernel; hyperparameters are estimated by maximum likelihood (Section 2.1).
- domain assumption The objective is single-objective, bound-constrained, deterministic, and expensive to evaluate (Section 2, Eq. 1).
- domain assumption CEC2017 benchmark functions are representative of expensive black-box optimization problems (Section 4.1).
- ad hoc to paper Random axis-aligned subspace selection with uniform s in {1,...,d} is a reasonable exploration strategy (Section 3.3).
- domain assumption The EI acquisition function is the right building block for the proposed ESSI (Section 2.2).
- standard math Wilcoxon signed-rank test assumptions hold for the 30 independent runs (Section 4.1).
Cite this review
Pith. "Pith review of Scalable Batch Bayesian Optimization Via Subspace Acquisition Functions." pith.science (2026). https://pith.science/paper/4KECLBQX
@misc{pith2026241116206,
author = {Pith},
title = {Pith review of: Scalable Batch Bayesian Optimization Via Subspace Acquisition Functions},
year = {2026},
howpublished = {\url{https://pith.science/paper/4KECLBQX}},
note = {Machine review of arXiv:2411.16206}
}
read the original abstract
Extending Bayesian optimization to batch evaluation can enable the designer to make the most use of parallel computing technology. However, most of current batch approaches do not scale well with the batch size. That is, their optimization efficiencies often deteriorate as the batch size increases. To address this issue, we propose a simple and efficient approach to extend Bayesian optimization to large-scale batch evaluation in this work. Different from existing batch approaches, the idea of the new approach is to draw a batch of axis-aligned subspaces of the original problem and select one point from each subspace using existing acquisition functions. Numerical experiments show that our proposed approach speedups the convergence significantly when compared with the sequential Bayesian optimization algorithm, and performs very competitively when compared with ten batch Bayesian optimization algorithms. The implementation of our proposed approach is available at https://github.com/zhandawei/SubSpace_Acquisition_Functions.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
J. Mockus, Application of bayesian approach to numerical methods of global and stochastic optimization, Journal of Global Optimization 4 (1994) 347–365
work page 1994
-
[2]
D. R. Jones, M. Schonlau, W. J. Welch, Efficient global optimization of expensive black-box functions, Journal of Global Optimization 13 (1998) 455–492. 29
work page 1998
- [3]
-
[4]
D. J. Lizotte, T. Wang, M. H. Bowling, D. Schuurmans, Automatic gait optimization with gaussian process regression, in: Proceedings of the 20th international joint conference on Artifical intelligence, volume 7, 2007, pp. 944–949
work page 2007
-
[5]
R. Lam, M. Poloczek, P. Frazier, K. E. Willcox, Advances in bayesian optimization with applications in aerospace engineering, in: 2018 AIAA Non-Deterministic Approaches Conference, 2018, p. 1656
work page 2018
-
[6]
B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, N. d. Freitas, Taking the human out of the loop: A review of bayesian optimization, Proceedings of the IEEE 104 (2016) 148–175. doi:10.1109/JPROC.2015.2494218
arXiv 2016
-
[7]
X. Wang, Y. Jin, S. Schmitt, M. Olhofer, Recent advances in bayesian optimization, ACM Computing Surveys 55 (2023) Article 287. doi:10.1145/3582078
doi:10.1145/3582078 2023
-
[8]
H. Kushner, A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise, Journal of Basic Engineering 86 (1964) 97–106
work page 1964
Show all 39 references
-
[9]
D. D. Cox, S. John, A statistical method for global optimization, in: IEEE interna- tional conference on systems, man, and cybernetics, 1997, pp. 1241–1246
1997
-
[10]
Schonlau, Computer experiments and global optimization, Thesis, University of Waterloo, 1997
M. Schonlau, Computer experiments and global optimization, Thesis, University of Waterloo, 1997
1997
-
[11]
Ginsbourger, R
D. Ginsbourger, R. Le Riche, L. Carraro, Kriging Is Well-Suited to Parallelize Op- timization, volume 2 of Adaptation Learning and Optimization , Springer Berlin Heidelberg, 2010, pp. 131–162. doi:10.1007/978-3-642-10701-6_6
2010 doi
-
[12]
Chevalier, D
C. Chevalier, D. Ginsbourger, Fast Computation of the Multi-Points Expected Im- provement with Applications in Batch Selection, volume 7997 of Lecture Notes in Computer Science , Springer Berlin Heidelberg, 2013, pp. 59–69. doi: 10.1007/ 978-3-642-44973-4_7
2013
-
[13]
Marmin, C
S. Marmin, C. Chevalier, D. Ginsbourger, Differentiating the multipoint expected improvement for optimal batch design, in: P. Pardalos, M. Pavone, G. M. Farinella, V. Cutello (Eds.), International Workshop on Machine Learning, Optimization and Big Data, volume 9432 ofLecture N...
2015
-
[14]
J. Wang, S. C. Clark, E. Liu, P. I. Frazier, Parallel bayesian global optimization of expensive functions, Operations Research 68 (2020) 1850–1865
2020
-
[15]
D. Zhan, Y. Meng, H. Xing, A fast multipoint expected improvement for parallel expensive optimization, IEEE Transactions on Evolutionary Computation 27 (2023) 170–184. doi:10.1109/TEVC.2022.3168060
2023
-
[16]
Gonzalez, Z
J. Gonzalez, Z. Dai, P. Hennig, N. Lawrence, Batch bayesian optimization via local penalization, in: Artificial Intelligence and Statistics, 2016, pp. 648–657
2016
-
[17]
Z. Li, S. Ruan, J. Gu, X. Wang, C. Shen, Investigation on parallel algorithms in efficient global optimization based on multiple points infill criterion and domain decomposition, Structural and Multidisciplinary Optimization 54 (2016) 747–773. doi:10.1007/s00158-016-1441-2
2016 doi
-
[18]
D. Zhan, J. Qian, Y. Cheng, Pseudo expected improvement criterion for parallel ego algorithm, Journal of Global Optimization 68 (2017) 641–662. doi: 10.1007/ s10898-016-0484-7
2017
-
[19]
Bischl, S
B. Bischl, S. Wessing, N. Bauer, K. Friedrichs, C. Weihs, MOI-MBO: Multi- objective Infill for Parallel Model-Based Optimization, volume 8426 of Lecture Notes in Computer Science , Springer International Publishing, 2014, pp. 173–186. doi:10.1007/978-3-319-09584-4_17
2014 doi
-
[20]
Z. W. Feng, Q. B. Zhang, Q. F. Zhang, Q. G. Tang, T. Yang, Y. Ma, A multiobjective optimization based framework to balance the global exploration and local exploitation in expensive optimization, Journal of Global Optimization 61 (2015) 677–694. doi:10.1007/s10898-014-0210-2
2015 doi
-
[21]
Zhang, H
Q. Zhang, H. Li, MOEA/D: A multiobjective evolutionary algorithm based on decomposition, IEEE Transactions on Evolutionary Computation 11 (2007) 712–
2007
-
[22]
W. Lyu, F. Yang, C. Yan, D. Zhou, X. Zeng, Batch bayesian optimization via multi- objective acquisition ensemble for automated analog circuit design, in: International Conference on Machine Learning, 2018, pp. 3312–3320
2018
-
[23]
Robi ˇc, B
T. Robi ˇc, B. Filipi ˇc, DEMO: Differential evolution for multiobjective optimization, in: International Conference on Evolutionary Multi-Criterion Optimization, 2005, pp. 520–533. 31
2005
-
[24]
J. Chen, F. Luo, G. Li, Z. Wang, Batch bayesian optimization with adaptive batch acquisition functions via multi-objective optimization, Swarm and Evolution- ary Computation 79 (2023) 101293. doi: https://doi.org/10.1016/j.swevo. 2023.101293
2023
-
[25]
Y. Jin, H. Wang, T. Chugh, D. Guo, K. Miettinen, Data-driven evolutionary optimiza- tion: An overview and case studies, IEEE Transactions on Evolutionary Computation 23 (2019) 442–458. doi:10.1109/TEVC.2018.2869001
2019
-
[26]
P. I. Frazier, A tutorial on bayesian optimization, 2018. URL: https://arxiv.org/abs/ 1807.02811. arXiv:1807.02811
2018 arXiv
-
[27]
C. E. Rasmussen, C. K. Williams, Gaussian processes for machine learning, The MIT Press, Cambridge, MA, USA, 2006
2006
-
[28]
P. I. Frazier, W. B. Powell, S. Dayanik, A knowledge-gradient policy for sequential information collection, SIAM Journal on Control and Optimization 47 (2008) 2410–
2008
-
[29]
Frazier, W
P. Frazier, W. Powell, S. Dayanik, The knowledge-gradient policy for correlated normal beliefs, INFORMS Journal on Computing 21 (2009) 599–613. doi: 10. 1287/ijoc.1080.0314
2009
-
[30]
Hennig, C
P. Hennig, C. J. Schuler, Entropy search for information-efficient global optimization, Journal of Machine Learning Research 13 (2012) 1809–1837
2012
-
[31]
J. M. Hern ´andez-Lobato, M. W. Hoffman, Z. Ghahramani, Predictive entropy search for efficient global optimization of black-box functions, in: Advances in neural information processing systems, volume 27, 2014
2014
-
[32]
D. Zhan, H. Xing, Expected improvement for expensive optimization: a review, Journal of Global Optimization 78 (2020) 507–544. doi: 10.1007/ s10898-020-00923-x
2020
-
[33]
Zhan, Expected coordinate improvement for high-dimensional bayesian opti- mization, Swarm and Evolutionary Computation 91 (2024) 101745
D. Zhan, Expected coordinate improvement for high-dimensional bayesian opti- mization, Swarm and Evolutionary Computation 91 (2024) 101745. doi: https: //doi.org/10.1016/j.swevo.2024.101745
2024
-
[34]
N. Awad, M. Ali, J. Liang, B. Qu, P. Suganthan, Problem definitions and evaluation criteria for the CEC special session and competition on single objective real-parameter 32 numerical optimization, Technical Report, Nanyang Technological University, Singa- pore, Jordan Univers...
2017
-
[35]
T. T. Joy, S. Rana, S. Gupta, S. Venkatesh, Batch bayesian optimization using multi- scale search, Knowledge-Based Systems 187 (2020) 104818. doi: https://doi. org/10.1016/j.knosys.2019.06.026
2020 doi
-
[36]
K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: Nsga-ii, IEEE Transactions on Evolutionary Computation 6 (2002) 182–
2002
-
[197]
doi:10.1109/4235.996017. 33
-
[731]
doi:10.1109/tevc.2007.892759
2007
-
[2439]
doi:10.1137/070693424
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.