Pith. sign in

REVIEW 3 major objections 5 minor 58 references

High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read High-dimensional Bayesian optimization can be reduced to a low-dimensional search plus a manifold back-projection.

desk verdict Promising BO recipe undermined by a false proof of its central theorem; the empirics are worth a serious revision. read the letter →

arxiv 2412.16554 v1 pith:JJ3HEK7J submitted 2024-12-21 cs.LG stat.ML

Quoc-Anh Hoang Nguyen , The Hung Tran This is my paper · ORCID
classification cs.LGstat.ML
keywords Bayesianoptimizationhigh-dimensionalrandomprojectionmanifoldlearningsemi-supervisedeffectivedimensionalityGaussianprocessback-projection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that high-dimensional Bayesian optimization becomes tractable when the objective function depends on a low-dimensional manifold: one random orthogonal matrix and one learned projection onto the manifold are enough to optimize in a low-dimensional space and map the chosen point back to the ambient space. The load-bearing identity is that the image of the manifold under the random projection equals the set of back-projections $A P_M(A^T q)$, so the acquisition function can be maximized over $q$ in a box. A semi-supervised consistency loss trains the projection $h$ when the manifold geometry is unknown, reducing the overfitting that plagues encoder-decoder approaches with few labeled points. Empirically, the method outperforms eight high-dimensional Bayesian optimization baselines on synthetic functions with spherical, mixed, and linear latent manifolds, and on Lasso and MuJoCo tasks.

What carries the argument

The central objects are the random orthogonal matrix $A \in \mathbb{R}^{m \times D}$ and the manifold projection $P_M$, approximated by a learned mapping $h$. The identity $M_{\mathcal{A}} = \{A P_M(A^T q)\}$ is the mechanism that turns the intractable acquisition optimization over the projected manifold into an unconstrained box search in $\mathbb{R}^m$ followed by the explicit back-projection $h(A^T z)$. The semi-supervised consistency loss $L_{us} = \frac{1}{pq}\sum_{j,i} \|h(\lambda_j x'_i + (1-\lambda_j)h(x'_i)) - h(x'_i)\|^2$ enforces that $h$ behaves like an orthogonal projection, which is the property used for the back-projection to land on $M$.

What would settle it

Take the mixed manifold $T^{2d_1} \times \mathbb{R}^{d_2}$ with $d_1=5$, $d_2=10$ in $\mathbb{R}^{500}$, set $m=15$, draw $A$ randomly, and sample points $x \in M$; check numerically whether for each $A x$ there is a $q$ with $A x = A P_M(A^T q)$ within box bounds. If any sampled point has no such $q$, the equality $M_{\mathcal{A}} = M_{\mathcal{A}}'$ fails and the low-dimensional acquisition provably covers only a subset of the projected manifold.

Watch

Extended reading notes

Core claim

Theorem 1 and Theorem 2 establish that, with probability 1, for a $d$-dimensional manifold $M$ embedded in $\mathbb{R}^D$ that can be embedded in $\mathbb{R}^m$, the set $M_{\mathcal{A}} = \{A x : x \in M\}$ equals $\{A P_M(A^T q) : q \in \mathbb{R}^m\}$. Therefore, instead of solving the acquisition maximization over the complicated domain $M_{\mathcal{A}}$, one can maximize $EI(A h(A^T q))$ over a box and set the next query to $x_{n+1} = h(A^T z_{n+1})$. The paper also shows that the low-dimensional Gaussian process regressor converges at the optimal posterior contraction rate $n^{-s/(2s+d)}$ and that the random projection compresses noise of near-manifold points.

Load-bearing premise

The proof relies on the premise that a manifold that can be folded into $m$ dimensions always lies entirely inside some $m$-dimensional flat plane of the original space; embeddability does not guarantee this.

Editorial extensions

If this is right

  • Acquisition optimization runs in $\mathbb{R}^m$ with $m \ll D$, so the per-iteration cost no longer scales with the ambient dimension.
  • The explicit back-projection $h(A^T z)$ replaces the decoder-network reconstruction used by VAE-based Bayesian optimization approaches, simplifying the pipeline.
  • When the manifold geometry is known (sphere, linear subspace), a closed-form $h$ gives lower test loss and needs no unlabeled data.
  • Training $h$ with the semi-supervised consistency loss reduces overfitting when few labeled points are available (10–100 labels).
  • On synthetic Ackley, Rotated Hyper-Ellipsoid, and Levy functions with latent dimensions 10–15, the method reaches lower function values than the eight baselines for $D \in \{500, 1000, 1500\}$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: The equality $M_{\mathcal{A}} = \{A P_M(A^T q)\}$ should be tested numerically on manifolds whose affine span exceeds $m$; the paper's own mixed manifold ($2d_1+d_2=20$, $m=15$) is a direct test case.
  • Extension: If coverage fails for such manifolds, a practical remedy is to increase $m$ to the affine span dimension or to add a second random projection restricted to the orthogonal complement.
  • Extension: The consistency loss could be strengthened with an explicit idempotence penalty $\|h(h(x))-h(x)\|^2$, which would make the learned $h$ closer to a true projection and tighten the approximation $h \approx P_M$.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RPM-BO, a high-dimensional Bayesian optimization algorithm for objective functions whose effective dependence lies on a low-dimensional manifold. The method combines a random orthogonal projection A from R^D to R^m with a learned feature map h (analytic when the manifold geometry is known, otherwise a neural network trained with a semi-supervised consistency loss). A Gaussian process surrogate is constructed on the projected points Ah(x), and the acquisition function is optimized in R^m with a back-projection step h(A^T z). The authors claim theoretical support via Theorems 1 and 2, which are supposed to show that optimizing EI in the projected space is equivalent to optimizing over the projected manifold, and they report empirical improvements over several baselines on synthetic and real benchmarks.

Significance. If the theoretical claims were correct, the back-projection scheme would be a valuable contribution because it avoids expensive decoder training and provides a simple way to map low-dimensional acquisition candidates back to the original space. The empirical study is broad, covering multiple synthetic manifolds and real applications with dimensions up to 1500, and the authors make their code available. The semi-supervised consistency loss is a reasonable practical idea for regularizing the feature map with unlabeled data. However, the central theoretical result, Theorem 1, has an invalid proof and appears false in the general setting claimed; since Theorem 2 and the acquisition optimization in Algorithm 1 rely directly on Theorem 1, the advertised theoretical support is not established.

major comments (3)
  1. [Appendix A, proof of Theorem 1] The proof contains a non-sequitur: from "M can be embedded in Euclidean space R^m" it concludes "therefore, there exists an m-dimensional affine space L ⊂ R^D such that M ⊂ L." Embeddability is an intrinsic property and does not imply containment in an affine subspace of the ambient space; for instance, a 1-dimensional helix in R^3 can be embedded in R^2 but is not contained in any 2-dimensional affine subspace. This invalidates the proof of Theorem 1. Consequently, Theorem 2, which relies on Theorem 1, is also not proven, and the equality M_A = \bar{M}_A that justifies optimizing EI(Ah(A^T z)) over z ∈ [-√m, √m]^m in Algorithm 1 (lines 9-10) is not established. Moreover, the paper's own mixed-manifold experiments (Section 4.1, Eq. (12), with d1=5, d2=10) have affine span 2d1+d2=20 in R^D, while the experiments set m=15; thus the proof's required affine-subspace condition is not satisfied in the main nonlinear experiments.
  2. [Appendix B.3] The claimed convergence rate O(n^{-1/m}) for RPM-BO is asserted rather than derived. The text states that "if we assume that the feature mapping h is also accurately learned, then the convergence rate of our proposed BO algorithm is equivalent to the convergence rate of a BO algorithm in low dimension m," and that the proof can be done similarly to [51]. This is not a proof, and the setting here differs from [51] in essential ways: h is learned from data, the acquisition optimization is performed in R^m and then back-projected, and the surrogate is built on projected points Ah(x). No analysis is given for the effect of approximation error in h, the mismatch between h(A^T z) and A P_M(A^T z), or the finite-sample behavior of the learned lengthscale parameter a. The claim that the algorithm "achieves the optimal posterior contraction rate" is therefore unsupported.
  3. [Section 3.3, Eq. (11)] The derivation of the acquisition optimization relies on the assumption that "h approximates P_M" and uses the approximation symbol to replace P_M by h. The paper does not provide a quantitative statement of this approximation, nor does it show that the consistency loss in Eq. (7) drives h toward P_M in any controlled way. In fact, Proposition 1 characterizes P_M, but the loss only encourages h to be invariant on line segments between x' and h(x'); it does not enforce that h(x) lies on M or that h equals the orthogonal projection. This leaves a gap between the theory and the actual algorithm, and the experiments do not verify that h is close to P_M for the neural-network cases (e.g., the mixed-manifold and MuJoCo experiments).
minor comments (5)
  1. [Eq. (5)] The spherical projection formula is undefined when the denominator ||B(B^T x - c)||_2 is zero; the paper does not discuss this edge case or how it is handled in the implementation.
  2. [Algorithm 1, line 10] The back-projection step is written as x_{n+1} = h(A^T z_{n+1}), which is consistent with using h as a replacement for P_M, but the relationship to Theorem 2's exact expression A P_M(A^T q) should be stated explicitly to avoid confusion.
  3. [Section 3.4] The discussion of the choice of m says that if m ≥ 2d then every d-dimensional manifold can be embedded in R^m (Whitney), while the random projection condition in Eq. (9) suggests m = O(d log D). The relationship between these two requirements is not clarified; the experiments set m = 15 for d = 15, which does not satisfy m ≥ 2d.
  4. [Appendix D.4] The sentence "We will show that training with semi-supervised loss function in Equation (8) can reduce the overfitting issue in Appendix D.4" is worded as a claim of a proof, but Appendix D.4 presents only a toy empirical illustration. The wording should be softened to "provide empirical evidence".
  5. [References] The reference [48] appears in the text as "Tran-The, H., Gupta, S., Rana, S., Venkatesh, S.: Regret bounds for expected improvement algorithms in Gaussian process bandit optimization." This is an odd citation for the noise-affected convergence rate; the authors should verify the precise statement and ensure the cited result supports the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RPM-BO's back-projection theorem, consistency loss, and convergence claims rest on external geometric/statistical results and the paper's own conditional derivations; the Appendix A proof gap is a correctness concern, not input-output circularity.

full rationale

The derivation chain contains no step in which a predicted quantity is equal by construction to a fitted input, and no load-bearing premise is justified only by a self-citation. Theorem 2 is a set-theoretic consequence of Theorem 1, and Theorem 1 is attempted in Appendix A using Lemma 1 and an affine-space argument. Even if that proof is mathematically invalid because embeddability in R^m does not imply containment in an m-dimensional affine subspace, the invalidity is a correctness or theoretical-support gap, not circularity: the theorem's conclusion is not assumed in its premises. The consistency loss in Eq. (7) is justified by Proposition 1 from Leobacher and Steinicke [27], an external geometric result, and the semi-supervised training is then evaluated against held-out loss in Appendix D.4 rather than being used as its own evidence. The convergence rates in Appendix B quote Yang and Dunson [54] and Guhaniyogi and Dunson [17], and the noisy-case EI rate quotes Tran-The et al. [48], which is a self-citation; however, it is invoked as a ready-made external regret bound, not as an input to the present model fit, so it does not make the argument circular. The empirical comparisons are against external baselines and do not claim to predict a fitted value. The proof gap in Appendix A should be flagged as a correctness risk because the stated support for Theorem 1 does not cover the paper's own mixed-manifold experiments where the affine span is 20 while m = 15, but this does not rise to circularity under the definitions used here.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The method relies on the manifold hypothesis, on the assumed validity of the projection property used in the consistency loss, on the Baraniuk-Wakin distance-preservation theorem, and on an unstated affine-subspace condition in Theorem 1. Hyperparameters (m, gamma, p, q, prior shape) are chosen by hand rather than fitted to the objective.

free parameters (5)
  • m (projection dimension) = 10, 15, 20 in experiments (D.2)
    Chosen by hand; the theory requires m >= dim(affine span of M) for Theorem 1 to hold, but experiments use m equal to intrinsic manifold dimension d in several cases.
  • gamma (consistency loss weight) = 1
    Set in Appendix C.2; balances supervised and unsupervised losses.
  • p, q (consistency loss sizes) = p=5, q=100
    Number of interpolation coefficients and unlabeled points; chosen without tuning.
  • a0, b0 (gamma prior on lengthscale) = a0=1, b0=0.15
    Prior for GP lengthscale; standard choices, but they affect the GP fit.
  • NN architecture = single hidden layer, 35 units, ReLU
    Used when geometry is unknown; no architecture search reported.
assumptions (6)
  • domain assumption The objective function has an effective low-dimensional manifold M embedded in R^D.
    Section 2 states this as the core assumption of the paper.
  • domain assumption The orthogonal projection P_M exists and is unique on the search space X.
    Section 2 assumes |S_M(x)| = 1 for all x to define the projection.
  • ad hoc to paper If M can be embedded in R^m, then M is contained in an m-dimensional affine subspace L of R^D.
    Appendix A, proof of Theorem 1: 'Therefore, ∃ a m-dimensional affine space L ⊂ R^D such that M ⊂ L'. This is not implied by embeddability; it is the actual condition needed, and it fails when dim(affine span) > m.
  • domain assumption A random projection A with m = O(d log(D)) preserves pairwise distances on the manifold and is a diffeomorphism onto its image.
    Section 3.2 and Appendix B.1 quote Baraniuk-Wakin; conditions on compactness and condition number are not verified for the NN-learned mappings.
  • domain assumption The Yang-Dunson posterior contraction theorem applies to the low-dimensional GP on projected data.
    Theorem 4 in Appendix B.1 is quoted from [54]; conditions (C^{gamma1} manifold, C^{gamma2} diffeomorphism, g in C^s) are assumed to hold.
  • standard math Proposition 1: the orthogonal projection of any point on the segment between x and its projection x_M is x_M.
    Quoted from [27] and used to design the consistency loss in Equation (7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces." pith.science (2026). https://pith.science/paper/JJ3HEK7J

@misc{pith2026241216554,
  author       = {Pith},
  title        = {Pith review of: High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JJ3HEK7J}},
  note         = {Machine review of arXiv:2412.16554}
}
read the original abstract

Bayesian Optimization (BO) is a popular approach to optimizing expensive-to-evaluate black-box functions. Despite the success of BO, its performance may decrease exponentially as the dimensionality increases. A common framework to tackle this problem is to assume that the objective function depends on a limited set of features that lie on a low-dimensional manifold embedded in the high-dimensional ambient space. The latent space can be linear or more generally nonlinear. To learn feature mapping, existing works usually use an encode-decoder framework which is either computationally expensive or susceptible to overfittting when the labeled data is limited. This paper proposes a new approach for BO in high dimensions by exploiting a new representation of the objective function. Our approach combines a random linear projection to reduce the dimensionality, with a representation learning of the nonlinear manifold. When the geometry of the latent manifold is available, a solution to exploit this geometry is proposed for representation learning. In contrast, we use a neural network. To mitigate overfitting by using the neural network, we train the feature mapping in a geometry-aware semi-supervised manner. Our approach enables efficient optimizing of BO's acquisition function in the low-dimensional space, with the advantage of projecting back to the original high-dimensional space compared to existing works in the same setting. Finally, we show empirically that our algorithm outperforms other high-dimensional BO baselines in various synthetic functions and real applications.

Figures

Figures reproduced from arXiv: 2412.16554 by the authors.

Figure 1
Figure 1. Left: The spherical 2−dimensional manifold S 2 embedded in R 3 . Right: The mixed 2−dimensional manifold M2 embedded in R 3 with (x, y) ∈ T 1 and z ∈ R . types we can easily exploit are spherical geometry, linear geometry, etc. For the linear manifold, which is the most used assumption in the existing works, we can construct the mapping h as: h(x) = hB(x) = BBT x (3) where column space of matrix B ∈ R D×d is a basis… view at source ↗
Figure 2
Figure 2. Performances on two standard functions with effective spherical manifold for 500, 1000, and 1500 input dimensions. For all cases, the dimension of the effective manifold is 10. The y−axis presents the value function (A smaller value is better). REMBO [51]; HeSBO [33]; SIRBO [55]; SILBO [9] ; VAE-BO [35]; TuRBO [14]; random search method [3] and SAASBO [13]. We use Expected Improvement as an acquisition function. For… view at source ↗
Figure 3
Figure 3. Performances on two standard functions with effective mixed-manifold for 500, 1000, and 1500 input dimensions. For all cases, the dimension of the effective manifold is 15. The y−axis presents the value function (A smaller value is better) [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Performance on three real applications. where    z2i−1 = √ x2i−1 x 2 2i−1+x 2 2i ∀i = 1, d1 z2i = √ x2i x 2 2i−1+x 2 2i ∀i = 1, d1 zi = xi ∀i = (d1 + 1), d2 The target function f depends only on the first (2d1 +d2) dimensions. However, the first 2d1 elements lie…
Figure 5
Figure 5. Figure 5: Performance on Ackley function with effective 15-dimensional mix manifold M for varying projection dimension m. D.3 Runtime experiment We measure the runtime of RPM-BO and each baseline method on the Ackley Mix 1000D test problem. We run each method for 300 evaluations…
Figure 6
Figure 6. Figure 6: Performances on two standard functions with effective linear manifold for 500, 1000, and 1500 input dimensions. For all cases, the dimension of an effective manifold is 10. The y-axis presents the regret spherical case, RPM-BO outperforms the other baseline within 300 …
Figure 7
Figure 7. Figure 7: Performances on Levy function with effective sphere manifold for 500, 1000, 1500 input dimensions. For all cases, the dimension of an effective manifold is 10. The y-axis presents the function vale 0 50 100 150 200 250 300 Evaluations 1.0 1.5 2.0 2.5 3.0 Function Value…
Figure 8
Figure 8. Figure 8: Performances on Levy function with effective mix manifold for 500, 1000, 1500 input dimensions. For all cases, the dimension of the effective manifold is 15. The y-axis presents the function value functions defined as: fAckley(x) = −20 exp  −0.2 sPd i=1 x 2 i d   −…
Figure 9
Figure 9. Figure 9: Performance on 2 functions with 2 types of latent effective manifold. dimension for all methods as 15 and also m = 15. The result is shown in [PITH_FULL_IMAGE:figures/full_fig_p030_9.png]
Figure 10
Figure 10. Figure 10: Performances on MuJoCo humanoid experiments. The y-axis show the nega￾tive reward (the lower is better) [PITH_FULL_IMAGE:figures/full_fig_p030_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 51 canonical work pages

  1. [51]

    Wang, Z., Zoghi, M., Hutter, F., Matheson, D., de Freitas, N.: Bayesian opti- mization in a billion dimensions via random embeddings. J. Artif. Intell. Res.55, 361–387 (2016)

    work page 2016

  2. [1]

    In: Advances in Neural Information Processing Systems 33 (2020)

    Balandat, M., Karrer, B., Jiang, D.R., Daulton, S., Letham, B., Wilson, A.G., Bak- shy, E.: BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. In: Advances in Neural Information Processing Systems 33 (2020)

    work page 2020

  3. [2]

    Foundations of Computational Mathematics9, 51–77 (2009)

    Baraniuk, R., Wakin, M.B.: Random projections of smooth manifolds. Foundations of Computational Mathematics9, 51–77 (2009)

    work page 2009

  4. [3]

    Journal of Machine Learning Research13(10), 281–305 (2012)

    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. Journal of Machine Learning Research13(10), 281–305 (2012)

    work page 2012

  5. [4]

    In: Advances in Neural Information Processing Systems 32, 2019

    Berthelot, D., Carlini, N., Goodfellow, I.J., Papernot, N., Oliver, A., Raffel, C.: Mixmatch: A holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems 32, 2019. pp. 5050–5060 (2019)

    work page 2019

  6. [5]

    In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H

    Borovitskiy, V., Terenin, A., Mostowsky, P., Deisenroth, M.P.: Matérn gaussian processes on riemannian manifolds. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual (2020)

    work page 2020

  7. [6]

    Calandra, R., Peters, J., Rasmussen, C., Deisenroth, M.: Manifold gaussian pro- cesses for regression (11 2016).https://doi.org/10.1109/IJCNN.2016.7727626

    arXiv 2016

  8. [7]

    IEEE Robotics & Automation Magazine27, 33–45 (2019) 16 Nguyen et al

    Calinon, S.: Gaussians on riemannian manifolds: Applications for robot learning and adaptive control. IEEE Robotics & Automation Magazine27, 33–45 (2019) 16 Nguyen et al

    work page 2019

Show all 58 references
  1. [8]

    Bulletin of the American Mathematical Society 46, 255–308 (2009)

    Carlsson, G.E.: Topology and data. Bulletin of the American Mathematical Society 46, 255–308 (2009)

    2009

  2. [9]

    arXiv preprint arXiv:2005.14601 (2020)

    Chen, J., Zhu, G., Yuan, C., Huang, Y.: Semi-supervised embedding learning for high-dimensional bayesian optimization. arXiv preprint arXiv:2005.14601 (2020)

    2020 arXiv

  3. [10]

    Chikuse, Y.: Statistics on special manifolds (2003)

    2003

  4. [11]

    In: NeurIPS (2022)

    Dai, Z., Shu, Y., Low, B.K.H., Jaillet, P.: Sample-then-optimize batch neural thompson sampling. In: NeurIPS (2022)

    2022

  5. [12]

    Dunson, D., Wu, N.: Inferring manifolds from noisy data using gaussian processes (10 2021)

    2021

  6. [13]

    In: Conference on Uncertainty in Artificial Intelligence (2021)

    Eriksson, D., Jankowiak, M.: High-dimensional bayesian optimization with sparse axis-aligned subspaces. In: Conference on Uncertainty in Artificial Intelligence (2021)

    2021

  7. [14]

    In: Advances in Neural Information Processing Systems

    Eriksson, D., Pearce, M., Gardner, J., Turner, R.D., Poloczek, M.: Scalable global optimization via local bayesian optimization. In: Advances in Neural Information Processing Systems. vol. 32 (2019)

    2019

  8. [15]

    In: Ad- vances in Neural Information Processing Systems

    Gardner, J., Pleiss, G., Weinberger, K.Q., Bindel, D., Wilson, A.G.: Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In: Ad- vances in Neural Information Processing Systems. vol. 31 (2018)

    2018

  9. [16]

    ACS Central Science 4, 268 – 276 (2018)

    Gómez-Bombarelli, R., Duvenaud, D.K., Hernández-Lobato, J.M., Aguilera- Iparraguirre, J., Hirzel, T.D., Adams, R.P., Aspuru-Guzik, A.: Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science 4, 268 – 276 (2018)

    2018

  10. [17]

    Journal of Machine Learning Research17(69), 1–26 (2016)

    Guhaniyogi, R., Dunson, D.B.: Compressed gaussian process for manifold regres- sion. Journal of Machine Learning Research17(69), 1–26 (2016)

    2016

  11. [18]

    Gupta, A., Dasgupta, S.: An elementary proof of the johnson-lindenstrauss lemma (1999)

    1999

  12. [19]

    In: NeurIPS (2021)

    Hutchinson, M., Terenin, A., Borovitskiy, V., Takao, S., Teh, Y.W., Deisenroth, M.P.: Vector-valued gaussian processes on riemannian manifolds via gauge inde- pendent projected kernels. In: NeurIPS (2021)

    2021

  13. [20]

    In: CoRL (2021)

    Jaquier, N., Borovitskiy, V., Smolensky, A., Terenin, A., Asfour, T., Rozo, L.D.: Geometry-aware bayesian optimization in robotics using riemannian matérn ker- nels. In: CoRL (2021)

    2021

  14. [21]

    In: Advances in Neural Information Processing Systems

    Jaquier, N., Rozo, L.: High-dimensional bayesian optimization via nested rieman- nian manifolds. In: Advances in Neural Information Processing Systems. vol. 33, pp. 20939–20951 (2020)

    2020

  15. [22]

    In: CoRL (2019)

    Jaquier, N., Rozo, L.D., Calinon, S., Bürger, M.: Bayesian optimization meets riemannian manifolds in robot learning. In: CoRL (2019)

    2019

  16. [23]

    Contemporary mathematics 26, 189–206 (1984)

    Johnson, W.B.: Extensions of lipschitz mappings into hilbert space. Contemporary mathematics 26, 189–206 (1984)

    1984

  17. [24]

    In: ICML (2015)

    Kandasamy, K., Schneider, J.G., Póczos, B.: High dimensional bayesian optimisa- tion and bandits via additive models. In: ICML (2015)

    2015

  18. [25]

    In: ICML (2019)

    Kirschner, J., Mutný, M., Hiller, N., Ischebeck, R., Krause, A.: Adaptive and safe bayesian optimization in high dimensions via one-dimensional subspaces. In: ICML (2019)

    2019

  19. [26]

    In: 5th In- ternational Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings

    Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. In: 5th In- ternational Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net (2017)

    2017

  20. [27]

    Annals of Global Analysis and Geometry60, 1–29 (10 2021)

    Leobacher, G., Steinicke, A.: Existence, uniqueness and regularity of the projection onto differentiable manifolds. Annals of Global Analysis and Geometry60, 1–29 (10 2021). https://doi.org/10.1007/s10455-021-09788-z HD-BO via Random Projection of Manifold Subspaces 17

    2021 doi

  21. [28]

    In: Advances in Neural Information Processing Systems

    Letham, B., Calandra, R., Rai, A., Bakshy, E.: Re-examining linear embeddings for high-dimensional bayesian optimization. In: Advances in Neural Information Processing Systems. vol. 33, pp. 1546–1558 (2020)

    2020

  22. [29]

    In: IJCAI (2017)

    Li, C., Gupta, S., Rana, S., Nguyen, V., Venkatesh, S., Shilton, A.: High dimen- sional bayesian optimization using dropout. In: IJCAI (2017)

    2017

  23. [30]

    In: AISTATS (2016)

    Li, C.L., Kandasamy, K., Póczos, B., Schneider, J.G.: High dimensional bayesian optimization via restricted projection pursuit models. In: AISTATS (2016)

    2016

  24. [31]

    In: Optimization Techniques IFIP Technical Conference Novosibirsk, July 1–7, 1974

    Močkus, J.: On bayesian methods for seeking the extremum. In: Optimization Techniques IFIP Technical Conference Novosibirsk, July 1–7, 1974. pp. 400–404. Berlin, Heidelberg (1975)

    1975

  25. [32]

    Machine Learning109(9), 1925– 1943 (2020)

    Moriconi, R., Deisenroth, M.P., Sesh Kumar, K.: High-dimensional bayesian op- timization using low-dimensional feature spaces. Machine Learning109(9), 1925– 1943 (2020)

    2020

  26. [33]

    In: ICML (2019)

    Nayebi, A., Munteanu, A., Poloczek, M.: A framework for bayesian optimization in embedded subspaces. In: ICML (2019)

    2019

  27. [34]

    Discrete & Computational Geometry39, 419–441 (2008)

    Niyogi, P., Smale, S., Weinberger, S.: Finding the homology of submanifolds with high confidence from random samples. Discrete & Computational Geometry39, 419–441 (2008)

    2008

  28. [35]

    In: NeurIPS (2021)

    Notin, P., Hernández-Lobato, J.M., Gal, Y.: Improving black-box optimization in vae latent space using decoder uncertainty. In: NeurIPS (2021)

    2021

  29. [36]

    In: Advances in Neural Information Processing Systems, NeurIPS 2022

    Papenmeier, L., Nardi, L., Poloczek, M.: Increasing the scope as you learn: Adap- tive bayesian optimization in nested subspaces. In: Advances in Neural Information Processing Systems, NeurIPS 2022. vol. 35 (2022)

    2022

  30. [37]

    Persson, M.: The whitney embedding theorem (2014)

    2014

  31. [38]

    In: 2018 IEEE Inter- national Conference on Robotics and Automation (ICRA)

    Rai, A., Antonova, R., Song, S., Martin, W., Geyer, H., Atkeson, C.: Bayesian optimization using domain knowledge on the atrias biped. In: 2018 IEEE Inter- national Conference on Robotics and Automation (ICRA). pp. 1771–1778 (2018). https://doi.org/10.1109/ICRA.2018.8461237

    2018

  32. [39]

    The MIT Press (2005)

    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press (2005)

    2005

  33. [40]

    In: Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (2018)

    Rolland, P., Scarlett, J., Bogunovic, I., Cevher, V.: High-dimensional bayesian optimization via additive models with overlapping groups. In: Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (2018)

    2018

  34. [41]

    In: Advances in Neural Information Processing Systems

    Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of ma- chine learning algorithms. In: Advances in Neural Information Processing Systems. vol. 25 (2012)

    2012

  35. [42]

    Constructive Approximation (2019)

    Sober, B., Levin, D.: Manifold approximation by moving least-squares projection (mmls). Constructive Approximation (2019)

    2019

  36. [43]

    IEEE Transactions on Information Theory58(5), 3250–3265 (2012).https://doi.org/10.1109/TIT

    Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.W.: Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory58(5), 3250–3265 (2012).https://doi.org/10.1109/TIT. 2011.2182033

    2012

  37. [44]

    In: 5th Interna- tional Conference on Learning Representations, 2017, Workshop Track Proceedings

    Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In: 5th Interna- tional Conference on Learning Representations, 2017, Workshop Track Proceedings

    2017

  38. [45]

    IEEE Transactions on Neural Networks20, 542 (01 2009)

    Thomas, P.: Semi-supervised learning by olivier chapelle, bernhard schölkopf, and alexander zien (review). IEEE Transactions on Neural Networks20, 542 (01 2009)

    2009

  39. [46]

    In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems

    Todorov, E., Erez, T., Tassa, Y.: Mujoco: A physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 5026–5033 (2012).https://doi.org/10.1109/IROS.2012.6386109 18 Nguyen et al

    2012

  40. [47]

    In: AAAI (2020)

    Tran-The, H., Gupta, S., Rana, S., Venkatesh, S.: Trading convergence rate with computational budget in high dimensional bayesian optimization. In: AAAI (2020)

    2020

  41. [48]

    In: Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (2022)

    Tran-The, H., Gupta, S., Rana, S., Venkatesh, S.: Regret bounds for expected improvement algorithms in gaussian process bandit optimization. In: Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (2022)

    2022

  42. [49]

    van der Vaart, A., Wellner, J.A.: Weak convergence and empirical processes: With applications to statistics (1996)

    1996

  43. [50]

    ArXivabs/1703.01973 (2017)

    Wang, Z., Li, C., Jegelka, S., Kohli, P.: Batched high-dimensional bayesian opti- mization via structural kernel learning. ArXivabs/1703.01973 (2017)

    2017 arXiv

  44. [52]

    In: Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (2016)

    Wilson, A.G., Hu, Z., Salakhutdinov, R., Xing, E.P.: Deep kernel learning. In: Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (2016)

    2016

  45. [53]

    In: Advances in Neural Information Processing Systems 33, 2020 (2020)

    Xie, Q., Dai, Z., Hovy, E.H., Luong, T., Le, Q.: Unsupervised data augmentation for consistency training. In: Advances in Neural Information Processing Systems 33, 2020 (2020)

    2020

  46. [54]

    The Annals of Statistics 44(2), 876 – 905 (2016).https://doi.org/10.1214/15-AOS1390

    Yang, Y., Dunson, D.B.: Bayesian manifold regression. The Annals of Statistics 44(2), 876 – 905 (2016).https://doi.org/10.1214/15-AOS1390

    2016 doi

  47. [55]

    In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 (2019)

    Zhang, M., Li, H., Su, S.: High dimensional bayesian optimization via supervised dimension reduction. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 (2019)

    2019

  48. [56]

    (eds.) International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA

    Ziomek, J.K., Bou-Ammar, H.: Are random decompositions all we need in high dimensional bayesian optimisation? In: Krause, A., Brunskill, E., Cho, K., En- gelhardt, B., Sabato, S., Scarlett, J. (eds.) International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Hon...

    2023

  49. [57]

    Let O ∈ Rn×k is the orthogonal matrix (i.e OT O = Ik×k)

    Šehić, K., Gramfort, A., Salmon, J., Nardi, L.: Lassobench: A high-dimensional hyperparameter optimization benchmark suite for lasso (2021).https://doi.org/ 10.48550/ARXIV.2111.02790 HD-BO via Random Projection of Manifold Subspaces 19 Supplementary Material A Proofs of Theore...

    2021 doi arXiv

  50. [58]

    Surprisingly, TuRBO shows the worst performance in this experiment

    As we can see, our method shows the best performance followed by HeSBO, REMBO, and SILBO. Surprisingly, TuRBO shows the worst performance in this experiment. 0 50 100 150 200 250 300 Iterations 400 350 300 250 200 150 Function Value 6392D Mujoco Humanoid RPM-BO HeSBO SILBO SIR...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.