{"id":"1253803c-f8dc-4b8b-9614-c550b2b10e67","arxiv_id":"2412.16554","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"RPM-BO combines random projection with a learned, semi-supervised manifold mapping to run Bayesian optimization in a low-dimensional space and projects candidates back to the original space.","lead":"This paper proposes RPM-BO, a Bayesian optimization method that reduces very high-dimensional search problems to a low-dimensional random projection and learns a mapping back to the original space with a neural network. The authors report strong results on synthetic functions up to 1500 dimensions and on real benchmarks up to 7129 dimensions, but the central theoretical justification has a gap that limits the stated generality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proof of Theorem 1 falsely equates embeddability in R^m with containment in an m-dimensional affine subspace, breaking the M_A = M_A argument that underpins Algorithm 1.","rationale":"The paper's central methodological novelty is optimizing the acquisition in R^m and mapping back via x = h(A^T z); the justification is Theorem 2, which relies on Theorem 1. The proof of Theorem 1 is invalid for the reason above. This is not a stylistic issue: Algorithm 1's line 9 chooses z in a box in R^m, and line 10 maps it back. If the image of h(A^T ·) does not cover the projected manifold M_A, the algorithm may permanently miss the effective subspace. The paper's own nonlinear experiments use a manifold whose affine span exceeds m, so the gap is not a corner case; it is the primary nonlinear regime. The empirical results, p-values, and runtime comparison are useful and the code is provided, so I would not reject the paper outright; but the theoretical claims need either a corrected proof (e.g., via normal-bundle transversality, which may hold under reach/curvature conditions) or an explicit restriction to manifolds with aff-dim ≤ m, plus controlled approximation error for h. The reader's CONDITIONAL verdict captures this correctly, and my concern does not shift the verdict.","tokens_in":21235,"tokens_out":9385,"duration_ms":81785,"concrete_test":"Use the paper's mixed manifold M = T^5 × R^{10} ⊂ R^{20} (then embedded in R^D by padding), with D = 500, d = 15, m = 15. Draw a random orthogonal A ∈ R^{15×D} as in Algorithm 1. Densely sample, say, 10^4 points x ∈ M. For each, compute the closed-form projection P_M(A^T q) (normalize each of the five 2D circle blocks and keep the last 10 coordinates) and minimize ||x - P_M(A^T q)|| over q ∈ [-√15, √15]^{15} using multi-start L-BFGS-B. Record the maximum achieved distance. If even one x has distance above 1e-6, then the equality M_A = M_A of Theorem 2 fails for the paper's own experimental setting, so line 9's acquisition is not optimizing over the projected manifold. If the cover succeeds, the theorem may be salvageable, but the affine-subspace proof in Appendix A is still invalid and must be replaced.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the proof of Theorem 1 in Appendix A. It states: 'M can be embedded in Euclidean space R^m. Therefore, ∃ an m-dimensional affine space L ⊂ R^D such that M ⊂ L.' This inference is invalid: embeddability in R^m provides a smooth injective map into R^m, not a linear coordinate subspace of the ambient R^D. The affine hull of the given submanifold M ⊂ R^D can have dimension much larger than m. The paper's own mixed-manifold test function (Section 4.1, Eq. (12)) is M = T^{2d1} × R^{d2} with d1 = 5, d2 = 10; its coordinate support has 2d1 + d2 = 20 ambient coordinates, so aff(M) has dimension 20, while the paper chooses m = 15. Thus the key condition of the proof fails in the main nonlinear experiments. Since Theorem 2 derives M_A = M_A from Theorem 1, the equality that justifies optimizing EI(Ah(A^T z)) over z ∈ [-√m, √m]^m (Algorithm 1 lines 9-10) is not established for exactly the regime where h is a neural net and no analytic projection is available. The proof gap is internal, not merely a disagreement with convention; the stated proof does not cover the stated theorem. Even if a different proof of Theorem 1 exists, the paper provides none, so the advertised theoretical support for the back-projection scheme is not currently valid.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RPM-BO, a high-dimensional Bayesian optimization algorithm for objective functions whose effective dependence lies on a low-dimensional manifold. The method combines a random orthogonal projection A from R^D to R^m with a learned feature map h (analytic when the manifold geometry is known, otherwise a neural network trained with a semi-supervised consistency loss). A Gaussian process surrogate is constructed on the projected points Ah(x), and the acquisition function is optimized in R^m with a back-projection step h(A^T z). The authors claim theoretical support via Theorems 1 and 2, which are supposed to show that optimizing EI in the projected space is equivalent to optimizing over the projected manifold, and they report empirical improvements over several baselines on synthetic and real benchmarks.","tokens_in":21603,"tokens_out":7443,"duration_ms":66319,"significance":"If the theoretical claims were correct, the back-projection scheme would be a valuable contribution because it avoids expensive decoder training and provides a simple way to map low-dimensional acquisition candidates back to the original space. The empirical study is broad, covering multiple synthetic manifolds and real applications with dimensions up to 1500, and the authors make their code available. The semi-supervised consistency loss is a reasonable practical idea for regularizing the feature map with unlabeled data. However, the central theoretical result, Theorem 1, has an invalid proof and appears false in the general setting claimed; since Theorem 2 and the acquisition optimization in Algorithm 1 rely directly on Theorem 1, the advertised theoretical support is not established.","major_comments":[{"comment":"The proof contains a non-sequitur: from \"M can be embedded in Euclidean space R^m\" it concludes \"therefore, there exists an m-dimensional affine space L ⊂ R^D such that M ⊂ L.\" Embeddability is an intrinsic property and does not imply containment in an affine subspace of the ambient space; for instance, a 1-dimensional helix in R^3 can be embedded in R^2 but is not contained in any 2-dimensional affine subspace. This invalidates the proof of Theorem 1. Consequently, Theorem 2, which relies on Theorem 1, is also not proven, and the equality M_A = \\bar{M}_A that justifies optimizing EI(Ah(A^T z)) over z ∈ [-√m, √m]^m in Algorithm 1 (lines 9-10) is not established. Moreover, the paper's own mixed-manifold experiments (Section 4.1, Eq. (12), with d1=5, d2=10) have affine span 2d1+d2=20 in R^D, while the experiments set m=15; thus the proof's required affine-subspace condition is not satisfied in the main nonlinear experiments.","section":"Appendix A, proof of Theorem 1"},{"comment":"The claimed convergence rate O(n^{-1/m}) for RPM-BO is asserted rather than derived. The text states that \"if we assume that the feature mapping h is also accurately learned, then the convergence rate of our proposed BO algorithm is equivalent to the convergence rate of a BO algorithm in low dimension m,\" and that the proof can be done similarly to [51]. This is not a proof, and the setting here differs from [51] in essential ways: h is learned from data, the acquisition optimization is performed in R^m and then back-projected, and the surrogate is built on projected points Ah(x). No analysis is given for the effect of approximation error in h, the mismatch between h(A^T z) and A P_M(A^T z), or the finite-sample behavior of the learned lengthscale parameter a. The claim that the algorithm \"achieves the optimal posterior contraction rate\" is therefore unsupported.","section":"Appendix B.3"},{"comment":"The derivation of the acquisition optimization relies on the assumption that \"h approximates P_M\" and uses the approximation symbol to replace P_M by h. The paper does not provide a quantitative statement of this approximation, nor does it show that the consistency loss in Eq. (7) drives h toward P_M in any controlled way. In fact, Proposition 1 characterizes P_M, but the loss only encourages h to be invariant on line segments between x' and h(x'); it does not enforce that h(x) lies on M or that h equals the orthogonal projection. This leaves a gap between the theory and the actual algorithm, and the experiments do not verify that h is close to P_M for the neural-network cases (e.g., the mixed-manifold and MuJoCo experiments).","section":"Section 3.3, Eq. (11)"}],"minor_comments":[{"comment":"The spherical projection formula is undefined when the denominator ||B(B^T x - c)||_2 is zero; the paper does not discuss this edge case or how it is handled in the implementation.","section":"Eq. (5)"},{"comment":"The back-projection step is written as x_{n+1} = h(A^T z_{n+1}), which is consistent with using h as a replacement for P_M, but the relationship to Theorem 2's exact expression A P_M(A^T q) should be stated explicitly to avoid confusion.","section":"Algorithm 1, line 10"},{"comment":"The discussion of the choice of m says that if m ≥ 2d then every d-dimensional manifold can be embedded in R^m (Whitney), while the random projection condition in Eq. (9) suggests m = O(d log D). The relationship between these two requirements is not clarified; the experiments set m = 15 for d = 15, which does not satisfy m ≥ 2d.","section":"Section 3.4"},{"comment":"The sentence \"We will show that training with semi-supervised loss function in Equation (8) can reduce the overfitting issue in Appendix D.4\" is worded as a claim of a proof, but Appendix D.4 presents only a toy empirical illustration. The wording should be softened to \"provide empirical evidence\".","section":"Appendix D.4"},{"comment":"The reference [48] appears in the text as \"Tran-The, H., Gupta, S., Rana, S., Venkatesh, S.: Regret bounds for expected improvement algorithms in Gaussian process bandit optimization.\" This is an odd citation for the noise-affected convergence rate; the authors should verify the precise statement and ensure the cited result supports the claim.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The false inference in the proof of Theorem 1 is a serious, load-bearing issue: the paper's theoretical claims about the back-projection scheme do not currently hold. I would normally lean toward rejection, but the empirical results are extensive and the method may be salvageable by correcting the theorem statement (e.g., assuming M is contained in an m-dimensional affine subspace) and adjusting the experiments and claims accordingly. I recommend major revision with the explicit requirement that the theoretical support be either repaired or clearly downgraded to a heuristic; if the authors cannot provide a correct theorem for the actual algorithm, the paper should be rejected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a practical BO paper with a genuinely new way to combine random projections with a learned nonlinear manifold map, but the central theorem that justifies the back-projection is not proven as written, and the empirical section has budget fairness issues. I'd send it to review but expect heavy revision.\n\nThe new pieces are real: RPM-BO's acquisition over z in R^m with back-projection h(A^T z), and the geometry-aware consistency loss (Eq. 7) are not in REMBO, SILBO, or compressed GP. The paper also does the right empirical due diligence: code is public, p-values reported, an ablation on m, and a runtime table. It genuinely scales to D=1500 and 7129 in some problems, which is more than most baselines.\n\nThe soft spot is not minor. Theorem 1 claims that if a d-manifold M embeds in R^m, then for a random orthogonal A in R^{m x D}, every x in M is the projection of some A^T z. The proof in Appendix A says embeddability gives an m-dimensional affine space L in R^D containing M. That is false: an abstract embedding is not a linear containment in the ambient space. The paper's own mixed manifold (Eq. 12) has 20-dimensional affine span in R^D while m=15, so their proof condition fails in exactly the nonlinear case where h is a neural net. Theorem 2 inherits the problem. So the equality M_A = M_A, which is what lets them optimize EI in the box, is not established by the arguments given. I wouldn't bet the mathematical claim is unfixable—with m >= d+1 or a transversality argument a version may hold—but the paper doesn't supply it.\n\nThe convergence rate claims in B.3 are also asserted rather than proven: they assume h is \"accurately learned\" and then quote the low-dimensional BO rate. The justification \"can be done similarly as in [51]\" is not a proof in a paper that leans on theory. On experiments, SAASBO is limited to 60 iterations while others get 300; that asymmetry weakens comparisons where SAASBO is competitive. The consistency-loss ablation is only a toy, not the main BO curves.\n\nWho this is for: people working on HD-BO who want a simple scalable method and are willing to wait for the theory to be fixed. Recommend: engage seriously, send to a good venue, but the revision must either repair Theorem 1 under honest assumptions or restrict the theoretical claims, and equalize the baseline budgets.","headline":"Promising BO recipe undermined by a false proof of its central theorem; the empirics are worth a serious revision.","tokens_in":22123,"tokens_out":5623,"would_cite":false,"duration_ms":49338,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"High-dimensional Bayesian optimization can be reduced to a low-dimensional search plus a manifold back-projection.","keywords":["Bayesian optimization","high-dimensional optimization","random projection","manifold learning","semi-supervised learning","effective dimensionality","Gaussian process","back-projection"],"falsifier":"Take the mixed manifold $T^{2d_1} \\times \\mathbb{R}^{d_2}$ with $d_1=5$, $d_2=10$ in $\\mathbb{R}^{500}$, set $m=15$, draw $A$ randomly, and sample points $x \\in M$; check numerically whether for each $A x$ there is a $q$ with $A x = A P_M(A^T q)$ within box bounds. If any sampled point has no such $q$, the equality $M_{\\mathcal{A}} = M_{\\mathcal{A}}'$ fails and the low-dimensional acquisition provably covers only a subset of the projected manifold.","tokens_in":21023,"feed_emoji":"📉","tokens_out":6253,"duration_ms":50442,"temperature":0.7,"pith_summary":"The paper claims that high-dimensional Bayesian optimization becomes tractable when the objective function depends on a low-dimensional manifold: one random orthogonal matrix and one learned projection onto the manifold are enough to optimize in a low-dimensional space and map the chosen point back to the ambient space. The load-bearing identity is that the image of the manifold under the random projection equals the set of back-projections $A P_M(A^T q)$, so the acquisition function can be maximized over $q$ in a box. A semi-supervised consistency loss trains the projection $h$ when the manifold geometry is unknown, reducing the overfitting that plagues encoder-decoder approaches with few labeled points. Empirically, the method outperforms eight high-dimensional Bayesian optimization baselines on synthetic functions with spherical, mixed, and linear latent manifolds, and on Lasso and MuJoCo tasks.","feed_headline":"Random projection lets Bayesian optimization scale to 1500 dimensions","feed_subtitle":"Solving the acquisition in the projected subspace and mapping back beats eight baselines on tests up to D=1500.","key_machinery":"The central objects are the random orthogonal matrix $A \\in \\mathbb{R}^{m \\times D}$ and the manifold projection $P_M$, approximated by a learned mapping $h$. The identity $M_{\\mathcal{A}} = \\{A P_M(A^T q)\\}$ is the mechanism that turns the intractable acquisition optimization over the projected manifold into an unconstrained box search in $\\mathbb{R}^m$ followed by the explicit back-projection $h(A^T z)$. The semi-supervised consistency loss $L_{us} = \\frac{1}{pq}\\sum_{j,i} \\|h(\\lambda_j x'_i + (1-\\lambda_j)h(x'_i)) - h(x'_i)\\|^2$ enforces that $h$ behaves like an orthogonal projection, which is the property used for the back-projection to land on $M$.","core_discovery":"Theorem 1 and Theorem 2 establish that, with probability 1, for a $d$-dimensional manifold $M$ embedded in $\\mathbb{R}^D$ that can be embedded in $\\mathbb{R}^m$, the set $M_{\\mathcal{A}} = \\{A x : x \\in M\\}$ equals $\\{A P_M(A^T q) : q \\in \\mathbb{R}^m\\}$. Therefore, instead of solving the acquisition maximization over the complicated domain $M_{\\mathcal{A}}$, one can maximize $EI(A h(A^T q))$ over a box and set the next query to $x_{n+1} = h(A^T z_{n+1})$. The paper also shows that the low-dimensional Gaussian process regressor converges at the optimal posterior contraction rate $n^{-s/(2s+d)}$ and that the random projection compresses noise of near-manifold points.","pith_inferences":["Extension: The equality $M_{\\mathcal{A}} = \\{A P_M(A^T q)\\}$ should be tested numerically on manifolds whose affine span exceeds $m$; the paper's own mixed manifold ($2d_1+d_2=20$, $m=15$) is a direct test case.","Extension: If coverage fails for such manifolds, a practical remedy is to increase $m$ to the affine span dimension or to add a second random projection restricted to the orthogonal complement.","Extension: The consistency loss could be strengthened with an explicit idempotence penalty $\\|h(h(x))-h(x)\\|^2$, which would make the learned $h$ closer to a true projection and tighten the approximation $h \\approx P_M$."],"forward_implications":["Acquisition optimization runs in $\\mathbb{R}^m$ with $m \\ll D$, so the per-iteration cost no longer scales with the ambient dimension.","The explicit back-projection $h(A^T z)$ replaces the decoder-network reconstruction used by VAE-based Bayesian optimization approaches, simplifying the pipeline.","When the manifold geometry is known (sphere, linear subspace), a closed-form $h$ gives lower test loss and needs no unlabeled data.","Training $h$ with the semi-supervised consistency loss reduces overfitting when few labeled points are available (10–100 labels).","On synthetic Ackley, Rotated Hyper-Ellipsoid, and Levy functions with latent dimensions 10–15, the method reaches lower function values than the eight baselines for $D \\in \\{500, 1000, 1500\\}$."],"supporting_citations":[{"why":"Establishes the random-embedding paradigm for Bayesian optimization and the box search domain used for the acquisition optimization.","marker":"[51]"},{"why":"Provides the compressed Gaussian process result that yields the optimal posterior contraction rate and the noise-compression property of random projections.","marker":"[17]"},{"why":"Supplies the projection idempotence property (Proposition 1) that underlies the semi-supervised consistency loss.","marker":"[27]"},{"why":"Gives the posterior contraction theorem for Bayesian manifold regression used to justify the low-dimensional surrogate convergence.","marker":"[54]"},{"why":"Supplies the distance-preserving and diffeomorphism properties of random projections onto smooth manifolds.","marker":"[2]"}],"fun_headline_variants":["Project, learn, optimize: BO scales to 1500 dimensions","Manifold subspaces + random projection push BO to D=1500","Random projections unlock manifold geometry for BO at D=1500","Geometry-aware semi-supervised BO for 1500 dimensions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof relies on the premise that a manifold that can be folded into $m$ dimensions always lies entirely inside some $m$-dimensional flat plane of the original space; embeddability does not guarantee this.","fun_headline_variants_meta":{"raw":{"variants":["Project, learn, optimize: BO scales to 1500 dimensions","Manifold subspaces + random projection push BO to D=1500","Random projections unlock manifold geometry for BO at D=1500","Geometry-aware semi-supervised BO for 1500 dimensions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000777,"raw_usage":{"total_tokens":3451,"prompt_tokens":978,"completion_tokens":2473,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":2399}},"tokens_in":594,"tokens_out":2473,"duration_ms":16406,"temperature":1.0,"reasoning_tokens":2399,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:29:45.560553+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the mixed manifold $T^{2d_1} \\times \\mathbb{R}^{d_2}$ with $d_1=5$, $d_2=10$ in $\\mathbb{R}^{500}$, set $m=15$, draw $A$ randomly, and sample points $x \\in M$; check numerically whether for each $A x$ there is a $q$ with $A x = A P_M(A^T q)$ within box bounds. If any sampled point has no such $q$, the equality $M_{\\mathcal{A}} = M_{\\mathcal{A}}'$ fails and the low-dimensional acquisition provably covers only a subset of the projected manifold.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the random-embedding paradigm for Bayesian optimization and the box search domain used for the acquisition optimization."},{"cited_title":"Journal of Machine Learning Research17(69), 1–26 (2016)","cited_arxiv_id":null,"evidence_quote":"Provides the compressed Gaussian process result that yields the optimal posterior contraction rate and the noise-compression property of random projections."},{"cited_title":"Foundations of Computational Mathematics9, 51–77 (2009)","cited_arxiv_id":null,"evidence_quote":"Supplies the distance-preserving and diffeomorphism properties of random projections onto smooth manifolds."}],"review_version":1}