{"id":"4c66c02e-18fc-44cf-8c7c-521562f1bfc8","arxiv_id":"2502.00854","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"EGORSE couples random and supervised linear embeddings inside Bayesian optimization, with a constraint-based feasible-domain formulation, and shows faster CPU convergence than several high-dimensional BO baselines on benchmark problems up to 600 variables.","lead":"This paper introduces EGORSE, a Bayesian optimization method for expensive black-box problems with hundreds of design variables, which combines random linear embeddings with supervised embeddings learned from data. A constrained formulation avoids solving a quadratic program during each acquisition, reducing CPU time relative to prior embedding methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed universal superiority is conditional on the low-effective-dimension assumption: all MB benchmarks have d_eff=2 by construction, and on Rover_600 (d_eff=60) EGORSE with d_e=2 stops near 2.4 instead of the known optimum -5, so the central claim is unsupported for the case where the…","rationale":"The reader's conditional verdict is appropriate, and their weakest_assumption identifies the same load-bearing concern. EGORSE's design and all its claimed advantages (small GP, cheap acquisition, fast CPU time) stem from optimizing in a subspace of dimension d_e, which the user must specify. The MB_d benchmarks are constructed with d_eff=2, so they cannot distinguish EGORSE from any random 2D embedding method; the only test with a larger effective dimension is Rover_600, where EGORSE fails to reach the known optimum and the paper openly acknowledges this. Therefore the central claim of outperforming 'by far all the state-of-the-art HDBO solvers' is an overstatement: the evidence supports at most 'competitive on problems with very small d_eff, and on Rover_600 only marginally better than HESBO while slower in wall-clock time.' The CPU-time improvement over RREMBO is a genuine incremental contribution, and the constraint-based reformulation of the feasibility test is clearly described. But the paper does not establish that the method works when the effective dimension is not tiny, and its own results contradict the strength of the conclusion. The proposed d_e-sweep test would directly quantify how performance degrades as the true effective dimension grows, and would also reveal whether the supervised embedding contributes anything beyond random sampling in that regime. The conditionality is therefore warranted; the verdict should remain CONDITIONAL pending rephrased claims, additional benchmarks with known and varied effective dimensions, and ideally code and seeds for reproducibility.","tokens_in":18436,"tokens_out":9515,"duration_ms":103208,"concrete_test":"Run EGORSE PLS+Gaussian on the Rover_600 problem with d_e in {2, 4, 8, 16, 32}, keeping the same evaluation budget and all other hyperparameters fixed, and record the best value and CPU time. Track whether the best value approaches the known optimum -5 as d_e grows; if the improvement beyond d_e=2 is negligible, then the algorithm cannot exploit additional effective dimensions, confirming that the reported advantage is an artifact of benchmarks with d_eff=2 rather than a general high-dimensional Bayesian optimization capability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV.A states that the objective is assumed to depend only on d_e effective dimensions, and EGORSE treats d_e as a fixed user-supplied hyperparameter, set to 2 in every experiment. On the MB_d benchmarks, the effective dimension is 2 by construction because f(x)=f1(A_d x) with A_d in R^{2×d}; thus those tests satisfy the assumption by design. The only non-toy experiment, Rover_600, has true effective dimension 60. With d_e=2, no 2D subspace can contain the active directions, and Figure 9 shows the best value is about 2.4, far from the known optimum -5; Section VII.B explicitly concedes that 'EGORSE cannot provide better results.' The conclusion nonetheless asserts that EGORSE 'outperforms by far all the state-of-the-art HDBO solvers.' That claim is not supported for problems whose effective dimension exceeds d_e. Moreover, the reported data themselves conflict with the 'by far' wording: on MB_10, EGO-KPLS and TuRBO are faster and reach lower values (Figures 7a and 7c), and on Rover_600, HESBO is faster in CPU time (Figure 9a). The assertion in Section IV.E that convergence properties from Binois et al. are 'preserved' is made without proof and does not address the adaptive resampling of embeddings over iterations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EGORSE, a high-dimensional Bayesian optimization method that adaptively combines several low-dimensional linear embeddings, including supervised dimension-reduction techniques (PLS, MGP) and random embeddings, and replaces the quadratic-program feasibility test of RREMBO with a constraint function in the embedded space. The method is evaluated on modified Branin problems with 10 and 100 variables and a 600-variable Rover path-planning problem, against TuRBO, EGO-KPLS, RREMBO, and HESBO. The central claim, stated in Section VIII, is that EGORSE 'outperforms by far all the state-of-the-art HDBO solvers' in both CPU time and number of expensive function evaluations.","tokens_in":18762,"tokens_out":4311,"duration_ms":39747,"significance":"If supported, the main contribution is practically useful: the constraint-based formulation of Eq. (9) avoids solving a quadratic programming problem at every acquisition-function evaluation, and the adaptive combination of supervised and random embeddings is a sensible way to balance exploitation of discovered active directions with exploration. The paper is clearly structured, the algorithms are described in detail, and the experiments include external baseline methods and a problem with 600 variables. However, the evidence supports only a modest version of the claim. The paper's own results in Sections VI and VII undercut the 'outperforms by far' wording, and the fixed effective dimension d_e=2 is a load-bearing assumption that is violated by the non-toy benchmark.","major_comments":[{"comment":"The method's core assumption is that the objective depends only on d_e effective directions, and d_e is set to 2 in all experiments. On Rover_600 the true effective dimension is 60, so no two-dimensional subspace can contain the active directions; Figure 9b shows the best value near 2.4 versus the known optimum -5, and Section VII.B explicitly concedes that 'EGORSE cannot provide better results.' The conclusion in Section VIII that EGORSE 'outperforms by far all the state-of-the-art HDBO solvers' is therefore unsupported for problems whose effective dimension exceeds d_e. The authors should either restrict the claim to the low-effective-dimension regime or provide experiments in which d_e is comparable to the problem's actual effective dimension.","section":"Section IV.A and Section VII.B"},{"comment":"The reported data do not support the sweeping conclusion. On MB_10, EGO-KPLS and TuRBO converge faster and to lower objective values, both by number of evaluations and by CPU time. On MB_100, only the CPU-time comparison clearly favors EGORSE. On Rover_600, HESBO reaches its final value faster in CPU time. The defensible claims are narrower: EGORSE PLS+Gaussian is CPU-efficient on MB_100 and achieves a lower value than HESBO on Rover_600 in similar wall-clock time. The 'by far all state-of-the-art HDBO solvers' statement should be revised to match these data.","section":"Section VI.B, Figures 7a, 7c, and 9a"},{"comment":"The sentence 'The convergence properties, given by Binois et al., are thus preserved' is asserted without proof. The adaptive setting here differs from Binois et al. in two substantive ways: the embedding is re-estimated from all evaluated points at each outer iteration, and the optimization is performed under the constraint function (9) rather than with the modified EI used in RREMBO. No theorem is cited that covers this variant, and the adaptive resampling of the embedding is precisely the part that their analysis does not address. Either provide a proof or a precise reference for this setting, or remove the claim.","section":"Section IV.E"},{"comment":"The comparison protocol is not apples-to-apples for the CPU-time claims. TuRBO uses five trust regions and its own initial design, RREMBO and HESBO are run as 20 concatenated independent optimizations, and EGORSE and EGO-KPLS use a different initial design. The total budget per method, including initialization and hyperparameter fitting, should be reported on a common basis, and the CPU-time comparisons should state all wall-clock components. Without this, the claim that EGORSE is the fastest in CPU time is difficult to interpret.","section":"Section VI.A"}],"minor_comments":[{"comment":"There are several typos: 'high-dimensionnal' in the abstract, 'Hach' instead of 'Hash' in Figures 5 and 6, and 'EGO-KPSL' instead of 'EGO-KPLS' in the caption of Figure 9.","section":"Abstract and figures"},{"comment":"The text says the constraint function is 'normalized to provide values in [-1,1]', but the infeasible branch -||u_A||_2^2 is not normalized as written; please state the normalization step explicitly.","section":"Eq. (9)"},{"comment":"The phrase 'using 10 initial DoE' should be clarified as 'using 10 independent initial designs', since each optimization run uses one initial DoE.","section":"Section V.C.2"},{"comment":"Calling TuRBO, EGO-KPLS, and RREMBO 'intractable in time' is too strong given that Figures 7c and 7d show them completing in several hours; 'substantially more expensive in wall-clock time' would be accurate.","section":"Section VI.B"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: EGORSE is a credible incremental contribution to high-dimensional Bayesian optimization. The genuinely new piece is the adaptive loop that alternates supervised embeddings (PLS, MGP) with random ones, plus the constraint-based formulation that avoids solving a quadratic program at every acquisition evaluation. That last idea is the real contribution: it explains the CPU time advantage you see on MB_100, where RREMBO, TuRBO and EGO-KPLS become expensive in wall-clock time. The paper also does a decent sensitivity study and is honest in Section VII.B about why Rover_600 stops at 2.4 instead of the known -5. Give credit for that.\n\nNow the soft spots, in order of importance. The conclusion says EGORSE 'outperforms by far all the state-of-the-art HDBO solvers.' That sentence is not supported by the paper's own Figures. On MB_10, EGO-KPLS and TuRBO converge faster and to lower values, both in iterations and CPU time. On Rover_600, HESBO is actually faster in CPU time, and EGORSE's final value is far from the optimum. The only test where EGORSE clearly wins is MB_100 in CPU time. That is a legitimate win, but it does not license the 'by far' claim.\n\nThe load-bearing assumption is the fixed effective dimension d_e, set to 2 everywhere. The MB_d benchmarks satisfy the assumption by construction: the random matrix maps R^d into R^2. Rover_600 has true effective dimension 60, and with d_e=2 the method silently optimizes in a subspace that cannot contain the optimum. The authors acknowledge this, but it means the method's success is conditional on a hyperparameter that users will not know in practice. A method that adaptively estimates d_e, or that evaluates with d_e=60 on a 600-dim problem, would be needed to support the general claim.\n\nTwo smaller issues. The assertion in Section IV.E that Binois et al.'s convergence properties are 'preserved' is just stated, not proved, and the adaptive resampling of embeddings over iterations likely breaks whatever assumptions the original proof made. And there is no code or seeds released, which makes the 10-run averages hard to check; the hyperparameters were also tuned on the same test functions used in the final comparison, so the comparison is a bit optimistic.\n\nOverall: the method is not circular, the description is clear, and the CPU-time advantage on MB_100 is a real data point. This paper deserves a serious referee, but the authors should be asked to moderate the central claim, release code/seeds, and either prove or drop the convergence-preservation statement. Who is this for? People working on HDBO, especially in aerospace MDAO, will get value from the constraint-based formulation even if they don't adopt EGORSE wholesale. I'd engage with it in a reading group.","headline":"A useful incremental HDBO method with one good trick, but the central 'outperforms by far' claim is not supported by the paper's own data.","tokens_in":19334,"tokens_out":2895,"would_cite":true,"duration_ms":28745,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","90C56"],"pacs":[],"model":"deepseek-v4-flash","headline":"EGORSE claims that high-dimensional Bayesian optimization becomes tractable when the optimizer adaptively learns a low-dimensional linear subspace and replaces a costly quadratic feasibility test with a constraint function.","keywords":["Bayesian optimization","high-dimensional optimization","linear embedding","random embedding","supervised embedding","partial least squares","Gaussian process","expensive black-box optimization"],"falsifier":"Run EGORSE PLS+Gaussian with $d_e=2$ on a 600-variable problem with a known 60-dimensional active subspace and a known global minimum, as in the Rover_600 benchmark, and check whether the best found value approaches the known optimum; in the paper's own run the best value is about 2.4 against an optimum of $-5$, which would falsify any claim that EGORSE finds global optima in such cases.","tokens_in":18226,"feed_emoji":"🎯","tokens_out":9014,"duration_ms":81446,"temperature":0.7,"pith_summary":"EGORSE is a Bayesian optimization method aimed at expensive black-box functions defined on high-dimensional spaces, here tested with 10 to 600 design variables. The paper's central claim is that such problems become tractable if the objective is assumed to depend on a low-dimensional linear subspace, with the optimization carried out inside that subspace while the subspace itself is updated as function evaluations accumulate. The method combines random Gaussian and hash embeddings with supervised embeddings built from partial least squares, so the search directions reflect observed variation of the function rather than being purely random. The authors argue that this yields faster convergence both in CPU time and in number of expensive evaluations than existing high-dimensional Bayesian optimization solvers, and they report that the comparison methods become time-intractable beyond roughly one hundred variables. A reader should care because simulation-based design, such as aircraft multidisciplinary optimization, is exactly the regime where expensive black-box evaluations in high dimension are the bottleneck.","feed_headline":"Adaptive embeddings speed high-dimensional Bayesian optimization","feed_subtitle":"The method learns a low-dimensional subspace as it runs, cutting CPU time on 100- and 600-variable benchmarks.","key_machinery":"The load-bearing object is the linear embedding subspace $\\mathcal{A}^{(t)} = \\{ \\mathbf{u} = \\mathbf{A}^{(t)} \\mathbf{x} : \\mathbf{x} \\in \\Omega \\}$, where the transfer matrix $\\mathbf{A}^{(t)} \\in \\mathbb{R}^{d_e \\times d}$ is produced by one of several dimension-reduction methods. The reduced optimization runs over the smallest hypercube $\\mathcal{B}^{(t)}$ containing $\\mathcal{A}^{(t)}$. Two backward maps carry the argument: $\\gamma_B$ maps $\\mathcal{A}^{(t)}$ bijectively onto $\\Omega$, and $\\gamma_W$ projects every point of $\\mathcal{B}^{(t)}$ onto $\\Omega$, which lets the objective be defined on all of $\\mathcal{B}^{(t)}$. The mechanism that saves computation is the constraint function $g^{(t)}(\\mathbf{u}) \\ge 0$, which takes the value $1 - \\|\\gamma_B(\\mathbf{u})\\|_2^2 / d$ inside the feasible embedded set and $-\\|\\mathbf{u}^{\\mathbf{A}^{(t)}}\\|_2^2$ outside it, turning the membership test into a cheap function evaluation instead of a quadratic program solved during acquisition.","core_discovery":"The paper's central claim is that a Bayesian optimizer can work reliably in a reduced linear subspace of dimension $d_e$ much smaller than $d$, provided the subspace is chosen by combining supervised and random dimension reduction and the reduced problem is formulated with a constraint rather than a feasibility oracle. The previous random-embedding approach required solving a quadratic program at every acquisition-function evaluation to decide whether a candidate point has a valid image in the original space; EGORSE removes that bottleneck by defining the objective and a constraint on the whole bounding hypercube, so a standard constrained Bayesian optimization subroutine can be used. At each outer iteration, transfer matrices are rebuilt from all previously evaluated points using partial least squares and a random Gaussian matrix, which is the adaptive part of the method. On the 100-variable benchmark the method is reported to converge fastest in CPU time, and on the 600-variable path-planning problem it reaches a lower objective value than the hash-embedding baseline in comparable time, although the best value it finds is far from the known global optimum.","pith_inferences":["The constraint-function replacement is a general trick: any embedding method that provides a transfer matrix and a projection can be plugged into the same reduced formulation, so the CPU-time argument is not limited to PLS or Gaussian embeddings.","The Rover_600 shortfall suggests a practical diagnostic: estimate the number of effective directions before choosing $d_e$ (for example by looking at the decay of the PLS variance), because a too-small $d_e$ can silently discard the subspace containing the optimum.","If the speed gain really comes from removing quadratic solves from the acquisition loop, then the gap between EGORSE and HESBO should widen as the dimension $d$ and the acquisition budget grow; a controlled scaling study would isolate that mechanism.","The claim of superiority rests on two benchmark families, both of which are linear embeddings of low-dimensional functions; testing on functions with genuinely high effective dimension or with nonlinear low-dimensional structure would show whether the method's advantage generalizes."],"forward_implications":["High-dimensional expensive black-box problems with hundreds of design variables can be optimized within a few thousand evaluations, with the expensive part of each iteration shifted from the acquisition sub-problem to a single quadratic solve per objective evaluation.","The standard comparison methods TuRBO, EGO-KPLS, and RREMBO would be impractical in CPU time for problems beyond about one hundred variables, leaving EGORSE as the plausible choice in that regime.","The recommended configuration is the PLS+Gaussian variant with an initial design of $d$ points, because the parametric study reports the best convergence speed and robustness for that setting.","When the true number of effective directions is much larger than the user-specified $d_e$, EGORSE will produce a valid but not globally optimal answer; the Rover_600 experiment is the paper's own demonstration of this limitation.","The same machinery is intended to extend to constrained black-box problems, which the authors identify as the necessary next step for realistic aerospace design."],"supporting_citations":[{"why":"Supplies the random-embedding backward application and the bounding hypercube that EGORSE builds on.","marker":"[17]"},{"why":"Defines the REMBO random-embedding baseline whose reduced bounds EGORSE improves.","marker":"[15]"},{"why":"Provides the partial least squares method used to build the supervised transfer matrix.","marker":"[27]"},{"why":"Provides the marginal Gaussian process embedding used as the alternative supervised method.","marker":"[30]"},{"why":"Introduces the hash-table embedding baseline HESBO, the main competitor on Rover_600.","marker":"[29]"},{"why":"Supplies the trust-region baseline TuRBO that EGORSE claims is time-intractable in high dimension.","marker":"[14]"},{"why":"Supplies the KPLS-based baseline EGO-KPLS used in the CPU-time comparison.","marker":"[5]"},{"why":"Provides the constrained Bayesian optimization solver that EGORSE calls to solve the reduced problem.","marker":"[4]"},{"why":"Contributes the Rover_60 path-planning problem from which the Rover_600 benchmark is built.","marker":"[16]"}],"fun_headline_variants":["Hybrid embeddings cut CPU time for high-dimensional Bayesian optimization","Learned low-dim subspaces slash expensive blackbox calls in BO","EGORSE: random + supervised embeddings adaptively accelerate BO","Constrained BO in learned subspaces scales to 600 variables","Random + supervised embeddings make BO practical in 600D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The objective function truly depends on a low-dimensional linear subspace of dimension $d_e$, and the user-chosen $d_e$ (two in all main experiments) captures enough of that variation for the optimum to lie in the searched subspace.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid embeddings cut CPU time for high-dimensional Bayesian optimization","Learned low-dim subspaces slash expensive blackbox calls in BO","EGORSE: random + supervised embeddings adaptively accelerate BO","Constrained BO in learned subspaces scales to 600 variables","Random + supervised embeddings make BO practical in 600D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000672,"raw_usage":{"total_tokens":3044,"prompt_tokens":914,"completion_tokens":2130,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":2044}},"tokens_in":530,"tokens_out":2130,"duration_ms":14494,"temperature":1.0,"reasoning_tokens":2044,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T17:30:04.286182+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run EGORSE PLS+Gaussian with $d_e=2$ on a 600-variable problem with a known 60-dimensional active subspace and a known global minimum, as in the Rover_600 benchmark, and check whether the best found value approaches the known optimum; in the paper's own run the best value is about 2.4 against an optimum of $-5$, which would falsify any claim that EGORSE finds global optima in such cases.","supporting_citations":[{"cited_title":"On the Choice of the Low-Dimensional Domain for Global Optimization via Random Embeddings,","cited_arxiv_id":null,"evidence_quote":"Supplies the random-embedding backward application and the bounding hypercube that EGORSE builds on."},{"cited_title":"Bayesian Optimization in a Billion Dimensions via Random Embeddings,","cited_arxiv_id":null,"evidence_quote":"Defines the REMBO random-embedding baseline whose reduced bounds EGORSE improves."},{"cited_title":"OntheStructureofPartialLeastSquaresRegression,","cited_arxiv_id":null,"evidence_quote":"Provides the partial least squares method used to build the supervised transfer matrix."},{"cited_title":"Active Learning of Linear Embeddings for Gaussian Processes,","cited_arxiv_id":null,"evidence_quote":"Provides the marginal Gaussian process embedding used as the alternative supervised method."},{"cited_title":"A Framework for Bayesian Optimization in Embedded Subspaces,","cited_arxiv_id":null,"evidence_quote":"Introduces the hash-table embedding baseline HESBO, the main competitor on Rover_600."},{"cited_title":"Scalable Global Optimization via Local Bayesian Optimization,","cited_arxiv_id":null,"evidence_quote":"Supplies the trust-region baseline TuRBO that EGORSE claims is time-intractable in high dimension."},{"cited_title":"Efficient Global Optimization for High-Dimensional ConstrainedProblemsbyUsingtheKrigingModelsCombinedwiththePartialLeastSquaresMethod,","cited_arxiv_id":null,"evidence_quote":"Supplies the KPLS-based baseline EGO-KPLS used in the CPU-time comparison."},{"cited_title":"Adaptive Modeling Strategy for Constrained Global Optimization with Application to Aerodynamic Wing Design,","cited_arxiv_id":null,"evidence_quote":"Provides the constrained Bayesian optimization solver that EGORSE calls to solve the reduced problem."},{"cited_title":"Batched High-Dimensional Bayesian Optimization via Structural Kernel Learning,","cited_arxiv_id":null,"evidence_quote":"Contributes the Rover_60 path-planning problem from which the Rover_600 benchmark is built."}],"review_version":1}