{"id":"026d2aed-1782-44cf-8b91-0945bbb75acd","arxiv_id":"2412.15679","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Compressing thousands of constraints into a low-dimensional latent space lets Bayesian optimization solve a 108D aeroelastic-tailoring problem with 1,786 black-box constraints, at the cost of slightly worse solution quality on a benchmark.","lead":"This paper combines Gaussian-process surrogates with PCA-based compression of thousands of constraints, so Bayesian optimization can run on high-dimensional engineering designs that previously had too many constraints to model individually. The method is tested on a 7D speed-reducer benchmark and a 108D aeroelastic wing-tailoring problem with 1,786 constraints, where it finds feasible designs that random search and CMA-ES cannot reach.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Latent feasibility is not original feasibility: requiring PCA scores <= 0 is a rotated half-space condition, so even a perfect low-rank approximation can reject feasible designs and admit infeasible ones.","rationale":"The reader's conditional verdict is sound; this stress-test sharpens the load-bearing assumption it identified. The method as written uses a coordinate-wise sign constraint on PCA scores as if it were the feasibility constraint, but this is not implied by c_i(x) <= 0. The final validation in original space does not repair the search geometry: it only labels evaluated points after the fact. The empirical wins on the 108D problem may be real, but under the stated mechanism they are not explained; a corrected algorithm would reconstruct the G original constraint predictions from the g latent GPs and threshold those, or explicitly justify a sign-preserving basis. Because this can be checked and fixed, I do not move the verdict beyond the reader's CONDITIONAL, but the revision conditions should include this correction and the proposed diagnostic. I did not elevate the secondary complexity discrepancy (the claimed O(G^3) eigendecomposition is O(N^3) for kPCA) because it does not affect the feasibility-proxy concern and can be corrected independently.","tokens_in":16661,"tokens_out":10960,"duration_ms":105415,"concrete_test":"Run the published or released DOE and optimisation logs (or re-run the 108D case) and, at each iteration, compute for every candidate x both the latent score vector \\tilde c(x) = P_k(c(x)) and the original constraint vector c(x). Tabulate (a) the fraction of original-feasible points with min_j \\tilde c_j(x) > 0 (false rejection) and (b) the fraction of latent-feasible points (all \\tilde c_j <= 0) that violate at least one original constraint (false admission). Do the same on 10^4 independent Latin-hypercube samples using the final P_k. If either fraction is materially above zero, the acquisition's feasibility criterion is not a proxy for the original feasibility set, and the benchmark and aeroelastic results must be reinterpreted as heuristic exploration rather than constrained optimisation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the g latent constraint surrogates used in the acquisition represent the original feasibility condition c_i(x) <= 0 for i = 1..G. In Algorithm 1, as invoked by Algorithm 2, a candidate is treated as feasible when every latent coordinate satisfies \\tilde c_j(x) <= 0 (j = 1..g), and when none does the fallback minimizes \\sum_j max(\\tilde c_j(x), 0). But \\tilde c(x) = P_k(c(x)) is a (centered) linear or kernel projection of the constraint vector, not the vector itself. The set {\\tilde c_j <= 0 for all j} is a different intersection of half-spaces in constraint space; even with g = G and an orthonormal basis, coordinate-wise nonpositivity in the rotated basis is not equivalent to c_i(x) <= 0. Consequently the acquisition can be confidently wrong about feasibility regardless of how well PCA captures variance: a truly feasible point with positive latent score is rejected, and an infeasible point can be selected. The paper only checks original feasibility after the expensive evaluation (Section 4.2), so this misdirection directly consumes the search budget. The eigenvalue decay in Figure 6 bounds reconstruction error of the constraint values, not boundary classification error, so it cannot validate the acquisition. This is not cured by 'consistency of physics across loadcases' unless that consistency also forces all feasible points into a common orthant of the PCA basis, which is neither stated nor tested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a dimensionality-reduction wrapper for Scalable Constrained Bayesian Optimization (SCBO) in which the G constraint outputs are projected via PCA or kernel PCA onto g latent coordinates, and g independent Gaussian processes are trained on these coordinates instead of one GP per original constraint. Algorithm 2 embeds this in the TuRBO/Thompson-sampling loop. The authors report a complexity reduction from O((G+1)N^3) to O((g+1)N^3+G^3) and validate the method on the 7D speed reducer benchmark with 11 black-box constraints and on a 108D aeroelastic tailoring problem with G=1786 constraints. On the benchmark, PCA-GP and kPCA-GP are 57-60% faster than SCBO and converge to solutions within 1.90-3.07% of the known optimum; on the aeroelastic problem, the latent-space methods find feasible designs while SCBO crashes from memory and random search/CMA-ES do not.","tokens_in":16994,"tokens_out":8740,"duration_ms":78256,"significance":"The paper targets a real bottleneck: constrained BO with thousands of constraints is currently impractical because independent GPs have O(GN^3) training cost and O(GN^2) storage. The 108D, G=1786 aeroelastic demonstration is a useful existence proof, and the reported speedups on the speed reducer are concrete. The eigenvalue decay analysis and the comparison with KS constraint aggregation are also informative. The main contribution is potentially valuable but, as it stands, the acquisition is not guaranteed to respect the original feasibility condition; the manuscript needs either a corrected formulation or direct evidence that latent-space feasibility approximates original feasibility.","major_comments":[{"comment":"As written, the acquisition selects candidates for which all latent scores \\tilde c_j(x) are nonpositive. Because \\tilde c(x)=P_k(c(x)) is a projection (a rotation followed by truncation), the condition \\tilde c(x) \\le 0 is not equivalent to c_i(x) \\le 0 for all i. Even if g=G and the basis is orthonormal, coordinatewise nonpositivity in the rotated basis is a different condition than nonpositivity in the original basis. A truly feasible design can therefore be rejected by the acquisition, and an infeasible design can be accepted. The paper only verifies original feasibility after the expensive evaluation (Sec. 4.2), so these misclassifications consume search budget without being corrected. The eigenvalue decay in Figure 6 bounds reconstruction error of the constraint values, not boundary classification error, so it cannot validate the acquisition. Please provide either a corrected acquisition that propagates latent GP uncertainty back to the original constraints or a direct empirical test that {x: \\Psi_g^T c(x) \\le 0} approximates {x: c(x) \\le 0} on held-out points.","section":"Sec. 3.3, Algorithm 1, Algorithm 2, Eqs. (20), (26)"},{"comment":"The premise stated in Section 4.2, namely that consistency of the physics across loadcases justifies compression, supports low-rankness of the constraint outputs, but it does not support the stronger claim that the feasible boundary is aligned with the PCA coordinate axes. Principal components maximize variance, not separation between feasible and infeasible designs. The paper should test boundary preservation directly by computing latent scores and original feasibility on a held-out set of points (both random and generated by the optimiser) and reporting the confusion matrix for g=35 and for several values of g.","section":"Sec. 4.2"},{"comment":"The sensitivity study of g in Figure 4 shows that g=1 fails to find a feasible point while g=2,4,6 succeed, but it does not identify whether the failure is due to reconstruction error, boundary misclassification, or simply a smaller search space for the acquisition. Without a per-iteration or per-candidate diagnostic, the benchmark cannot separate the validity of the latent-feasibility model from the optimisation dynamics. A false-positive/false-negative analysis on the benchmark's known feasible region would make the empirical claim much stronger.","section":"Sec. 4.1, Fig. 4"},{"comment":"The complexity statement O((g+1)N^3+G^3) is incomplete. Algorithm 2 recomputes the projection P_k at every iteration, so the G^3 (or, for kPCA, the N^3 kernel eigendecomposition) cost is incurred per iteration and not only once; the total cost is O(T(G^3+(g+1)N^3)) where T is the number of iterations. The authors should state whether the reported timings include the per-iteration projection cost and how this cost scales with N and G.","section":"Sec. 3.4, Algorithm 2"}],"minor_comments":[{"comment":"The kernel matrix is defined as K_{ij} := (\\phi(c(x_j)), \\phi(c(x_j))), which appears to be a typo; it should be \\phi(c(x_i))^T \\phi(c(x_j)).","section":"Sec. 3.2, Eq. (24)"},{"comment":"The reference to 'Mathoron's rule' should read 'Matheron's rule'.","section":"Sec. 3.4"},{"comment":"The heading 'CONSTRAINED THOMSPON SAMPLING' contains a typo; it should read 'CONSTRAINED THOMPSON SAMPLING'.","section":"Algorithm 1"},{"comment":"The phrase 'principle components' should be 'principal components' throughout, including the subsection heading and the discussion of Figure 3.","section":"Sec. 3.1, Sec. 4.1"},{"comment":"In the right panel, the horizontal axis is labeled 'Sample Index' but the plotted quantity is eigenvalues indexed by component number; the axis label should be 'Component Index'.","section":"Fig. 3"},{"comment":"The data set D0 is introduced with only objective values, while later sections include constraint vectors in D; the notation should be adjusted so that the constrained-data setting is introduced explicitly before Eq. (2).","section":"Sec. 2.1"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The empirical demonstration is real, but the methodological gap around latent-space feasibility is substantive and will require new experiments or a reformulated acquisition. I do not think the paper should be rejected outright, because the computational idea is promising and the benchmark evidence is useful; however, in its current form the paper overclaims that it solves the large-scale constrained BO problem. If the authors can provide held-out classification evidence or correct the acquisition, the paper would be a good fit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What's new: they take Higdon-style output PCA/kPCA and apply it to constraints inside SCBO, replacing G constraint GPs with g latent GPs. On the speed reducer they show 57-60% time savings with modest objective degradation (1.9-3.1% above the known optimum), and on a 108D aeroelastic problem with 1,786 constraints, SCBO crashes while their method runs and finds feasible designs. That is a real practical gain and worth discussing.\n\nThe soft spots. The load-bearing one is exactly the stress-test you ran. The acquisition treats a point as feasible when every latent score is <=0. But those scores are linear combinations of the original constraints in a rotated basis. The set {Psi_g^T c <= 0} is a different intersection of half-spaces from {c_i <= 0}. Even with g=G this is not equivalent unless the basis is a signed permutation. So a feasible point can have positive latent scores and be rejected, and an infeasible point can look feasible. The paper checks original constraints only after the expensive evaluation; that filters the final batch, but the acquisition is optimized against the wrong condition, so the search is systematically misdirected. The 'consistency of physics' argument addresses low-rank structure, not sign preservation, and Figure 6's reconstruction error does not bound classification error. The benchmark still works, which suggests the heuristic may survive in practice, but the paper neither identifies this mismatch nor analyzes when it breaks.\n\nOther soft spots: no code or data released; the text says N=20 initial samples while the Figure 3 caption says N=10; and the aeroelastic case has no reference optimum, so 'convergence' is just a decreasing curve with no ground truth. The complexity claim O((g+1)N^3 + G^3) is fine, though the G^3 eigendecomposition can still be a bottleneck at larger G.\n\nCredit where it is earned: the components are prior art, but the combination is new and the application is substantial. The benchmark results are honestly reported, including the solution-quality loss. The paper is readable and the authors clearly know the surrounding literature.\n\nRecommendation: send to peer review, but expect major revision. A referee should push them to fix the acquisition (e.g., model the projected feasible region explicitly, or add a classification head) or at least quantify the misclassification rate of the latent feasibility test on real data. Without that, the method's benchmark success is unexplained rather than justified.","headline":"A promising speed-up for constrained BO undercuts itself: the latent-space feasibility test is not equivalent to original feasibility, and the paper never confronts that.","tokens_in":17545,"tokens_out":2681,"would_cite":false,"duration_ms":24022,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A low-dimensional constraint subspace lets Bayesian optimisation scale to 1,786 constraints and a 108-variable aeroelastic design problem.","keywords":["Bayesian optimisation","Gaussian processes","large-scale constraints","latent space","principal component analysis","kernel PCA","aeroelastic tailoring","constrained optimisation"],"falsifier":"Take a benchmark with G independent or near-independent constraint functions (for instance, each design variable has its own unrelated constraint) and run PCA-GP SCBO with a small g. If the eigenvalue decay is slow and the method returns designs that violate many original constraints while still appearing feasible in the latent space, the central compression claim fails. A more quantitative version compares the fraction of original constraints satisfied at the returned optimum with the latent feasibility prediction: a mismatch indicates the subspace has not captured the feasible boundary.","tokens_in":16456,"feed_emoji":"⚙️","tokens_out":5052,"duration_ms":39136,"temperature":0.7,"pith_summary":"The paper argues that the main obstacle to applying Bayesian optimisation (BO) to high-dimensional engineering design is not the number of design variables but the number of constraints, each of which normally needs its own Gaussian-process surrogate. It proposes projecting the constraint outputs onto a low-dimensional latent subspace via PCA or kernel PCA and training only g of surrogate Gaussian processes on that subspace instead of G. In the 108-dimensional aeroelastic tailoring problem with 1,786 black-box constraints, this makes constrained BO run successfully on a standard workstation, while the baseline SCBO method crashes from memory demands after one iteration. The method also finds feasible designs where random search and CMA-ES find none, and it reaches near-optimal values on a 7D speed-reducer benchmark with 11 constraints.","feed_headline":"Low-rank constraints let Bayesian optimisation handle 1,786 of them","feed_subtitle":"Projecting 1,786 constraint outputs onto 35 latent dimensions finds feasible designs where standard SCBO runs out of memory.","key_machinery":"The load-bearing object is the truncated projection matrix Ψ_g ∈ $R^{{G×g}}$ obtained from an eigendecomposition of the constraint covariance matrix (PCA) or of the kernel matrix (kPCA). The projection maps each design's constraint vector c(x) ∈ R^G to a latent coordinate vector c~(x) ∈ R^g, and the optimisation then builds g+1 independent Gaussian-process surrogates (objective plus the g latent constraints) inside the SCBO trust-region framework. Because feasibility is always checked against the original 1,786 constraints during batch evaluation, the latent GPs only guide the acquisition; the projection is recomputed each iteration as data accumulate. The complexity reduction is the argument's spine: O((g+1)$N^{3}$ + $G^{3}$) versus O((G+1)$N^{3}$) for independent-constraint surrogates.","core_discovery":"The central claim is that constraint outputs in many large-scale design problems are compressible: the G-dimensional vector of constraint values c(x) lies close to a g-dimensional subspace, so modelling the g latent coordinates with independent GPs is an adequate proxy for the feasibility of all G original constraints. Using PCA (or its kernel extension) to build a projection Ψ_g from the constraint matrix C ∈ $R^{{N×G}}$, the paper reduces the surrogate-training cost from O((G+1)$N^{3}$) to O((g+1)$N^{3}$ + $G^{3}$), where the $G^{3}$ term is the one-time eigendecomposition. In the aeroelastic case, g = 35 latent GPs replace 1,786 surrogates; with g = 35 the approach converges to a feasible, reduced-mass design, whereas the standard SCBO implementation crashes from insufficient memory and both random search and CMA-ES fail to find a feasible point. The paper further shows that adding a second loadcase doubles the number of constraints but barely increases the number of principal components needed.","pith_inferences":["One could estimate the achievable compression up front by computing the PCA eigenvalue decay of a small pilot design-of-experiments; problems without a fast decay, such as nearly independent constraints, would be poor candidates for this method.","The same latent-space surrogate idea could extend to other acquisition functions and to multi-fidelity or design-under-uncertainty settings, where the variance of the latent GPs would propagate back through Ψ_g to the original constraints.","A testable extension is an adaptively chosen g during optimisation, which the paper lists as future work and would eliminate the user-specified principal-components count.","The failure of random search and CMA-ES on the 108D problem suggests that the main benefit lies in focusing samples on the feasible boundary via the latent surrogates, a claim one could isolate by ablating the trust-region mechanism."],"forward_implications":["Problems with thousands of black-box constraints that were previously out of reach for SCBO become tractable on ordinary hardware.","Adding loadcases multiplies the number of constraints but adds little to the required number of latent components, since the underlying physics is shared across loadcases.","The method applies to any large-scale constrained BO problem, not just aeroelastic tailoring, provided the constraint outputs are low-rank.","The choice of g matters: too small a subspace (g = 1 on the benchmark) loses feasibility information, while larger g reduces the computational savings.","Recomputing the projection each iteration lets the method adapt as data accumulate, though in the low-dimensional benchmark fixing the initial projection performed even better."],"supporting_citations":[{"why":"Supplies the idea of constructing GP surrogates on a low-dimensional output subspace rather than on the full output space.","marker":"Higdon et al. [2008]"},{"why":"Provides kernel PCA, used as the nonlinear extension for building the latent constraint projection.","marker":"Schölkopf et al. [1998]"},{"why":"Defines the SCBO algorithm that the paper extends with latent-space constraints.","marker":"Eriksson and Poloczek [2021]"},{"why":"Supplies the PCA formulation and eigenvalue decomposition used to construct Ψ_g.","marker":"Jolliffe and Cadima [2016]"},{"why":"Provides the 7D speed-reducer benchmark with 11 black-box constraints used for validation.","marker":"Lemonge et al. [2010]"},{"why":"Defines the 108D aeroelastic tailoring problem with 1,786 constraints to which the method is applied.","marker":"Maathuis et al. [2024]"},{"why":"Provides the comparison showing that (k)PCA-GP accuracy is acceptable relative to higher-order GPs, used to justify latent-space surrogates.","marker":"Zhe et al. [2019]"}],"fun_headline_variants":["1,786 constraints collapsed to 35 latent GPs for aeroelastic design","Latent GPs make 1,786-constraint aeroelastic optimisation feasible","35 latent GPs replace 1,786 surrogates for design constraints","Memory-friendly BO: 35 latent GPs for 1,786 constraints","Bayesian optimisation scales to 1,786 constraints via latent space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method works only if the constraint outputs across the design space lie close to a low-dimensional subspace, so that the dominant principal components carry enough information about feasibility; if the constraints are effectively independent or high-rank, latent-feasible designs will violate many original constraints and the advantages disappear.","fun_headline_variants_meta":{"raw":{"variants":["1,786 constraints collapsed to 35 latent GPs for aeroelastic design","Latent GPs make 1,786-constraint aeroelastic optimisation feasible","35 latent GPs replace 1,786 surrogates for design constraints","Memory-friendly BO: 35 latent GPs for 1,786 constraints","Bayesian optimisation scales to 1,786 constraints via latent space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001028,"raw_usage":{"total_tokens":4359,"prompt_tokens":998,"completion_tokens":3361,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":3259}},"tokens_in":614,"tokens_out":3361,"duration_ms":22495,"temperature":1.0,"reasoning_tokens":3259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:11:42.524770+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a benchmark with G independent or near-independent constraint functions (for instance, each design variable has its own unrelated constraint) and run PCA-GP SCBO with a small g. If the eigenvalue decay is slow and the method returns designs that violate many original constraints while still appearing feasible in the latent space, the central compression claim fails. A more quantitative version compares the fraction of original constraints satisfied at the returned optimum with the latent feasibility prediction: a mismatch indicates the subspace has not captured the feasible boundary.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the comparison showing that (k)PCA-GP accuracy is acceptable relative to higher-order GPs, used to justify latent-space surrogates."}],"review_version":1}