{"id":"46a8b848-5ff0-4bde-8c2b-24ee3794d2c0","arxiv_id":"1908.11272","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Bayesian optimization method that builds Gaussian process surrogates in a PCA-derived eigenshape basis, focusing on output-relevant directions, improves low-budget shape optimization.","lead":"This paper proposes a way to optimize shapes with many CAD parameters using only a small number of expensive simulations. It maps shapes to a compact eigenshape basis, identifies which shape directions matter for the objective, and builds a surrogate model that focuses on those directions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pre-image quality is the load-bearing risk: the claimed gain of EI-in-eigenspace plus replication depends on solving Eq. 16 well, but the paper never measures pre-image error.","rationale":"The reader's weakest_assumption correctly identifies the pre-image problem as load-bearing, so I partially agree. However, I sharpen the concern beyond the existence of a good pre-image: the paper explicitly relies on the replication strategy to beat its own no-replication baseline (Table 16: AddGP embed with replication 26.19 vs no replication 27.58), and replication is only defensible if the virtual point (α*, f(x*)) is placed at coordinates that the GP treats as close to the true evaluated point α(x*). The paper provides no numerical evidence on pre-image residuals, no sensitivity analysis of the threshold d0 used to trigger replication, and no report of how often the pre-image solver fails or lands on a different local optimum. This is an internal-consistency check, not a disagreement with community consensus: the method could still be correct, but the current evidence does not establish that the optimized α* is actually realized in the CAD space. Because the central claim is a low-budget speedup claim, and the main reported advantage on the CFD case is that the eigenshape run starts optimizing during the other method's initial DoE, the quality of the proposed x* shapes matters for the claimed speedup too. I also note the paper honestly reports negative results (e.g., GP(α1:2) failing for f4, Table 12) and shares much-desired detail on the additive GP, but those strengths do not close the specific pre-image gap. A conditional verdict is appropriate: the method is promising and well tested at the metamodeling level, but the optimization-level claim needs one additional experiment that reports or bounds the pre-image error. I do not recommend rejection because the evidence for improved R² in reduced eigenspaces (Tables 11-14) is independent and convincing for the modeling claim; the concern applies specifically to the optimization branch and the replication benefit. The proposed test is a single reproducible check that would either confirm or close the weak link.","tokens_in":37446,"tokens_out":2384,"duration_ms":21444,"concrete_test":"For the catenoid (Example 5) and NACA 22 (Example 7) experiments, instrument the optimization loop to record, at every iteration, the pre-image residual r_n = ‖(φ(x_{n+1})−φ̄)−Vα_{n+1}*‖ / ‖Vα_{n+1}*‖ together with the corresponding distance d(α_{n+1}*, α(x_{n+1})) in the reduced space relevant to the GP's length scales. Then (1) compute the fraction of iterations where r_n exceeds a threshold equal to 0.25 times the typical GP length scale in the active dimensions, and (2) rerun the AddGP-EI embed comparison from Table 16 with replication disabled but only accepting EI proposals whose pre-image residual is below that threshold. If the replication advantage disappears or the acceptance-filtered runs fail to reproduce the reported best values (26.19 vs 27.58), then the central claim depends on uncontrolled pre-image quality rather than on the eigenshape modeling idea.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that at low budgets the eigenshape workflow gives a more accurate model and faster optimization than working directly in CAD parameters. The optimization branch of that claim depends critically on the pre-image step (Eq. 16 in Section 4.2): the EI maximizer α* is proposed in eigenspace, then the actual evaluated point is x* = argmin ‖φ(x)−(Vα*+φ̄)‖, and the GP is updated at α(x*) and possibly at the virtual point (α*, f(x*)) under the replication strategy. For the acquisition function to make sense, α(x*) must be close to α*, and for replication not to corrupt the surrogate, the distance between φ(x*) and Vα*+φ̄ must be small relative to the GP's length scales. The paper only says the pre-image problem is 'relatively not costly' to solve, but it never reports the residual norm achieved, how often the EI maximizer lies outside the manifold, or which optimizer/settings were used for Eq. 16. The issue is not merely practical: if φ is injective but the image φ(X) is strongly curved in R^D, the orthogonal projection of an off-manifold α* onto the manifold can land far from the α-region that the EI actually targeted, and the replication point (α*, f(x*)) then teaches the GP a false value at unattainable coordinates. This is a genuine correctness risk for the central claim because the replication strategy's benefit is observed experimentally (Table 16, e.g. 26.19 with replication vs 27.28 no replication), but those comparisons would be misleading if the pre-image projection error was large for the same runs. None of the reported experiments include a check that the returned x* actually realizes the intended α*, so the improvement attributed to 'replication' and 'embedding' may be partly an artifact of matching poor projections to poor acquisition targets.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end method for Bayesian optimization of expensive parametric shape functions. CAD shapes are represented by high-dimensional discretizations φ(x); a PCA on a large database of such representations yields an eigenshape coordinate system α; an L1-penalized GP likelihood selects a small set of output-relevant active coordinates αa; an additive GP with an anisotropic kernel over αa and an isotropic kernel over the remaining coordinates models the objective; and expected improvement is maximized over the active coordinates plus a random line in the inactive coordinates. The EI maximizer is mapped back to CAD parameters by solving the pre-image problem (Eq. 16), and a replication strategy adds a virtual point at the off-manifold EI maximizer. The paper reports R² comparisons on four benchmarks (Tables 11–14), optimization comparisons on the catenoid and rectangular-heart problems (Tables 15–17), and two NACA 22 airfoil runs (Figures 40–41). The abstract claims that at low budgets the methodology delivers a more accurate model and faster optimization than working directly with the CAD parameters.","tokens_in":37833,"tokens_out":7785,"duration_ms":76310,"significance":"The central claim is practically valuable and, for the metamodeling branch, largely supported: the additive GP achieves the best R² on every benchmark, test sets are independent, and the authors honestly report cases where unsupervised dimension reduction fails (e.g., GP(α1:2) on the heart problem). The optimization experiments also show consistent gains for the additive GP with embedding and replication on the catenoid and heart problems. If the pre-image accuracy issue is resolved, the method is a useful contribution to high-dimensional shape optimization, since the shape mapping and PCA are cheap compared with simulator calls. The paper does not ship code or data, but the analytic benchmarks are specified in enough detail to be reproducible.","major_comments":[{"comment":"The optimization branch of the central claim rests on the pre-image step, but its quality is never measured. The EI maximizer α* selects a shape in eigenspace; the evaluated point is x* = argmin ‖φ(x)−(Vα*+φ̄)‖, and the GP is updated at α(x*) and possibly at the virtual point (α*, f(x*)). If ‖φ(x*)−(Vα*+φ̄)‖ is comparable to the GP length scales, the acquisition evaluates a different point than intended and replication teaches the surrogate a value at an unattainable coordinate. The manuscript only says that solving Eq. (16) is 'relatively not costly' and never reports the achieved residual, the optimizer and bounds used for Eq. (16), the fraction of iterations with α* outside the manifold, or the distance between α(x*) and α* during optimization. These diagnostics should be provided for Tables 16–17 and the NACA runs; if the residuals turn out to be non-negligible, the comparison between replication and no replication needs to be revisited.","section":"Section 4.2, Eq. (16)"},{"comment":"The active-set procedure depends on several hyperparameters that are set once without sensitivity analysis: the L1 weight λ = n/D in Eq. (8), the factor 10 in Definition 1, and the eigenvalue-truncation criterion for d′ in Section 2.3 (the text gives '99.9, 99.95 or 99.99%' without stating the value used in each benchmark). Since the additive model's performance and the EI embedding dimension are driven by the selected active set, the central comparisons in Tables 12–14 and 16–17 could partly reflect these choices. A sensitivity study over reasonable ranges of λ and the active threshold, reporting both R² and optimization traces, would let the reader assess robustness.","section":"Section 3.2.1, Definition 1"},{"comment":"Replication is presented as a constant prolongation of the surrogate outside the manifold, but the validity of that prolongation is an assumption rather than a demonstrated fact. The only evidence is the aggregate improvement in Table 16 (26.19 with replication versus 27.58 without). If the pre-image residual is large, the replicated point (α*, f(x*)) is a false observation at an unattainable coordinate and can bias the GP. The threshold d0 in Definition 2, the minimal pairwise distance among the 5000 database shapes, is likewise not related to GP length scales and can strongly affect replication frequency. Please provide, for the experiments of Table 16, the distribution of ‖φ(x*)−(Vα*+φ̄)‖, the fraction of replication events, and a comparison against an alternative that updates only at the projected point but avoids repeated sampling of the same α* region; this would isolate the benefit of the virtual point from the benefit of variance reduction at α*.","section":"Section 4.2, Definition 2"},{"comment":"The engineering demonstration on the NACA 22 airfoil consists of only two runs per objective, and the two algorithms use different initial-design sizes (10 initial points for AddGP(αa+ᾱa)-EI embed versus 50 for GP(X)-EI(X)). With two runs and a confounded initialization budget, the 'faster optimization' conclusion for the realistic CFD case is not statistically established. I recommend either more repeats, possibly on a cheaper surrogate of the CFD code, or explicit framing of the NACA results as an illustrative case study rather than as support for the general claim.","section":"Section 4.3.2, Figures 40–41"}],"minor_comments":[{"comment":"The text states, 'Without formal proof, we think that this is related to the observed property that the d first eigencomponents … make a convex set'; since convexity is relevant to the projection discussion in Section 4.2, this conjecture should be either proven, cited, or explicitly marked as non-essential.","section":"Section 2.3.1"},{"comment":"The passages 'Numerical experiments not reported here for reasons of brevity…' and 'In experiments that are not reported here…' assert robustness properties (local optima of the penalized likelihood, behavior with normalized αj's) without evidence; these claims should be removed or the supporting experiments reported in an appendix.","section":"Section 3.2.1"},{"comment":"The signed-distance function and the contour discretization are both denoted D, which makes headings such as 'Signed Distance Discretization' ambiguous; using distinct symbols such as D_SD and D_cont would improve readability.","section":"Section 2.1, Tables 1–5"},{"comment":"The one-dimensional embedding coordinate α and the inactive vector ᾱa are visually similar, and the Kronecker symbol δ overlaps with the active dimension δ; a different symbol for the embedding coordinate would reduce confusion.","section":"Section 4.1"},{"comment":"There are typographical artifacts such as 'a vector ofd Computer Aided Design' in Section 1 and an unbalanced parenthesis in Definition 1 ('and{ααα(1:n), y(1:n))}'); a careful proofread is needed.","section":"Section 1 and Definition 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read of the Gaudrie et al. extended manuscript. The paper is a legitimate methods contribution: it combines PCA on contour-discretized shape representations, L1-penalized GP length-scale selection, an additive GP that keeps inactive eigenshapes in an isotropic term, and an EI maximization that embeds the inactive coordinates along a random line. The specific pipeline is new, and the empirical finding that contour discretization recovers intrinsic dimension almost exactly (eigenvalues drop to zero beyond the true dimension) is genuinely useful for shape parameterization work. The modeling comparisons are also honest: independent test sets, several benchmarks, and negative results are reported rather than hidden—GP(alpha_1:2) fails on the rectangle because the first two modes affect a quantity the objective ignores, and naive variance-based truncation is shown to be unreliable.\n\nThe soft spots are mostly about evidence strength rather than hidden circularity. There is no released code or data, hyperparameters such as lambda = n/D, the active-dimension threshold, and the replication distance threshold are set without sensitivity analysis, and the CFD demonstration is only two runs per objective. Those are fixable. More substantive is the pre-image issue. The optimization branch depends on solving Eq. (16) well enough that the proposed alpha* is close to the alpha(x) actually evaluated, and the replication strategy deliberately adds the virtual point (alpha*, y). The paper never reports pre-image residual norms, how often EI maximizers fall off the manifold, or how far off they are. Without those numbers, the observed gains from replication (Table 16) cannot be cleanly attributed to the method rather than to matching a poor projection with a poor acquisition. This does not sink the paper—the modeling results stand on their own—but it is exactly the kind of gap a reviewer should ask to close.\n\nOverall, the central idea is sound, the paper is honest about its own limitations, and the contribution is real. I would send it to review, with a request for code/data, pre-image diagnostics, and sensitivity analysis. It deserves a serious referee.","headline":"A solid, honest methods paper whose modeling claims hold up, but the optimization branch needs pre-image diagnostics and sensitivity analysis before I'd fully trust the headline gains.","tokens_in":38356,"tokens_out":2678,"would_cite":true,"duration_ms":28897,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that building Gaussian-process surrogates in a PCA-derived eigenshape basis gives more accurate models and faster optimization than building them in the original CAD parameter basis, at low evaluation budgets.","keywords":["eigenshape basis","Gaussian process surrogate","shape optimization","Bayesian optimization","dimension reduction","principal component analysis","random embedding","pre-image problem"],"falsifier":"Run the full pipeline on a parametric family where the map from eigenshape coefficients back to shapes is known to be poor, such as shapes with many independent local bumps that create a highly non-convex or high-dimensional manifold; if the distance $\\|\\alpha^{(n+1)*}-\\alpha(x^{(n+1)})\\|$ does not decay as the budget grows and the additive eigenshape GP's best value stays worse than a CAD-parameter GP at equal budget, the central claim fails.","tokens_in":37249,"feed_emoji":"📐","tokens_out":9032,"duration_ms":82428,"temperature":0.7,"pith_summary":"This paper tries to show that when the objective is an expensive simulation of a parametric shape, the surrogate model and the search should live in a reduced coordinate system made of \"eigenshapes,\" not in the original CAD parameter space. The authors construct that coordinate system by mapping shapes into high-dimensional contour descriptions, applying PCA, and selecting the eigen-directions that most affect the output through a penalized likelihood. They then fit a Gaussian process that is anisotropic on those active directions and isotropic on the rest, optimize expected improvement in the active directions plus a random line through the inactive ones, and map proposed eigen-coordinates back to CAD parameters via a pre-image problem. On circle, rectangle, catenoid, and airfoil benchmarks, they report that at small evaluation budgets this pipeline gives more accurate models and faster convergence than the classical CAD-parameter GP. If true, this would make Bayesian optimization practical for engineering shape design with hundreds of parameters and only tens to hundreds of expensive simulator calls.","feed_headline":"Eigenshape coordinates beat CAD parameters on low budgets","feed_subtitle":"A PCA-derived coordinate system plus an additive GP finds better shapes in fewer simulator calls.","key_machinery":"The load-bearing machinery is a chain of four objects. First, a high-dimensional shape mapping $\\varphi(x)$ — in this paper the discretization of the contour, not the characteristic function or signed distance — gives a representation space in which PCA eigenvalues drop to zero at the true effective dimension and the eigenvectors $v_j$ are interpretable \"eigenshapes.\" Second, an $L_1$-penalized concentrated log-likelihood selects the active eigenshape components $\\alpha_a$ by driving inverse length-scales to zero; this is the supervised step that prevents geometric modes irrelevant to the output from dominating the surrogate. Third, the additive GP $Y(\\alpha)=\\beta+Y_a(\\alpha_a)+Y_{\\bar a}(\\alpha_{\\bar a})$ splits the fit into an anisotropic GP on $\\alpha_a$ and an isotropic GP on the inactive components $\\alpha_{\\bar a}$, keeping the number of learned hyperparameters small enough for $n\\ll D$. Fourth, optimization couples expected improvement maximization in $\\alpha_a$ with a random embedding along one line in $\\alpha_{\\bar a}$, and the pre-image problem together with the replication strategy ties proposed eigen-coordinates back to evaluable CAD parameters without letting repeated off-manifold proposals fool the surrogate.","core_discovery":"The paper's central claim is that the effective coordinates for shape surrogate modeling and optimization are the PCA coefficients $\\alpha = V^\\top(\\varphi(x)-\\bar\\varphi)$ of a discretized contour representation, and that the objective's variation is concentrated on a small subset of these coordinates. To exploit this, the authors maximize an $L_1$-penalized GP log-likelihood to declare a subset $\\alpha_a$ of eigenshape components active, and model $Y(\\alpha)=\\beta+Y_a(\\alpha_a)+Y_{\\bar a}(\\alpha_{\\bar a})$, an additive GP with an anisotropic kernel on the active components and an isotropic kernel on the inactive ones. Optimization maximizes expected improvement over $\\alpha_a$ together with a random direction in the inactive space, solves the pre-image problem $x^{(n+1)}=\\arg\\min_x\\|\\varphi(x)-\\bar\\varphi-V\\alpha^{(n+1)*}\\|^2$ to obtain an evaluable design, and, when the proposed $\\alpha^*$ is far from the recovered $\\alpha(x)$, updates the GP with both points (\"replication\"). Across all benchmark problems, including a CFD NACA airfoil, the additive eigenshape GP with this acquisition strategy yields higher $R^2$ and lower objective values at low budgets than a GP built directly in the CAD parameters $x$.","pith_inferences":["Inference: Because the method only requires a fast auxiliary map $\\varphi$ and a database of plausible inputs, it should transfer to any high-dimensional input whose output is governed by a low-dimensional geometric description, such as images, signals, or material microstructures, not just CAD contours.","Inference: The replication strategy suggests a general principle for latent-space Bayesian optimization: whenever the acquisition maximizer cannot be exactly realized, add the virtual observation at the proposed latent point; the paper's experiments indicate this improves exploration but do not prove it in general.","Inference: If the observed near-convexity of the eigenshape manifold for contour discretizations holds more generally, the pre-image projection would be unique and the whole loop could be accelerated; proving or disproving convexity of $\\mathcal{A}$ is a testable mathematical question."],"forward_implications":["At evaluation budgets of 20 to 200 calls, the additive eigenshape GP gives higher prediction $R^2$ than a GP in the CAD parameter space on every benchmark tested, including a CFD airfoil with 22 parameters.","When the first eigenshapes do not affect the objective (as in the rectangle case), selecting active components by penalized likelihood avoids the failure mode of truncating to the first principal components.","Optimizing expected improvement in the active subspace plus a random line in the inactive subspace reaches lower objective values in fewer calls than optimizing over all variables or only active variables.","Replicating the objective value at off-manifold proposed points removes their acquisition uncertainty and prevents the optimizer from repeatedly proposing the same unreachable shape.","The method's advantage is largest at small budgets: as the number of evaluations grows, higher-dimensional eigenshape models catch up, so the claim is specifically about early-stage optimization."],"supporting_citations":[{"why":"Supplies the Efficient Global Optimization loop that the proposed algorithm extends to eigenshape coordinates.","marker":"[24]"},{"why":"Defines the Expected Improvement acquisition criterion maximized in the reduced space.","marker":"[31]"},{"why":"Introduces the random-embedding idea that motivates optimizing the inactive eigenshapes along a random line.","marker":"[51]"},{"why":"Frames the mapping-plus-PCA construction as kernel PCA in a known feature space.","marker":"[41]"},{"why":"Provides the Point Distribution Model contour discretization used as the shape mapping.","marker":"[12]"},{"why":"Is the Kriging-with-PLS dimension-reduced baseline that the proposed method is compared against.","marker":"[8]"},{"why":"Is the active-subspace dimension-reduction approach that motivates output-driven reduced bases.","marker":"[11]"},{"why":"Supplies the $L_1$-penalized GP likelihood used for selecting active eigenshape dimensions.","marker":"[53]"},{"why":"Is a related GP with built-in dimensionality reduction, providing a baseline for the supervised dimension-reduction comparison.","marker":"[47]"},{"why":"Provides the derivative-aware genetic optimizer used to maximize the Expected Improvement.","marker":"[29]"}],"fun_headline_variants":["Eigenshape GP finds better shapes with fewer simulator calls","Additive GP in eigenshape space speeds up shape optimization","Active eigenshapes cut simulator calls for shape design","PCA eigenshapes beat CAD params for cheap surrogate modeling","Shape optimization: eigenshape GP beats CAD coordinates at low budget"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pre-image problem can be solved well enough: the CAD parameters recovered from a proposed eigenshape vector must produce a shape whose actual eigen-coordinates are close to the proposed ones, otherwise the GP is updated at points the acquisition function did not choose and replicated data mislead the surrogate.","fun_headline_variants_meta":{"raw":{"variants":["Eigenshape GP finds better shapes with fewer simulator calls","Additive GP in eigenshape space speeds up shape optimization","Active eigenshapes cut simulator calls for shape design","PCA eigenshapes beat CAD params for cheap surrogate modeling","Shape optimization: eigenshape GP beats CAD coordinates at low budget"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1299,"prompt_tokens":1017,"completion_tokens":282,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":208}},"tokens_in":633,"tokens_out":282,"duration_ms":3202,"temperature":1.0,"reasoning_tokens":208,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:19:41.362125+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the full pipeline on a parametric family where the map from eigenshape coefficients back to shapes is known to be poor, such as shapes with many independent local bumps that create a highly non-convex or high-dimensional manifold; if the distance $\\|\\alpha^{(n+1)*}-\\alpha(x^{(n+1)})\\|$ does not decay as the budget grows and the additive eigenshape GP's best value stays worse than a CAD-parameter GP at equal budget, the central claim fails.","supporting_citations":[{"cited_title":"Kernel principal component analysis","cited_arxiv_id":null,"evidence_quote":"Frames the mapping-plus-PCA construction as kernel PCA in a known feature space."},{"cited_title":"Active shape models-their training and application","cited_arxiv_id":null,"evidence_quote":"Provides the Point Distribution Model contour discretization used as the shape mapping."},{"cited_title":"Efficient Global Optimization of expensive black-box functions","cited_arxiv_id":null,"evidence_quote":"Supplies the Efficient Global Optimization loop that the proposed algorithm extends to eigenshape coordinates."},{"cited_title":"On Bayesian methods for seeking the extremum","cited_arxiv_id":null,"evidence_quote":"Defines the Expected Improvement acquisition criterion maximized in the reduced space."},{"cited_title":"Bayesian optimization in high dimensions via random embeddings","cited_arxiv_id":null,"evidence_quote":"Introduces the random-embedding idea that motivates optimizing the inactive eigenshapes along a random line."},{"cited_title":"Im- proving kriging surrogates of high-dimensional design models by partial least squares dimension reduction","cited_arxiv_id":null,"evidence_quote":"Is the Kriging-with-PLS dimension-reduced baseline that the proposed method is compared against."},{"cited_title":"Active subspace methods in theory and practice: applications to kriging surfaces","cited_arxiv_id":null,"evidence_quote":"Is the active-subspace dimension-reduction approach that motivates output-driven reduced bases."},{"cited_title":"Penalized Gaussian process regression and classification for high- dimensional nonlinear data","cited_arxiv_id":null,"evidence_quote":"Supplies the $L_1$-penalized GP likelihood used for selecting active eigenshape dimensions."},{"cited_title":"Gaussian processes with built-in di- mensionality reduction: Applications to high-dimensional uncertainty propagation","cited_arxiv_id":null,"evidence_quote":"Is a related GP with built-in dimensionality reduction, providing a baseline for the supervised dimension-reduction comparison."},{"cited_title":"Genetic optimization using derivatives: the rgenoud package for R","cited_arxiv_id":null,"evidence_quote":"Provides the derivative-aware genetic optimizer used to maximize the Expected Improvement."}],"review_version":1}