{"id":"baebf4d6-e6fc-491a-afa8-dfd461553fde","arxiv_id":"2412.09183","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A VAE-based latent-space Bayesian optimisation framework with Matérn-5/2 kernels and Sequential Domain Reduction solves more 100D benchmark problems than BO-SDR and REMBO in small numerical experiments.","lead":"This paper integrates Sequential Domain Reduction (SDR) into Bayesian Optimisation run in the low-dimensional latent space of a Variational Autoencoder, and reports benchmark results on 100-dimensional problems. It also tests VAE retraining and soft triplet loss as ways to structure the latent space, comparing against random-embedding baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 'structured latent manifolds improve BO performance' is confounded: Algorithm 4 (DML, no SDR) is compared mainly against Algorithms 1 and 3 (no DML, with SDR), so the gain could come from omitting SDR rather than from DML; the paper itself notes SDR and DML conflict.","rationale":"The reader's weakest assumption (decoder coverage) is real but not the first-order threat to this paper's central claim: the benchmark optima lie near the centre of the scaled domain, where a pretrained VAE on normal samples is likely to have support. The more immediate problem is experimental identification. Algorithm 4 modifies two things at once relative to Algorithm 1 (DML and no SDR), and Appendix B.3 concedes that SDR and DML conflict. Without the DML-off/SDR-off control, the 'structured latent manifolds improve BO' claim is confounded by the presence/absence of SDR. Figure 1 provides a no-SDR/no-DML point for vanilla VAE, but the aggregate Tables 1-2 and Figure 2 do not use it to isolate DML. I recommend keeping the reader's CONDITIONAL verdict: the proposed 2x2 control would settle whether the attribution survives. This is a design/causal-identification concern, not an internal mathematical contradiction; I agree with the reader that coverage is a secondary gap, hence partial agreement.","tokens_in":11737,"tokens_out":7009,"duration_ms":67562,"concrete_test":"Run a 2x2 control on Test Set 1, D=100, d=2, budget 350, q=50, with 5 runs per cell: (i) plain ELBO without SDR (Algorithm 1 with the SDR block disabled), (ii) DML without SDR (Algorithm 4), (iii) plain ELBO with SDR (Algorithm 1), (iv) DML with SDR if stable. Compare mean best value and tau=1e-3 success rates. If (i) matches (ii) within run-to-run variance, DML is not the driver and the headline claim should be downgraded; if (ii) beats (i), the structured-latent attribution survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that DML-structured latent spaces cause S-BOVAE's superior performance. But S-BOVAE (Algorithm 4) differs from R-BOVAE (Algorithm 1) in two variables: it replaces the ELBO with the DML objective and it removes SDR from the latent-space loop. Appendix B.3 explicitly says Algorithm 4 is not implemented with SDR because 'SDR and DML methods conflict with each other in excluding the global optimum.' Thus the reported comparisons (Tables 1-2, Figure 2) are missing the control cell DML-off/SDR-off. If SDR is harmful in the latent space--which the paper's own remark suggests--then Algorithm 4 could outperform simply because it does not use SDR, with no contribution from structured latent manifolds. Figure 1 does compare vanilla VAE with and without SDR, but those data are not used to isolate DML in the aggregate solved-problem tables. Consequently, the headline attribution 'due to its structured latent spaces' is not supported by the experimental design as reported. This is load-bearing because it targets exactly the paper's central claim: structured latent manifolds improve BO performance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies latent-space Bayesian optimisation (LSBO) built on variational autoencoders. It proposes a reformulation of the VAE-based BO algorithm of Grosnit et al. for general black-box optimisation, introducing variants that use periodic VAE retraining and deep metric loss (soft triplet loss). It also implements Sequential Domain Reduction (SDR) inside BoTorch, both in the original ambient space and in VAE-generated latent spaces, and compares the resulting BO-VAE algorithms against BO-SDR and REMBO on 100-dimensional test problems. The central claim is that structured latent manifolds improve BO performance and that Algorithm 4 (S-BOVAE) performs best because of its structured latent space.","tokens_in":12044,"tokens_out":4456,"duration_ms":44082,"significance":"If substantiated, the paper would be a useful empirical contribution: it brings VAE-based LSBO beyond its original molecule-generation setting, shows the first SDR-in-latent-space implementation, and provides a direct comparison with random-embedding methods. The authors are transparent that the only theoretical regret bound they borrow does not apply to the Matérn kernel they use. However, the central empirical claim is not currently supported by the experimental design, because the algorithm comparison changes two variables at once and the statistical evidence is thin. The significance of the work therefore depends on whether the missing control experiments and uncertainty quantification can be supplied.","major_comments":[{"comment":"The comparison between Algorithm 4 (S-BOVAE) and Algorithm 1 (R-BOVAE) changes two variables simultaneously: the training objective (DML objective instead of ELBO) and the presence of SDR (Algorithm 1 includes SDR; Algorithm 4 does not). Appendix B.3 explicitly states that Algorithm 4 is not implemented with SDR because 'SDR and DML methods conflict with each other in excluding the global optimum.' Therefore, the reported superiority of Algorithm 4, attributed in Section 4 to 'its structured latent spaces', could be entirely due to the removal of SDR from the latent-space loop. The experimental design lacks the control cell with DML off and SDR off; Figure 1 compares the vanilla VAE with and without SDR but does not isolate the DML component. Please add this control, or implement Algorithm 4 with SDR, or rephrase the causal claim.","section":"Section 4 and Appendix B.3, Algorithm 4 vs Algorithm 1"},{"comment":"The reduction of problem (P) to min_{z in Z} E_{p_theta*(x|z)}[f(x)] is valid only if the decoder's latent representation covers the ambient global minimizer x*. The paper assumes there exists z with P[x* ~ p_theta*(·|z)] > 0 but provides no theoretical or numerical verification. Since the pretraining set D_U is sampled from a multivariate normal with a large covariance matrix (Appendix C) and the benchmark domains are bounded (e.g., [-30,30]^D for Ackley), it is not automatic that the decoder can generate points near x*. The authors note that the regret bound from Grosnit et al. relies on a Gaussian kernel and does not apply to the Matérn kernel used here, so the coverage assumption is the main remaining safety net and it is untested. Please report, for example, the reconstruction error of the known global minimizers for the benchmark problems, or otherwise justify that the latent-space optimum maps back to the ambient optimum.","section":"Section 3, Eq. (1) and Appendix C"},{"comment":"The headline numerical claim rests on very small samples: each algorithm was run twice on each problem in Test Set 1 and twice on each randomized problem in Test Set 2 (10 problems), while Figures 1 and 2 average only 5 runs. No confidence intervals or standard errors are reported for the success rates in Tables 1 and 2; at tau = 10^-3, Table 1 shows 50% for all three VAE variants and Table 2 shows 40%, 30%, and 20% for the three variants. With 10 problems, these differences are not shown to be statistically significant, and the statement that Algorithm 4 'consistently performed best' is not supported. The paper also does not provide code, seeds, or per-problem traces. Please include per-problem results, confidence intervals for success proportions, and make the implementation or at least the per-run data available.","section":"Section 4, Tables 1-2 and Figures 1-2"}],"minor_comments":[{"comment":"The notation 'V AE', 'BO-V AE', and 'V AE-generated' is inconsistently spaced; please use a single consistent form throughout.","section":"Throughout"},{"comment":"There is a typo in the sentence introducing gamma_p: 'Meanwhiel' should be 'Meanwhile'.","section":"Appendix B.1"},{"comment":"The phrase 'background knowlegde' should be 'background knowledge'.","section":"Appendix B.3"},{"comment":"The text 'BoT orchframework' contains a spacing typo and should read 'BoTorch framework'.","section":"Section 1"},{"comment":"The caption of Table 5 introduces (beta_i, beta_f, beta_s, beta_a) but the notation is not defined before the table; please define each symbol in the caption.","section":"Table 5 and Appendix C"},{"comment":"The text says the low-rank functions are scaled to [-1,1]^{d_e}, but the domain column of Table 4 still lists the original domains; this should be clarified to avoid confusion.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical workshop-length study, and the main issue is experimental design rather than mathematical correctness. The central claim is confounded by the simultaneous change of DML and SDR between Algorithms 1 and 4, and the sample sizes are too small to distinguish the reported success rates. These issues are fixable with additional experiments and more complete reporting, so a major revision is appropriate. I have no concerns about citation practice or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhiwei, quick take on 2412.09183. It's a workshop-scale paper from Cartis's group on putting Sequential Domain Reduction inside latent-space BO with VAEs. That combination is new as far as I know, and there's some decent engineering in the BoTorch implementation. The authors also reformulate Grosnit et al.'s algorithms for general optimization, which is a legitimate move. I give them credit for being upfront: they explicitly say Theorem 1 doesn't cover the Matérn kernel, and they explicitly note that SDR and DML conflict in their experiments.\n\nThe problem is the central empirical claim. The paper says 'structured latent manifolds improve BO performance' and says Algorithm 4 (S-BOVAE) does best 'due to its structured latent spaces'. But Algorithm 4 differs from Algorithm 1 in two ways: it uses DML and it drops SDR. There's no control cell for vanilla retraining without SDR, and none for DML with SDR. The paper's own remark that SDR and DML conflict suggests SDR might be harmful in that setting. So the reported comparisons can't tell you whether the gain comes from DML or from just not using SDR. Figure 1 does show SDR helps for the vanilla VAE case, but that's a different base algorithm and doesn't resolve the attribution. This is load-bearing, not a quibble.\n\nBeyond that, the statistical support is thin: two runs per problem in Tables 1 and 2, five runs for the figures, no confidence intervals, no code. For a workshop paper that's slightly under the bar for a strong claim like 'consistently performed best'. The decoder-coverage assumption (that x* is reachable from some latent z) is stated but untested; given the pretraining data is sampled from a wide Gaussian, it's plausible the latent space misses the region containing the optimum, and then BO is solving the wrong problem.\n\nAll that said, the paper isn't confused. The extension is coherent, the writing is clear, and the limitations are acknowledged rather than hidden. If a referee asks for the missing control cell, more runs, and code, the paper could become a solid empirical note. I'd send it to review rather than desk reject, but with a clear request for major revision. I'd not cite it as evidence for the DML claim until the confound is resolved.","headline":"A useful workshop-level extension—SDR in VAE latent space—but the headline claim is confounded by a missing control cell, and the empirical base is thinner than the prose.","tokens_in":12611,"tokens_out":2618,"would_cite":false,"duration_ms":24841,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","90C56"],"pacs":[],"model":"deepseek-v4-flash","headline":"Structured latent manifolds from variational autoencoders improve Bayesian optimisation in 100 dimensions, with sequential domain reduction in the latent space accelerating convergence.","keywords":["Bayesian optimisation","latent space optimisation","variational autoencoder","dimensionality reduction","deep metric learning","sequential domain reduction","high-dimensional optimisation","random embeddings"],"falsifier":"Take a benchmark with a known global minimiser $x^*$, train the VAE exactly as in the paper, and compute $\\max_z P[x^* \\sim p_{\\theta^*}(\\cdot|z)]$ over the latent space; if this maximum is negligible for the regions the acquisition function explores, the latent-space search cannot locate $x^*$. A cheaper experiment is to place the minimiser far outside the support of the multivariate-normal pretraining distribution and check whether the reported success rate of S-BOVAE collapses.","tokens_in":11516,"feed_emoji":"🎯","tokens_out":10775,"duration_ms":99718,"temperature":0.7,"pith_summary":"This paper argues that using variational autoencoders to build a low-dimensional latent space makes Bayesian optimisation work in 100-dimensional black-box problems where standard BO and random-embedding methods struggle. The authors build on an earlier VAE-based latent-space BO algorithm, correct parts of its implementation, and add periodic retraining and a deep metric (soft triplet) loss so that the latent space is structured by objective value. They also integrate sequential domain reduction into the latent search, the first such combination, and test with the Matérn-5/2 kernel rather than the Gaussian kernel used in the earlier regret analysis. On full-rank and low-rank $D=100$ benchmarks, the retrained VAE algorithms solve more problems than BO in the original space and than REMBO, and the version with deep metric loss performs best. If this holds, expensive high-dimensional optimisation no longer needs to assume the objective is low-rank to benefit from dimensionality reduction.","feed_headline":"Structured VAE latents improve 100-D Bayesian optimisation","feed_subtitle":"Retraining and soft-triplet loss make latent-space BO beat random embeddings on high-dimensional benchmarks.","key_machinery":"The load-bearing object is the VAE encoder–decoder pair $q_\\phi(z|x): \\mathcal{X} \\to \\mathcal{Z}$ and $p_\\theta(x|z): \\mathcal{Z} \\to \\mathcal{X}$, which replaces the original problem with $f^* = \\min_{z \\in \\mathcal{Z}} \\mathbb{E}_{p_{\\theta^*}(x|z)}[f(x)]$. The paper's mechanism for making that replacement effective has three parts: periodic retraining of the VAE on the growing labelled dataset, a soft triplet loss added to the ELBO that pulls latent points with similar objective values together and pushes dissimilar ones apart, and sequential domain reduction that contracts the latent search box as the optimisation proceeds. The soft triplet loss is what creates the structured latent manifolds the paper credits for the performance gain; SDR is what accelerates early convergence; retraining is what keeps the decoder adapted to the region BO actually explores.","core_discovery":"The paper's central claim is that VAE-based latent-space Bayesian optimisation, reformulated for general optimisation rather than molecule generation, outperforms both ambient-space BO and random-embedding BO on 100-dimensional benchmarks. Concretely, the paper reports that applying sequential domain reduction inside the VAE latent space accelerates convergence (Figure 1), and that the algorithm combining VAE retraining with soft triplet deep metric loss, called S-BOVAE, consistently finds better global optima than the vanilla and retraining-only variants (Figure 2). On two test sets, the VAE-based algorithms reach the $\\tau = 10^{-1}$ accuracy level on 90–100% of problems, whereas BO-SDR solves 10–20% and REMBO solves 50%, and the advantage persists at $\\tau = 10^{-3}$. The paper also finds that small latent dimensions ($d=2,5$) work better than $d=50$, attributing this to VAE generalisation capacity, and it shows the Matérn-5/2 kernel works in this setting despite lacking the Gaussian-kernel regret bound of the earlier framework.","pith_inferences":["The paper leaves a coverage condition untested: if the unlabelled pretraining distribution or the decoder's smoothing assigns negligible probability to the region containing $x^*$, latent-space BO is optimising a different function; one concrete extension is to compute $\\max_z P[x^* \\sim p_{\\theta^*}(\\cdot|z)]$ for known minimisers and reject latent spaces with zero coverage.","Because SDR and deep metric loss conflict, a natural scheduling extension is to retrain with soft triplet loss during the exploration phase and switch on SDR only after the latent region has stabilised; the paper identifies the conflict but does not resolve it.","The full-rank benchmark success suggests VAE-based LSBO may be a general high-dimensional strategy rather than a low-effective-dimensionality one; a testable consequence is that its advantage over BO without dimensionality reduction should grow as the number of active variables increases."],"forward_implications":["If the central claim is correct, VAE-based LSBO with retraining and deep metric loss solves more $D=100$ benchmark problems than BO-SDR and REMBO at both $\\tau = 10^{-1}$ and $\\tau = 10^{-3}$ accuracy levels.","The structured latent spaces produced by the soft triplet loss are the main reported source of improvement, so the algorithm's success depends on the metric loss being included, not just on dimensionality reduction.","Smaller latent dimensions ($d=2,5$) outperform larger ones, which makes the choice of latent dimension a primary tuning lever for VAE-based LSBO at fixed ambient dimension.","SDR can be moved into the VAE latent space and accelerates convergence for the vanilla VAE-BO algorithm, although the paper reports that SDR and deep metric loss conflict in the current implementation.","The empirical success with the Matérn-5/2 kernel suggests the earlier Gaussian-kernel regret bound is not the practical bottleneck, but a Matérn-specific regret analysis is needed to match the theory to the experiments."],"supporting_citations":[{"why":"supplies the VAE-LSBO framework, the soft triplet loss, and the sub-linear regret analysis this paper extends and corrects.","marker":"[12]"},{"why":"introduces REMBO, the random-embedding baseline and the low-rank problem construction used for comparison.","marker":"[27]"},{"why":"supplies the weighted-retraining procedure used to adapt the VAE during optimisation.","marker":"[25]"},{"why":"supplies the sequential domain reduction scheme that is integrated into BO and into the VAE latent space.","marker":"[23]"},{"why":"fixes the embedding scale and the low-rank test-function construction used in the REMBO comparisons.","marker":"[4]"},{"why":"provides the GPU-based BO implementation in which all algorithms and SDR are evaluated.","marker":"[1]"},{"why":"defines the performance and data profiles used to measure how many benchmark problems each algorithm solves.","marker":"[20]"},{"why":"supplies the full-rank test problems and the benchmarking methodology for the numerical study.","marker":"[6]"}],"fun_headline_variants":["VAE latents beat random embeddings in 100-D BO","Latent-space BO: retraining and metric loss win","SDR inside VAE space speeds up Bayesian optimisation","Why structured VAEs outperform in global optimisation","High-dim BO gets a VAE boost with SDR integration"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the true best point $x^*$ being reachable through the autoencoder's decoder from some latent point; if the pretraining samples or the decoder's smoothing miss that region, the method is solving a different problem, and the paper gives no guarantee or test for this.","fun_headline_variants_meta":{"raw":{"variants":["VAE latents beat random embeddings in 100-D BO","Latent-space BO: retraining and metric loss win","SDR inside VAE space speeds up Bayesian optimisation","Why structured VAEs outperform in global optimisation","High-dim BO gets a VAE boost with SDR integration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000147,"raw_usage":{"total_tokens":1220,"prompt_tokens":1016,"completion_tokens":204,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":125}},"tokens_in":632,"tokens_out":204,"duration_ms":2663,"temperature":1.0,"reasoning_tokens":125,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:13:24.914576+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a benchmark with a known global minimiser $x^*$, train the VAE exactly as in the paper, and compute $\\max_z P[x^* \\sim p_{\\theta^*}(\\cdot|z)]$ over the latent space; if this maximum is negligible for the regions the acquisition function explores, the latent-space search cannot locate $x^*$. A cheaper experiment is to place the minimiser far outside the support of the multivariate-normal pretraining distribution and check whether the reported success rate of S-BOVAE collapses.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces REMBO, the random-embedding baseline and the low-rank problem construction used for comparison."},{"cited_title":"Sample-efficient opti- mization in the latent space of deep generative models via weighted retraining","cited_arxiv_id":null,"evidence_quote":"supplies the weighted-retraining procedure used to adapt the VAE during optimisation."},{"cited_title":"On the robustness of a simple domain reduction scheme for simulation-based optimization","cited_arxiv_id":null,"evidence_quote":"supplies the sequential domain reduction scheme that is integrated into BO and into the VAE latent space."},{"cited_title":"A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality","cited_arxiv_id":"2003.09673","evidence_quote":"fixes the embedding scale and the low-rank test-function construction used in the REMBO comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the performance and data profiles used to measure how many benchmark problems each algorithm solves."}],"review_version":1}