{"id":"4669f468-60d1-4d0f-b9bd-cef2bce495d1","arxiv_id":"2507.20533","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"KOBO learns a continuous latent space of GP kernels with a VAE and optimizes kernel choice inside that space, reducing the number of function evaluations needed to reach the optimum.","lead":"This paper trains a variational autoencoder to map many possible kernel functions into a smooth latent space, then runs Bayesian optimization in that space to pick the best Gaussian process kernel. The result is a method that finds optima of black-box functions using fewer user or simulation queries than existing kernel-selection approaches.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"KerVAE latent space continuity and completeness are asserted, not demonstrated; a 2D latent space without validity or smoothness checks can break KerGPR's optimization and undermine the sample-efficiency claim.","rationale":"The paper's central claim is that KOBO achieves lower sample budgets by learning a continuous latent space of kernels and optimizing model evidence in that space. The viability of this claim depends entirely on the KerVAE latent space having two properties: continuity (nearby latent points yield similar kernels) and completeness (every latent point decodes to a valid kernel). The paper explicitly states that V AEs are 'expected' to provide these properties, but provides no formal guarantee or targeted evaluation. The reconstruction check in the appendix shows near-perfect reconstruction of the training kernel codes, but that only verifies the decoder on the training distribution; it says nothing about the latent space away from the encoded training points. KerGPR's optimization (Equation 8) assumes the model evidence is a smooth function of z, which would follow from continuity of the decoding map and continuity of model evidence in the kernel parameters, but this is not established. A 2D latent space mapping to a 15-dimensional grammar code is especially susceptible to 'holes' where the decoder produces invalid outputs, a known failure mode of VAEs. If the latent space has dead regions, KerGPR may waste evaluations on invalid kernels or miss the optimal kernel entirely, and the observed sample-efficiency gains could instead stem from other algorithmic details (e.g., the ALEBO embedding or the specific acquisition strategy). The concern is concrete and testable; the proposed grid experiment would directly measure latent-space validity and smoothness. Since the paper provides some supporting evidence (reconstruction accuracy and empirical success on benchmarks) but not enough to rule out the failure mode, the appropriate verdict remains conditional: the central mechanism should be accepted only after the validity and smoothness of the latent space are demonstrated.","tokens_in":17197,"tokens_out":8946,"duration_ms":102393,"concrete_test":"Sample a dense 100x100 grid over [-3,3]^2 in the KerVAE latent space, covering the N(0,I) prior's high-probability region. For each z, decode rc, construct the kernel matrix on a fixed set of 20 random input points, and record (i) the minimum eigenvalue of that matrix, (ii) the proportion of valid PSD kernels, and (iii) the model evidence L(z) for a fixed dataset D. Then measure the empirical Lipschitz constant of L over the grid and identify any isolated spikes. If more than 5% of decoded kernels are non-PSD, or if the Lipschitz estimate is large relative to the SE lengthscale used in KerGPR, the continuity/completeness assumption fails and the proposed mechanism is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central mechanism (Section 3) rests on the unverified claim that the KerVAE latent space is continuous and complete: 'two close points in the latent space cannot decode to completely different results, and a point sampled from the latent space must decode to a valid result.' No experiment establishes this. The reconstruction check (Appendix, Figure 9) only measures reconstruction error on the training kernel codes; it does not sample arbitrary latent points and test PSD-validity or model-evidence smoothness. KerGPR (Equation 8) then optimizes model evidence over this latent space with an SE kernel, implicitly assuming L(z)=log P(F|X, Dec(z)) is smooth in z. Because the latent space is only 2-dimensional and the decoder maps to a 15-dimensional grammar code, a large fraction of the prior's mass may decode to invalid kernels (negative or fractional powers that violate PSD) or to regions where L is discontinuous. If so, KerGPR can select a poor kernel, and the reported sample-efficiency gains (17 versus 28 evaluations on Staircase) may not be attributable to the proposed mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KOBO, a Gaussian-process kernel learning method for black-box optimization under tight sample budgets. KOBO defines a discrete grammar-based space of composite kernels, augments each kernel's grammar code with a data-dependent distance vector, trains a variational autoencoder (KerVAE) to map this space into a low-dimensional continuous latent space, and runs a second GP (KerGPR) over the latent space to maximize model evidence. The resulting kernel is used by the main function-level GP. Experiments compare KOBO with static base kernels and with MCMC, CKS, and BOMS on synthetic benchmarks, a CO2 time-series structure-learning task, a hearing-aid personalization study, and a prompt-based image recommendation study.","tokens_in":17411,"tokens_out":5771,"duration_ms":61818,"significance":"If the method works as claimed, KOBO is a useful contribution to sample-constrained BO: it replaces discrete kernel search with continuous optimization, includes a sanity check for ground-truth kernel recovery (Table 1), an ablation of kernel encoding length, and an honest limitations section, and it reports a public audio demo URL. However, the empirical support is incomplete in several load-bearing places, and the central latent-space continuity/completeness assumption is asserted rather than demonstrated. The idea of embedding a grammar-generated kernel space into a continuous VAE latent space is reasonable and potentially valuable, but the paper currently does not provide sufficient evidence that the proposed mechanism, rather than the specific experimental setup, is responsible for the reported gains.","major_comments":[{"comment":"The central mechanism assumes that the KerVAE latent space is continuous and complete: 'two close points in the latent space cannot decode to completely different results, and a point sampled from the latent space must decode to a valid result.' This assumption is load-bearing because KerGPR with an SE kernel optimizes model evidence over Z, implicitly assuming L(z) is smooth and well-defined everywhere. The paper provides no experiment that samples arbitrary latent points and checks PSD-validity of the decoded kernels or smoothness of the model-evidence surface. Appendix Figure 9 only measures reconstruction error on training kernel codes; it does not test untrained or dead regions of the latent space. Without such a validation, the claimed sample-efficiency gains (e.g., 17 vs. 28 function evaluations on Staircase in Section 4) cannot be confidently attributed to the continuous-space mechanism. Please add an explicit latent-space validity/smoothness study, or state conditions under which the VAE guarantees these properties.","section":"Section 3, 'Intuition and Overview', and Eq. (8)"},{"comment":"The ground-truth kernel recovery test is presented as evidence that KOBO learns the structure of the objective function, but the table shows only partial recovery. For f1, the learned kernel at Q=25 is A*A*B + C*D instead of the ground-truth A*A*B + C; for f2, the learned kernel is A*C + D instead of C + D. Only f3 is recovered exactly (up to commutativity of multiplication). The limitations paragraph mentions overfitting for f1, but not the persistent mismatch for f2. Since this table is the primary sanity check for the core claim, the text should either report the recovery rate across runs, acknowledge the imperfect recovery more prominently, or demonstrate that the mismatched kernels still yield equivalent model evidence on the observed data.","section":"Table 1 and Section 4, 'Is K* indeed learning the structure of f(x)?'"},{"comment":"The abstract claims 'real-world audio experiments with U=6 users,' but Table 2 reports results for only U1, U2, and U3, and the appendix, despite stating that six volunteers were recruited, does not present the remaining three users' data. The main text says 'rest in Appendix,' but no such table appears in the submitted material. Similarly, the image recommendation study (Table 3 and Table 5) is qualitative, with no quantitative metric or inter-rater protocol. Thus the real-world claims that KOBO personalizes hearing aids with fewer queries or converges to desirable images with limited ratings are not fully supported by the reported data. Please either include the complete user data or revise the claims to match what is actually shown.","section":"Section 4 and Technical Appendix, 'User Experiment: Audio Personalization'"},{"comment":"All numerical comparisons are reported as averages over 10 runs, but no error bars, standard deviations, confidence intervals, or significance tests are provided anywhere in the paper. For example, the Staircase comparison reports 17 function evaluations for KOBO versus 28 for MCMC; without variance information it is impossible to assess whether this difference is robust across random initializations and ALEBO projections. Given that the paper's central claim is a quantitative reduction in sample budget, the lack of uncertainty quantification is a substantive gap. Please add error bars or statistical tests to all main results.","section":"Section 4 and Technical Appendix, 'Synthetic Function Evaluation Parameters'"},{"comment":"The kernel-selection objective in Eq. (8) is defined as exact model evidence P(F|X, Dec(z)), but the evaluation replaces it with a BIC approximation, log P(F|X,K) = -1/2 f^T K^{-1} f - 1/2 log((2*pi)^N |K|). In the low-sample regime used throughout the paper (B <= 25), the BIC approximation may be inaccurate, and KerGPR may therefore be optimizing a different objective than the one stated. The relationship between BIC-optimal and exact-marginal-likelihood-optimal kernels in this regime is not discussed. Please justify the BIC approximation for the sample sizes used, or report whether exact marginal likelihood gives similar results.","section":"Eq. (4) and Section 4, 'Another Metric'"}],"minor_comments":[{"comment":"The grammar code in Eq. (5) allows fractional exponents, and the appendix restricts the sum of exponents to 0 <= sum <= 3, but it does not state whether individual exponents can be negative. Negative powers of base kernels are generally not positive semi-definite, so the validity of decoded kernels is unclear; please clarify the allowed range of individual exponents.","section":"Section 3, Kernel Combiner"},{"comment":"The objective in Eq. (1) is written as argmin ||f(hat x) - f(x*)||_2, but f(x*) is unknown. The paper later uses regret f(hat x) - f(x*) as the evaluation metric. Please align the problem formulation with the metric actually used.","section":"Section 2, Eq. (1)"},{"comment":"There are several typos and reference inconsistencies: 'MICHALEWICZ' should be 'Michalewicz'; 'Helliger distance' should be 'Hellinger distance'; 'Forbenius norm' should be 'Frobenius norm'; and 'Figure 5(a) and (b) in the Appendix' is incorrect because Figure 5 appears in the main text and the appendix figures are numbered 7-10.","section":"Section 4 and Appendix"},{"comment":"The image recommendation results in Table 3 do not specify how many raters were used, how scores were collected, or how the qualitative match to the prompt was assessed. Please describe the rating protocol and the number of users.","section":"Section 4, Table 3"},{"comment":"Figure 9 shows reconstruction error only on the training kernel codes; please also report reconstruction error on held-out kernel codes, which would be a more meaningful check of generalization.","section":"Technical Appendix, 'KerVAE Reconstruction'"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The abstract promises 6-user audio results while the main text reports only 3, and the appendix does not provide the promised supplementary table; please verify the supplementary materials contain the missing data. The paper fits an applied ML venue, but the missing uncertainty quantification and the unvalidated latent-space assumption need to be addressed before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you care about kernel learning for BO. The core idea is fresh: instead of searching a discrete grammar space of composite kernels, KOBO trains a VAE on grammar-based kernel codes plus a data-dependent distance feature, then runs a second GP in the 2D latent space to pick kernels for the main optimizer. I haven't seen that exact combination before.\n\nWhat I liked: the ground-truth kernel recovery check in Table 1 is a good sanity test, and the CO2 example shows the learned kernels track structure (periodic, trend) in a way that static kernels miss. The limitations section is honest—overfitting, computational overhead, latent interpretability are all acknowledged. The synthetic comparisons against MCMC, BOMS, and CKS are plausible, and the paper is clearly written.\n\nSoft spots, in proportion. First, every headline number is an average of 10 runs with no error bars or significance tests. The central comparison—17 vs 28 evaluations on Staircase—could be noise, and with no code or data released, I can't check. That's the biggest concrete issue. Second, the load-bearing assumption in Section 3—that the VAE latent space is continuous and complete—is never actually tested. The reconstruction check in Appendix Figure 9 only measures error on training codes, not on arbitrary latent points. Nothing samples a random z, decodes it, and verifies the result is PSD or that model evidence is smooth in z. KerGPR uses an SE kernel on that latent space, so if there are dead regions or discontinuities, the method silently fails. The stress-test note is right about this gap. It may be fixable with an experiment, but as written it's an unsupported axiom. Third, the real-world studies are thin: only three users in the main text, no SOTA kernel-search baselines there, and the image results are qualitative. Finally, the baseline setup is underspecified—do MCMC/CKS/BOMS also run on the ALEBO embedding? If not, the comparison may be apples to oranges.\n\nNone of this is fatal. The idea deserves a serious referee and the authors can address the concerns. But it's a conditional accept at best in current form.\n\nRecommendation: send to peer review. I'd want to see error bars, a latent-space validity/smoothness ablation, and a clearer baseline setup before believing the sample-efficiency claim.","headline":"A genuinely novel VAE-based kernel selection mechanism with plausible synthetic gains, but the continuity/completeness assumption that carries the method is asserted rather than tested, and the real-world evidence is too thin to confirm the headline claim.","tokens_in":17941,"tokens_out":2582,"would_cite":true,"duration_ms":29850,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Kernel selection in black-box optimization can be recast as a black-box optimization problem in the continuous latent space of a VAE, and doing so reaches optima in far fewer function evaluations.","keywords":["black-box optimization","kernel learning","Gaussian process regression","variational autoencoder","Bayesian optimization","composite kernels","sample efficiency","audio personalization"],"falsifier":"Take a target function drawn from a known composite kernel that is in the training grammar, run KOBO to a fixed budget, and compare the decoded $K^*$ with the true kernel and with the best kernel found by exhaustive enumeration in that grammar. If the decoded kernel either fails to match the true one or is outperformed by a grammar kernel that KerGPR never approaches, the latent map is not faithful and the sample-efficiency claim would not generalize.","tokens_in":1778,"feed_emoji":"🎯","tokens_out":3306,"duration_ms":80686,"temperature":0.7,"pith_summary":"Black-box optimization assumes each evaluation of the unknown function is costly, so the paper asks whether the surrogate model's kernel can be learned from the same samples the optimizer already collects. It proposes KOBO, which turns the discrete space of composite kernels into a continuous latent space of a variational autoencoder, then runs a second Gaussian-process search in that latent space to maximize model evidence: how plausible the observed samples are under a candidate kernel. The paper claims this finds the right kernel shape earlier than greedy or random kernel searches, so the main optimizer's surrogate matches the function's structure and the optimum is reached with fewer queries. On a discontinuous staircase benchmark, KOBO reaches the global minimum in about 17 function evaluations, versus 28 for MCMC search, 32 for BOMS, and 43 for CKS. The same pipeline is demonstrated on hearing-aid filter personalization with real users and on rating-based image recommendation.","feed_headline":"A learned-kernel method reaches black-box minima in 17 evaluations","feed_subtitle":"By optimizing kernels in a VAE-built continuous space, it finds the global minimum 11–26 queries sooner than prior kernel searches.","key_machinery":"The load-bearing object is the continuous latent kernel space $\\mathcal{Z}$ learned by KerVAE. KerVAE is trained on composite kernels encoded as $r = [r_c, r_d]$: $r_c$ is a 15-slot grammar vector recording which base kernels (squared-exponential, periodic, rational-quadratic, Matérn, linear) are added or multiplied, and $r_d$ is the vector of Frobenius distances between the composite kernel's covariance matrix and each base kernel's covariance matrix on the current observations. The continuity of this representation is what lets KerGPR treat model evidence as a smooth black-box function and optimize it with a standard GP, instead of exhaustively expanding a discrete kernel tree. The decoder maps the resulting optimal latent point back to an interpretable composite kernel for the function-space GPR.","core_discovery":"The paper's central claim is that kernel selection for Gaussian-process black-box optimization can be treated as a black-box optimization problem in a continuous latent space rather than as a search over a discrete grammar. Composite kernels are described by a grammar-based code $r_c$ concatenated with a data-based code $r_d$ that records Frobenius distances between the composite kernel's covariance matrix and those of the base kernels on observed samples. A variational autoencoder (KerVAE) maps this coded kernel space to a low-dimensional continuous latent space $\\mathcal{Z}$; a second Gaussian process (KerGPR) maximizes model evidence $L(K) = P(F|X,K)$ over $\\mathcal{Z}$, and the decoder turns the best latent point into the kernel $K^*$ that the main function-space GPR uses. The paper reports that $K^*$ converges to ground-truth kernels when the target function is drawn from a known GP kernel, that it recovers periodic-plus-trend structure in real CO2 data, and that it lowers regret faster than the MCMC, BOMS, and CKS baselines.","pith_inferences":["If the latent space generalizes, the same encode-decode-and-optimize loop is a template for other discrete model-selection problems, such as choosing acquisition functions or prompting strategies, wherever a model-evidence-like score can be computed.","A practical extension would be to freeze KerVAE after training and only recompute the data-based part $r_d$ for each new objective, making kernel selection nearly free at deployment time.","One testable risk: if the latent space contains regions that decode to degenerate or invalid kernels, KerGPR might waste latent evaluations there; a dense-coverage sanity check that samples many latent points and measures reconstruction validity and evidence smoothness would reveal whether such dead regions limit the reported gains."],"forward_implications":["For optimization problems whose hidden function has flat plateaus or periodic trends, learned composite kernels should reach better or equal solutions within a fixed query budget than any single base kernel, because the surrogate can represent the same structure.","The auxiliary kernel search runs in latent space and does not consume user or function-evaluation budget, so the sample-efficiency gain should persist even when KerVAE retraining is frequent.","Decoded $K^*$ remains interpretable as a grammar composition (e.g., $SE \\times PER + RQ$), so the method doubles as structure discovery: it names the function's components, not just a fitted matrix.","If a target function's true kernel is expressible in the grammar, KOBO should recover it exactly as sample count grows, as the paper's ground-truth kernel experiments indicate."],"supporting_citations":[{"why":"Supplies the variational autoencoder and ELBO objective used to build the continuous latent kernel space.","marker":"(Kingma and Welling 2013)"},{"why":"Defines grammar-based composite kernel construction and the CKS baseline that searches the discrete kernel tree greedily.","marker":"(Duvenaud et al. 2013)"},{"why":"Provides the BOMS baseline, which performs Bayesian optimization over kernels using a Hellinger-distance kernel between kernels and the same model-evidence objective.","marker":"(Malkomes, Schaff, and Garnett 2016)"},{"why":"Provides the MCMC kernel-search baseline that moves over composite kernels via Metropolis-Hastings proposals.","marker":"(Gardner et al. 2017)"},{"why":"Supplies the tutorial-level formulation of Bayesian optimization and Gaussian-process surrogates that the function-space GPR builds on.","marker":"(Frazier 2018)"},{"why":"Provides the standard Gaussian-process posterior and model-evidence computation used to score candidate kernels.","marker":"(Rasmussen, Williams et al. 2006)"},{"why":"Supplies ALEBO, the random-projection embedding baseline for high-dimensional sparse Bayesian optimization that KOBO builds on and extends.","marker":"(Letham et al. 2020)"}],"fun_headline_variants":["KOBO: learn kernels in latent space to cut BBO sample budget","Latent-space kernel learning needs fewer black-box samples","Optimize kernels with a GP to lower BBO query count","KOBO finds black-box optima with far fewer evaluations","Learn composite kernels via VAE: less sampling for BBO"],"cache_read_input_tokens":20096,"weakest_assumption_plain":"The argument collapses if the VAE latent space is not continuous and complete: nearby latent points must decode to similar kernels and every latent point must decode to a valid kernel, because KerGPR's optimum is only as good as the map from Z back to kernels.","fun_headline_variants_meta":{"raw":{"variants":["KOBO: learn kernels in latent space to cut BBO sample budget","Latent-space kernel learning needs fewer black-box samples","Optimize kernels with a GP to lower BBO query count","KOBO finds black-box optima with far fewer evaluations","Learn composite kernels via VAE: less sampling for BBO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000798,"raw_usage":{"total_tokens":3498,"prompt_tokens":921,"completion_tokens":2577,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":2506}},"tokens_in":537,"tokens_out":2577,"duration_ms":19338,"temperature":1.0,"reasoning_tokens":2506,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:42:00.967293+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a target function drawn from a known composite kernel that is in the training grammar, run KOBO to a fixed budget, and compare the decoded $K^*$ with the true kernel and with the best kernel found by exhaustive enumeration in that grammar. If the decoded kernel either fails to match the true one or is outperformed by a grammar kernel that KerGPR never approaches, the latent map is not faithful and the sample-efficiency claim would not generalize.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MCMC kernel-search baseline that moves over composite kernels via Metropolis-Hastings proposals."}],"review_version":2}