Latent Bayesian Optimization via Autoregressive Normalizing Flows

Hyunwoo J. Kim; Jaewon Chu; Jinyoung Park; Minseo Yoon; Seunghun Lee

REVIEW 4 major objections 5 minor 1 cited by

Latent Bayesian Optimization via Autoregressive Normalizing Flows

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Latent Bayesian optimization can be made lossless by an invertible flow that maps discrete inputs one-to-one into latent space, eliminating the reconstruction gap that corrupts VAE-based latent search.

desk verdict Solid NF-based latent BO with a real but unenforced reconstruction guarantee; strong experiments, minor tuning concerns. read the letter →

arxiv 2504.14889 v1 pith:PQM7PVE2 submitted 2025-04-21 cs.LG cs.AI

Seunghun Lee , Jinyoung Park , Jaewon Chu , Minseo Yoon , Hyunwoo J. Kim This is my paper · ORCID

classification cs.LGcs.AI

keywords latentBayesianoptimizationnormalizingflowsvaluediscrepancyproblemautoregressiveflowforsequencesmoleculardesigndiscretesequencetoken-levelcandidatesamplingpointwisemutualinformation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Latent Bayesian optimization embeds discrete inputs, such as molecules, into a continuous space, runs Bayesian optimization there, and decodes the result; when the generative model is a variational autoencoder, the reconstruction is imperfect and the objective value attached to a latent point can disagree with the value of the decoded input. The paper claims that this value discrepancy problem is avoidable: replace the VAE with an invertible normalizing flow, SeqFlow, whose autoregressive coupling layers transform discrete token embeddings into a latent code, with a nearest-embedding argmax as the left-inverse decoder. Under distinct, L2-normalized token embeddings, every input $x$ is reconstructed exactly from its latent code, so latent space and input space support the same objective values throughout optimization. On molecular design benchmarks, the resulting NF-BO, augmented with token-importance sampling, ranks first on five of six PMO metrics, outperforming both VAE-based latent optimization methods and strong generative baselines.

What carries the argument

The carrier of the argument is SeqFlow, an autoregressive normalizing flow for discrete sequences, together with its left-inverse decoding rule. Each token index $x_i$ is first represented by an L2-normalized embedding $e_{x_i}$ from a learned embedding set $E$; an invertible stack of $K$ autoregressive coupling blocks maps the embedding sequence $v$ into a latent code $z$, and decoding is the inverse flow followed by an argmax over cosine similarities to $E$. Propositions 1 and 2 establish that when the embeddings are pairwise distinct, $h$ is injective with $h^{-1}(v) = [\arg\max_j \mathrm{sim}(v_i, e_j)]$ as its left inverse, so $f = g \circ h$ is left-invertible and reconstruction is exact. The second component, token-level adaptive candidate sampling, uses the pointwise mutual information between each latent coordinate and the decoded sequence to define a sampling probability over which tokens to perturb around an anchor point, biasing search toward positions with high influence on the objective.

What would settle it

Train SeqFlow on any sequence dataset, then compute the reconstruction failure rate: the fraction of training inputs $x$ for which $h^{-1}(g^{-1}(g(h(x))))$ differs from $x$, or for which two distinct tokens have cosine similarity equal to 1. Any non-zero failure rate, or any observed embedding collapse, would falsify the exact-reconstruction claim central to the method.

Watch

Extended reading notes

Core claim

The central discovery is that perfect reconstruction in latent Bayesian optimization follows from combining an invertible flow with an injective token lookup rather than from any approximation. Writing $h$ for the map that sends a discrete sequence $x$ to the sequence of learned token embeddings, and $g$ for a normalizing flow, the composition $f = g \circ h$ is left-invertible because $h$ is injective (distinct L2-normalized embeddings) and $g$ is invertible by construction; decoding via $h^{-1} \circ g^{-1}$ returns exactly $x$. SeqFlow instantiates this with an autoregressive flow over the sequence of embeddings, so the surrogate model in latent space always refers to a well-defined input and a well-defined objective value. Empirically, the paper reports that this exactness transfers to performance: NF-BO outperforms six latent Bayesian optimization baselines on Guacamol and takes first rank on five out of six PMO scoring metrics, improving the SELFIES VAE baseline's average rank from 19th to 1st.

Load-bearing premise

The guarantee of perfect reconstruction rests on the learned token embeddings remaining pairwise distinct and L2-normalized; if any two embeddings collapse together, the argmax decoding fails and the exact-reconstruction claim collapses.

Editorial extensions

If this is right

Latent Bayesian optimization no longer needs re-encoding or iterative inversion: because every latent code decodes to a unique input, an acquisition function can trust that the surrogate value it sees at $z$ is the objective value of the decoded molecule.
The one-to-one mapping turns the latent space into a faithful coordinate system for discrete structures, so trust-region local search and Thompson sampling behave as they do in continuous domains.
The exact-reconstruction property removes the need for repeated re-encoding of newly found data during optimization, which the paper notes previously caused additional value discrepancy.
The method is not tied to molecule-specific inductive biases, and the same SeqFlow/TACS pipeline is applicable to other discrete sequence optimization problems where exact reconstruction matters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If embedding distinctness degrades under training, the argmax decoder is still locally stable, so monitoring the minimum pairwise embedding distance during optimization could serve as a cheap early warning for reconstruction failures; this diagnostic is not reported in the paper.
Because the latent code is an invertible transformation of the input rather than a stochastic summary, interpolations and small perturbations in $z$ may be more semantically faithful, which could make NF-BO useful for property-guided molecular interpolation; this is an extension the paper does not explore.
The PMI-based token importance, computed with a trained flow, gives a per-position attribution of which parts of a molecule drive the objective, so TACS doubles as an interpretability tool for medicinal chemists beyond its role in candidate sampling.
The same construction transfers to other discrete structures with learned embeddings, such as DNA/RNA sequences or program tokens, where the left-inverse guarantee could replace approximate decoders; this is a conjecture, not a claim of the paper.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: this is a solid, well-executed application of normalizing flows to latent BO. It won’t reorganize the field, but it solves the reconstruction gap in a direct way and supports it with strong empirical work. The main caveat is that the perfect-reconstruction guarantee depends on an unenforced distinctness condition on learned token embeddings.

The genuinely new part is using an invertible normalizing flow as the encoder-decoder inside LBO. SeqFlow adapts existing autoregressive flows (TextFlow) to this setting, and TACS is a sensible token-level candidate sampling scheme. Props 1 and 2 are correct: with distinct, L2-normalized embeddings, the argmax decoder is a left inverse, so reconstruction is exact. That is a real argument, not hand-waving. The empirical evidence is also good: the y-vs-yhat mismatch is 0.000 on the top-1000 initial points, and NF-BO ranks first on five of six PMO metrics and beats the VAE-based LBO baselines on Guacamol. The code is available.

The soft spots are real but fixable. The distinct-embedding assumption is load-bearing but not enforced. The contrastive loss encourages separation, but nothing guarantees ei != ej during or after the iterative retraining in Algorithm 1. So the exact-reconstruction claim is an empirical property, not a proven invariant. Table 5 only checks the top-1000 initial data, not the retraining loop. That’s not fatal, but it should be stated honestly. Second, the TACS temperature was tuned on one Guacamol task and then fixed for everything else; the sensitivity analysis shows the result is fairly robust, but it is still benchmark tuning. Third, the valt task has a disclosed modification (two nonzero initial points added), which slightly muddies that comparison. And Proposition 2 quantifies over N^L while the embedding set is finite; the intended statement should be over the token vocabulary.

Who is it for? People doing latent BO, discrete sequence optimization, or molecular design. It’s a useful baseline and a clear proof of concept for NF-based latent spaces. I’d definitely give it a serious referee; the core idea holds up and the weaknesses are addressable. If I were the reviewer, I’d ask for an explicit separation constraint or post-training distinctness check, and a finite-vocabulary statement of the proposition. I’d also want the temperature selection reported as a minor limitation. But none of that undermines the central finding.

Referee Report

4 major / 5 minor

Summary. The paper proposes NF-BO, a latent Bayesian optimization method that uses an autoregressive normalizing flow (SeqFlow) to map discrete sequences to a continuous latent space. The key claimed advantage is a one-to-one encoding function from the input space to the latent space with a left-inverse decoding function, which is said to eliminate the value discrepancy problem that arises from imperfect reconstruction in VAE-based LBO. The method also introduces Token-level Adaptive Candidate Sampling (TACS) to focus trust-region search on important tokens. The authors evaluate NF-BO on Guacamol and PMO molecular-design benchmarks, reporting strong performance against existing LBO and generative-model baselines.

Significance. If the central claims hold, the paper would make a useful contribution to latent Bayesian optimization by providing a principled alternative to VAE-based encoders and by proposing a concrete way to address the value discrepancy problem. The empirical evaluation is extensive, covering Guacamol and PMO benchmarks with multiple oracle-budget settings, and the manuscript provides code, pseudocode, and proofs in the appendix. However, the theoretical guarantee of perfect reconstruction rests on an unenforced distinctness assumption on the learned embeddings, and the construction does not fully address the consistency between arbitrary latent-space candidates and re-encoded evaluated points. The TACS temperature is also tuned on a benchmark task whose results are then reported. These issues are load-bearing for the paper's main claims, so the manuscript needs substantial revision before the claims can be considered fully established.

major comments (4)

[Section 4.2, Proposition 2 and Eqs. (13)-(14)] The injectivity of the mapping h is proven under the assumption that the embedding set E consists of distinct, L2-normalized vectors. This assumption is not enforced during training: the contrastive loss in Eq. (13) encourages separation but does not guarantee it, and Algorithm 1 has no projection or pairwise-distinctness step after gradient updates. If two embeddings collapse to the same vector, the argmax in Eq. (8) is not unique, h^{-1} is not a well-defined function, and the claimed guarantee h^{-1}(h(x)) = x fails. Table 5 reports a zero y != y_hat ratio on the top-1000 initial data, but because Algorithm 1 updates E jointly with the flow, this measurement does not cover the embeddings used later in optimization. The proof of Proposition 2 also quantifies over x in N^L while E is finite; the intended statement should be over the finite token vocabulary.
[Section 4.2 and Section 4.4 / Algorithm 1] The construction gives a left-inverse for the encoding h, but it does not give a two-sided inverse between the discrete input space and the latent space. For an arbitrary latent candidate z sampled by TACS (e.g., within a trust region), the decoded input x = h^{-1}(g^{-1}(z)) may encode to a different latent vector z' = g(h(x)) that is not equal to z. The surrogate is trained on the latent encodings z^{(i)} = g(e_{x^{(i)}}) of the evaluated points, while acquisition is performed at the candidate z. Thus the latent coordinate at which the surrogate is queried can differ from the latent coordinate at which the resulting evaluation is stored, potentially reintroducing a form of value inconsistency that the paper claims to remove. Propositions 1-2 only establish h^{-1}(h(x)) = x, not that decoding an arbitrary z and re-encoding the result returns the same z; the paper should either restrict the candidate set to the image of the discrete embeddings under g, or analyze/control this discrepancy.
[Section 5.3 and Appendix K] The TACS temperature tau is tuned on a single Guacamol task (searching {400, 200, 100}) and then fixed at tau=400 for all benchmarks, including the same seven Guacamol tasks whose results are reported in the paper. This constitutes hyperparameter selection on the evaluation tasks, which can bias the comparison with baselines. The sensitivity analysis in Table 7 shows that tau=200 achieves a higher summed score (5.565) than the selected tau=400 (5.544), so the chosen value is not clearly the best even within the same benchmark. The authors should either tune hyperparameters on a separate validation set or report results for all settings without selecting on the test tasks.
[Section 5.1 and Figure 8] For the valt task, the authors state that 'we also added two nonzero data points in the initial dataset of 100 for all methods.' This is a modification to the standard Guacamol benchmark setup. Although applied uniformly to all methods, it should be reported as a limitation and its impact on the ranking should be discussed, especially because the y-axis is on a log scale and without these points the task may be degenerate for all methods.

minor comments (5)

[Section 4.2, Eq. (9)] In Eq. (9), the notation \hat{x}_i is used both for the argmax index and for the reconstructed token; this is confusing and should be clarified (e.g., by writing the argmax index explicitly as j^*).
[Section 4.2, Eq. (11)] The description of the constrained sampling distribution q_1(v_i|x_i) is terse; the relationship between the normalization constant Z and the acceptance probability should be spelled out so that the rejection step is unambiguous.
[Algorithm 1, line 2] The notation D[r - N_q:] is not defined; presumably r refers to the current iteration index, but this should be stated explicitly.
[Appendix K, Table 7] The sentence 'TACS temperatures above 200 consistently show better optimization results compared to not using TACS' is accurate according to the table, but the conclusion would be clearer if the authors also noted that temperatures 100, 50, and 20 perform worse than the no-TACS baseline, indicating that the method has a limited favorable range.
[Appendix A, Table 2] Several PMO tasks have very large standard deviations (e.g., celecoxibrediscovery 0.799+-0.164, scaffoldhop 0.769+-0.172), which may weaken the statistical significance of some reported improvements; the authors should discuss this variation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the reconstruction guarantee is a construction-level property, not a fitted prediction; self-citations are contextual and not load-bearing.

full rationale

The central claim that SeqFlow guarantees exact reconstruction follows directly from the definitions in Section 4.2: h maps each token to a distinct L2-normalized embedding and h^{-1} is the argmax-cosine left inverse (Eqs. 8-9, Propositions 1-2). This is a mathematical consequence of the stated injectivity assumption, not a quantity fitted to benchmark data. The empirical y != y_hat ratio of 0.000 in Table 5 is a consistency check of that construction, not a fitted prediction, and it is measured only on the initial top-1000 data. The proof is conditional on the distinctness and L2-normalization of the learned embedding set E, which the training loss in Eqs. (13)-(14) does not formally enforce after gradient updates; that is an unproven premise and a correctness risk, not circularity. Citations to the authors' prior CoBO (Lee et al., 2023) and Inversion-based LBO (Chu et al., 2024) appear as related work and baselines and are not load-bearing for the reconstruction guarantee or the benchmark comparisons. The proposed method is applied to external molecular-design benchmarks and compared with external baselines; no uniqueness theorem or self-citation chain forces the result. No equation in the paper reduces to its own inputs or renames a fitted parameter as a prediction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the invertibility of the normalizing flow and the distinctness of learned token embeddings. The former is standard for NF architectures; the latter is assumed in Proposition 2 but not enforced during training. Several hyperparameters (TACS temperature, similarity loss weight, variational noise, anchor softmax temperature) are chosen by hand or tuned on the benchmark and are not part of the theoretical guarantee. No new physical entities are introduced.

free parameters (6)

TACS temperature tau = 400
Searched over {400, 200, 100} on one Guacamol task, then fixed for all benchmarks.
TACS scaling factor kappa = 0.1 * L
Set relative to sequence length; no tuning reported.
Variational distribution std sigma = 0.1
Fixed for all experiments.
Similarity loss coefficient lambda = 1
Sensitivity analysis shows robustness in [0.1, 10].
Anchor softmax temperature tau1 = 0.1
Used for anchor point selection in trust regions.
Top-k data points for training = 1000
Number of best-scoring points selected for retraining each iteration.

assumptions (4)

standard math The normalizing flow g and its inverse g^{-1} are exactly implementable with autoregressive coupling layers.
Standard property of normalizing flows; required for Eq. (15) and Proposition 1.
domain assumption The learned embedding vectors remain distinct and L2-normalized after training.
Needed for Proposition 2's left-inverse theorem; contrastive loss encourages but does not guarantee distinctness.
domain assumption The surrogate Gaussian process with deep kernel adequately models the composite function f composed with p_theta in latent space.
Standard BO assumption; not verified for all tasks.
domain assumption The constrained variational sampling q1 in Eq. (11) yields a valid approximation of the discrete likelihood.
The ELBO derivation depends on this approximation; acceptance procedure is described but not analyzed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Latent Bayesian Optimization via Autoregressive Normalizing Flows." pith.science (2026). https://pith.science/paper/PQM7PVE2

@misc{pith2026250414889,
  author       = {Pith},
  title        = {Pith review of: Latent Bayesian Optimization via Autoregressive Normalizing Flows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQM7PVE2}},
  note         = {Machine review of arXiv:2504.14889}
}

read the original abstract

Bayesian Optimization (BO) has been recognized for its effectiveness in optimizing expensive and complex objective functions. Recent advancements in Latent Bayesian Optimization (LBO) have shown promise by integrating generative models such as variational autoencoders (VAEs) to manage the complexity of high-dimensional and structured data spaces. However, existing LBO approaches often suffer from the value discrepancy problem, which arises from the reconstruction gap between input and latent spaces. This value discrepancy problem propagates errors throughout the optimization process, leading to suboptimal outcomes. To address this issue, we propose a Normalizing Flow-based Bayesian Optimization (NF-BO), which utilizes normalizing flow as a generative model to establish one-to-one encoding function from the input space to the latent space, along with its left-inverse decoding function, eliminating the reconstruction gap. Specifically, we introduce SeqFlow, an autoregressive normalizing flow for sequence data. In addition, we develop a new candidate sampling strategy that dynamically adjusts the exploration probability for each token based on its importance. Through extensive experiments, our NF-BO method demonstrates superior performance in molecule generation tasks, significantly outperforming both traditional and recent LBO approaches.

Figures

Figures reproduced from arXiv: 2504.14889 by the authors.

**Figure 1.** Visualization of value discrepancy problem. However, the reconstruction of VAE is not always perfect, leading to value discrepancy problem, which indicates that given a sample encoded as an embedding in the latent space, its decoding may not result in the same sample in the input space [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** (a) Most existing LBO approaches suffer from the value discrepancy problem y ‰ yˆ induced by the reconstruction gap, pθpqϕpxqq ‰ x. This results in that the latent representation z corresponds to different evaluation values y and yˆ due to the reconstruction error, where x ‰ xˆ. (b) Our NF-BO effectively addresses the value discrepancy problem by employing a normalizing flow model that ensures one-to-one mapping bet… view at source ↗

**Figure 3.** Overall pipeline of SeqFlow. Given the input space of a sequence discrete values x, SeqFlow first maps the discrete values x to continuous representation v and efficiently transforms them via autoregressive transformations tg i u K´1 i“0 to a latent representation z 0 in the encoding phase (top pathway). In the decoding phase (bottom pathway), SeqFlow reconstructs x from z 0 through the inverse of transformations. S… view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: Overview of NF-BO. We employ our normalizing flows, SeqFlow, as a mapping function between a discrete input space and a continuous latent space. Each discrete input token xi is mapped to its corresponding embedding vector vi from the dictionary. A surrogate model is t…

**Figure 5.** Figure 5: Optimization results of NF-BO on Guacamol benchmarks comparing performance with [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 7.** Figure 7: Comparison of performance with and without TACS [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: Optimization results on Guacamol benchmarks under 500 additional oracle settings. Note [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Optimization results on Guacamol benchmarks under 10K additional oracle settings. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

**Figure 10.** Figure 10: Optimization results on Guacamol benchmarks under 70K additional oracle settings. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

**Figure 11.** Figure 11: Distinct sample ratio with various TACS temperatures on Guacamol benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

**Figure 12.** Figure 12: Pointwise Mutual Information (PMI) value [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

**Figure 13.** Figure 13: Sampling Distribution Visualization. Voronoi cells represent different regions, and color intensity indicates the likelihood of accepting a sample. For each block g k , the input is represented by z k,0 “ z k , and the output of the final layer in each block sets the …

**Figure 14.** Figure 14: Visualizations of the value discrepancy problem across 5 Guacamol tasks. ver et al., 2024) (e.g., MAE) operate differently since they utilize the original x in the decoder to restore inputs for unmasked input tokens, thus cannot define the value discrepancy problem, w…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

A Language-Guided Bayesian Optimization for Efficient LoRA Hyperparameter Search
cs.CL 2026-01 conditional novelty 5.0 of 10

LLM embeddings plus Bayesian optimization find better LoRA hyperparameters in ~30 proxy trials than standard published settings.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages · cited by 1 Pith paper

[1]

Light sky blue areas indicate regions with lower density compared to the darker regions

Darker regions indicate higher Gaussian values, and hence higher likelihoods of sample acceptance. Light sky blue areas indicate regions with lower density compared to the darker regions. This visualization demonstrates the selective nature of our sampling method, focusing only on feasible solutions during optimization. H A RCHITECTURE DETAILS Each autore...

work page 2025
[3]

Table 3: Fixed parameters for all tasks and settings. Parameter Value Scaling factorκ in TACS 0.1¨ Sequence lengthL Standard deviationσ of variational distributionq 0.1 # of topk data for training 1000 Coefficient of similarity loss Lsim 1 Typically, the anchor point within a trust region is selected based on the current best observed value from the accum...

work page 2019
[4]

The table presents scores and standard deviations across 6 evaluation metrics, with each score representing the mean of 5 independent runs

13 Published as a conference paper at ICLR 2025 Table 2: Detailed results on PMO benchmarks. The table presents scores and standard deviations across 6 evaluation metrics, with each score representing the mean of 5 independent runs. Addition- ally, the sum for each column is computed to summarize the overall performance. Top-1 Top-10 Top-100 AUC Top-1 AUC...

work page 2025
[7]

For other settings, these parameters were adjusted to (10, 100). 19 Published as a conference paper at ICLR 2025 We explored the temperatureτ for the Token-level Adaptive Candidate Sampling (TACS) across the values {400, 200, 100} to find the optimal setting. The sequence lengthL was determined based on the longest sequence in the initial dataset. For det...

work page 2025
[9]

From the table, our SeqFlow model achieves better perfor- mance with fewer parameters compared to the baseline model

Please note that we do not apply TACS solely to compare generative models. From the table, our SeqFlow model achieves better perfor- mance with fewer parameters compared to the baseline model. SeqFlow and TextFlow use the same NF model, but TextFlow includes more components and therefore has more parameters. Although TextFlow has more parameters, our SeqF...

work page 2025
[10]

Both the oracle budget and the number of initial data were set to 10,000

This analysis is performed across seven Guacamol tasks, with results averaged over five runs per task and summed. Both the oracle budget and the number of initial data were set to 10,000. From the table, TACS temperatures above 200 consistently show better optimization results compared to not using TACS, highlighting the robustness of our approach to the ...

work page 2025
[11]

Each task’s performance is averaged over five trials

NF-BO w/ TACS (init 10K) adip 0.765˘ 0.038 0.818˘ 0.051 0.809˘ 0.059 med2 0.306˘ 0.014 0.307˘ 0.027 0.380˘ 0.014 osmb 0.848˘ 0.037 0.855˘ 0.007 0.897˘ 0.016 pdop 0.564˘ 0.045 0.623˘ 0.043 0.759˘ 0.023 rano 0.846˘ 0.019 0.848˘ 0.021 0.941˘ 0.006 valt 0.198˘ 0.443 0.786˘ 0.439 0.995˘ 0.004 zale 0.586˘ 0.013 0.589˘ 0.033 0.760˘ 0.012 Table 7: Sensitivity ana...

work page 2000
[500]

Figures 8, 9, and 10 display the results for the (100, 500), (10,000, 10,000), and (10,000, 70,000) oracle settings, respectively

and (10,000, 10,000) oracle settings, which were not covered in the main section, along with the results for the (10,000, 70,000) oracle settings. Figures 8, 9, and 10 display the results for the (100, 500), (10,000, 10,000), and (10,000, 70,000) oracle settings, respectively. In the case of PG-LBO (Chen et al., 2024), we were unable to include results fo...

work page 2024

Show all 12 references

[2018]

A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences

11 Published as a conference paper at ICLR 2025 Miguel Gonz´alez-Duque, Richard Michael, Simon Bartels, Yevgen Zainchkovskyy, Søren Hauberg, and Wouter Boomsma. A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences. arXiv:2406.04739,

2025 arXiv
[2019]

Variational inference with normalizing flows

12 Published as a conference paper at ICLR 2025 Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In ICML,

2025
[2020]

High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning.arXiv:2106.03609,

Antoine Grosnit, Rasul Tutunov, Alexandre Max Maraval, Ryan-Rhys Griffiths, Alexander I Cowen- Rivers, Lin Yang, Lin Zhu, Wenlong Lyu, Zhitang Chen, Jun Wang, et al. High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning.arXiv:2106.03609,

arXiv
[2022]

ver et al.,

and LaMBO-2 (Gru- 22 Published as a conference paper at ICLR 2025 Figure 14: Visualizations of the value discrepancy problem across 5 Guacamol tasks. ver et al.,

2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Light sky blue areas indicate regions with lower density compared to the darker regions

Darker regions indicate higher Gaussian values, and hence higher likelihoods of sample acceptance. Light sky blue areas indicate regions with lower density compared to the darker regions. This visualization demonstrates the selective nature of our sampling method, focusing only on feasible solutions during optimization. H A RCHITECTURE DETAILS Each autore...

work page 2025

[2] [3]

Table 3: Fixed parameters for all tasks and settings. Parameter Value Scaling factorκ in TACS 0.1¨ Sequence lengthL Standard deviationσ of variational distributionq 0.1 # of topk data for training 1000 Coefficient of similarity loss Lsim 1 Typically, the anchor point within a trust region is selected based on the current best observed value from the accum...

work page 2019

[3] [4]

The table presents scores and standard deviations across 6 evaluation metrics, with each score representing the mean of 5 independent runs

13 Published as a conference paper at ICLR 2025 Table 2: Detailed results on PMO benchmarks. The table presents scores and standard deviations across 6 evaluation metrics, with each score representing the mean of 5 independent runs. Addition- ally, the sum for each column is computed to summarize the overall performance. Top-1 Top-10 Top-100 AUC Top-1 AUC...

work page 2025

[4] [7]

For other settings, these parameters were adjusted to (10, 100). 19 Published as a conference paper at ICLR 2025 We explored the temperatureτ for the Token-level Adaptive Candidate Sampling (TACS) across the values {400, 200, 100} to find the optimal setting. The sequence lengthL was determined based on the longest sequence in the initial dataset. For det...

work page 2025

[5] [9]

From the table, our SeqFlow model achieves better perfor- mance with fewer parameters compared to the baseline model

Please note that we do not apply TACS solely to compare generative models. From the table, our SeqFlow model achieves better perfor- mance with fewer parameters compared to the baseline model. SeqFlow and TextFlow use the same NF model, but TextFlow includes more components and therefore has more parameters. Although TextFlow has more parameters, our SeqF...

work page 2025

[6] [10]

Both the oracle budget and the number of initial data were set to 10,000

This analysis is performed across seven Guacamol tasks, with results averaged over five runs per task and summed. Both the oracle budget and the number of initial data were set to 10,000. From the table, TACS temperatures above 200 consistently show better optimization results compared to not using TACS, highlighting the robustness of our approach to the ...

work page 2025

[7] [11]

Each task’s performance is averaged over five trials

NF-BO w/ TACS (init 10K) adip 0.765˘ 0.038 0.818˘ 0.051 0.809˘ 0.059 med2 0.306˘ 0.014 0.307˘ 0.027 0.380˘ 0.014 osmb 0.848˘ 0.037 0.855˘ 0.007 0.897˘ 0.016 pdop 0.564˘ 0.045 0.623˘ 0.043 0.759˘ 0.023 rano 0.846˘ 0.019 0.848˘ 0.021 0.941˘ 0.006 valt 0.198˘ 0.443 0.786˘ 0.439 0.995˘ 0.004 zale 0.586˘ 0.013 0.589˘ 0.033 0.760˘ 0.012 Table 7: Sensitivity ana...

work page 2000

[8] [500]

Figures 8, 9, and 10 display the results for the (100, 500), (10,000, 10,000), and (10,000, 70,000) oracle settings, respectively

and (10,000, 10,000) oracle settings, which were not covered in the main section, along with the results for the (10,000, 70,000) oracle settings. Figures 8, 9, and 10 display the results for the (100, 500), (10,000, 10,000), and (10,000, 70,000) oracle settings, respectively. In the case of PG-LBO (Chen et al., 2024), we were unable to include results fo...

work page 2024