{"id":"6ff8efb1-aab1-48bf-a135-b844f30cb56e","arxiv_id":"2501.18768","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DynAMO adds a KL distribution-matching term to offline model-based optimization, producing more diverse candidate designs without sacrificing the best-found score.","lead":"Researchers propose DynAMO, a wrapper that adds a diversity incentive to any offline design-optimization algorithm by asking the optimizer to match the spread of good designs already in the training data. On six scientific design tasks, the method yields more varied candidate sets than standard baselines while keeping the best discovered design competitive.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (5) and Algorithm 1 rely on DKL(qπ||pτ_D) where pτ_D is a finite empirical measure over a continuous VAE latent space; with no density estimator specified, the KL term is infinite/undefined, so the provable-optimality analysis does not cover the algorithm that produced Table 1.","rationale":"The paper's empirical contribution is real: Table A1 shows large Pairwise Diversity gains and Best@128 values at or above baselines, code is public, and ablations are reported. But the paper's stated central derivation, including the abstract's 'provably optimal' language, is built on Eq. (5)'s KL penalty. That penalty is not defined for the pair of objects the paper actually uses: a continuous generative/optimization distribution over a VAE latent space and an empirical τ-weighted measure over a finite dataset. This is not a stylistic gap; it makes the objective, the entropy-divergence equivalence in Lemma 3.3, and the dual lower bound in Lemma 3.4 formally undefined (infinite). The paper explicitly says 'pτ_D(x) ≈ pτ(x)' but never provides the density needed to make the ratio qπ/pτ_D meaningful. The code likely resolves this somehow, but a reader cannot verify that the implemented divergence has the properties used in the proof, and the paper does not state a limiting argument (e.g., kernel bandwidth or dataset size) under which the KL becomes finite. A second tension, W0=0 in Section 4, would also need reconciliation because with a Wasserstein-optimal critic the constraint could force qπ close to pτ_D; however, the undefined KL is the more fundamental blocker because it affects the statement of the objective itself. The right outcome is to keep the reader's CONDITIONAL verdict: the empirical claims may survive, but the theoretical claim should be revised to describe the actual implemented divergence, or explicitly downgraded to a heuristic.","tokens_in":78386,"tokens_out":17670,"duration_ms":189250,"concrete_test":"Take pτ_D to be the finite empirical measure from Definition 3.2 and any continuous qπ on the VAE latent space; compute DKL(qπ||pτ_D) and observe that it is +∞, so Eq. (5), Lemma 3.3, Lemma 3.4, and Algorithm 1 are not finite as written. Then inspect the public DynAMO code (github.com/michael-s-yao/DynAMO) for the actual KL estimator used when scoring candidates in Algorithm 1. If it uses a kernel density estimate, histogram binning, or a discriminator-based density-ratio approximation, re-derive Lemmas 3.3-3.4 for that estimator and check whether Eq. (13) still lower-bounds the dual. If no KL estimator is present, or if the code uses a surrogate divergence without a matching proof, the concern lands and the central theoretical claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central derivation starts from Eq. (5), J(π)=E_{qπ}[rθ]-(β/τ)DKL(qπ||pτ_D), and Definition 3.2 defines pτ_D as the empirical τ-weighted distribution over the finite offline dataset D. The paper states that optimization is performed in a continuous space (the VAE latent space, Appendix B). For any continuous qπ, DKL(qπ||pτ_D)=∫ qπ log(qπ/pτ_D) is +∞ because log pτ_D(x)=-∞ for x outside the finite support of the dataset; if qπ is instead supported only on dataset atoms, the policy can never propose the novel designs that the experiments report. The paper never specifies a density estimator, kernel, histogram, or smoothing for pτ_D in the main text or in Algorithm 1. Lemma 3.3, Lemma 3.4, and the λ update in Algorithm 1 are all derived from this KL term, so the 'provably optimal solution' claim is not attached to a well-defined objective. The actual implementation must be using some surrogate (e.g., a critic or density-ratio estimate); until that surrogate is specified and shown to satisfy the same dual derivation, the theoretical contribution does not explain the empirical results in Table 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DynAMO, a wrapper for offline model-based optimization (MBO) that adds a KL-divergence-based distribution-matching term to the surrogate objective, encouraging the sampling distribution to match a τ-weighted empirical distribution over the offline dataset, and adds an adversarial source-critic constraint to limit out-of-distribution queries. The authors derive DynAMO as a solution to a Lagrangian-dual reformulation of this objective, claiming a provably optimal and optimizer-agnostic method. Experiments on six MBO tasks across six backbone optimizers (gradient ascent, Adam, CMA-ES, CoSyNE, BO-qEI, BO-qUCB) report that DynAMO substantially improves pairwise diversity while keeping Best@128 competitive with existing MBO baselines.","tokens_in":78715,"tokens_out":5961,"duration_ms":64085,"significance":"If the claims held, this would be a practically attractive contribution: a simple, optimizer-agnostic modification of the MBO objective that yields more diverse candidate batches without sacrificing the best oracle-scored design. The empirical study is extensive and well executed: six tasks spanning discrete and continuous design spaces, six backbone optimizers, ten seeds, comparisons to both model-based and model-free baselines, ablations of the two algorithmic components, hyperparameter and budget sensitivity analyses, and a demonstration of downstream secondary-objective exploration. The authors also release code. These strengths make the empirical claims credible even though the theoretical derivation, as written, has load-bearing gaps that prevent the 'provably optimal' claim from being supported by the presented mathematics.","major_comments":[{"comment":"The central objective DKL(qπ||pτ_D) is not well-defined for the setting actually used in the experiments. pτ_D is defined as the empirical τ-weighted distribution over the finite dataset D, while optimization is performed in the continuous VAE latent space (Appendix B). For any continuous generative distribution qπ, DKL(qπ||pτ_D) is infinite because log pτ_D(x) = -∞ for x outside the finite support of D; if qπ is restricted to dataset atoms, the policy cannot propose the novel designs reported in Table 1. The main text and Algorithm 1 never specify a density estimator, kernel smoother, or density-ratio surrogate for pτ_D. Since Lemmas 3.3 and 3.4 and the λ update in Algorithm 1 are all built on this KL term, the theoretical derivation does not currently apply to the algorithm that produced the empirical results. The manuscript must either specify a smoothed/estimated reference density and prove the same dual results for it, or clearly state that the implemented algorithm uses a different surrogate for the KL term.","section":"§3.2, Definition 3.2, Eq. (5), Algorithm 1"},{"comment":"Lemma 3.4 establishes only a lower bound gℓ(λ) ≤ g(λ), and the text correctly says that maximizing gℓ bounds the dual maximum from below. However, the surrounding claims then call the resulting λ the 'globally optimal λ' and describe the overall procedure as a 'provably optimal solution' to (7). A lower-bound dual function does not in general yield the optimal Lagrange multiplier nor a certificate of primal optimality, especially because the primal is non-convex and Slater's condition is not assumed. The abstract and Section 3.5 overstate the formal result; the paper should either strengthen the derivation to an exact dual solution under stated conditions or temper the optimality claims accordingly.","section":"§3.4, Lemma 3.4 and the following paragraph"},{"comment":"Setting W0 = 0 in the constraint Epτ_D[c*(x)] - Eqπ[c*(x)] ≤ W0 is theoretically problematic. The source critic c* is trained, as in Eq. (4), to maximize exactly this difference subject to a Lipschitz bound, so if c* is close to optimal the constraint forces W1(pτ_D, qπ) ≤ 0, i.e., qπ = pτ_D almost surely. That would preclude discovering designs beyond the offline dataset, contradicting the method's goal and its reported Best@128 improvements. Since W0 = 0 is fixed for all experiments, the paper should explain why the constraint is not vacuous or binding in practice (e.g., because the critic is only approximately trained, or because the constraint is enforced over a finite batch rather than the full distribution) and reconcile this with the theoretical formulation.","section":"§4, implementation of W0=0, Algorithm 1"},{"comment":"The Lagrangian in Eq. (9) is defined over distributions π, with terms Eqπ[rθ(x)] and Eqπ[c*(x)] and the KL divergence DKL(qπ||pτ_D). Algorithm 1, however, evaluates a per-candidate score -L(xnew; λ) and stores it in Dgen, without specifying how the distribution-level KL term is converted into a pointwise objective. A per-point evaluation of the KL divergence is not defined unless qπ is given a density and pτ_D is smoothed. This is not merely a presentation issue: the final top-k selection in Algorithm 1 is based on these per-candidate scores, so the reader cannot reproduce the method without guessing the per-point surrogate. The algorithm must state the exact per-candidate objective used.","section":"Algorithm 1, line 'Evaluate and cache the candidates according to (9)'"}],"minor_comments":[{"comment":"There is a typo: 'singla optima' should read 'single optimum' or 'single optima'.","section":"Figure 1 caption"},{"comment":"Weak duality gives a lower bound on the primal optimum; the text says it provides a 'tight lower bound,' which is not guaranteed. Please rephrase to 'a lower bound'.","section":"§3.4, Eq. (10)"},{"comment":"The proof of Lemma 3.3 relies on the approximation rθ(x) ≈ r(x) over the dataset to replace e^{τ rθ(x)}/Zτ with pτ_D(x). The lemma statement should explicitly state this assumption, since the 'equivalence' is not algebraic but holds only under that approximation.","section":"Lemma 3.3 proof, Eq. (22)-(23)"},{"comment":"There are typographical errors in the proof: in Eq. (81), the absolute difference should be between DKL(qπ||pτ(x)) and DKL(qπ||pτ_D(x)), not between two identical terms; and 'log(|4Π|/δ)' in Eq. (83) should be 'log(4|Π|/δ)' for dimensional consistency with Eq. (81).","section":"Appendix D.4, Theorem D.9, Eqs. (81) and (83)"},{"comment":"The definition of pτ(x) uses an integral over X; for the discrete tasks (TFBind8, UTR, ChEMBL, Molecule), this should be a sum over the discrete design space, or the paper should clarify that the integral notation is understood in a measure-theoretic sense.","section":"Definition 3.2, Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is strong and the practical idea is appealing, but the theoretical core currently does not cover the implemented algorithm: the KL term is undefined for an empirical reference distribution on a continuous space, the dual result is a lower bound rather than an exact solution, and the W0=0 constraint is in tension with the stated objective. These are fixable in principle—by specifying a smoothed density estimator or a density-ratio surrogate, proving the dual result for that surrogate, and choosing W0 or an alternative constraint formulation—so I recommend major revision rather than rejection. The novelty is incremental over GAMBO (the source critic is reused), but the diversity-via-distribution-matching framing plus the extensive external comparisons make the empirical contribution publishable if the theory is brought in line with the algorithm."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the experimental contribution is real and useful, but the paper oversells the theory. The idea of adding a KL distribution-matching term to the MBO objective and using a source critic to keep candidates in distribution is a sensible new combination, and the results back it up: across six tasks and six backbone optimizers, DynAMO consistently improves pairwise diversity without hurting Best@128. That is a genuinely practical result for anyone who wants a diverse final batch for downstream multi-objective evaluation. Code is public, and the limitations section honestly flags that Median@128 is worse.\n\nThe formal part is where I part ways. Eq. (5) writes DKL(qπ||pτ_D) with pτ_D defined as an empirical distribution over a finite dataset. In the continuous VAE latent space used for all discrete tasks, that KL is infinite for any continuous qπ, unless they implicitly smooth pτ_D. The paper never says how. So the derivation in Section 3 does not formally apply to the algorithm that actually ran. That's a load-bearing gap, but it doesn't sink the empirical claim; the algorithm might still be doing something reasonable (like a density-ratio or critic approximation) and the experiments show it works. The authors need to say what they actually compute.\n\nTwo smaller issues. First, Lemma 3.4 gives a lower bound on the dual function, but the text later calls it an \"exact solution for λ\" — that's overstated; maximizing a lower bound is fine, but it's not exact. Second, W0=0 with a 1-Lipschitz critic trained to maximize Ep[c*] − Eq[c*] theoretically forces W1=0, i.e., q = pτ_D. In practice the critic is retrained and the constraint is a soft Lagrangian penalty, so it behaves, but the theory as stated contradicts the stated goal of exceeding the dataset. These are fixable with clearer writing and a few caveats.\n\nWho is this for? Practitioners in offline MBO who want diversity as a first-class objective and don't want to redesign their optimizer. They'll get value from the algorithm and the public code, even if they skip the proofs.\n\nVerdict: send it to peer review. The empirical work is solid enough to warrant referee time, but only with major revisions to the theory section. I'd cite it for the method, not for the formal guarantees.","headline":"DynAMO is a useful empirical plug-in for diversity in offline MBO, but the theoretical justification is rougher than the paper claims.","tokens_in":79182,"tokens_out":2485,"would_cite":true,"duration_ms":30152,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DynAMO makes offline model-based optimization produce diverse designs without sacrificing the best candidate.","keywords":["offline model-based optimization","design diversity","distribution matching","KL divergence","Lagrangian duality","source critic","generative design","Bayesian optimization"],"falsifier":"On a continuous task such as Superconductor or D'Kitty, compute the objective in Eq. (5) literally with the empirical $p^\\tau_D$ as a sum of Dirac masses and a Gaussian or neural policy $q_\\pi$; the KL divergence will be $+\\infty$ for any continuous $q_\\pi$, showing the algorithm as stated cannot run without additional smoothing. Separately, train the source critic to near-optimality with $W_0 = 0$ and check whether the constraint $\\mathbb{E}_{p^\\tau_D}[c^*] - \\mathbb{E}_{q_\\pi}[c^*] \\le 0$ forces the Wasserstein-1 distance to zero, which would imply $q_\\pi = p^\\tau_D$ and eliminate reward optimization.","tokens_in":78211,"feed_emoji":"🧬","tokens_out":5127,"duration_ms":49436,"temperature":0.7,"pith_summary":"The paper proposes DynAMO, a way to modify any offline model-based optimization (MBO) formulation so that the final batch of proposed designs is diverse while still containing high-quality candidates. The central idea is to add a KL-divergence penalty that pushes the distribution of generated designs toward a temperature-weighted distribution of the offline dataset, thereby inheriting the dataset's diversity. The paper derives an explicit, concave lower bound for the Lagrangian dual function using the Fenchel conjugate of $f(u) = u \\log u$, which makes the constrained objective tractable with any standard optimizer. Across six benchmark tasks and six backbone optimizers, DynAMO consistently improves pairwise diversity while keeping Best@128 quality competitive. If the paper is right, design diversity can be treated as a drop-in objective term rather than a separate algorithm.","feed_headline":"One objective term makes offline design search far more diverse","feed_subtitle":"DynAMO adds a KL penalty toward the dataset's own distribution, keeping top candidates while spreading the batch.","key_machinery":"The load-bearing objects are the $\\tau$-weighted distribution $p^\\tau_D(x) \\propto \\exp(\\tau r(x))$ over the offline dataset, which serves as the reference distribution whose diversity the policy is pushed to match, and the KL-divergence penalty $D_{\\mathrm{KL}}(q_\\pi \\| p^\\tau_D)$, which is the diversity term. The proof machinery is the Fenchel conjugate of the KL generator $f_{\\mathrm{KL}}(u) = u \\log u$, namely $f^*_{\\mathrm{KL}}(v) = e^{v-1}$, which converts the Lagrangian dual into a closed-form concave function of $\\lambda$. A 1-Lipschitz source critic $c^*$ supplies the out-of-distribution constraint, and together these pieces make the Lagrangian explicit enough that any standard optimization algorithm can be applied.","core_discovery":"The central claim is that the offline MBO objective $J(\\pi) = \\mathbb{E}_{q_\\pi}[r_\\theta(x)] - \\frac{\\beta}{\\tau} D_{\\mathrm{KL}}(q_\\pi \\| p^\\tau_D)$, subject to a source-critic constraint $\\mathbb{E}_{p^\\tau_D}[c^*(x)] - \\mathbb{E}_{q_\\pi}[c^*(x)] \\le W_0$, admits an explicit dual lower bound $g_\\ell(\\lambda) = \\beta\\big[\\lambda(\\mathbb{E}_{p^\\tau_D}[c^*(x)] - W_0) - \\mathbb{E}_{p^\\tau_D} e^{\\lambda c^*(x) - 1}\\big]$. Because $g_\\ell(\\lambda)$ is concave in $\\lambda$, the optimal Lagrange multiplier can be found by gradient ascent, leaving any backbone optimizer free to solve the resulting unconstrained Lagrangian. The paper argues that maximizing this modified objective makes a generative policy capture the diversity of high-scoring designs in the offline dataset, and it reports experiments showing that DynAMO-augmented optimizers discover both diverse and high-quality candidates.","pith_inferences":["The 'provably optimal' claim should be read as optimal for the lower-bounded surrogate dual, not necessarily for the original non-convex primal, since the derivation only guarantees a lower bound on the dual function.","A reader checking the theory should look for the missing density estimator: with a finite empirical $p^\\tau_D$ and a continuous generative policy, $D_{\\mathrm{KL}}(q_\\pi \\| p^\\tau_D)$ is infinite or undefined as written, so the derivation requires an implicit smoothing step to hold literally.","A direct testable extension is to use a synthetic design space with known modes and measure whether the number and location of modes recovered by DynAMO match the high-weight modes of $p^\\tau_D$; the Branin figure suggests this behavior but does not quantify it.","The source-critic constraint with $W_0 = 0$ pins the Wasserstein-1 distance between $q_\\pi$ and $p^\\tau_D$ to zero when the critic is near-optimal, which would force $q_\\pi = p^\\tau_D$ and contradict reward maximization; the algorithm's practical success likely depends on the critic not being trained to optimality, an implicit gap between theory and implementation."],"forward_implications":["Any existing MBO pipeline can be augmented with DynAMO by replacing its objective with the derived Lagrangian; the paper reports diversity gains across gradient ascent, Adam, CMA-ES, CoSyNE, BO-qEI, and BO-qUCB.","The final batch of 128 candidates retains competitive Best@128 oracle scores, so the diversity improvement does not come at the cost of best-candidate quality in the tested benchmarks.","The method applies to both discrete design spaces, through a VAE latent space, and continuous design spaces, making it portable across scientific domains.","The framework extends to mixed $f$-divergences; the appendix derives an analogous dual lower bound for a mixed $\\chi^2$-KL divergence, though the experiments show KL alone is non-inferior.","A more diverse batch enables better downstream evaluation of secondary objectives such as manufacturing cost, toxicity, or DNA GC content, as demonstrated on vehicle safety, welded beam, and UTR tasks."],"supporting_citations":[{"why":"Defines offline model-based optimization and the baseline objective that DynAMO modifies.","marker":"Trabucco et al. (2021)"},{"why":"Supplies the 1-Lipschitz source critic and the Wasserstein distance formulation used in the DynAMO constraint.","marker":"Arjovsky et al. (2017)"},{"why":"Establishes source-critic regularization for offline MBO, the constraint approach DynAMO builds on.","marker":"Yao et al. (2024)"},{"why":"Provides the Fenchel conjugate of $f(u) = u \\log u$, the key algebraic step in the explicit dual function.","marker":"Borwein & Lewis (2006)"},{"why":"Motivates distribution matching as a way to capture the diversity of an expert or reference distribution.","marker":"Ho & Ermon (2016)"},{"why":"Defines the pairwise diversity metric and motivates the importance of diverse design candidates.","marker":"Jain et al. (2022)"},{"why":"Provides the Design-Bench benchmark tasks used for the paper's experimental evaluation.","marker":"Trabucco et al. (2022)"}],"fun_headline_variants":["DynAMO: diverse designs from offline data via matching","KL penalty term diversifies offline design optimization","Diversity by design: matching the data distribution","Offline MBO gets a diversity boost with DynAMO","Add a penalty, keep quality, gain design diversity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derivation assumes the KL divergence between a continuous generative policy and the finite empirical $\\tau$-weighted dataset distribution is a well-defined finite quantity, but as written that divergence is infinite or undefined for continuous design spaces without an explicit density estimator; a second fragile premise is that the source-critic constraint with $W_0=0$ does not collapse the solution to $q_\\pi = p^\\tau_D$.","fun_headline_variants_meta":{"raw":{"variants":["DynAMO: diverse designs from offline data via matching","KL penalty term diversifies offline design optimization","Diversity by design: matching the data distribution","Offline MBO gets a diversity boost with DynAMO","Add a penalty, keep quality, gain design diversity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000151,"raw_usage":{"total_tokens":1178,"prompt_tokens":903,"completion_tokens":275,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":198}},"tokens_in":519,"tokens_out":275,"duration_ms":3680,"temperature":1.0,"reasoning_tokens":198,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T22:32:35.993991+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a continuous task such as Superconductor or D'Kitty, compute the objective in Eq. (5) literally with the empirical $p^\\tau_D$ as a sum of Dirac masses and a Gaussian or neural policy $q_\\pi$; the KL divergence will be $+\\infty$ for any continuous $q_\\pi$, showing the algorithm as stated cannot run without additional smoothing. Separately, train the source critic to near-optimality with $W_0 = 0$ and check whether the constraint $\\mathbb{E}_{p^\\tau_D}[c^*] - \\mathbb{E}_{q_\\pi}[c^*] \\le 0$ forces the Wasserstein-1 distance to zero, which would imply $q_\\pi = p^\\tau_D$ and eliminate reward optimization.","supporting_citations":[],"review_version":1}