REVIEW 2 major objections 5 minor 15 references
Composing diffusion models by ratio-of-densities can silently produce non-normalizable intermediate paths; the paper derives a criterion that predicts this collapse and a time-varying-exponent correction that provably prevents it.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 17:11 UTC pith:OF6ZETLJ
load-bearing objection A genuinely useful diagnosis of path collapse in heterogeneous diffusion steering, with a correction that is promising but whose headline guarantee overreaches its coordinate-wise proof. the 2 major comments →
On the Collapse of Generative Paths: A Criterion and Correction for Diffusion Steering
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
For compactly supported targets, a ratio-of-densities composition h_t = ∏ (q_t^(i))^{γ_i(t)} stays integrable exactly when C_k(t) := Σ_{i:k∈I_i} γ_i(t)/(α_t^(i))² > 0 for every coordinate k (given h_1 integrable). If some C_k(t) < 0, Marginal Path Collapse occurs: an intermediate density is non-normalizable, so the sampler follows a different, unintended path. ACE fixes this by adding a bump B t(1−t) to one exponent, preserving the endpoints while restoring C_k(t)>0, and provides a weighted Feynman–Kac SDE/ODE whose marginal is the corrected path. Because the criterion depends only on schedules and exponents, path validity can be certified before any network evaluation.
What carries the argument
The load-bearing object is the coordinate-wise path-existence criterion C_k(t) = Σ_{i:k∈I_i} γ_i(t)/(α_t^(i))², which converts the question 'is the composed intermediate density normalizable?' into a sign check on schedules and exponents. The correction is the bump-function protocol: adding B·t(1−t) to one positive-exponent schedule lifts C_k(t) above zero on (0,1) without altering the endpoint distributions. The sampling engine is a weighted Feynman–Kac SDE/ODE whose weight update includes the new time-derivative terms γ̇_i(t) log q_t^(i), together with resampling on effective sample size; setting the exponents constant recovers the earlier Feynman–Kac corrector as a special case.
Load-bearing premise
The guarantee assumes each expert's velocity and score fields are the exact generators of its density path; real pretrained models supply learned approximations, and the paper itself notes that the correction parameter B linearly amplifies their errors.
What would settle it
For a compactly supported target with exact (analytic) expert densities, pick schedules and exponents so that C_k(t*) < 0 at some interior time; the theorem predicts the integral of the composed density at t* diverges. Numerically computing that integral and finding it finite would refute the criterion.
If this is right
- Inference-time steering of heterogeneous pretrained models (e.g., de-novo, conformer, and pocket-conditioned molecular experts) can be run at high guidance scales without the generated molecules fragmenting into invalid structures.
- Practitioners can pre-screen any composition: compute C(t) on the sampler's time grid from schedules and exponents alone; if it stays positive, the path is guaranteed valid, so costly failures are caught before sampling.
- Because ACE keeps the endpoint densities fixed and only reshapes intermediate marginals, it can replace constant-exponent ratio-of-densities recipes (classifier-free guidance, product-of-experts, contrastive decoding) whenever experts use mismatched schedules.
- Even when no collapse occurs, increasing C(t) via a small bump tightens the intermediate distributions' tails, which empirically improves attribute accuracy in compositional image generation.
- The weighted sampler with resampling removes out-of-distribution trajectories from the batch, which the no-resampling heuristic leaves in place.
Where Pith is reading between the lines
- The criterion also suggests a quantitative safety margin: the minimum positive value of C(t) could serve as an a priori 'stability score' for a composition, guiding how much bump is needed before running any model.
- Because the proof of the criterion relies on compact support and exact score fields, a testable extension is to measure how empirical sample validity degrades as the learned scores' error grows, and whether that degradation tracks the dip in C(t).
- The paper notes that the bump parameter B linearly amplifies network approximation errors; this implies an optimal-B trade-off curve (peak near B=30 in their experiments) that could be predicted from the score-error profile of each expert, rather than tuned per task.
- Extending ACE to mixed continuous-categorical molecular spaces would need a discrete analogue of the Gaussian-envelope argument; the paper explicitly leaves this open, so the current guarantee does not cover models that evolve atom types as categorical variables.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies inference-time steering of diffusion/flow models through heterogeneous ratio-of-densities paths p_t^* ∝ ∏_i (q_t^{(i)})^{γ_i(t)}. It identifies Marginal Path Collapse, where an intermediate density becomes non-normalizable despite valid endpoints; gives a coordinate-wise path-existence criterion C_k(t) for compactly supported targets (Theorem 2.1); proposes ACE, a bump-function correction of the exponents (Theorem 2.2) together with a weighted Feynman–Kac SDE/ODE sampler with time-varying exponents (Theorem 2.3); and reports experiments on a synthetic checkerboard, flexible-pose scaffold decoration, and compositional image generation.
Significance. The Gaussian-envelope proof of Theorem 2.1 is a genuine conceptual contribution: it converts an informal instability into a simple, checkable condition, and the converse direction is valid. The extension of Feynman–Kac steering to time-dependent exponents in Theorem A.1 is useful and appears correctly derived. The empirical evaluation is broad, and the paper is candid in Appendix F about several limitations. However, the headline claim that ACE ``guarantees a valid probability path whenever endpoints are valid'' is stronger than what the supplied proofs establish: the constructive one-bump theorem is proved only for a scalar criterion, not for the coordinate-wise C_k(t), and the sampling guarantee assumes exact expert score/velocity fields. These gaps are fixable, but they are load-bearing for the paper's central theoretical promise.
major comments (2)
- [§2.4, Appendix B.3 (Theorem B.2)] The constructive proof of the one-bump protocol is given for the scalar S(t)=Σ_i γ_i(t)/(α_t^(i))^2, not for the coordinate-wise C_k(t)=Σ_{i:k∈I_i} γ_i(t)/(α_t^(i))^2 in Eq. (4). Adding a bump Bt(1−t) to one expert j changes only the coordinates k∈I_j; if a coordinate k* has C_{k*}(t)<0 and k*∉I_j, that C_{k*} is unchanged. Since §2.1 explicitly permits arbitrary coordinate subsets I_i, the stated conclusion of Theorem 2.2 (``there always exists ... choosing one index j ... satisfies the path existence criterion for all t'') does not follow from the supplied proof. The existential claim may be salvageable by bumping a set of experts whose coordinate sets cover all coordinates where C_k dips below zero, but the current statement is overbroad. This gap is load-bearing because the abstract and §2.4 claim a guaranteed valid path for arbitrary heterogeneous experts.
- [Appendix A.1 (Theorem A.1) and §4] Theorem 2.3's sampling guarantee assumes exact velocity and score fields v_t^(i), s_t^(i) generating each q_t^(i). For pretrained networks these fields are learned approximations, and the paper itself notes in §4 that a large bump B ``linearly amplifies inherent network approximation errors.'' Therefore the abstract's ``guarantees a valid probability path whenever endpoints are valid'' should be scoped to the exact mathematical path, not to the path realized with approximate scores. This is the standard idealization in the guidance literature and is not fatal, but the unconditional wording in the abstract and §2.4 should be corrected.
minor comments (5)
- [Abstract and §1 vs §2.4, Theorem 2.2] The phrase ``whenever the endpoints are valid'' is not equivalent to Theorem 2.2's hypothesis, which requires C_k(0)>0 and lim_{t→1−}C_k(t)>0 for every coordinate. In particular, integrability of h_1 for compactly supported q_1^(i) can hold even when the limiting coefficient is non-positive. The wording should be aligned with the actual theorem assumptions.
- [§3.1] The sentence ``ACE differs from NR and FKC only by its adaptive exponent correction (with B=30 for all experiments)'' conflicts with Table 2, which sweeps B=10,20,30,40,50,100, and with Figure 6. Presumably B=30 is used in the molecular and main synthetic experiments; please make this explicit.
- [Algorithm 1, line 8] The stochastic term is written as σ_t \tilde{s}_t·ξ_j (1/√Δt), which looks dimensionally inconsistent until one multiplies by Δt. A brief note that the displayed quantity is the per-unit-time form of the Itô increment would improve readability.
- [Table E.10] The table reports CLIP and Local CLIP, but the caption does not define them. Please add definitions or a reference so that the metric is unambiguous.
- [Appendix F] The limitations listed in Appendix F (extension to arbitrary transport, hybrid continuous-categorical domains, and error propagation in composed models) are appropriate and welcome. The main text's ``complete solution'' language should be tempered to acknowledge these scopes.
Circularity Check
No significant circularity; the derivation is self-contained and uses standard external Feynman–Kac identities.
full rationale
The paper's central chain defines h_t = Π_i (q̃_t^(i))^{γ_i(t)}, proves the path-existence criterion via Gaussian-envelope integrability bounds (Theorem B.1), constructs exponent corrections that preserve endpoint values (Theorems 2.2/B.2), and derives the weighted SDE/ODE sampler from the Feynman–Kac identity (Proposition A.1, cited to external literature: Skreta et al. 2025a and Stoltz et al. 2010). No input is defined in terms of the target result: C_k(t) is computed analytically from the schedules and exponents rather than fitted to observed samples, and the bump parameter B is a hand-set hyperparameter pre-screened on the same analytical criterion. The endpoint distributions are preserved by imposing γ̃(0)=γ(0) and γ̃(1)=γ(1) before the correction, so no boundary quantity is a renamed fitted parameter. The argument does not rely on a load-bearing self-citation: the Feynman–Kac weighting comes from a standard external identity, and the rest of the proof is self-contained. The reviewer-identified issue is a rigor gap, not circularity: Theorem B.2 proves positivity of the scalar S(t) rather than the coordinate-wise C_k(t) of Eq. 4, so the one-bump protocol in Theorem 2.2 may be under-proved for coordinates outside the bumped expert's support. That is a proof/completeness concern, not a reduction of the conclusion to the assumptions. Likewise, the assumption of exact expert score/velocity fields is a transfer assumption to learned models, not a circular step. Therefore no circular step meets the required evidentiary standard.
Axiom & Free-Parameter Ledger
free parameters (1)
- Bump magnitude B in gamma_j(t) := gamma_j(t) + B t(1-t) =
B = 30 for synthetic and molecular experiments; B = 5 for image experiments
axioms (5)
- domain assumption Target densities q_1^(i) have compact support.
- domain assumption Each expert path q_t^(i) is exactly the marginal of X_t^(i) = α_t^(i) X_0^(i) + β_t^(i) X_1^(i) with X_0^(i) ~ N(0,I) and known differentiable schedules.
- domain assumption Endpoint integrability of h_1 and support inclusion for negative exponents.
- domain assumption Velocity and score fields v_t^(i), s_t^(i) are exact C¹ generators of their density paths.
- standard math Feynman–Kac expectation identity (Proposition A.1) from prior literature holds.
read the original abstract
Inference-time steering adapts pretrained diffusion and flow models to new tasks without retraining, often utilizing ratio-of-densities constructions that reweight time-indexed marginals with fixed exponents. We identify Marginal Path Collapse, a failure mode in which the intermediate density defined by such compositions becomes non-normalizable despite valid endpoints. This collapse can arise when composing heterogeneous experts trained with mismatched noise schedules (and/or negative exponents / partial supports). To address this, we provide (i) a sharp sufficient Path Existence Criterion that characterizes when the composed intermediate densities are mathematically well-defined, and (ii) Adaptive Path Correction with Exponents (ACE), which generalizes Feynman-Kac steering to support time-varying exponents. Our analysis reveals that ACE controls the quantile radius of the intermediate distributions, providing a theoretical mechanism for path stabilization observed in experiments. On flexible-pose scaffold decoration, a drug design task composed of de-novo, conformer, and protein-conditioned experts, ACE prevents collapse and significantly outperforms constant-exponent baselines. Furthermore, ACE improves attribute success rates in compositional image generation, establishing it as a general framework for compositional sampling. Project Page: https://ziseoklee.github.io/projects/ACE/
Figures
Reference graph
Works this paper leans on
-
[1]
We can ensure that this exists, since there existsδ 1, δ2 >0 such that∀t∈[0, δ 1)∪(δ 2,1),S(t,{γ i}i)>0
Find the timet min where the sumS(t,{γ i}i) is at its minimum. We can ensure that this exists, since there existsδ 1, δ2 >0 such that∀t∈[0, δ 1)∪(δ 2,1),S(t,{γ i}i)>0. This comes from our assumptionS(0,{γ i}i)>0,lim t→1− S(t,{γ i}i)>0 and the continuity ofS(t,{γ i}i) with respect tot. Then,S(t,{γ i}i) is a continuous function on a compact set [δ 1, δ2] wh...
-
[2]
We construct a diverging lower bound
Sufficiency for Marginal Path Collapse att ∗. We construct a diverging lower bound. Applying Proposition B.2 using the lower bound formula forγ i(t)>0 and the upper bound formula forγ i(t)<0, we derive: ht(x)≥ Y γi>0 C−,ie −∥πi (x)−µ(i) t ∥2 +∥µ(i) t ∥2 −V (i) t 2(α(i) t )2 γi Y γi<0 C+,ie −∥πi (x)∥2 +2∥πi (x)∥R(i) t 2(α(i) t )2 γi The quadr...
-
[3]
This will have a greater positive impact when the denominator is small
We will choose the indexjsuch that γj (tmin) (α(j) tmin )2 is the largest. This will have a greater positive impact when the denominator is small
-
[4]
Sincep 1 is compactly supported,ρ t is supported on a compact set Ω t
Letρ t(y) be the density ofY t =β tX1. Sincep 1 is compactly supported,ρ t is supported on a compact set Ω t. The path density is the convolutionp t =N t ∗ρ t.We analyze the ratiop t(x)/Nt(x): pt(x) Nt(x) = Z Ωt Nt(x−y) Nt(x) ρt(y)dy= Z Ωt exp 2x·y− ∥y∥2 2σ2 t ρt(y)dy=E Y∼ρ t exp 2x·Y− ∥Y∥ 2 2σ2 t . Upper Bound:LetR t = sup y∈Ωt ∥y∥<∞. Using Cauchy-Schwar...
-
[5]
Marginal Path Collapse
Finally, define theadaptive exponentvia ˜γ j(t) :=γ j(t) +B·b(t) where B= |Smin| cmin + 1 All other functions are preserved: ˜γi =γ i ∀i̸=j. Now we verify thatS(t,{˜γ i}i) with our adaptive exponent ˜γj(t) is strictly positive for allt∈[0,1]. Fort∈[0, δ 1)∪(δ 2,1],S(t,{˜γ i}i)≥S(t,{γ i}i)>0 is already positive. Fort∈[δ 1, δ2], S(t,{˜γi}i) =S(t,{γ i}i) + |...
2019
-
[7]
We denote the minimum value ofb(t) ont∈[δ 1, δ2] byb min >0
Define thebump functionbyb(t) =t(1−t) such thatb(0) =b(1) = 0 andb(t) is always nonnegative on [0,1]. We denote the minimum value ofb(t) ont∈[δ 1, δ2] byb min >0
-
[9]
This exists and is positive sinceα (j) t is positive and continuous on [δ1, δ2]
Letc min := min t∈[δ1,δ2] bmin/(α(j) t )2. This exists and is positive sinceα (j) t is positive and continuous on [δ1, δ2]. We also have the inequality, cmin ≤ b(t) (α(j) t )2 ∀t∈[δ 1, δ2]
-
[12]
Heterogeneity in Representations.Unlike images, molecules admit multiple incompatible representations. Some models encode atom types as continuous one-hot vectorsH one-hot ∈R N evolving under Gaussian convolution; others use categorical variables with discrete states (Dunn and Koes, 2025; Lin et al., 2023) or operate in latent spaces (Ketata et al., 2024;...
2025
-
[13]
Empty Intersection
The “Empty Intersection” of Schedules.Even if we restrict our scope to models with compatible continuous representations, a homogeneous triplet often does not exist. In our scaffold decoration experiment, all applicable DN models (EDM, GCDM) employquadraticschedulers, while SBDD models (DiffSBDD, DualDiff) use either quadratic or sigmoid schedules. Howeve...
2022
-
[14]
that modify model’s internal signals. In contrast, ACE offers a sampling strategy that achieves unbiased modular composition using a single fundamental expert model without requiring retraining, external data, or architectural changes. Empirical results on COCO-MIG.We evaluate on the COCO-MIG benchmark (Zhou et al., 2024b) and report the instance attribut...
2021
-
[15]
59 with no path collapse
For the image experiments, it sufficed to resample once att s = 0.3 due to a small batchsize ofN= 3. 59 with no path collapse. Remarkably, ACE matches or exceeds a task-specific adapter GLIGEN (Li et al., 2023b) on several metrics, while remaining completely training-free. Qualitatively, Fig. E.13 shows that ACE produces sharply localized objects with cor...
-
[2015]
undruggable
success. (ii) Docking Score, computed with QVina2 (Alhossary et al., 2015), assesses pocket compatibility. For each protein pocket, we report the Mean, Median, Best, Top-3, and Worst scores, as well as the Optimization Success Rate (OSR), defined as the proportion of generated molecules 36 with docking scores better than the reference ligand. We further c...
2015
-
[2022]
URLhttps://openreview.net/forum?id=PzcvxEMzvQC. H. Ye, H. Lin, J. Han, M. Xu, S. Liu, Y. Liang, J. Ma, J. Zou, and S. Ermon. TFG: Unified training-free guidance for diffusion models. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=N8YbGX98vc. J. Yu, Y. Wang, C. Zhao, B. Ghanem, and J...
Pith/arXiv arXiv 2024
-
[2024]
doi: 10.1021/acs.jcim.3c01466. Early Access. G. Xu, Y. Yu, Y. Jiang, Y. Yang, and Y. Bian. CoFM: Molecular conformation generation via flow matching in SE(3)-invariant latent space. InICML 2025 Generative AI and Biology (GenBio) Workshop, 2025. URLhttps://openreview.net/forum?id=C0jrjy4F1D. M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, and J. Tang. Geodiff: A ...
-
[2025]
URLhttps://api.semanticscholar.org/CorpusID:280677042. D. Epstein, A. Jabri, B. Poole, A. A. Efros, and A. Holynski. Diffusion self-guidance for controllable image generation. InThirty-seventh Conference on Neural Information Processing Systems, 2023. URLhttps://openreview.net/forum?id=qgv56R2YJ7. P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M¨ uller...
Pith/arXiv arXiv 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.