{"id":"6b5c7543-002d-446d-bf78-5b1b4fe32b62","arxiv_id":"2412.07877","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An optimal diffusion discretization spends equal Fisher-divergence \"distance\" per step, phi*(s) = Lambda^-1(Lambda s), and the resulting schedule recovers hand-tuned samplers on image benchmarks.","lead":"This paper derives a rule for spacing the steps of a diffusion model sampler: each step should do the same amount of \"work\" measured by how much the score function changes between adjacent noise levels. The rule is computed from the model's own score estimates, removes manual schedule search, and reproduces schedules that previously required hand tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (14) omits a ∇F^{-T} factor and is false for non-isometric predictors; because the proof of Theorem 2.1 relies on it, the general φ* schedule is not rigorously established as written.","rationale":"The reader's stated weakest assumption is Assumption 3.1, concerning the transfer of the optimality result to Heun solvers with imperfect scores; my concern is different and more internal: the change-of-variables identity Eq. (14) is false for non-isometric predictor maps, and the proof of Theorem 2.1 depends on it. I nonetheless do not move the verdict because the leading-order conclusion appears salvageable: the missing ∇F^{-T} factor is I+O(∆t), so the local coefficient δ(t) in Theorem 2.1 is correct to the order used in Theorem 3.1, and the corrector-optimized cost Lc—the cheaper and recommended variant—uses F=identity where Eq. (14) is exact. Thus the main schedule formula is likely correct, but the paper's proof is incomplete as written; this is exactly the kind of revision a conditional acceptance should require. My agreement is partial because the reader mentioned Eq. (14)'s exactness as questionable in the rationale but did not make it the weakest assumption; I consider the proof gap equally load-bearing as the Assumption 3.1 transfer issue. The empirical FID results remain suggestive rather than conclusive, and the lack of error bars is a real but secondary concern. Overall, the conditional verdict remains appropriate.","tokens_in":21589,"tokens_out":16507,"duration_ms":152948,"concrete_test":"In 1D, take p_t = p_{t'} = N(0,1), F(x)=a x with a=1.1, so the predictor is an exact scaling. Compute both sides of Eq. (14): the left side v(t')^2 D(p_{t'} || F^# p_t) equals v^2(a^2-1)^2/a^2, while the right side v^2 E_{X~p_t}[(d/dx log G(X))^2] equals v^2(a^2-1)^2. These differ by a factor a^2, demonstrating Eq. (14) is not an identity. Then independently re-derive Theorem 2.1's leading coefficient from the corrected expression L = v^2 E[||∇F^{-T}∇log G||^2]; if the resulting δ(t) matches Eq. (16) to leading order, the schedule formula survives but the proof still needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central schedule formula φ*(s)=Λ^{-1}(Λs) rests on the local cost δ(t) of Theorem 2.1. Theorem 2.1 is proved via Eq. (14), which claims L(t,t') = v(t')^2 E_{X_t~p_t}[||∇ log G_{t,t'}(X_t)||^2] with G_{t,t'}(x)=p_{t'}(F_{t,t'}(x))|det ∇F_{t,t'}(x)|/p_t(x). This identity is not correct for a general predictor map. Using z=F(x), the pushforward density satisfies q(F(x))=p_t(x)/|det∇F(x)|, and one obtains ∇_z log q(F(x)) = ∇F^{-T}(∇log p_t(x) - ∇log|det∇F(x)|). Combining this with ∇log G = ∇F^T ∇log p_{t'}(F(x)) + ∇log|det∇F(x)| - ∇log p_t(x) shows that the integrand on the right side of Eq. (13) equals ||∇F^{-T}∇log G(x)||^2 after the change of variables, not ||∇log G(x)||^2. The paper's Eq. (14) drops the ∇F^{-T} factor. For the identity predictor F(x)=x this factor is harmless, so the corrector-optimized cost Lc is exact. For the Euler probability-flow predictor of Example 2.2, ∇F = I + O(∆t), so the missing factor only affects O(∆t^3) terms; the leading-order coefficient δ(t) stated in Theorem 2.1 is in fact correct. But the proof in Appendix A.1 uses the false identity in the mean-value-theorem argument (Eqs. (23)-(26)), so the paper does not actually prove Theorem 2.1 for general predictors, and the claimed optimality of φ* for the predictor-optimized cost is not rigorously supported. The authors should either correct Eq. (14) (e.g., state it as an asymptotic leading-order relation) or re-prove Theorem 2.1 directly from Eq. (13).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a principled method for choosing the discretisation schedule used when simulating the reverse diffusion in denoising diffusion models. The authors define an incremental cost L(t,t') based on the work done by a hypothetical predictor-corrector step, derive a local quadratic approximation delta(t) (Theorem 2.1), and then show that, in the dense-schedule limit, the total cost is minimized by the schedule generator phi*(s) = Lambda^{-1}(Lambda s), where Lambda(t) = int_0^t sqrt(delta(u)) du (Theorem 3.1). Algorithm 1 estimates this schedule using score evaluations only, and Algorithm 2 extends it to online training. Experiments on a mollified Cantor distribution, a bimodal example, and CIFAR-10/FFHQ/AFHQv2/ImageNet show that the learned schedules recover the shape of the best hand-tuned Karras schedules and achieve competitive FID scores.","tokens_in":22026,"tokens_out":5817,"duration_ms":74120,"significance":"If the theoretical result is fully established, the paper makes a useful contribution: it gives a parameter-free, data-dependent criterion for schedule selection in diffusion samplers, replacing manual hyperparameter sweeps, and it connects schedule design to a Fisher-divergence geometry of the diffusion path. The proposed cost is derived rather than fit, and the resulting schedule update is cheap and scalable, requiring only score evaluations. The paper also honestly states the main limitations in Section 5: the theory assumes perfect scores and the predictor-corrector formulation, while the image experiments use deterministic Heun solvers with learned scores. The empirical finding that the optimized schedule matches the manually tuned Karras schedule is striking and well presented, although the formal transfer of the optimality theorem to the actual sampler remains an assumption.","major_comments":[{"comment":"Equation (14) is stated as an exact change of variables, but it is not correct for a general predictor map. With z = F_{t,t'}(x), the pushforward density satisfies q(F(x)) = p_t(x)/|det∇F(x)|, and the correct transformation introduces a factor ∇F^{-T} in the score difference. Consequently the integrand after the change of variables is ||∇F^{-T}∇log G_{t,t'}(x)||^2, not ||∇log G_{t,t'}(x)||^2. This identity is used in the proof of Theorem 2.1 in Appendix A.1, specifically in the mean-value-theorem argument around Eqs. (23)-(26). For the identity predictor the missing factor is harmless, and for the Euler probability-flow predictor of Example 2.2 the discrepancy is O(Δt^3), so the leading-order coefficient δ(t) in Theorem 2.1 is likely correct in the cases used for the main experiments. However, as written, the proof does not establish Theorem 2.1 for arbitrary predictors, and the predictor-optimized cost L_p is therefore not rigorously supported. The authors should either correct Eq. (14) and re-prove Theorem 2.1 directly from Eq. (13), or explicitly state Eq. (14) as an asymptotic leading-order relation and restrict the theorem accordingly.","section":"Section 2.3, Eq. (14); Appendix A.1"},{"comment":"The optimality of φ* is proved under Assumption 3.1, which requires that every predictor-corrector step reaches stationarity, and the paper also assumes perfect score estimation throughout the theoretical derivation. The image experiments in Section 4.4, however, use deterministic second-order Heun/ODE solvers with learned scores and no Langevin corrector, as described in Appendix C.2. Thus Theorem 3.1 does not formally imply that the schedules used in Table 1 minimize FID for these samplers. The authors acknowledge this limitation in Section 5, but the abstract and Section 4.4 state that the method recovers the optimal schedule without this caveat. I recommend that the authors either soften the claim or provide additional empirical evidence that schedules minimizing L_c also minimize the actual sampling error beyond the six-schedule correlation plot in Figure 4 (left).","section":"Section 3, Assumption 3.1; Section 4.4"},{"comment":"Theorem 3.1 is a dense-limit statement: as T→∞, T L(φ,T) → E(φ) and L(φ,T) ≳ Λ^2/T. Algorithm 1, however, outputs a finite schedule by interpolating and inverting a finite approximation of Λ. The paper states that for finite T the optimal schedule makes incremental costs approximately constant, but this is only an approximation inherited from the local quadratic expansion. The finite-T schedule produced by the interpolation step is not proved to be the exact minimizer of the finite sum of costs. This is not a fatal issue if presented as an approximation, but the text should distinguish the asymptotic theorem from the finite-T algorithm more carefully.","section":"Section 3.1 and Algorithm 1"}],"minor_comments":[{"comment":"In the expression for log G_{t,t'}(x), the term '- pt(x)' should read '- log p_t(x)'. The printed formula is a typo, but it appears in a proof and should be corrected.","section":"Appendix A.1, Eq. (28)"},{"comment":"The type of the corrector is written as L_{t,τ}: X × P(X) → [0,1], which is not the usual notation for a Markov transition kernel. The intended meaning is clear, but the notation is nonstandard and should be cleaned up.","section":"Definition 2.2"},{"comment":"The FID values are reported without error bars or multiple seeds, and the authors justify this in the checklist. This is acceptable practice for this type of experiment, but the paper should explicitly state that the FID differences between the top schedules are within the typical run-to-run variability of a single FID evaluation.","section":"Section 4.4 and Table 1"},{"comment":"The monotone spline interpolation used to construct Λ^{-1} is referenced to Fritsch and Carlson (1980), but the implementation details are not given. In particular, the handling of endpoints and the number of quadrature points for the cumulative sums would be useful for reproducibility.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of the journal and the core idea is attractive. The main concern is the correctness gap in the proof of Theorem 2.1 for general predictors, which affects the predictor-optimized cost. This is fixable by either correcting the change of variables or stating the result as a leading-order asymptotic, and the empirical section already shows the corrector-optimized variant performs well. The mismatch between Assumption 3.1 and the actual samplers should be addressed explicitly but does not require new theory."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's central object—an explicit dense-limit optimal schedule φ*(s)=Λ⁻¹(Λs) under a Fisher-divergence cost—is new and, for the corrector-optimized cost, sound and practically useful. The predictor-optimized extension is not rigorously proved as written: Eq (14) is a change of variables that drops a ∇F⁻ᵀ factor.\n\nCredit where due. The cost story is well motivated: a hypothetical predictor-corrector step gives an incremental cost equal to the expected squared score gap between target and predicted pushforward; the identity predictor reduces it to the Fisher divergence between pt′ and pt; Theorem 2.1 gives the local quadratic cost δ(t); and Theorem 3.1 identifies φ* by equalizing per-step cost. Algorithm 1 is a clean adaptation of Syed et al.'s SMC schedule update, and the self-citation is appropriate there. Experiments are solid: on CIFAR-10, FFHQ, AFHQv2, and ImageNet the learned schedules recover the manually tuned ρ=7 Karras schedules, improve small-step-count FID, and the cost ranks schedules better than Sabour et al.'s KLUB. The method is cheap for the corrector-optimized case and needs no hyperparameter search.\n\nNow the soft spots. The stress-test note is correct: Eq (14) is not an identity for general predictors. With z=F(x), the score difference transforms like ∇F⁻ᵀ∇logG, not ∇logG. For identity predictor this factor is absent, so the corrector-optimized cost Lc is exact. For the Euler probability-flow predictor, ∇F=I+O(Δt), so the missing factor changes only O(Δt³) terms and the leading δ(t) remains correct. But because the proof of Theorem 2.1 uses the false identity inside the mean-value argument (Eqs. 23–26), the general predictor-optimized formula is unproved as stated. This is fixable, not fatal, and it does not take down the main corrector-optimized contribution.\n\nSecond caveat: Assumption 3.1 (exact corrector stationarity, perfect scores) does not describe the FID experiments, which use deterministic Heun/ODE solvers and learned scores. So the formal optimality theorem does not transfer to the reported samples; the strong empirical agreement with ρ=7 mitigates but does not eliminate this. No error bars on FID are a minor issue, and the paper acknowledges it in the checklist.\n\nBottom line: the paper deserves a serious referee. I would cite the corrector-optimized schedule result and bring it to a reading group. Ask the authors to fix Eq (14), state it as a leading-order asymptotic, or re-prove Theorem 2.1 directly; then the general φ* claim will be on solid footing.","headline":"A genuinely useful optimal-schedule formula with a fixable Jacobian gap in the predictor-optimized proof.","tokens_in":22586,"tokens_out":3740,"would_cite":true,"duration_ms":33896,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The optimal diffusion sampling schedule is the one that spends equal Fisher-divergence cost per step, computable from score evaluations alone.","keywords":["diffusion models","discretisation schedule","score-based generative models","Fisher divergence","schedule optimisation","predictor-corrector","image generation"],"falsifier":"Take a target with a known score (for example a Gaussian mixture) and compute $\\delta(t)$ exactly; simulate the reverse diffusion with a fixed ODE solver and compare the $\\phi^*$ schedule against a schedule that equalises the solver's local truncation error. If $\\phi^*$ does not give the lowest end-to-end error for small step counts, the transfer from the perfect-corrector cost to practical samplers fails.","tokens_in":21359,"feed_emoji":"⚖️","tokens_out":7118,"duration_ms":62971,"temperature":0.7,"pith_summary":"The paper aims to replace hand-tuned discretisation schedules in denoising diffusion models with an automatic, data-dependent rule. It derives a cost for moving samples from one point on the diffusion path to the next — a Fisher divergence between predicted and target distributions — and proves that in the dense-step limit the optimal schedule is the one that equalises this cost per step. The optimal schedule has the explicit form $\\phi^*(s)=\\Lambda^{-1}(\\Lambda s)$, where $\\Lambda(t)$ is the cumulative local cost, and it attains the lower bound on total cost. If correct, this turns schedule design from a hyperparameter sweep into a cheap computation from score evaluations, usable both when sampling a pretrained model and during training, and it explains why particular hand-tuned schedules such as Karras et al.'s $\\rho=7$ work.","feed_headline":"A Fisher-divergence cost sets optimal diffusion steps","feed_subtitle":"An adaptive algorithm equalizes per-step cost and matches hand-tuned ρ=7 without searching.","key_machinery":"The load-bearing object is the local cost $\\delta(t)=v(t)^2\\mathbb{E}_{X_t\\sim p_t}[\\|\\nabla \\dot G_t(X_t)\\|^2]$ from Theorem 2.1, where $\\dot G_t$ is the time derivative of the incremental transport weight $G_{t,t'}$ associated with the predictor map. Theorem 2.1 proves that the incremental cost of a step from $t$ to $t'$ is $L(t,t')=\\delta(t)\\Delta t^2+O(\\Delta t^3)$; the schedule generator $\\phi$ is then interpreted as a curve through the diffusion path with metric $\\delta$, making $\\Lambda(t)$ the path length and $E(\\phi)=\\int_0^1 \\delta(\\phi(s))\\dot\\phi(s)^2\\,ds$ the kinetic energy. The optimal $\\phi^*$ is the constant-speed geodesic, obtained by inverting $\\Lambda$, and the algorithm estimates $\\Lambda$ by Monte Carlo using only the learned score (with a Hutchinson trace for the predictor-optimised variant).","core_discovery":"The paper's central claim is that the reverse diffusion should be discretised so that every step does an equal amount of 'work,' and that the schedule achieving this is explicit: write $\\delta(t)$ for the local cost of Theorem 2.1, set $\\Lambda(t)=\\int_0^t \\sqrt{\\delta(u)}\\,du$, and define $\\phi^*(s)=\\Lambda^{-1}(\\Lambda s)$. Theorem 3.1 shows that as the number of steps grows, any schedule has total cost at least $\\Lambda^2/T$, with equality exactly for $\\phi^*$; equivalently $\\phi^*$ equalises the incremental costs $L(t^*_{i+1},t^*_i) \\approx \\Lambda^2/T^2$. The paper further claims that Algorithm 1, which estimates the cumulative cost $\\Lambda$ from score evaluations alone, recovers schedules that match or beat hand-tuned image-generation schedules (notably the $\\rho=7$ Karras schedule) without any hyperparameter search, and that optimising the cost correlates with FID better than the competing KL-based objective of Sabour et al. (2024).","pith_inferences":["The constant-speed-geodesic view suggests the same equalise-cost principle could design schedules for other sequential Monte Carlo or tempering samplers, where a reparameterisation of the path can be chosen to minimise the same kind of transport work.","Because $\\delta(t)$ is estimated from score evaluations, the schedule could in principle be updated from the denoising score-matching loss already computed during training, making online schedule adaptation nearly free; the paper's Algorithm 2 already reuses function evaluations but does not explore this fully.","If Assumption 3.1 is violated strongly, the same cost could instead be used to set the corrector integration time or the stochasticity level per step; equalising the cost of actual updates may then restore optimality for deterministic solvers.","The observed dataset-dependence of optimal schedules (higher-resolution datasets favour shorter early steps) implies that universal schedules cannot be optimal across datasets, and a cheap per-model schedule computation like Algorithm 1 could plausibly become a standard part of the sampling pipeline."],"forward_implications":["Schedule optimisation needs no hyperparameter search: Algorithm 1 updates time points from estimated incremental costs, and the resulting schedules match the FID of the best hand-tuned $\\rho=7$ schedule on CIFAR-10, FFHQ, AFHQv2 and ImageNet.","With the optimal schedule, coarse sampling is stable: on CIFAR-10 with only 10 steps the optimised schedule keeps FID near 2.5, while a poor schedule degrades to 50.75.","At finite step counts the optimal schedule makes per-step costs approximately equal, so the user can choose the number of steps by budget and the same principle gives the step locations.","The corrector-optimised cost, which only needs score evaluations at the two endpoints, is cheap enough for online training and is sufficient for competitive FID on image data.","The learned cost correlates with sample quality better than the KL upper bound of Sabour et al. (2024), suggesting the objective is not prone to the over-optimisation failure of that baseline."],"supporting_citations":[{"why":"Supplies the forward/backward diffusion formulation, the probability-flow ODE / Langevin decomposition, and the denoising score matching framework the cost is built on.","marker":"Song et al. (2021)"},{"why":"Provides the pretrained image models, the $\\sigma(t)$ noising parameterisation, the polynomial $\\rho$-schedule baselines, and the Heun solver used in the FID comparisons.","marker":"Karras et al. (2022)"},{"why":"Introduces the linear noising schedule used as a default baseline and the DDPM formulation the paper contrasts with cosine schedules.","marker":"Ho et al. (2020)"},{"why":"Supplies the cosine schedule and the training codebase used for the online schedule-learning experiments.","marker":"Nichol and Dhariwal (2021)"},{"why":"Algorithm 1 adapts this paper's schedule-update procedure for parallel tempering, which is the mechanism for updating the diffusion schedule.","marker":"Syed et al. (2021)"},{"why":"Defines the incremental weight $G_{t,t'}$ and annealed flow transport, used to rewrite the incremental cost in terms of a transport map.","marker":"Arbel et al. (2021)"},{"why":"Provides the KL-upper-bound schedule objective that the paper compares against and whose over-optimisation problem motivates the new cost.","marker":"Sabour et al. (2024)"},{"why":"Provides the LogLinear schedule and the DPM-solver ODE framework used as baselines in the image experiments.","marker":"Lu et al. (2022)"},{"why":"Introduces annealed Langevin dynamics, which is the corrector-optimised predictor (identity map) case of the cost.","marker":"Song and Ermon (2019)"},{"why":"Supplies the stochastic trace estimator used to compute the predictor-optimised cost in high dimensions.","marker":"Hutchinson (1989)"}],"fun_headline_variants":["Optimal diffusion steps equalize per-step cost","Adaptive schedule matches hand-tuned diffusion without tuning","No-search schedule makes each diffusion step do equal work","Score-only optimization finds optimal diffusion schedules","Equalize diffusion step work for optimal sampling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that $\\phi^*$ is optimal assumes that after every predictor–corrector step the sampler is exactly at the target distribution and that the score function is perfect, while the image experiments use a deterministic Heun solver with a learned score.","fun_headline_variants_meta":{"raw":{"variants":["Optimal diffusion steps equalize per-step cost","Adaptive schedule matches hand-tuned diffusion without tuning","No-search schedule makes each diffusion step do equal work","Score-only optimization finds optimal diffusion schedules","Equalize diffusion step work for optimal sampling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1475,"prompt_tokens":964,"completion_tokens":511,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":441}},"tokens_in":580,"tokens_out":511,"duration_ms":5228,"temperature":1.0,"reasoning_tokens":441,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:27:25.447279+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a target with a known score (for example a Gaussian mixture) and compute $\\delta(t)$ exactly; simulate the reverse diffusion with a fixed ODE solver and compare the $\\phi^*$ schedule against a schedule that equalises the solver's local truncation error. If $\\phi^*$ does not give the lowest end-to-end error for small step counts, the transfer from the perfect-corrector cost to practical samplers fails.","supporting_citations":[],"review_version":1}