{"id":"94856194-9c4b-4f41-8564-90da31f3fa5d","arxiv_id":"2412.10786","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A diffusion model can be made faster by alternately optimizing the noise schedule and finetuning the denoiser, yielding lower FID at small step counts on three datasets.","lead":"This paper proposes a two-stage finetuning method for diffusion image models: first learn an optimized noise schedule, then finetune the denoiser with a schedule-derived weighting. On ImageNet64, FFHQ64 and CIFAR-10, it reports lower FID for few-step sampling compared with the EDM baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 14 is not the full gradient of L_disc: it omits the sigma-dependence of the weights lambda_t and of the Euler trajectory points hat x_t, so the claimed schedule optimum and Theorem 1 are unsupported.","rationale":"Reading in good faith, the paper's contribution is a theoretical guarantee (Theorem 1) plus an algorithm that instantiates it. The empirical FID improvements, if reproduced, would be useful, but they do not by themselves establish the stated mechanism. The reader's weakest assumption is exactly the correctness of Eq. 14, and my independent check agrees. The omitted terms are not small or merely technical: the trajectory dependence is the whole point of a sampling schedule, since changing sigma_t changes every later iterate through Eq. 7. A gradient that ignores this cannot be used to characterize the minimizer of L_disc. Consequently, the claimed first-order global error and the justification for stage-1 are unsupported. This is an internal-correctness issue in the derivation, not a disagreement with scientific consensus. If the authors can supply a full derivation or a proof of Theorem 1 that accounts for all terms, the verdict could change. As written, the paper should not be accepted as a theoretical contribution; at best it is an empirical recipe with an unverified schedule-optimization step. I therefore keep the reader's REJECT verdict.","tokens_in":10076,"tokens_out":6259,"duration_ms":56941,"concrete_test":"On a small pretrained EDM (e.g., CIFAR-10, T=5), fix theta and choose a random sigma. Compute L_disc by rolling out the full Euler chain Eq. 7 and evaluating Eq. 10 exactly. For each coordinate t, approximate dL_disc/dsigma_t by central finite differences with small perturbations, recomputing the entire trajectory and the lambda_t weights each time. Compare this numerical gradient to the RHS of Eq. 14 evaluated at the same sigma. If the two disagree beyond numerical tolerance (e.g., cosine similarity below 0.9 or relative L2 error above 20 percent), then Eq. 14 is not the gradient of Eq. 10, and the proof of Theorem 1 cannot rest on it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that minimizing L_disc in Eq. 10 over the sampling schedule yields a first-order-accurate Euler sampler, via Theorem 1 in Section 3.1. The only derivation offered is Eq. 14, which is stated to be the gradient of L_disc with respect to each sigma_t. Eq. 14 writes dL_disc/dsigma_t = 2 lambda_t <hat x_1 - x, d(D_theta(hat x_t, sigma_t) - x)/dsigma_t>, capturing only the explicit sigma_t argument of the t-th denoiser. But L_disc also depends on sigma through: (i) the weights lambda_t = sigma_1/sigma_{t-1} - sigma_1/sigma_t; (ii) the trajectory points hat x_t, which are generated recursively by Eq. 7 and therefore depend on all earlier schedule values; and (iii) the factor sigma_1/sigma_T multiplying x_0 in Eq. 10. The full derivative is dL_disc/dsigma_t = 2 <hat x_1 - x, d hat x_1/dsigma_t>, with d hat x_1/dsigma_t containing terms from dlambda_s/dsigma_t, from lambda_s dD/dhat x_s * dhat x_s/dsigma_t, and from the explicit sigma_t argument. Eq. 14 retains only the last of these. Thus setting Eq. 14 to zero is not equivalent to nabla_sigma L_disc = 0; Theorem 1, whose proof is also absent from the appendix, does not follow from the stated calculation. The stage-1 update in Algorithm 1 therefore minimizes an approximation of Eq. 10, not Eq. 10 itself, breaking the line from theory to the reported FID gains. Additionally, Algorithm 1 line 6 sets L_disc = ||D_theta(x + sigma_T epsilon, sigma_T) - x||^2, not the weighted sum over all steps in Eq. 10, so even the computed loss does not match the stated objective.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage finetuning method for diffusion models with few-step sampling. In stage-1, the sampling schedule (noise levels) is optimized by minimizing a discretization loss L_disc derived from the Euler discretization of the probability-flow ODE; in stage-2, the denoising model is finetuned with a weighted diffusion loss induced by the learned schedule. The authors claim a theorem that the optimal schedule yields a first-order global error, and they report FID improvements on CIFAR-10, FFHQ-64, and ImageNet-64, e.g., ImageNet-64 FID at T=10 improving from 22.42 to 12.28.","tokens_in":10500,"tokens_out":5074,"duration_ms":42000,"significance":"If the theoretical claims were correct, the paper would offer a principled and practical way to optimize the sampling schedule and finetune the model jointly, with substantial gains in few-step generation. The reported improvements are meaningful and the connection between the sampling schedule and the training weighting is an interesting perspective. However, the central theory is not established as stated: the proof of Theorem 1 is missing, the gradient in Eq. 14 drops key dependencies, and the implementation in Algorithm 1 does not match the stated discretization loss. The empirical results are therefore not backed by the paper's theoretical narrative.","major_comments":[{"comment":"Eq. (14) is not the gradient of L_disc with respect to sigma_t. The loss in Eq. (10) depends on the schedule through the trajectory points xhat_t, the weights lambda_t, and the prefactor sigma_1/sigma_T, in addition to the explicit sigma_t argument of D_theta. The full derivative is dL_disc/dsigma_t = 2 < xhat_1 - x, d xhat_1/dsigma_t >, with d xhat_1/dsigma_t containing contributions from dlambda_s/dsigma_t and from d xhat_s/dsigma_t for all s. Eq. (14) retains only the explicit dependence inside D_theta, so setting it to zero does not minimize the stated L_disc. Consequently, the premise of Theorem 1—that sigma* minimizes L_disc—is not established by the given calculation, and the theorem's proof is deferred to an appendix that is absent from the manuscript.","section":"§3.1, Theorem 1 and Eq. (14)"},{"comment":"The implemented stage-1 loss in line 6 is L_disc = ||D_theta(x + sigma_T epsilon, sigma_T) - x||^2, which is a single denoising error at the minimum noise level. This is not the weighted sum over all steps in Eq. (10), which includes lambda_t D_theta(xhat_t, sigma_t) for t=1,...,T and the boundary term sigma_1/sigma_T x_T. The optimization performed in stage-1 therefore does not minimize the theoretical L_disc as written, breaking the link between the theory in §3.1 and the algorithm.","section":"Algorithm 1, line 6"},{"comment":"The definition of the weights lambda_t is inconsistent. Eq. (8) defines lambda_t = sigma_1/sigma_{t-1} - sigma_1/sigma_t, while the text immediately below and Eq. (12) use lambda_i = sigma_T/sigma_{i+1} - sigma_T/sigma_i. Under the first definition, the sum of the weights is sigma_1/sigma_0 - sigma_1/sigma_T, which is not 1 for typical schedules (sigma_0 >> sigma_T). The Jensen bound in Eq. (12) requires nonnegative weights summing to 1, so the claimed upper bound does not follow with the stated definition.","section":"Eqs. (8) and (12)"},{"comment":"The statement that the optimal schedule 'guarantees a first-order global error' is not derived. Theorem 1 concerns the backward differences of the sequence {tau^{sigma*}_i(x_0)} and their relation to first-order increments in tau(1/sigma; x_0). Even if this theorem were true, the paper does not provide the intermediate steps connecting this sequence-level property to a bound on ||xhat - x||_2, which is the quantity relevant to sample quality. The claim of first-order global accuracy is therefore unsupported by the given arguments.","section":"§3.1, Theorem 1 and global error claim"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and grammatical errors, e.g., 'purposed' for 'proposed', 'pretrinaed' for 'pretrained', 'hyperparameter[]' with an empty bracket, and 'perfomr' for 'perform'. These should be corrected in a revision.","section":"Throughout"},{"comment":"The appendix containing the proof of Theorem 1 is mentioned in §3.1 but is not included in the manuscript. The authors should either provide the full proof in the main text or include a complete appendix.","section":"References"},{"comment":"Figures 2 and 3 are referenced in the text, but the captions and the figures themselves are not visible in the manuscript text provided. Please ensure that all figures are included and that the caption text is complete.","section":"Figures"},{"comment":"The parameterization in Eq. (13) defines sigma_t as a cumulative softmax over v_j, but the text states that the learning variable has T-2 degrees of freedom; the relationship between the dimension of v and the number of free schedule parameters should be clarified.","section":"Eq. (13)"}],"recommendation":"reject","confidential_remarks":"The manuscript is not ready for publication. The central theoretical result lacks a verifiable proof, the main gradient formula is not the true gradient, and the algorithm implements a different objective than the one analyzed. These are load-bearing issues that would require a reworking of both the theory and the experiments. The authors may wish to consult the work of Watson et al. (2021) more carefully to position their contribution, and to provide full experimental details (e.g., training steps, hyperparameters, compute) in any future submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague --\n\nYou should know two things about this paper. First, the empirical recipe is genuinely interesting: writing the Euler sampler output as a convex combination of denoiser outputs and then jointly tuning the schedule and the model gives real FID gains, most strikingly on ImageNet64 at 10 steps (22.42 to 12.28). Second, the theory as written does not support these gains. Eq. 14 is not the full gradient of L_disc: it drops the sigma-dependence of the weights lambda_t and of the trajectory points xhat_t, so the stage-1 update minimizes an approximation, not Eq. 10. Theorem 1's proof is deferred to an appendix that is absent from the preprint, and Algorithm 1's line 6 computes L_disc as a single-step denoising loss, not the weighted sum in Eq. 10. These are load-bearing gaps.\n\nWhat is actually new: the convex-combination expansion and the Jensen upper bound are clean observations, and the two-stage finetuning is a sensible engineering choice. The paper also avoids the memory cost of backprop-through-time, which is a practical plus. But most of the core recipe -- jointly optimizing schedule and model -- is already in Watson et al. [24], which the paper cites but never compares against. That is a big omission, and it makes the reported improvements hard to attribute.\n\nThe experiments themselves: no error bars, no code, no baseline comparison to Watson. With the theory broken, the empirical results are intriguing but not independently assessable. The ablation does show both stages help, and the learned schedule's shape (aggressively skipping high-noise steps) is consistent with prior findings.\n\nIn sum: this is an early-stage method paper with a promising empirical signal and an unsupported theoretical narrative. A careful reader gets some useful ideas, but the paper needs a complete proof, a corrected gradient, and a proper comparison to schedule-learning baselines before the claims can be taken at face value.\n\nMy recommendation: I would send it to peer review, because the ImageNet64 number at T=10 is the kind of result referees should see, but I'd expect heavy revision. If the author can fix the theory and add the Watson comparison, this could become a solid contribution.","headline":"Promising empirical recipe, but the theory is broken: Eq. 14 is not the true gradient of the discretization loss and the proof of Theorem 1 is missing.","tokens_in":11050,"tokens_out":2655,"would_cite":false,"duration_ms":23851,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimizing the sampling schedule yields a first-order-accurate Euler sampler, and two-stage fine-tuning cuts ImageNet64 FID from 22.42 to 12.28 at 10 steps.","keywords":["diffusion probabilistic models","sampling schedule","discretization loss","few-step sampling","probability-flow ODE","Euler solver","fine-tuning","FID"],"falsifier":"Compute the exact gradient of $L_\\mathrm{disc}$ with respect to $\\sigma_t$ by backpropagating through the entire Euler trajectory (including the dependence of $\\hat{x}_t$ and $\\lambda_t$ on $\\sigma$) and compare it with Eq. 14; if the two differ, stage-one does not minimize the stated loss and Theorem 1 does not apply.","tokens_in":9808,"feed_emoji":"🎨","tokens_out":11600,"duration_ms":85806,"temperature":0.7,"pith_summary":"The paper argues that the sampling schedule of a first-order diffusion ODE solver can be optimized directly. It shows that the final output of an Euler sampler is a convex combination of denoiser outputs at each scheduled noise level, so the discretization error can be written as a per-sample loss over the schedule. Minimizing this loss, the paper claims, makes the sequence of denoiser outputs first-order accurate in the noise-level grid. Practical consequence: a two-stage algorithm—schedule optimization followed by model fine-tuning—consistently lowers FID for few-step sampling, e.g., from 22.42 to 12.28 on ImageNet64 at T=10 steps.","feed_headline":"First-order accuracy for few-step diffusion by optimizing the schedule","feed_subtitle":"A schedule tuned to the model's own denoiser gives better few-step images, and a short fine-tune adds more.","key_machinery":"The load-bearing object is the discretization loss $L_\\mathrm{disc}$ (Eq. 10), built from the observation that the final Euler iterate $\\hat{x}_0$ is a convex combination of denoiser outputs: $\\hat{x}_0 = \\sum_{t=T}^1 \\lambda_t D_\\theta(\\hat{x}_t,\\sigma_t) + \\frac{\\sigma_1}{\\sigma_T} x_T$ with $\\lambda_t = \\frac{\\sigma_1}{\\sigma_{t-1}} - \\frac{\\sigma_1}{\\sigma_t}$. This expansion turns schedule optimization into a tractable Monte-Carlo objective. The schedule is parameterized by softmax increments, a trick borrowed from earlier work on learning fast samplers, and the gradient of $L_\\mathrm{disc}$ with respect to each $\\sigma_t$ (Eq. 14) is obtained as a by-product of backpropagating the diffusion loss. Stage two uses the upper bound $L_\\mathrm{disc} \\le \\sum_t \\lambda_t \\|D_\\theta(\\hat{x}_t,\\sigma_t) - x\\|^2 + 2\\sigma_T^2$, so the fine-tuning objective is a weighted diffusion loss with weights dictated by the learned schedule.","core_discovery":"The central claim is that minimizing the discretization loss $L_\\mathrm{disc}$ (Eq. 10) with respect to the sampling schedule $\\sigma$ yields a first-order-accurate Euler sampler: the backward differences of the denoiser outputs $\\{D_\\theta(\\hat{x}_t,\\sigma_t)\\}$ equal the first-order increments of the ground-truth denoiser trajectory $\\tau(1/\\sigma; x_0)$ (Theorem 1). Because the end point of that sequence is also the sample output, the schedule guarantees first-order global error. On top of this, the paper proposes a two-stage fine-tuning procedure: stage one optimizes the schedule for a fixed pre-trained model using a Monte-Carlo estimate of $L_\\mathrm{disc}$, and stage two fine-tunes the model on the selected noise levels using a weighted diffusion loss derived as an upper bound of $L_\\mathrm{disc}$. Reported results on ImageNet64 show FID dropping from 22.42 to 12.28 at $T=10$ and from 6.448 to 5.824 at $T=20$.","pith_inferences":["If Eq. 14 is an incomplete gradient, the reported gains might stem partly from the schedule's inductive bias rather than from truly minimizing $L_\\mathrm{disc}$; a numerical gradient check would separate these.","The same schedule-loss idea could be extended to higher-order ODE solvers, where an analogous convex-combination structure might make schedule optimization tractable; the paper analyzes only the first-order Euler case.","Because stage-two fine-tuning couples the training objective to the chosen sampling budget, jointly learning the schedule and the model from scratch is a natural next step beyond fine-tuning a pre-trained checkpoint."],"forward_implications":["For a fixed pre-trained diffusion model, the sampling schedule alone can be optimized to reduce discretization error, without retraining the model.","The learned schedule skips most high-noise steps and concentrates computation near low noise levels, formalizing earlier empirical observations as a consequence of the discretization loss.","Fine-tuning the model with weights induced by the learned schedule is what yields the full FID gain; fine-tuning with the original continuous-time weights hurts performance.","The method improves FID across step budgets from 5 to 20 on ImageNet64, CIFAR-10, and FFHQ, with the largest gains at the smallest budgets."],"supporting_citations":[{"why":"supplies the canonical PF-ODE, the pretrained EDM baseline models, and the baseline FID values the experiments improve on.","marker":"[8]"},{"why":"provides the probability-flow ODE formulation and the denoiser-score relation that the Euler-step analysis starts from.","marker":"[21]"},{"why":"introduces the softmax-increment schedule parameterization and the idea of optimizing the schedule by differentiating through the generation process.","marker":"[24]"},{"why":"defines the denoising diffusion training objective and the denoiser output that the discretization-loss expansion is built from.","marker":"[6]"},{"why":"gives the deterministic few-step sampling framework that the paper reframes as Euler integration with an optimizable schedule.","marker":"[18]"}],"fun_headline_variants":["Optimized schedule boosts few-step diffusion accuracy","Few-step diffusion gets a first-order boost from schedule tuning","Schedule optimization sharpens few-step diffusion sampling","Tune the sampler schedule to improve few-step diffusion","Better few-step diffusion via optimized sampling schedules"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument stands on the gradient formula Eq. 14 being the complete derivative of the discretization loss with respect to each noise level, which ignores the schedule-dependence of the trajectory points and weights; the proof of the first-order guarantee is also deferred to an appendix that is missing.","fun_headline_variants_meta":{"raw":{"variants":["Optimized schedule boosts few-step diffusion accuracy","Few-step diffusion gets a first-order boost from schedule tuning","Schedule optimization sharpens few-step diffusion sampling","Tune the sampler schedule to improve few-step diffusion","Better few-step diffusion via optimized sampling schedules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000241,"raw_usage":{"total_tokens":1529,"prompt_tokens":957,"completion_tokens":572,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":501}},"tokens_in":573,"tokens_out":572,"duration_ms":5802,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:37:28.368244+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the exact gradient of $L_\\mathrm{disc}$ with respect to $\\sigma_t$ by backpropagating through the entire Euler trajectory (including the dependence of $\\hat{x}_t$ and $\\lambda_t$ on $\\sigma$) and compare it with Eq. 14; if the two differ, stage-one does not minimize the stated loss and Theorem 1 does not apply.","supporting_citations":[{"cited_title":"Elucidating the design space of diffusion-based generative models","cited_arxiv_id":null,"evidence_quote":"supplies the canonical PF-ODE, the pretrained EDM baseline models, and the baseline FID values the experiments improve on."},{"cited_title":"Learning fast samplers for diffusion models by differentiating through sample quality","cited_arxiv_id":null,"evidence_quote":"introduces the softmax-increment schedule parameterization and the idea of optimizing the schedule by differentiating through the generation process."},{"cited_title":"Denoising diffu- sion probabilistic models","cited_arxiv_id":null,"evidence_quote":"defines the denoising diffusion training objective and the denoiser output that the discretization-loss expansion is built from."}],"review_version":1}