{"id":"17aa4bee-2579-41c4-ab8c-f0f569620c1f","arxiv_id":"2611.17194","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"eDMD trains a fake score model only on the student's final outputs to eliminate a reference residual that distorts distribution matching in multi-step diffusion distillation.","lead":"This paper introduces eDMD, a method for distilling diffusion models into few-step generators by training a fake score model exclusively on the student's final outputs. The key insight is that existing methods train the fake model on intermediate states, which can distort the optimization and prevent the student from matching the teacher distribution even when it already does.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The theoretical guarantee (Prop. 1) assumes exact denoisers; the paper acknowledges this but the central claim that eDMD 'eliminates the reference residual' in practice depends on how well the endpoint fake approximates D_q under neural approximation error and finite data.","rationale":"The reader's weakest_assumption correctly identifies that the theoretical analysis assumes exact denoisers, and that the paper acknowledges this limitation. The reader's verdict is ACCEPT with medium correctness risk, which is appropriate. My stress-test does not identify a new load-bearing concern that would change the verdict. The concern about fake model approximation error is a natural extension of the acknowledged assumption, but the paper provides strong empirical evidence (ablations, synthetic experiments, multiple backbones) that the endpoint reference works well in practice. The concrete test I propose would further strengthen the evidence, but its absence does not invalidate the central claim. The paper is honest about limitations and provides sufficient empirical support. Therefore, the verdict should remain UNCHANGED. I agree with the reader's assessment.","tokens_in":39086,"tokens_out":807,"duration_ms":13015,"concrete_test":"Train eDMD on a small-scale problem (e.g., 2D synthetic or CIFAR-10) where the exact denoiser D_q can be computed analytically or via a sufficiently large model. Compare the correction C = A - F against the theoretical correction C_theory = A - D_q. If the difference ||C - C_theory|| is large relative to the norm of C_theory, then fake model approximation error is significant and the 'elimination' of the reference residual is incomplete. Alternatively, on the same small-scale problem, train a student with the endpoint reference but replace F with a Monte Carlo estimate of D_q (e.g., by averaging many samples). If the performance gap between this 'oracle' endpoint reference and the learned F is small, then approximation error is not a major concern; if large, it is.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central theoretical result (Proposition 1, Sec. 3.2) proves that the reference residual R = D_q - D_ν vanishes if and only if ν = q, assuming exact conditional denoisers under a shared forward noising process. The paper acknowledges this assumption (b85, b290) and provides empirical evidence that the endpoint reference works well. However, the load-bearing step for the central claim is that training the fake model exclusively on final outputs (Eq. 7) yields F ≈ D_q, and that this approximation is sufficiently accurate to eliminate the reference residual in practice. The paper does not analyze how approximation error in the fake model (due to finite capacity, finite data, or optimization) propagates into the correction signal C = A - F. In particular, if F is a biased estimate of D_q, the residual R = D_q - F may not vanish even when ν = q, and the correction may not vanish when q = p. The paper's ablations (Tab. 3, w/o E) show that removing the endpoint reference causes the largest degradation, but this does not isolate the effect of fake model approximation error from the effect of the reference distribution mismatch. The concern is that the practical benefit of eDMD may be partly due to other components (PT, calibrated query density) or to the endpoint reference providing a better-conditioned learning signal, rather than to the elimination of the reference residual per se. The paper's own analysis of the Gaussian case (Lemma 4, App. A.5) shows that even with an exact endpoint fake, the student's equilibrium variance can deviate from the teacher's due to the teacher target A not being a pure denoiser D_p. This suggests that the theoretical guarantee is fragile under practical approximations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes eDMD, a method for few-step diffusion distillation that trains the fake score model exclusively on the student's final outputs (endpoint distribution) to eliminate a 'reference residual' that distorts the distribution matching signal in multi-step DMD. The paper provides theoretical analysis (Proposition 1) showing that under exact denoisers, the reference residual vanishes iff the fake model matches the endpoint distribution. It also introduces a precise teacher (PT) using short multi-step trajectories for low-noise queries and a calibrated query density. Experiments on four backbones (SD3-Medium, FLUX.2-klein 4B/9B, Qwen-Image-2.1) show consistent improvements over baselines (DMD2, TDM, D-DMD, CDM) on multiple metrics, with ablations confirming the importance of each component.","tokens_in":39605,"tokens_out":1860,"duration_ms":18440,"significance":"If the results hold, eDMD provides a principled solution to a previously unrecognized objective mismatch in multi-step DMD. The theoretical analysis is clean and the empirical gains are substantial and consistent across multiple large-scale models. The method is practical (no real images needed, moderate training cost). This could become a standard component in few-step distillation pipelines.","major_comments":[{"comment":"The theoretical guarantee assumes exact conditional denoisers under a shared forward noising process. The paper acknowledges this (b85, b290) and provides empirical evidence that the endpoint reference works well. However, the central claim that eDMD 'eliminates the reference residual' in practice depends on how well the endpoint fake approximates D_q under neural approximation error and finite data. The paper does not analyze how approximation error in the fake model propagates into the correction signal C = A - F. In particular, if F is a biased estimate of D_q, the residual R = D_q - F may not vanish even when ν = q, and the correction may not vanish when q = p. The ablations (Tab. 3, w/o E) show that removing the endpoint reference causes the largest degradation, but this does not isolate the effect of fake model approximation error from the effect of the reference distribution mismatch. The concern is that the practical benefit of eDMD may be partly due to other components (PT, calibrated query density) or to the endpoint reference providing a better-conditioned learning signal, rather than to the elimination of the reference residual per se. The paper's own analysis of the Gaussian case (Lemma 4, App. A.5) shows that even with an exact endpoint fake, the student's equilibrium variance can differ from the teacher's due to the interplay of PT and query density, suggesting that the endpoint reference alone does not guarantee exact matching in practice. The authors should discuss this limitation more explicitly and ideally provide an analysis of how approximation error in the fake model affects the correction signal.","section":"§3.2, Proposition 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The ablation study removes one component at a time, but the 'w/o E' ablation replaces the endpoint fake with a stage-based fake (TDM's weighting). This changes both the reference distribution and the fake training objective. It would be more informative to also ablate by training the fake on final outputs but with a different loss (e.g., the same loss as w/o E but on final outputs only) to isolate the effect of the reference distribution from the effect of the training objective. The paper's claim that 'removing E causes the largest overall degradation' is supported, but the ablation conflates multiple changes.","section":"§5.3, Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The PT component uses short multi-step integration for low-noise queries. The paper reports that extending PT to all queries causes training collapse (App. C). This suggests that the benefit of PT is specific to low-noise levels and that the multi-step integration may introduce bias at higher noise levels. The paper should discuss why PT is harmful at higher noise levels and how the threshold σ_{K-1} was chosen. Is there a principled way to set this threshold, or is it a hyperparameter that needs tuning per teacher?","section":"§4.2, Precise Teacher","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The calibrated query density is computed once before training using teacher samples. The paper reports that on FLUX.2-klein-4B, the calibrated density is nearly uniform and performance is similar to uniform sampling (Tab. 3). This suggests that the calibration may not be necessary for all teachers. The paper should discuss when calibration is expected to be beneficial and whether the calibration procedure could be simplified or omitted for some architectures.","section":"§4.3, Calibrated query density","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The phrase 'the fake model should represent the distribution of the samples that distillation ultimately aims to optimize' is a key insight but could be stated more precisely. Consider rephrasing to clarify that the fake model should be trained on the student's final outputs, not on intermediate predictions.","section":"§1, Introduction","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The derivation of Eq. (5) for the mixture reference residual is given in App. A.3. It would be helpful to include a brief intuitive explanation in the main text of why the residual pushes final outputs toward regions where the last step's density exceeds that of earlier steps.","section":"§3.2, Eq. (5)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The fake training loss uses a clipped SNR weight. The paper should specify the clipping range (min{max{SNR(τ),1},10}) in the main text or at least reference the appendix where it is given (App. C).","section":"§4.1, Eq. (7)","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The evaluation uses DrawBench-1000 (200 prompts × 5 seeds). The paper should report whether the 200 prompts are a standard subset or were selected specifically for this study. If a subset, how was it chosen?","section":"§5.1, Evaluation","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The table includes 'Distilled (4 steps)*' for FLUX.2-klein models as an unranked reference. It would be helpful to note in the caption or text what method was used to produce these official checkpoints (e.g., adversarial distillation, progressive distillation, etc.).","section":"§5.2, Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The rerun noise analysis reports standard deviations of 0.074 on SD3-Medium and 0.11 on FLUX.2-klein-4B. The paper should clarify whether these are standard deviations of the HPSv3++ score across checkpoints or across seeds. If across seeds, the differences between ablations (e.g., w/o E vs. full) are much larger than the noise, supporting the conclusions.","section":"§5.3, Ablations","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The Gaussian analysis is insightful but assumes a scale-only final step. The paper should note that this is a simplified setting and that the conclusions may not directly generalize to high-dimensional, non-Gaussian distributions.","section":"App. A.5, Lemma 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The paper uses LoRA for the student and full parameters for the fake on FLUX.2-klein-9B. The choice of LoRA rank (32) and alpha (64) should be justified or referenced to prior work.","section":"App. C, Training settings","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The experiment adding the endpoint reference to TDM shows improvements, but the gains are smaller than those of full eDMD. The paper should discuss why the endpoint reference alone does not fully close the gap to eDMD, i.e., what additional benefits PT and calibrated query density provide.","section":"App. H, Endpoint reference on TDM","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"This paper identifies a genuine problem in multi-step DMD—the fake model trained on intermediate states creates a 'reference residual' that can push the student away from the teacher even when final outputs already match—and proposes a simple fix: train the fake model only on final outputs. The theoretical analysis (Proposition 1) is clean and the synthetic 2D experiments convincingly show the effect. The main experiments on four large backbones (SD3, FLUX.2, Qwen-Image) are extensive and show consistent improvements over strong baselines. The ablations confirm that the endpoint reference is the most important component. The paper is well-written and the authors are honest about the limitations (exact denoisers assumption, Gaussian analysis showing equilibrium variance can deviate).\n\nThe stress-test concern about approximation error in the fake model is valid but the paper addresses it empirically: the ablations show that removing the endpoint reference causes the largest degradation, and the endpoint reference improves even when added to TDM. The Gaussian analysis (Lemma 4) actually shows that even with an exact endpoint fake, the student's equilibrium variance can deviate from the teacher's due to the teacher target not being a pure denoiser—this is a separate issue from the reference residual, and the paper acknowledges it. The practical benefit is clear from the experiments.\n\nThe calibrated query density and precise teacher (PT) are useful engineering contributions but less central. The paper does not overclaim; it states that eDMD 'consistently outperforms' baselines, which the data support.\n\nWho this is for: anyone working on diffusion distillation, especially DMD-based methods. The paper deserves a serious referee; the claims are well-supported and the method is likely to be adopted. I would cite it and bring it to a reading group.\n\nRecommendation: accept with minor revisions. The main thing to check is whether the Gaussian analysis (Lemma 4) is correctly interpreted—it shows a limitation of the endpoint reference under guided/PT targets, which the paper should discuss more prominently. Also, the paper should clarify that the theoretical guarantee (Prop. 1) is for exact denoisers, and the empirical evidence shows it holds in practice.","headline":"A clean theoretical fix for a known DMD issue, backed by strong experiments; the practical benefit is real even if the theoretical guarantee is conditional on exact denoisers.","tokens_in":40140,"tokens_out":540,"duration_ms":9484},"pith_extraction":null,"created_at":"2026-11-13T09:26:57.594300+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":null,"supporting_citations":[],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}