{"id":"ada2ad74-bde7-471a-bf6e-f5cd2006a293","arxiv_id":"2507.00687","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A moving-average stabilization of denoised-image classifier gradients lets non-robust classifiers guide diffusion sampling.","lead":"The authors show that ordinary image classifiers, which normally fail on noisy diffusion inputs, can still steer a diffusion model if their gradients are computed on one-step denoised images and then smoothed with a moving average. This removes the need to retrain a noise-robust classifier for every diffusion model, which could make classifier-guided generation cheaper and more flexible.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CelebA 'outperforms robust classifier' claim is confounded: the robust baseline lacks the EMA stabilization, so the improvement may come from the EMA ramp, not from enabling non-robust classifiers.","rationale":"The reader identified the zero-initialized EMA's early-gradient suppression as the weakest assumption; that is related and untested, but it concerns optimality of the design rather than the central comparison. The more decisive flaw is the missing control: the robust classifier is never given the same xhat0+EMA stabilization. Since the EMA acts as a guidance-scale ramp, the reported cFID advantage may be attributable to the ramp benefiting any classifier, not to the method enabling non-robust classifiers. This is a concrete, testable experimental omission. The paper otherwise provides useful analysis (noise sensitivity metrics, three datasets, released code) and the core feasibility claim appears credible, so conditional acceptance remains appropriate. The proposed test would settle the confound; if it fails, the paper should be revised to weaken the 'outperforms robust' statement and to frame the contribution as a general stabilization technique. The suspicious 'nabla_x_t g' in Algorithm 1 is likely a typo but should be corrected for reproducibility.","tokens_in":11756,"tokens_out":12078,"duration_ms":142129,"concrete_test":"Run the robust classifier on CelebA with the exact proposed stabilization (xhat0-prediction + 0.99-EMA, same protocol as Table 1) and sweep guidance scales, e.g., s = 10, 40, 75, 150, 250. Report target accuracy, FID, and cFID over 50176 samples. If any robust+EMA configuration attains cFID <= 13.9 at FID <= 29.37, the 'outperforms robust' claim is unsupported; the correct conclusion would be that EMA stabilization generally improves classifier guidance regardless of robustness. If no such configuration reaches that trade-off, the claim is supported and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest quantitative claim (Section 3.3, Table 1, Figures 5-7) is that non-robust + xhat0 + 0.99-EMA achieves the best cFID (13.9) and 'outperforms even the unmodified robust classifier'. However, the robust arm is always run with standard guidance: gradient on xt, no xhat0, no EMA. The proposed EMA is zero-initialized and de-biasing-free (Section 2.5), so it acts as a time-varying ramp that suppresses guidance in the first ~100 reverse steps. This means the comparison mixes two effects: classifier robustness and the guidance-ramp/stabilization. A robust classifier given the same ramp might achieve equal or better cFID at comparable FID, which would invalidate the attribution that non-robust classifiers, 'when appropriately stabilized', surpass robust ones. The paper does not include a robust + xhat0 + EMA arm, nor a non-robust + xhat0 + equivalent-scale-ramp arm, so the reported advantage could be due to the EMA stabilization benefiting any classifier rather than to the method's ability to handle non-robust classifiers. This missing control is the most load-bearing weakness because it directly undermines the headline comparison while the core feasibility claim (non-robust guidance works) is less threatened.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for using non-robust classifiers (trained on clean data, without diffusion noise) for classifier guidance in DDPM sampling. The method applies the classifier to the one-step denoised estimate xhat0(xt) and stabilizes the resulting guidance gradients with a zero-initialized, de-biasing-free exponential moving average (or, alternatively, an ADAM-style update). The authors analyze classifier accuracy, logit sensitivity, and gradient sensitivity along the diffusion forward process, then evaluate conditional sampling on CelebA, SportBalls, and CelebA-HQ. They report that the non-robust classifier with xhat0 and a 0.99 EMA achieves the best class-conditional FID on CelebA (13.9) and state that this setup outperforms the unmodified robust classifier.","tokens_in":12035,"tokens_out":5365,"duration_ms":61112,"significance":"If the central claim holds, the paper removes a practical obstacle: off-the-shelf non-robust classifiers could steer pre-trained diffusion models without robust retraining. The paper ships code and trained weights, reports consistent qualitative trends across three datasets, and includes a useful negative result for ADAM-style gradient rescaling. The forward-process sensitivity metrics are a reasonable diagnostic tool. However, the headline comparison is confounded by an asymmetric baseline, the target-class accuracy metric is partly self-confirming, and the quantitative results lack error bars or multiple seeds, so the strongest claims are not yet established.","major_comments":[{"comment":"The claim that the non-robust setup 'outperforms even the unmodified robust classifier' is confounded. The robust baseline is run without xhat0 and without EMA stabilization (standard guidance on xt), while the proposed EMA is zero-initialized and de-biasing-free (Section 2.5), so it acts as a time-varying ramp that suppresses guidance in early reverse steps. The comparison therefore mixes classifier robustness with the EMA ramp; a robust classifier given the same ramp might achieve equal or better cFID at comparable FID, which would invalidate the attribution. Please add a robust + xhat0 + EMA arm (and ideally a non-robust + xhat0 arm with an equivalent ramp) before making the outperformance claim.","section":"§3.3, Table 1, Figures 5–7"},{"comment":"Target-class accuracy is measured with the same classifier that performed the guidance. Since guidance pushes samples toward that classifier's decision boundary, high accuracy is partly self-confirming and does not establish that the samples are recognizable to an independent observer. The cFID metric, which uses Inception features, is less circular, but the paper should also report accuracy with an independently trained classifier (e.g., a different architecture or training seed) or a human evaluation to support the claim of successful conditional generation.","section":"§3.2, Figures 5–7, Table 1"},{"comment":"The pseudocode as written is not the guidance rule from Eq. (1). Line 5 assigns xt−1 as a sample from N(µθ(xt), Σt(xt)), and line 6 adds s Σt(xt) ∇xt g, where g already equals ∇xt log pcl(y | xhat0(xt)). Taken literally, line 6 computes a second-order gradient (∇xt g) and the shift is applied after sampling rather than to the mean. The text and Eq. (1) indicate the intended update is x′t−1 = xt−1 + s Σt(xt) g with sampling performed after shifting the mean; the pseudocode must be corrected for reproducibility.","section":"Algorithm 1, lines 5–6"},{"comment":"The metric S_g measures smoothness of the guidance gradients along the diffusion trajectory, not their informativeness. A classifier with constant output would have S_g ≈ 0 but no class signal, so the inference 'low S_g ⇒ informative gradients' does not logically follow. The design choice to suppress early-step gradients (Section 2.5) is therefore supported only indirectly by downstream FID/cFID results. Please provide an analysis of the class-direction content of early gradients (e.g., cosine similarity to the class-conditional mean shift) or temper the claim.","section":"§2.3, Eq. (4)"},{"comment":"All FID, cFID, and target-class accuracy values are point estimates from single runs, and the guidance scales reported as 'best' are selected from the same curves used for the headline numbers. The claim that the proposed setup achieves the best cFID (13.9) may reflect selection noise. Please report multiple seeds (at least for the compared setups) with means and standard errors, or explicitly state the single-run nature of the evaluation in the main text.","section":"§3.3, Tables 1–3, Figures 5–8"}],"minor_comments":[{"comment":"Replace the assignment 'xt−1 = N(xt−1; µθ(xt), Σt(xt))' with an explicit sampling statement such as 'sample xt−1 ∼ N(µθ(xt), Σt(xt))'.","section":"Algorithm 1, line 5"},{"comment":"The text refers to 'figure 2' and 'figure 3' for SportBalls and CelebA-HQ sample images, but the actual sample images appear in Tables 2 and 3; please fix the cross-references.","section":"§3.3 and Tables 2–3"},{"comment":"The notation g^2 in νema_t(g^2, β = 0.999) is not defined; please state that squaring is elementwise.","section":"Eq. (8)"},{"comment":"The hardware is described as 'NVIDIA A80' multiple times; the commonly used GPU is the A100, so please verify the hardware name.","section":"§2 and §3.2"},{"comment":"The data set name is spelled inconsistently as 'Celeba-HQ' and 'CelebA-HQ'; please unify.","section":"Throughout"},{"comment":"The Limitations paragraph already acknowledges hyperparameter sensitivity and the lack of FID optimization; this is good self-assessment, but the main text should also connect this to the absence of error bars in the headline comparisons.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The central feasibility claim (non-robust classifiers can be used for guidance when combined with xhat0 and EMA stabilization) appears plausible and is supported by consistent qualitative trends. The main obstacle is the asymmetric comparison against the robust baseline and the self-confirming accuracy metric; these are fixable with additional controlled experiments. If the authors add the missing robust + EMA arm and an independent evaluation, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful core of this paper is a simple stabilization trick: when guiding a DDPM with a classifier that was not trained on diffusion noise, compute gradients on the one-step denoised prediction x̂0 and then run an EMA (zero-initialized, no debiasing) over the guidance vectors. That combination makes non-robust guidance work on CelebA, SportBalls, and an off-the-shelf CelebA-HQ model. The paper also introduces two sensitivity metrics (S_l on logits, S_g on gradients) that cleanly show why non-robust classifiers fail—the gradients are unstable across time steps—and the forward-process analysis is genuinely informative. Code and weights are released. This is a legitimate extension of the x̂0-prediction prior art, and the EMA contribution is clearly presented.\n\nThe main soft spot is exactly what your stress-test note says. The claimed 'outperforms even the unmodified robust classifier' result (Section 3.3, Table 1) compares the full proposed pipeline (non-robust + x̂0 + 0.99-EMA) against a robust classifier with none of that stabilization. So the 13.9 cFID advantage could come from the EMA ramp helping any classifier, not specifically from enabling non-robust ones. The paper never runs a robust + x̂0 + EMA arm, so the headline attribution is not supported. The core feasibility claim—non-robust guidance works when stabilized—is on much firmer ground, and that is what the title promises. The missing controls should be a mandatory revision request, not a reason to reject.\n\nOther issues are minor. Target-class accuracy is measured with the guiding classifier itself, which is partly self-confirming; cFID against real target images partially mitigates this. FID/cFID are single point estimates, no error bars or seeds. Guidance scales are chosen from the reported curves, though the full sweep is shown, which is more transparent than typical. And Algorithm 1, line 6, literally writes ∇xt g where it should be just g—that looks like a typo to fix in revision. The authors' Limitations section is candid about hyperparameter sensitivity and single-architecture scope.\n\nOverall, a solid empirical paper useful to anyone doing classifier guidance for counterfactual explanations or conditional generation. It deserves peer review, with the caveat that the comparison confound and evaluation robustness need to be addressed. I would send it out.","headline":"A simple, useful trick for using non-robust classifiers in diffusion guidance, but the headline comparison against robust guidance is confounded by a missing control.","tokens_in":12532,"tokens_out":4812,"would_cite":true,"duration_ms":51929,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that classifier guidance can work with non-robust classifiers by scoring one-step denoised predictions and stabilizing guidance gradients with a deliberately biased exponential moving average.","keywords":["diffusion models","classifier guidance","non-robust classifiers","exponential moving average","gradient stabilization","class-conditional sampling","DDPM","FID"],"falsifier":"Compute, for a non-robust classifier and a diffusion model, the alignment between the one-step-denoised guidance gradient $\\nabla_{x_t}\\log p_{cl}(y\\mid\\hat{x}_0(x_t))$ at large $t$ and the clean-image classifier gradient $\\nabla_{x_0}\\log p_{cl}(y\\mid x_0)$. If those early gradients carry strong class-aligned signal, then a de-biased EMA that keeps them should improve class-conditional FID, contradicting the paper's assumption that they should be suppressed.","tokens_in":11555,"feed_emoji":"🎯","tokens_out":6181,"duration_ms":60052,"temperature":0.7,"pith_summary":"The paper tries to remove a standard restriction on classifier guidance for diffusion models: the classifier must be trained on diffusion-noise-corrupted data (robust) for its gradients to steer sampling. It claims that any ordinary, non-robust classifier can be used instead if two changes are made: score the one-step denoised estimate $\\hat{x}_0(x_t)$ rather than the noisy sample, and pass the guidance gradients through an exponential moving average with no de-biasing. On CelebA this combination reaches a class-conditional FID of 13.9 while holding an unconditional FID of 29.37, beating the robust classifier. The point is not a new diffusion model, but a drop-in sampling-side fix that widens which classifiers can be used for conditional generation.","feed_headline":"One gradient fix lets ordinary classifiers steer diffusion","feed_subtitle":"One-step denoising plus moving-average gradients beats robust-classifier guidance on CelebA.","key_machinery":"The central object is the stabilized guidance gradient $\\hat{g}_t = \\mathrm{EMA}_t(\\nabla_{x_t}\\log p_{cl}(y \\mid \\hat{x}_0(x_t)))$ computed from the one-step denoised estimate $\\hat{x}_0(x_t) = x_t/\\sqrt{\\bar\\alpha_t} - (\\sqrt{1-\\bar\\alpha_t}/\\sqrt{\\bar\\alpha_t})\\epsilon_\\theta(x_t,t)$, with the EMA update $\\nu_t = \\beta \\nu_{t-1} + (1-\\beta) g$ and no de-biasing. This replaces the raw classifier gradient in the DDPM reverse mean shift $x'_{t-1} = x_{t-1} + s \\Sigma_t \\hat{g}_t$. The denoised estimate turns the classifier input into a clean-image prediction so a noise-unaware classifier can score it; the biased EMA suppresses guidance in the early high-noise steps and enforces smooth feature changes over time.","core_discovery":"The central claim is that the failure of non-robust classifier guidance is caused by unstable guidance gradients, not by the classifier being useless on clean images, and that this instability can be measured and smoothed away. The authors define sensitivity scores $S_l$ and $S_g$ that compare logit and gradient changes between adjacent steps on the same diffusion trajectory; non-robust classifiers show far higher sensitivity than robust ones. A one-step denoised prediction $\\hat{x}_0(x_t)$ restores accuracy and lowers gradient sensitivity, and an exponential moving average over guidance gradients with $\\beta=0.99$, deliberately zero-initialized and without de-biasing, closes the remaining gap. In the reverse process this stabilized guidance achieves the best class-conditional FID (13.9) on CelebA while keeping unconditional FID at 29.37, and the paper reports it outperforms the unmodified robust classifier. The same recipe transfers to SportBalls and to an off-the-shelf CelebA-HQ diffusion model.","pith_inferences":["If the early-step suppression is the active ingredient, a time-varying EMA schedule (small $\\beta$ early, larger $\\beta$ later) could adapt to per-step noise and may improve conditioning further; the paper does not test this.","The sensitivity metrics $S_l$ and $S_g$ double as a cheap pre-sampling diagnostic: a classifier whose gradient sensitivity stays high after $\\hat{x}_0$-prediction is likely to need stronger stabilization.","The same stabilization recipe should transfer to other iterative samplers such as flow-matching or consistency models, since it only modifies the guidance vector, not the denoiser; that transfer is untested in the paper."],"forward_implications":["A classifier trained on clean data can be plugged into classifier guidance without robust retraining; only the sampling procedure changes.","On CelebA, the proposed EMA-stabilized non-robust guidance reaches cFID 13.9 with FID 29.37, beating the robust-classifier baseline on class-conditional quality.","The method works on a third-party pre-trained DDPM (CelebA-HQ) and on the synthetic multi-class SportBalls set, so it transfers off the shelf.","ADAM-style second-moment rescaling amplifies gradient differences and is the wrong stabilizer; EMA with a large window ($\\beta=0.99$) is the recommended choice.","Omitting EMA de-biasing is deliberate: it biases guidance toward zero early, preventing unreliable high-noise gradients from breaking the reverse process."],"supporting_citations":[{"why":"Supplies the classifier-guidance update rule and the requirement that guidance classifiers be robust to diffusion noise, the baseline being extended.","marker":"[4]"},{"why":"Introduces the one-step denoised prediction $\\hat{x}_0(x_t)$ used to score noise-unaware classifiers.","marker":"[2]"},{"why":"Provides the $\\hat{x}_0$-prediction formalism and prior gradient analysis that this paper extends with stabilization.","marker":"[20]"},{"why":"Defines the DDPM forward process, noise schedule, and denoising objective used throughout.","marker":"[9]"},{"why":"Motivates the exponential moving average (momentum) stabilization of guidance gradients.","marker":"[17]"},{"why":"Motivates the ADAM-style second-moment stabilization variant and its failure mode.","marker":"[13]"},{"why":"Provides the SportBalls dataset used for controlled multi-class conditional sampling.","marker":"[19]"},{"why":"Provides the CelebA-HQ dataset and the off-the-shelf high-resolution setting.","marker":"[12]"}],"fun_headline_variants":["Stable gradients let non-robust classifiers guide diffusion","One-step denoising and EMA make classifier guidance robust","Gradient instability fixed, ordinary classifiers guide diffusion","Non-robust classifiers steer diffusion via stabilized gradients","EMA gradient smoothing unlocks classifier guidance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that early high-noise reverse-sampling steps contain no useful class information, so deliberately biasing guidance toward zero there (by omitting EMA de-biasing) cannot hurt conditioning; if informative class signal appears only early, this design would weaken conditioning.","fun_headline_variants_meta":{"raw":{"variants":["Stable gradients let non-robust classifiers guide diffusion","One-step denoising and EMA make classifier guidance robust","Gradient instability fixed, ordinary classifiers guide diffusion","Non-robust classifiers steer diffusion via stabilized gradients","EMA gradient smoothing unlocks classifier guidance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000733,"raw_usage":{"total_tokens":3271,"prompt_tokens":931,"completion_tokens":2340,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":2267}},"tokens_in":547,"tokens_out":2340,"duration_ms":15765,"temperature":1.0,"reasoning_tokens":2267,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:09:51.404804+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, for a non-robust classifier and a diffusion model, the alignment between the one-step-denoised guidance gradient $\\nabla_{x_t}\\log p_{cl}(y\\mid\\hat{x}_0(x_t))$ at large $t$ and the clean-image classifier gradient $\\nabla_{x_0}\\log p_{cl}(y\\mid x_0)$. If those early gradients carry strong class-aligned signal, then a de-biased EMA that keeps them should improve class-conditional FID, contradicting the paper's assumption that they should be suppressed.","supporting_citations":[{"cited_title":"NeurIPS (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the classifier-guidance update rule and the requirement that guidance classifiers be robust to diffusion noise, the baseline being extended."},{"cited_title":"CVPR (2022)","cited_arxiv_id":null,"evidence_quote":"Introduces the one-step denoised prediction $\\hat{x}_0(x_t)$ used to score noise-unaware classifiers."},{"cited_title":"NeurIPS (2020)","cited_arxiv_id":null,"evidence_quote":"Defines the DDPM forward process, noise schedule, and denoising objective used throughout."},{"cited_title":"Nature (1986)","cited_arxiv_id":null,"evidence_quote":"Motivates the exponential moving average (momentum) stabilization of guidance gradients."},{"cited_title":"ICLR (2015)","cited_arxiv_id":null,"evidence_quote":"Motivates the ADAM-style second-moment stabilization variant and its failure mode."},{"cited_title":"Generative Example-Based Explanations: Bridging the Gap between Generative Modeling and Explainability","cited_arxiv_id":"2410.20890","evidence_quote":"Provides the SportBalls dataset used for controlled multi-class conditional sampling."},{"cited_title":"ICLR (2018)","cited_arxiv_id":null,"evidence_quote":"Provides the CelebA-HQ dataset and the off-the-shelf high-resolution setting."}],"review_version":1}