Pith. sign in

REVIEW 5 major objections 4 minor 5 cited by

Gradient-Free Classifier Guidance for Diffusion Model Sampling

T0 review · 5 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A pretrained classifier used only for inference can guide diffusion sampling toward a target class, and does so without gradient backpropagation.

desk verdict Adaptive reference-class guidance is a real idea, but the Precision headline is circular until an independent classifier scores it. read the letter →

arxiv 2411.15393 v1 pith:7CHFD5PI submitted 2024-11-23 cs.CV cs.AI

Rahul Shenoy , Zhihong Pan , Kaushik Balakrishnan , Qisen Cheng , Yongmoon Jeon , Heejune Yang , Jaewon Kim This is my paper · ORCID
classification cs.CVcs.AI
keywords gradient-freeguidancediffusionmodelsclassifierclass-conditionalgenerationtext-to-imageadaptivescalereferenceclassFD_DINOv2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion models generate diverse images, but steering them toward a target class usually costs either gradient backpropagation or training an unconditional model. This paper proposes a third route: use a pretrained classifier in inference mode only, and at each denoising step ask it to classify a denoised estimate of the final image. If the desired class is not confident, the sampler picks the most confusable competing class as a reference and steers away from it, with a guidance scale that adapts to the classifier's confidence. Across class-conditional and text-to-image models, the method improves class prediction accuracy and combines with existing guidance methods to improve fidelity while preserving diversity.

What carries the argument

The load-bearing mechanism is the time-adaptive pair $(\omega, c_{\rm ref})$ that a pretrained classifier derives from a denoised estimate of the image. Equation (5) turns classifier confidence into a continuous guidance scale, and the reference-class rule makes the guidance target not a fixed unconditional model but the class the model is most likely to confuse with the desired one. Algorithm 3 plugs this into the contrast update $bD = \omega D^m(x_t,t,c_{\rm des}) - (\omega-1)D^g(x_t,t,c_{\rm ref})$, with optional multi-step denoising for $\hat{x}_0$ and stochastic reference-class sampling. This lets a single off-the-shelf classifier replace backpropagation while keeping the contrast-based guidance form of classifier-free guidance.

What would settle it

Compare the classifier's top competing class on intermediate denoised estimates with the actual failure mode of the final image as judged by a held-out classifier or human labels; if the intermediate top competitor frequently is not what the final image gets confused with, the steering target is wrong and GFCG should degrade. A direct control is to rerun GFCG with the classifier's softmax replaced by a fixed random reference class: if precision still rises, the classifier signal is not the cause.

Watch

Extended reading notes

Core claim

The paper claims that a pretrained classifier, used only in inference mode, can guide diffusion sampling toward a desired class by adaptively choosing a reference class and guidance strength at each time step. For a desired class $c_{\rm des}$ and noisy sample $x_t$, the sampler estimates the clean image $\hat{x}_0$, obtains probabilities $p(c|\hat{x}_0)$, and when $p(c_{\rm des}|\hat{x}_0) < \tau$ it sets $\omega = 1 + \alpha \exp(-\beta(p(c_{\rm des}|\hat{x}_0)-\tau))$ and selects as reference class the highest-probability competing class (or the second-highest when the desired class is already most probable). The denoising update then takes the contrast form $bD = \omega D^m(x_t,t,c_{\rm des}) - (\omega-1)D^g(x_t,t,c_{\rm ref})$, the same linear extrapolation as classifier-free guidance but with a classifier-chosen, time-adaptive reference. The paper reports that this consistently raises classification precision and, combined with Autoguidance on ImageNet 512, achieves a record FD_DINOv2 of 23.09 with 94.3% precision versus Autoguidance's 24.83 and 90.2%; on a 525-species fine-grained bird task it lifts precision from 5.3% (no guidance) to 32.3%.

Load-bearing premise

The method assumes that classifier probabilities computed on an intermediate denoised estimate of the image reliably indicate the final image's class content, so steering away from the top competing class improves the final sample; if those probabilities are noisy or biased, the guidance will optimize the classifier rather than the image.

Editorial extensions

If this is right

  • Classifier guidance no longer requires a differentiable classifier or backpropagation, so any off-the-shelf classifier can steer generation at roughly the cost of one extra forward pass per guidance prediction.
  • GFCG layers onto existing gradient-free methods: in mixed mode it adds no additional function evaluations, and in additive mode it improves both fidelity and diversity at the cost of doubled evaluations.
  • On the paper's ImageNet 512 evaluation, GFCG combined with Autoguidance reaches FD_DINOv2 23.09 and 94.3% precision, beating Autoguidance's 24.83 and 90.2% under identical random seeds.
  • The same mechanism transfers to text-to-image models, where a fine-grained 525-species bird classifier lifts precision from 5.3% without guidance and 27.3% with CFG to 32.3% with GFCG.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If intermediate classifier confidence is a reliable signal, the same adaptive reference idea should work for any attribute with a pretrained predictor, not just class labels; one could steer style, sentiment, or even regression targets by defining a 'most confusing' alternative.
  • The confidence threshold $\tau$ could double as a stopping rule: guidance is active only while the model is genuinely confused, which may reduce over-correction artifacts that plain CFG exhibits at high scales.
  • Because the headline Precision is measured with the same classifier family used for guidance, a stronger test would use a held-out classifier or human labels; the method's true fidelity gain may be smaller or larger than reported.
  • A testable scaling prediction is that better classifiers produce better guidance; varying classifier architecture and accuracy should move both FD_DINOv2 and precision monotonically.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes Gradient-free Classifier Guidance (GFCG), a sampling-time method that uses a pretrained classifier in inference mode to adaptively select a reference class cref and a guidance scale omega at each denoising step. When the classifier confidence in the desired class cdes falls below a threshold, the method steers the sample away from the most competing class via the update bD = omega*Dm(x_t,t,cdes) - (omega-1)*Dg(x_t,t,cref). The method is combined with CFG and ATG in mixed or additive modes, and is evaluated on EDM2 ImageNet 512x512 and on Stable Diffusion 1.5 with a fine-grained Bird Species classifier. The main claims are consistent improvements in class prediction accuracy (Precision), complementarity with CFG/ATG, and a record FD_DINOv2 of 23.09 on ImageNet 512 with GFCG+ATG.

Significance. If the results hold, GFCG is a practically valuable contribution: it avoids backpropagation through the diffusion network, makes adaptive use of an off-the-shelf classifier, and can be layered on top of CFG or ATG. The manuscript has real strengths: it builds on a public code base (EDM2), reports extensive ablations over alpha, beta, tau, ts, scp, guidance model choice, and classifier choice, includes a compression-based justification for preferring FD_DINOv2 over FID, and provides a random-seed variation study in the supplement. However, the headline Precision gains are measured with the same classifier that generates the guidance signal, so they are not independent evidence of class-label fidelity; the FD_DINOv2 results are independent and do suggest a real distributional improvement, but the 'record' claim is benchmarked only against a local rerun of ATG, and the 'no additional computational overhead' assertion is contradicted by the supplementary NFE accounting. These issues affect the central empirical claims and need to be addressed.

major comments (5)
  1. [§3.2 (Eq. 5), §4, Tables 1–2] The Precision metric is circular with respect to the guidance signal. Guidance in Eq. (5) uses the classifier's confidence p(c_des|x0_hat) to set omega and select cref, and Algorithm 3 uses that same classifier during sampling. Precision is then defined in Section 4 as the percentage of generated samples classified correctly by 'the real image classifier', which for ImageNet is the same ResNet-101 described in Section 4.1 and for SD 1.5 is the same Bird Species classifier used for guidance. Consequently, any method that raises this classifier's confidence on the desired class will increase Precision by construction, even if the image is not semantically closer to the class from an independent viewpoint. The paper should re-measure Precision with an independent classifier (different architecture and/or training data) that is not used for guidance, and should report the difference; without this, the Precision gains in Tables 1 and 2 (e.g., 95.4% vs. 90.6% and 32.3% vs. 27.3%) cannot be interpreted as evidence of genuine class-label alignment.
  2. [Abstract and §5 vs. Supp. §8.4 (Table 6)] The claim of 'without additional computational overhead' is contradicted by the implementation details. Supp. §8.4 states that the multi-step x0_hat estimation with T'=4 'introduces 7 additional NFEs' (63 to 70), and Table 6 confirms that the main-paper configuration uses 70 NFEs versus a 63-NFE baseline. Since the headline ImageNet results in Table 1 are all reported for scp set to its maximum and T'=4, the extra cost is part of the reported configuration, not an optional add-on. Please either remove the 'no additional computational overhead' wording, report runtime and NFE side-by-side for all compared methods, or clearly restrict the claim to configurations with a single-step x0_hat estimate.
  3. [§4.1, Table 1; Abstract] The 'record FD_DINOv2 of 23.09' is not supported by the reported comparison. The only ATG baseline in Table 1 for EDM2-XXL is the authors' rerun at 24.83, and the text acknowledges that this rerun is 'a little worse than the published value' in the ATG paper. A record claim should be benchmarked directly against the best published number for the same model, dataset, and evaluation protocol; otherwise 23.09 is merely an improvement over a locally rerun baseline. The random-seed study in Supp. §8.1 is a good step, but it is shown for the EDM2-S configuration and does not cover the XXL headline. Please state the published ATG FD_DINOv2 for this setting and demonstrate that 23.09 improves on it.
  4. [§3.2 (Eq. 6), §5] The resource-requirement narrative is also misleading regarding the need for a second diffusion model. GFCG's update in Eq. (6) requires a forward pass Dg_phi(x_t,t,cref) from a separate guidance network; for the reported ImageNet experiments this is the (XS,T/16) or (M,T/3.5) model, which must be available or trained. The statement in Section 5 that GFCG avoids 'requiring the training of an extra unconditional model, as seen with classifier-free guidance' is technically about an unconditional model, but it obscures that GFCG still requires an auxiliary conditional model for the reference class. Please clarify this second-model requirement, and state explicitly whether GFCG can be run with Dg = Dm when no smaller guidance model is available.
  5. [§3.2, Eq. (5); Supp. §10.1] The core steering assumption is that classifier probabilities computed on an intermediate denoised estimate x0_hat reliably indicate the final image's class content. The paper does not provide validation of this assumption; the failure cases in Supp. §10.1 (e.g., the TIT MOUSE and TEAL examples) show that GFCG can amplify incorrect features rather than correct them. Please report an empirical check of this assumption, for example by measuring the agreement between the classifier's argmax on x0_hat at intermediate steps and the final classified class of the generated sample, or by analyzing how the reliability of p(c|x0_hat) depends on the denoising schedule and the step budget T'.
minor comments (4)
  1. [Abstract] The sentence 'Image generation using diffusion models have demonstrated...' has a subject-verb agreement error; 'have' should agree with 'models' or the sentence should be rephrased to 'Image generation using diffusion models has demonstrated...'.
  2. [§3.3] There is a typographical error in 'Forscp, it is used to determine...'; it should read 'For scp, it is used to determine...'.
  3. [§4.1] The sentence 'The best FD DINOv2 of all is achieved when the additive guidance of GFCG and CFG, setting a SOTA performance of 33.39' is grammatically incomplete and also appears to refer to GFCG+ATG+CFG (the configuration with omega_ATG=2.45 and omega_CFG=1.60 in Table 1), not merely GFCG+CFG; please rephrase and make the naming consistent.
  4. [Equation (7)] The notation 'j∼{1,2,...,N}−{des}' is unclear; it should be written as j ∈ {1,...,N} \ {des} to denote all classes except the desired one.

Circularity Check

1 steps flagged · score 6.0 of 10

Precision gains are self-fulfilling: GFCG steers with the same classifier that later defines Precision, so the headline class-accuracy advantage over ATG is not independent evidence of fidelity.

  1. self definitional [Section 3.2, Eqs. (5)-(6); Section 4, Precision definition; also Supp. Section 7.1 Algorithm 3]
    "if p(cdes|bx0) < τ, we identify a reference class cref following two criteria: (1)cref = the class with highest probability if cdes does not have the highest probability; (2) cref = the class with second highest probability if cdes has the highest probability. We then recast the denoising step in Equation 3 as follows: bD = ωDmθ(x, t, cdes) − (ω − 1)Dgϕ(x, t, cref). (6) ... The image fidelity metric Precision is computed as the percentage of generated samples that fall into the data manifold (assessed using the real image classifier)."

    Eqs. (5)-(6) use classifier C to pick cref and ω from p(c|x0_hat), then steer bD = ωDm(cdes) − (ω − 1)Dg(cref) away from C's current top or runner-up class. Section 4 then defines Precision as the percentage of generated samples falling into the data manifold 'assessed using the real image classifier'; for ImageNet this is the same ResNet-101 used for guidance, and for SD 1.5 the paper states 'a classifier trained from the Bird Species dataset was used for guided sampling and the samples were evaluated against the same training dataset.' Hence Precision is C's own top-1 accuracy after GFCG has moved the sample in the direction that changes C's decision toward cdes.

full rationale

GFCG's headline Precision advantage is partly circular: the same classifier both selects the reference class and guidance scale (Eqs. 5-6) and defines the Precision metric (Section 4). Because GFCG is triggered whenever the classifier does not already favor cdes and steers away from that classifier's competing class, the reported Precision gain is the classifier rewarding its own steering signal. The FD_DINOv2 numbers are computed from DINOv2 embeddings and are genuinely independent; they support a real distributional-fidelity improvement (23.09 vs 24.83 for ATG on EDM2-XXL). Thus the circularity is partial: the 'improves class prediction accuracy' claim is substantially self-fulfilling, while the FD_DINOv2 and Recall claims retain independent content. No load-bearing self-citation or imported-uniqueness pattern exists. The limitations section concedes GFCG is 'most beneficial to generation of images which are in distribution of the pre-trained classifier,' reinforcing that the metric is tied to the same classifier's biases. Separately, and not a circularity issue, the abstract's 'without additional computational overhead' conflicts with Supp. Section 8.4, which reports 7 additional NFEs from the 4-step x0_hat estimation.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of classifier softmax scores on a denoised estimate, on the validity of extending CFG to a class reference, and on an evaluation metric that shares the classifier with the guidance signal. The numerical hyperparameters are tuned per dataset, so the method has a large fitted-parameter count.

free parameters (7)
  • alpha (guidance strength) = 0.5 to 2.5 depending on experiment
    Per Eq. 5, controls overall guidance strength and is tuned per dataset via ablations (Fig. 4, Fig. 8).
  • beta (confidence sensitivity) = 1.25 (ImageNet), 1.0 (text-to-image birds)
    Controls exponential falloff of omega with classifier confidence; determined by ablation (Fig. 4c).
  • tau (confidence threshold) = 1.0 for text-to-image; not explicitly reported for ImageNet
    Threshold in Eq. 5 deciding when to apply guidance; the SD experiments keep it at 1 without fine-tuning, but the ImageNet value is not stated.
  • ts (guidance start step) = 17 (ImageNet), 20 or 50 (birds)
    Starting time step for GFCG; tuned in Fig. 4d and Fig. 6c.
  • scp (classifier prediction interval) = max for ImageNet, 1 or 2 for birds
    Frequency of classifier calls; tuned in Fig. 6b and described in Sec. 3.3.
  • T-prime (multi-step denoising steps for x0_hat) = 4
    Number of extra denoising steps for clear-image estimation in Algorithm 3; set to 4 for ImageNet to boost classifier accuracy.
  • Guidance model choice Mg = (XS,T/16) for EDM2-S, (M,T/3.5) for EDM2-XXL
    The guidance network used as Dg in Eq. 6 is the reduced model from ATG; choice affects FD_DINOv2 and Precision (Table 4).
assumptions (4)
  • standard math Score approximation: D_theta(x,t,c) approximates the conditional score (Eq. 2)
    Standard score-based diffusion theory from Song et al. [32]; the paper uses it as background.
  • domain assumption CFG combination formula Eq. 3 is valid for an arbitrary reference condition
    The method extends Eq. 3 by substituting a classifier-chosen c_ref for the unconditional or null reference; this assumes the combination rule remains valid for a non-unconditional reference.
  • ad hoc to paper Classifier probabilities on x0_hat are reliable indicators of final image class fidelity
    The entire guidance signal is p(c_des|x0_hat) from Eq. 5; no theory links classifier confidence on an approximate denoised image to final sample quality.
  • ad hoc to paper Precision measured by the same classifier used for guidance is a valid image fidelity metric
    The paper defines Precision with the real-image classifier and uses the same classifier for guidance, making the metric partially circular.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gradient-Free Classifier Guidance for Diffusion Model Sampling." pith.science (2026). https://pith.science/paper/7CHFD5PI

@misc{pith2026241115393,
  author       = {Pith},
  title        = {Pith review of: Gradient-Free Classifier Guidance for Diffusion Model Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7CHFD5PI}},
  note         = {Machine review of arXiv:2411.15393}
}
abstract

Image generation using diffusion models have demonstrated outstanding learning capabilities, effectively capturing the full distribution of the training dataset. They are known to generate wide variations in sampled images, albeit with a trade-off in image fidelity. Guided sampling methods, such as classifier guidance (CG) and classifier-free guidance (CFG), focus sampling in well-learned high-probability regions to generate images of high fidelity, but each has its limitations. CG is computationally expensive due to the use of back-propagation for classifier gradient descent, while CFG, being gradient-free, is more efficient but compromises class label alignment compared to CG. In this work, we propose an efficient guidance method that fully utilizes a pre-trained classifier without using gradient descent. By using the classifier solely in inference mode, a time-adaptive reference class label and corresponding guidance scale are determined at each time step for guided sampling. Experiments on both class-conditioned and text-to-image generation diffusion models demonstrate that the proposed Gradient-free Classifier Guidance (GFCG) method consistently improves class prediction accuracy. We also show GFCG to be complementary to other guided sampling methods like CFG. When combined with the state-of-the-art Autoguidance (ATG), without additional computational overhead, it enhances image fidelity while preserving diversity. For ImageNet 512$\times$512, we achieve a record $\text{FD}_{\text{DINOv2}}$ of 23.09, while simultaneously attaining a higher classification Precision (94.3%) compared to ATG (90.2%)

Figures

Figures reproduced from arXiv: 2411.15393 by the authors.

Figure 1
Figure 1. Qualitative comparisons of the proposed gradient-free classifier guidance (GFCG) with other methods, including no-guidance [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Schematic diagram of the proposed GFCG, which use a pre-trained classifier to guide diffusion sampling away from inaccurate [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visual examples of generated ImageNet class images, combining GFCG with ATG as a function of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Ablation studies for ATG (a) and GFCG (b-d) for class-conditional image generation. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visual examples of generated fine-grained class images from SD 1.5 using GFCG and other gradient-free guidance methods. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: (a-c): Ablation studies for text-to-image generations (8,400 samples); (d): Experiments of detailed prompts (21,000 samples). [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Ablation study for impact on Precision and FD [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Ablation studies for GFCG method: text-to-image generations (8,400 samples) [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Ablation studies for GFCG+CFG method: text-to-image generations (8,400 samples) [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Ablation studies for other guidance methods reported in Table [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Visual examples of generated ImageNet class images, comparing GFCG with other guidance methods in EDM2-S sampling. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Visual examples of generated ImageNet class images comparing GFCG and GFCG [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Representative visual examples which demonstrate the benefits of GFCG over others in text-to-image generation using generic [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Representative visual examples where GFCG fails to improve class accuracy using generic prompts, where the incorrect [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Representative visual examples where GFCG improves class accuracy using detailed text prompts: 1) GFCG enhances the bird [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Representative visual examples where GFCG fails to improve class accuracy using detailed prompts. [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]
Figure 17
Figure 17. Figure 17: Visual examples which demonstrate the benefits of GFCG in text-to-image generation using pixel-level diffusion model (i.e. [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: More visual examples of generated ImageNet class images for different guidance methods in EDM2-S sampling. [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]
Figure 19
Figure 19. Figure 19: More visual examples of generated ImageNet class images comparing GFCG and GFCG [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: More visual examples from SD 1.5 model using generic text prompts. [PITH_FULL_IMAGE:figures/full_fig_p023_20.png]
Figure 21
Figure 21. Figure 21: More visual examples from SD 1.5 model using detailed text prompts. [PITH_FULL_IMAGE:figures/full_fig_p024_21.png]
Figure 22
Figure 22. Figure 22: More visual examples from DeepFloyd IF model using detailed text prompts. [PITH_FULL_IMAGE:figures/full_fig_p025_22.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approach

    cs.AI 2026-02 conditional novelty 6.0 of 10

    By adding drift g(t)^2 ∇log h(t,y) with h estimated via martingale and covariation losses, diffusion samples can be hard-conditioned on an event.

  2. UniDB: A Unified Diffusion Bridge Framework via Stochastic Optimal Control

    cs.CV 2025-02 conditional novelty 6.0 of 10

    A stochastic optimal control formulation of diffusion bridges, where Doob's h-transform is the infinite-penalty limit and a finite penalty yields a tunable detail-preserving bridge.

  3. Visual Generation Without Guidance

    cs.CV 2025-01 conditional novelty 6.0 of 10

    GFT trains a single β-conditioned network that reproduces Classifier-Free Guidance's sampling distribution, matching CFG FID scores across five model families with half the inference cost.

  4. DiffIER: Optimizing Diffusion Models with Iterative Error Reduction

    cs.CV 2025-08 reject novelty 4.0 of 10

    DiffIER claims that iteratively minimizing the distance between a diffusion model's predicted noise and a random Gaussian sample at each inference step improves generation quality.

  5. Contrastive Flow Matching

    cs.CV 2025-06 reject novelty 2.0 of 10

    Contrastive Flow Matching adds a negative flow-target term to the standard flow-matching loss, reporting large empirical gains, but the closed-form solution shows the term only applies a global rescaling and shift, no...

Reference graph

Works this paper leans on

41 extracted references · 26 canonical work pages · cited by 5 Pith papers

  1. [1]

    Self-rectifying dif- fusion sampling with perturbed-attention guidance

    Donghoon Ahn, Hyoungwon Cho, Jaewon Min, Wooseok Jang, Jungwoo Kim, SeonHwa Kim, Hyun Hee Park, Ky- ong Hwan Jin, and Seungryong Kim. Self-rectifying dif- fusion sampling with perturbed-attention guidance. arXiv preprint arXiv:2403.17377, 2024. 3

    arXiv 2024

  2. [2]

    Universal guidance for diffusion models

    Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geip- ing, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 843–852,

  3. [3]

    Diffusion posterior sam- pling for general noisy inverse problems

    Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sam- pling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022. 3

    arXiv 2022

  4. [4]

    Stable diffusion

    Katherine Crowson, Maxwell Ingham, Adam Letts, and Alex Spirin. Stable diffusion. https : / / github . com / CompVis/stable-diffusion, 2022. 1, 7

    work page 2022

  5. [5]

    ImageNet: A large-scale hierarchical im- age database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical im- age database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 5

    work page 2009

  6. [6]

    Diffusion models beat GANs on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Infor- mation Processing Systems, 34:8780–8794, 2021. 2, 3

    work page 2021

  7. [7]

    A smaller subset of 10 easily classified classes from imagenet, and a little more french

    fast.ai. A smaller subset of 10 easily classified classes from imagenet, and a little more french. https://github. com/fastai/imagenette, 2022. 6

    work page 2022

  8. [8]

    Zico Kolter, Ruslan Salakhutdinov, and Ste- fano Ermon

    Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov, and Ste- fano Ermon. Manifold preserving guided diffusion. ArXiv, abs/2311.16424, 2023. 3

    arXiv 2023

Show all 41 references
  1. [9]

    GANs trained by a two time-scale update rule converge to a local nash equi- librium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equi- librium. Advances in neural information processing systems, 30, 2017. 2, 5

    2017

  2. [10]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 1, 2, 3

    2021

  3. [11]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 2, 3, 4

    2020

  4. [12]

    Video diffu- sion models

    Jonathan Ho, Tim Salimans, Alexey A Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffu- sion models. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022. 2

    2022

  5. [13]

    Smoothed energy guidance: Guiding dif- fusion models with reduced energy curvature of attention

    Susung Hong. Smoothed energy guidance: Guiding dif- fusion models with reduced energy curvature of attention. arXiv preprint arXiv:2408.00760, 2024. 3

    2024 arXiv

  6. [14]

    Improving sample quality of diffusion models us- ing self-attention guidance

    Susung Hong, Gyuseong Lee, Wooseok Jang, and Seungry- ong Kim. Improving sample quality of diffusion models us- ing self-attention guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7462– 7471, 2023. 3

    2023

  7. [15]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 6, 1, 2

    2022

  8. [16]

    Guiding a dif- fusion model with a bad version of itself

    Tero Karras, Miika Aittala, Tuomas Kynk ¨a¨anniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a dif- fusion model with a bad version of itself. arXiv preprint arXiv:2406.02507, 2024. 1, 2, 3, 6

    2024 arXiv

  9. [17]

    Analyzing and improving the training dynamics of diffusion models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24174–24184, 2024. 1, 5, 3

    2024

  10. [18]

    Diffwave: A versatile diffusion model for audio synthesis

    Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021. 2

    2021

  11. [19]

    Applying guidance in a limited interval improves sample and distribution quality in diffusion models

    Tuomas Kynk ¨a¨anniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. arXiv preprint arXiv:2404.07724, 2024. 4, 6

    2024 arXiv

  12. [20]

    Pseudo numerical methods for diffusion models on manifolds.arXiv preprint arXiv:2202.09778, 2022

    Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds.arXiv preprint arXiv:2202.09778, 2022. 7

    2022 arXiv

  13. [21]

    Diffusion probabilistic models for 3d point cloud generation

    Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2837–2845, 2021. 2

    2021

  14. [22]

    GLIDE: Towards photorealistic image gener- ation and editing with text-guided diffusion models

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image gener- ation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2022. 2

    2022 arXiv

  15. [23]

    Arbitrary style guid- ance for enhanced diffusion-based text-to-image generation

    Zhihong Pan, Xin Zhou, and Hao Tian. Arbitrary style guid- ance for enhanced diffusion-based text-to-image generation. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, pages 4461–4471, 2023. 2

    2023

  16. [24]

    On aliased resizing and surprising subtleties in gan evaluation

    Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11410–11420, 2022. 5, 1 9

    2022

  17. [25]

    Dif- fusion motion: Generate text-guided 3d human motion by diffusion model

    Zhiyuan Ren, Zhihong Pan, Xin Zhou, and Le Kang. Dif- fusion motion: Generate text-guided 3d human motion by diffusion model. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023. 2

    2023

  18. [26]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 2

    2022

  19. [27]

    Palette: Image-to-image diffusion mod- els

    Chitwan Saharia, William Chan, Huiwen Chang, Chris A Lee, Jonathan Ho, Tim Salimans, David J Fleet, and Mo- hammad Norouzi. Palette: Image-to-image diffusion mod- els. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. 2

    2021

  20. [28]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  21. [29]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 2, 3, 4

    2021

  22. [30]

    Loss-guided diffusion models for plug-and-play controllable generation

    Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mar- dani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Ma- chine Learning, pages 32483–32498. PMLR, 2023. 3

    2023

  23. [31]

    Generative modeling by esti- mating gradients of the data distribution.Advances in Neural Information Processing Systems, 32, 2019

    Yang Song and Stefano Ermon. Generative modeling by esti- mating gradients of the data distribution.Advances in Neural Information Processing Systems, 32, 2019. 2

    2019

  24. [32]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations, 2020. 2, 3

    2020

  25. [33]

    Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models

    George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances i...

    2024

  26. [34]

    Geometric latent diffusion models for 3d molecule generation

    Minkai Xu, Alexander S Powers, Ron O Dror, Stefano Er- mon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning, pages 38592–38610. PMLR, 2023. 2

    2023

  27. [35]

    TFG: Unified training-free guidance for diffusion models

    Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Zou, and Stefano Ermon. TFG: Unified training-free guidance for diffusion models. arXiv preprint arXiv:2409.15761, 2024. 2, 3, 7

    2024 arXiv

  28. [36]

    FreeDoM: Training-free energy-guided condi- tional diffusion model

    Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. FreeDoM: Training-free energy-guided condi- tional diffusion model. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 23174– 23184, 2023. 2, 3 10 Gradient-Free Classifier Guid...

    2023

  29. [37]

    To further validate this choice, we also conducted a lossy compression test as in [24] to compare FID and FDDINOv2

    Evaluation Metrics We have explained the reasoning for choosing FD DINOv2 over FID as the overall image quality metric in the main paper. To further validate this choice, we also conducted a lossy compression test as in [24] to compare FID and FDDINOv2 . As shown in Table 3, F...

  30. [38]

    Class-Conditional Generation: Pseudo Code For Algorithm 1, specifics like timestep t, noise schedule and sampling method are illustrated using DDIM as the example

    Additional Implementation Details 7.1. Class-Conditional Generation: Pseudo Code For Algorithm 1, specifics like timestep t, noise schedule and sampling method are illustrated using DDIM as the example. In implementations, GFCG is applicable to dif- ferent models and sampling ...

  31. [39]

    Effects of Random Seed Variation The results presented in Table 1 of the main paper were gen- erated using the same random seed for image generation

    More Experimental Results 8.1. Effects of Random Seed Variation The results presented in Table 1 of the main paper were gen- erated using the same random seed for image generation. As the random seeds used in the ATG study [16] were not disclosed, we were unable to exactly rep...

  32. [40]

    Additionally, we compare GFCG to the additive method GFCGATG+CFG, which achieves state- of-the-art performance in FD DINOv2 for EDM2-S

    Class-Conditional Visual Examples Visual examples from class-conditional image generation using existing guidance methods (refer to Table 1) are com- pared with our GFCG method in EDM2-S sampling, as il- lustrated in Figure 11. Additionally, we compare GFCG to the additive met...

  33. [41]

    Additionally, we also conducted experiments using another popular model, DeepFloyd IF model5 from Stability AI

    Text-to-Image Visual Examples For text-to-image generations, the results presented in the main paper were all based on samples from SD 1.5 and more visual examples are included here. Additionally, we also conducted experiments using another popular model, DeepFloyd IF model5 f...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.