Fine-tuning a pretrained diffusion model with a GAN objective and most weights frozen yields a one-step generator that matches or beats prior distillation methods on several datasets.
Plug-and-Play Diffusion Distillation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Diffusion models have shown tremendous results in image generation. However, due to the iterative nature of the diffusion process and its reliance on classifier-free guidance, inference times are slow. In this paper, we propose a new distillation approach for guided diffusion models in which an external lightweight guide model is trained while the original text-to-image model remains frozen. We show that our method reduces the inference computation of classifier-free guided latent-space diffusion models by almost half, and only requires 1\% trainable parameters of the base model. Furthermore, once trained, our guide model can be applied to various fine-tuned, domain-specific versions of the base diffusion model without the need for additional training: this "plug-and-play" functionality drastically improves inference computation while maintaining the visual fidelity of generated images. Empirically, we show that our approach is able to produce visually appealing results and achieve a comparable FID score to the teacher with as few as 8 to 16 steps.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation
Fine-tuning a pretrained diffusion model with a GAN objective and most weights frozen yields a one-step generator that matches or beats prior distillation methods on several datasets.