Pith. sign in

REVIEW 7 cited by

One-step Diffusion Models with $f$-Divergence Distribution Matching

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.15681 v2 pith:JTDV5LHI submitted 2025-02-21 cs.LG cs.AIcs.CV

Yilun Xu , Weili Nie , Arash Vahdat This is my paper
classification cs.LGcs.AIcs.CV
keywords divergencedistributiongenerationscorediffusiondistilldistillationdivergences
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Sampling from diffusion models involves a slow iterative process that hinders their practical deployment, especially for interactive applications. To accelerate generation speed, recent approaches distill a multi-step diffusion model into a single-step student generator via variational score distillation, which matches the distribution of samples generated by the student to the teacher's distribution. However, these approaches use the reverse Kullback-Leibler (KL) divergence for distribution matching which is known to be mode seeking. In this paper, we generalize the distribution matching approach using a novel $f$-divergence minimization framework, termed $f$-distill, that covers different divergences with different trade-offs in terms of mode coverage and training variance. We derive the gradient of the $f$-divergence between the teacher and student distributions and show that it is expressed as the product of their score differences and a weighting function determined by their density ratio. This weighting function naturally emphasizes samples with higher density in the teacher distribution, when using a less mode-seeking divergence. We observe that the popular variational score distillation approach using the reverse-KL divergence is a special case within our framework. Empirically, we demonstrate that alternative $f$-divergences, such as forward-KL and Jensen-Shannon divergences, outperform the current best variational score distillation methods across image generation tasks. In particular, when using Jensen-Shannon divergence, $f$-distill achieves current state-of-the-art one-step generation performance on ImageNet64 and zero-shot text-to-image generation on MS-COCO. Project page: https://research.nvidia.com/labs/genair/f-distill

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Parallel Decoding Distillation for Fast Image and Video Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A trajectory-based distillation method trains a student to predict multiple mean velocities per network evaluation, enabling 4-8 step generation with competitive quality and improved diversity.

  2. Cross-Resolution Distribution Matching for Diffusion Distillation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Cross-resolution distribution matching with logSNR timestep alignment and predicted-noise re-injection enables high-fidelity few-step multi-resolution cascaded diffusion distillation.

  3. Transition Matching Distillation for Fast Video Generation

    cs.CV 2026-01 conditional novelty 6.0 of 10

    Splitting a video diffusion model into a fixed feature extractor and a small recurrent flow head lets TMD generate videos in one to two effective steps with better VBench scores than prior distilled models.

  4. Distribution Matching Distillation Meets Reinforcement Learning

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Combining DMD distillation with RL during training produces few-step text-to-image models that outperform their multi-step teacher on several benchmarks.

  5. TeEFusion: Blending Text Embeddings to Distill Classifier-Free Guidance

    cs.CV 2025-07 conditional novelty 6.0 of 10

    TeEFusion distills classifier-free guidance into text embeddings via linear fusion, enabling a student model to generate images in one forward pass instead of two.

  6. DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Matching init-to-DMD mode coverage and jointly training DMD with consistency distillation improves AR video distillation quality, coverage, and diversity enough that a 1.3B teacher can beat 14B baselines.

  7. FVGen: Accelerating Novel-View Synthesis with Adversarial Video Diffusion Distillation

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    FVGen uses GAN-based adversarial distillation and softened reverse KL divergence to compress a video diffusion teacher for novel-view synthesis into a four-step student with comparable quality.

Pith tools