Pith. sign in

REVIEW 7 cited by

Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.14919 v4 pith:B4ASGIDA submitted 2024-10-19 cs.CV cs.LG

Mingyuan Zhou , Huangjie Zheng , Yi Gu , Zhendong Wang , Hai Huang This is my paper · ORCID
classification cs.CVcs.LG
keywords sidaadversarialgenerationlossmodeldistillationperformancediffusion
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Score identity Distillation (SiD) is a data-free method that has achieved SOTA performance in image generation by leveraging only a pretrained diffusion model, without requiring any training data. However, its ultimate performance is constrained by how accurate the pretrained model captures the true data scores at different stages of the diffusion process. In this paper, we introduce SiDA (SiD with Adversarial Loss), which not only enhances generation quality but also improves distillation efficiency by incorporating real images and adversarial loss. SiDA utilizes the encoder from the generator's score network as a discriminator, allowing it to distinguish between real images and those generated by SiD. The adversarial loss is batch-normalized within each GPU and then combined with the original SiD loss. This integration effectively incorporates the average "fakeness" per GPU batch into the pixel-based SiD loss, enabling SiDA to distill a single-step generator. SiDA converges significantly faster than its predecessor when distilled from scratch, and swiftly improves upon the original model's performance during fine-tuning from a pre-distilled SiD generator. This one-step adversarial distillation method establishes new benchmarks in generation performance when distilling EDM diffusion models, achieving FID scores of 1.110 on ImageNet 64x64. When distilling EDM2 models trained on ImageNet 512x512, our SiDA method surpasses even the largest teacher model, EDM2-XXL, which achieved an FID of 1.81 using classifier-free guidance (CFG) and 63 generation steps. In contrast, SiDA achieves FID scores of 2.156 for size XS, 1.669 for S, 1.488 for M, 1.413 for L, 1.379 for XL, and 1.366 for XXL, all without CFG and in a single generation step. These results highlight substantial improvements across all model sizes. Our code is available at https://github.com/mingyuanzhou/SiD/tree/sida.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Amortized Moment Matching for Visual Generation

    cs.LG 2026-07 accept novelty 6.0 of 10

    Amortized Fréchet Distance uses neural nets to match conditional means and covariances, yielding stronger one-step visual generators than explicit FD-loss or multi-step teachers.

  2. Parallel Decoding Distillation for Fast Image and Video Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A trajectory-based distillation method trains a student to predict multiple mean velocities per network evaluation, enabling 4-8 step generation with competitive quality and improved diversity.

  3. Bridging Diffusion Pruning and Step Distillation with Teacher-Aligned Repair

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A short teacher-alignment repair stage between structured pruning and one-step distillation yields a 20% pruned one-step generator that improves FID from 3.53 to 3.12 on ImageNet-512 while reducing NFE from 63 to 1.

  4. End-to-End Training for Autoregressive Video Diffusion via Self-Resampling

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Resampling Forcing trains autoregressive video diffusion models on self-resampled degraded histories with a causal mask, achieving stable long-horizon generation without a teacher or discriminator.

  5. Visual Generation Without Guidance

    cs.CV 2025-01 conditional novelty 6.0 of 10

    GFT trains a single β-conditioned network that reproduces Classifier-Free Guidance's sampling distribution, matching CFG FID scores across five model families with half the inference cost.

  6. DiFA: Inference-Time Forward-Process Alignment for Diffusion Models

    cs.LG 2026-07 conditional novelty 5.0 of 10

    DiFA produces a consensus of historical denoiser predictions weighted by structural and noise-level compatibility, then applies SNR-gated residual guidance, improving few-step FID/IS of diffusion and flow-matching sam...

  7. Controllable diffusion-based generation for multi-channel biological data

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A diffusion model with multi-resolution conditioning and channel-wise attention achieves state-of-the-art accuracy for protein and gene imputation in spatial and single-cell biological data.

Pith tools