Pith. sign in

REVIEW 16 cited by

Analyzing and Improving the Training Dynamics of Diffusion Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.02696 v2 pith:BI7CMDXC submitted 2023-12-05 cs.CV cs.AIcs.LGcs.NEstat.ML

Tero Karras , Miika Aittala , Jaakko Lehtinen , Janne Hellsten , Timo Aila , Samuli Laine This is my paper · ORCID
classification cs.CVcs.AIcs.LGcs.NEstat.ML
keywords trainingdiffusionnetworkarchitectureimbalancesmodelsseveralsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Diffusion models currently dominate the field of data-driven image synthesis with their unparalleled scaling to large datasets. In this paper, we identify and rectify several causes for uneven and ineffective training in the popular ADM diffusion model architecture, without altering its high-level structure. Observing uncontrolled magnitude changes and imbalances in both the network activations and weights over the course of training, we redesign the network layers to preserve activation, weight, and update magnitudes on expectation. We find that systematic application of this philosophy eliminates the observed drifts and imbalances, resulting in considerably better networks at equal computational complexity. Our modifications improve the previous record FID of 2.41 in ImageNet-512 synthesis to 1.81, achieved using fast deterministic sampling. As an independent contribution, we present a method for setting the exponential moving average (EMA) parameters post-hoc, i.e., after completing the training run. This allows precise tuning of EMA length without the cost of performing several training runs, and reveals its surprising interactions with network architecture, training time, and guidance.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 16 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The GAN is dead; long live the GAN! A Modern GAN Baseline

    cs.LG 2025-01 conditional novelty 7.0 of 10

    A minimalist GAN with a regularized relativistic loss and modern backbone matches or beats StyleGAN2 and several diffusion models on standard FID benchmarks.

  2. FairDiffuseVQVAE: Sampling-Time Fairness in Tabular Diffusion via Conditional Refinement of Vector-Quantized Latents

    cs.LG 2026-07 conditional novelty 6.0 of 10

    FairDiffuseVQVAE reaches state-of-the-art fairness on the standard tabular benchmark (DPR 0.702, EOR 0.686) by uniform protected-attribute sampling at inference, paying ~15 AUC points of utility.

  3. Diffusion Models for Sampling Near Criticality in Lattice Field Theories

    hep-lat 2026-07 accept novelty 6.0 of 10

    Fully convolutional diffusion models trained on small lattices transfer to unseen larger volumes for 2D/3D phi^4 sampling across phases, matching or beating same-size training on most observables.

  4. Bridging Diffusion Pruning and Step Distillation with Teacher-Aligned Repair

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A short teacher-alignment repair stage between structured pruning and one-step distillation yields a 20% pruned one-step generator that improves FID from 3.53 to 3.12 on ImageNet-512 while reducing NFE from 63 to 1.

  5. Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Splitting weight matrices into a fixed-norm direction and learnable per-row/column magnitudes improves LLM training over AdamW/Muon, removes weight decay and warmup, and transfers the optimal LR across width.

  6. SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SRC-Flow compresses RAE features via a Semantic Representation Compressor into a low-dimensional space, enabling normalizing flows to reach gFID 1.65 on ImageNet 256x256 and 2.07 on 512x512 while retaining exact likelihoods.

  7. SeqTex: Generate Mesh Textures in Video Sequence

    cs.CV 2025-07 conditional novelty 6.0 of 10

    SeqTex adapts a pretrained video diffusion model to directly generate complete UV texture maps by jointly predicting four multi-view images and the UV map as a five-frame sequence.

  8. FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A hybrid U-Net and Transformer diffusion backbone trained in residual space with a velocity parametrization outperforms baselines on 2D turbulence super-resolution and forecasting, and generalizes zero-shot to unseen ...

  9. SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A co-designed 4-bit quantization and temporal sparsity scheme, with a matching dense-sparse accelerator, speeds up diffusion model inference by 6.91x with 51.5% energy reduction at near-FP32 image quality.

  10. Image Generation Diversity Issues and How to Tame Them

    cs.CV 2024-11 conditional novelty 6.0 of 10

    The paper proposes a retrieval-based diversity metric (IRS), finds that state-of-the-art diffusion models retrieve at most 77% of training images, and introduces feature-conditioned DiADM to improve unconditional diversity.

  11. Guiding a diffusion model using sliding windows

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Masked sliding window guidance improves diffusion sample quality by guiding the model with its own crop-based predictions, without training.

  12. EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes

    cs.LG 2025-07 unverdicted novelty 5.0 of 10

    A bias-corrected exponential moving average (BEMA) is claimed to remove the lag of standard EMA weight averaging during LLM fine-tuning, improving convergence and final performance over EMA and vanilla training.

  13. Rethinking Oversaturation in Classifier-Free Guidance via Low Frequency

    cs.CV 2025-06 conditional novelty 5.0 of 10

    LF-CFG down-weights low-frequency, low-change regions in classifier-free guidance to reduce oversaturation, with reported gains over CFG and APG on SD-XL, SD2.1, SD3.0, SD3.5, and SiT-XL.

  14. Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget

    cs.CV 2026-07 conditional novelty 4.0 of 10

    An open-source image-generation family shows that agentic prompt rewriting and a stronger text encoder can lift quality to near closed-source levels with only 208.62M images and about $400K of training compute.

  15. Pinterest Canvas: Large-Scale Image Generation at Pinterest

    cs.CV 2026-03 conditional novelty 4.0 of 10

    A FLUX-style base diffusion model plus task-specific fine-tunes and product-preserving pipelines yields double-digit Pinterest ads engagement lifts and higher no-defect rates than GPT-Image, FLUX Kontext, and Nano Banana.

  16. The Principles of Diffusion Models

    cs.LG 2025-10 unverdicted novelty 3.0 of 10

    A principled monograph showing that variational, score-based, and flow-based diffusion models are instances of one continuous-time transport backbone, with sampling equal to solving a differential equation governed by...

Pith tools