Pith. sign in

REVIEW 14 cited by

EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.03270 v4 pith:E4APQ2ZV submitted 2023-10-05 cs.CV

Yefei He , Jing Liu , Weijia Wu , Hong Zhou , Bohan Zhuang This is my paper · ORCID
classification cs.CV
keywords diffusionmodelsdataefficientdmperformancequantizationefficiencyfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Diffusion models have demonstrated remarkable capabilities in image synthesis and related generative tasks. Nevertheless, their practicality for real-world applications is constrained by substantial computational costs and latency issues. Quantization is a dominant way to compress and accelerate diffusion models, where post-training quantization (PTQ) and quantization-aware training (QAT) are two main approaches, each bearing its own properties. While PTQ exhibits efficiency in terms of both time and data usage, it may lead to diminished performance in low bit-width. On the other hand, QAT can alleviate performance degradation but comes with substantial demands on computational and data resources. In this paper, we introduce a data-free and parameter-efficient fine-tuning framework for low-bit diffusion models, dubbed EfficientDM, to achieve QAT-level performance with PTQ-like efficiency. Specifically, we propose a quantization-aware variant of the low-rank adapter (QALoRA) that can be merged with model weights and jointly quantized to low bit-width. The fine-tuning process distills the denoising capabilities of the full-precision model into its quantized counterpart, eliminating the requirement for training data. We also introduce scale-aware optimization and temporal learned step-size quantization to further enhance performance. Extensive experimental results demonstrate that our method significantly outperforms previous PTQ-based diffusion models while maintaining similar time and data efficiency. Specifically, there is only a 0.05 sFID increase when quantizing both weights and activations of LDM-4 to 4-bit on ImageNet 256x256. Compared to QAT-based methods, our EfficientDM also boasts a 16.2x faster quantization speed with comparable generation quality. Code is available at \href{https://github.com/ThisisBillhe/EfficientDM}{this hrl}.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 14 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. StrideDiffusion: Accelerating Diffusion Models for Time-series Generation

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A training-free sampler that adapts diffusion denoising strides to spectral band activity, cutting inference steps from 500-1000 to 14-66 with mostly comparable quality.

  2. MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation

    cs.CV 2025-07 conditional novelty 6.0 of 10

    MPQ-DMv2 adds binary residual quantization, temporal relation distillation, and SVD-initialized LoRA to mixed-precision quantization, improving low-bit diffusion model generation quality.

  3. Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Q-VDiT quantizes video diffusion transformers to 3-4 bit weights by adding a learned rank-1 error correction (TQE) and a temporal distribution distillation loss (TMD), nearly doubling VBench scene consistency at W3A6 ...

  4. Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A mix of signed and unsigned 4-bit floating-point formats, timestep-aware LoRA experts, and a denoising-weighted loss keeps diffusion-model image quality close to full precision.

  5. DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    DGQ quantizes text-to-image diffusion models to 4-8 bits without fine-tuning by preserving activation outliers and applying prompt-specific log quantization to cross-attention scores.

  6. 1.58-bit FLUX

    cs.CV 2024-12 reject novelty 6.0 of 10

    A post-training method reduces 99.5% of FLUX.1-dev's transformer weights to ternary values and reports roughly comparable text-to-image quality with large storage and memory savings.

  7. MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    MPQ-DM combines kurtosis-based intra-layer mixed-precision weight quantization with time-smoothed relation distillation to keep diffusion models accurate at 2 to 4 bit widths.

  8. BiDM: Pushing the Limit of Quantization for Diffusion Models

    cs.CV 2024-12 conditional novelty 6.0 of 10

    BiDM achieves the first fully binarized (W1A1) diffusion model, with FID 22.74 on LSUN-Bedrooms, by adding timestep-aware binary structures and patch-based distillation.

  9. PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution

    cs.CV 2024-11 conditional novelty 6.0 of 10

    PassionSR quantizes one-step diffusion super-resolution models to 6-8 bits via learnable quantizer boundaries, equivalent transformations, and a two-stage calibration, while keeping quality close to full precision.

  10. Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Quaff shows that activation outlier channels keep their spatial positions during LLM fine-tuning, and exploits this stability to cut fine-tuning memory and latency with INT8 quantization while matching or beating full...

  11. D$^2$-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models

    cs.CV 2025-01 conditional novelty 5.0 of 10

    Modeling quantization noise in compressed diffusion models as a time-step-dependent joint Gaussian, then correcting its mean and variance during sampling, improves FID over prior PTQ methods and can beat the full-prec...

  12. Efficiency Meets Fidelity: A Novel Quantization Framework for Stable Diffusion

    cs.CV 2024-12 conditional novelty 4.0 of 10

    A serial-to-parallel distillation pipeline with per-timestep activation quantization makes W4A8 quantized Stable Diffusion models generate images much closer to the full-precision model than prior PTQ methods.

  13. Efficient Diffusion Models: A Survey

    cs.LG 2025-02 conditional novelty 2.0 of 10

    The paper organizes research on efficient diffusion models into a taxonomy spanning algorithms, systems, and frameworks, and provides a curated reference list.

  14. From Noise to Nuance: Advances in Deep Generative Image Models

    cs.CV 2024-12 conditional

    A broad literature review of deep generative image models from GANs to diffusion and transformer architectures, with no new empirical results.

Pith tools