REVIEW 13 cited by
Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Recent advancements in diffusion models, particularly the architectural transformation from UNet-based models to Diffusion Transformers (DiTs), significantly improve the quality and scalability of image and video generation. However, despite their impressive capabilities, the substantial computational costs of these large-scale models pose significant challenges for real-world deployment. Post-Training Quantization (PTQ) emerges as a promising solution, enabling model compression and accelerated inference for pretrained models, without the costly retraining. However, research on DiT quantization remains sparse, and existing PTQ frameworks, primarily designed for traditional diffusion models, tend to suffer from biased quantization, leading to notable performance degradation. In this work, we identify that DiTs typically exhibit significant spatial variance in both weights and activations, along with temporal variance in activations. To address these issues, we propose Q-DiT, a novel approach that seamlessly integrates two key techniques: automatic quantization granularity allocation to handle the significant variance of weights and activations across input channels, and sample-wise dynamic activation quantization to adaptively capture activation changes across both timesteps and samples. Extensive experiments conducted on ImageNet and VBench demonstrate the effectiveness of the proposed Q-DiT. Specifically, when quantizing DiT-XL/2 to W6A8 on ImageNet ($256 \times 256$), Q-DiT achieves a remarkable reduction in FID by 1.09 compared to the baseline. Under the more challenging W4A8 setting, it maintains high fidelity in image and video generation, establishing a new benchmark for efficient, high-quality quantization in DiTs. Code is available at \href{https://github.com/Juanerx/Q-DiT}{https://github.com/Juanerx/Q-DiT}.
Forward citations
Cited by 13 Pith papers
-
Importance-Aware OBS Pruning for Diffusion Models
Injecting spatial importance maps (e.g., CFG delta) into the OBS Hessian improves subject preservation in pruned diffusion models at high sparsity, but gains over the baseline are small and without error bars.
-
Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling
Q-Sched's quantization-aware scheduler with a reference-free JAQ loss lets 2-8 step quantized diffusion models reach lower FID than full-precision baselines.
-
Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers
Q-VDiT quantizes video diffusion transformers to 3-4 bit weights by adding a learned rank-1 error correction (TQE) and a temporal distribution distillation loss (TMD), nearly doubling VBench scene consistency at W3A6 ...
-
FPQVAR: Floating Point Quantization for Visual Autoregressive Model with FPGA Hardware Co-design
A post-training floating-point quantization framework with grouped Hadamard rotation and learned smoothing brings 4-bit visual autoregressive image generation to near-FP16 quality, plus a matching FPGA accelerator.
-
Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scales
A data-free method that tunes only the diagonal scales of Winograd transform matrices enables accurate 8-bit fully quantized Winograd convolution for diffusion models and ResNets.
-
1.58-bit FLUX
A post-training method reduces 99.5% of FLUX.1-dev's transformer weights to ternary values and reports roughly comparable text-to-image quality with large storage and memory savings.
-
Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
TeaCache speeds up video diffusion models by caching model outputs when the difference between timestep-embedding-modulated inputs is small, after polynomial rescaling calibrated on a small prompt set.
-
Importance-Based Token Merging for Efficient Image and Video Generation
A token-merging method that anchors computation on high-CFG-importance tokens improves generation quality at fixed inference speedups.
-
Diffusion Product Quantization
DPQ applies product quantization with compressed codebooks and DDPM-loss calibration to compress DiT diffusion models to 1-bit weights with FID 14.03, far better than VQ's 91.71.
-
Bag of Design Choices for Inference of High-Resolution Masked Generative Transformer
A set of inference-time design choices improves image quality, memory, and speed of masked generative Transformers, with combined tricks winning about 70% of human-preference comparisons against vanilla sampling.
-
QwT-v2: Practical, Effective and Efficient Post-Training Quantization
A post-training quantization method that replaces QwT's full matrix compensation with per-channel affine compensation, matching or nearly matching QwT accuracy with just 1-2% extra parameters and zero runtime overhead.
-
Quantization without Tears
Attaching per-block linear compensators, fit by closed-form least squares, improves the accuracy of post-training quantized models with minimal latency overhead.
-
Diffusion Model Quantization: A Review
A structured review and benchmark of methods for quantizing diffusion models, with a taxonomy of post-training and quantization-aware approaches and an analysis of quantization artifacts.
Discussion (0). Continue with ORCID to comment.