REVIEW 3 major objections 4 minor 28 references
DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A post-training quantization scheme for text-to-image diffusion models preserves image quality and text alignment at bit widths down to 4-6 bits without fine-tuning weight quantization parameters.
desk verdict Solid, well-ablated quantization paper with a genuinely new cross-attention observation; the headline numbers need error bars and the outlier-stability assumption needs measuring, but this deserves a serious referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two distributional diagnostics carry the argument. First, the activation metric $$D_d = \left(\max_i $a^{{\max}}$_{i,d} - \min_i $a^{{\max}}$_{i,d}\right) + \left(\max_i $a^{{\min}}$_{i,d} - \min_i $a^{{\min}}$_{i,d}\right)$$ measures how much the per-vector activation ranges spread along dimension $d \in \{\text{channel},\text{pixel}\}$; the dimension with larger $D_d$ is selected, and K-means clustering on that dimension splits activations into $K$ groups, each with its own scale $s_k$ and zero-point $z_k$, so outlier values keep a scale matched to their range while the bulk of values get finer quantization. Second, for cross-attention, the paper isolates the <start> token's attention scores, applies a logarithmic quantizer $A^q = \mathrm{clamp}(\lfloor -\log_2(A/s)\rceil, 0, 2^b-1)$ with a dynamic scale $s = \max(A[:,1:])$ computed at inference time from the non-start scores, and leaves the start-token scores in full precision. The per-group activation scales are stored for each timestep, adding roughly 2.29 MB (about 0.1% of UNet memory) at 25 steps with 16 groups.
What would settle it
Take a trained DGQ model and compare, across 100 diverse prompts, the per-layer indices of the channels and pixels that the $D_d$ metric flags as outliers; if those indices vary across prompts more than they vary across timesteps for one prompt, the grouping premise fails. A sharper test: calibrate DGQ on 64 captions from a domain with systematically different outlier structure (for example, medical or satellite imagery) and evaluate on MS-COCO; if FID or CLIP degrades far beyond the MS-COCO-calibrated model, the transferability of the calibrated groups is falsified, and the reported gains are calibration-set-specific.
Extended reading notes
Core claim
The core discovery is that a text-to-image diffusion model can be aggressively quantized in a post-training, hardware-friendly way if two distributional facts are respected. First, activation outliers are not random: they sit on a small number of identifiable channels or pixels, so a metric $D_d$ that measures how much the per-channel or per-pixel min/max ranges vary can pick the right grouping dimension, and K-means on that dimension yields per-group scales that keep the outliers exact while shrinking overall quantization error. Second, cross-attention scores are not log-normal like self-attention: the <start> token forms its own peak near 1.0 and drives background attention, while the remaining scores concentrate or disperse depending on the prompt, so the paper applies a logarithmic quantizer whose scale is set per prompt to the maximum of the non-<start> scores and leaves the <start> token in full precision. On MS-COCO with Stable Diffusion v1.4, the method records FID 13.15 at W8A8—below the full-precision 14.44—and a CLIP score within 0.001 of full precision, with a 93.7% reduction in bit operations. The authors state this is the first time low-bit (<8 bit activation) quantization of text-to-image diffusion models has been achieved without additional fine-tuning of weight quantization parameters.
Load-bearing premise
The load-bearing premise is that activation outliers stay on the same few channels or pixels across unseen prompts and timesteps, so the groupings and scales calibrated from 64 MS-COCO captions remain valid at inference; if the outliers move, the per-group scales miss the very values they were designed to preserve.
Editorial extensions
If this is right
- At W8A8 on MS-COCO, DGQ achieves FID 13.15 (full precision: 14.44) and CLIP 0.297 (full precision: 0.298), while reducing bit operations by 93.7% from 694 TBOPs to 43.4 TBOPs.
- At W4A6, where Q-Diffusion and TFMQ-DM produce FID above 200 and CLIP below 0.16, DGQ reaches FID 43.66 and CLIP 0.274, keeping generation usable in a setting the baselines cannot handle.
- The method generalizes to PartiPrompts, a dataset of prompts unlike the MS-COCO calibration captions, indicating the calibrated scales and groupings are not overfit to the 64 calibration prompts.
- On SDXL Turbo with only 4 denoising steps, DGQ at W8A8 keeps FID 22.46 versus 21.25 full precision, whereas TFMQ-DM degrades to 111.69, so the approach transfers to a different architecture and sampling procedure.
- The overhead of storing timestep- and group-dependent quantization parameters is about 2.29 MB (about 0.1% of UNet memory) for 25 steps and 16 groups, making the scheme practical for edge deployment.
Reading between the lines
- A natural extension: if outlier locations stay stable across broader prompt distributions, the same $D_d$-and-K-means recipe could be applied to video or audio diffusion backbones with only a small recalibration set.
- A natural extension: isolating semantically special tokens before quantizing attention (as done for <start>) may also help LLM quantization, where CLS or separator tokens attract disproportionate attention mass.
- A testable extension: replacing K-means with a per-layer outlier budget could reduce sensitivity to the 64-prompt calibration set at 4-bit activations while preserving DGQ's group-wise format.
- A testable extension: since DGQ only quantizes activations, pairing it with a weight-only sub-4-bit quantizer is a direct next experiment that the paper does not run.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes DGQ, a post-training quantization method for text-to-image diffusion models. It makes two distributional observations: activation outliers appear on specific channels or pixels and are critical for image quality, while cross-attention scores have a distinct peak for the <start> token and prompt-dependent ranges. The method selects a grouping dimension via the Dd metric, uses K-means to group channels or pixels per layer and per timestep with per-group scales, and applies a logarithmic quantizer with a prompt-specific dynamic scale to attention scores while keeping the <start> token in full precision. Experiments on Stable Diffusion v1.4 and SDXL Turbo report improved FID and CLIP scores over Q-Diffusion and TFMQ-DM, including FID 13.15 versus 14.44 for full precision, and the paper claims the first sub-8-bit activation quantization of text-to-image diffusion models without fine-tuning weight quantization parameters.
Significance. If the reported results are reproducible, DGQ is a valuable PTQ contribution: it avoids fine-tuning, the components are simple and hardware-oriented, and the paper includes component ablations, 30K-sample evaluation, and code release. The main significance hinges on two points that need strengthening: a fair comparison with baselines as originally configured, and evidence that the calibration-derived grouping transfers to unseen prompts. The outlier-stability concern raised by the stress-test is real; it is currently addressed with qualitative observations rather than quantitative diagnostics.
major comments (3)
- [Section 4.1, Table 2] The claim that DGQ outperforms Q-Diffusion and TFMQ-DM is weakened by the baseline protocol. Section 4.1 states that the attention-score quantizer of both baselines is forced to match the activation bit width (8 or 6 bits), whereas Q-Diffusion and TFMQ-DM were originally designed to keep attention scores at 16 bits. Because text-image alignment is the quantity most affected by attention quantization, this changes the baseline methods rather than merely equalizing the comparison. Please rerun the baselines with their original 16-bit attention quantizer and report both settings, and ideally compare DGQ against a variant that uses the same attention quantizer as the baseline, so that the benefit of the attention-aware component is not conflated with the baseline handicap.
- [Section 3.3, Eqs. (4)-(7)] The transferability of the Dd-based dimension choice and the K-means grouping from 64 MS-COCO calibration prompts to unseen prompts is load-bearing but is only supported qualitatively. Equation (7) makes the scales per-timestep, not per-prompt, and group assignment at inference must rely on the calibration clusters; if the outlier channels or pixels shift across prompts, the per-group scales will be misassigned. The only held-out evidence is the CLIP score on PartiPrompts in Table 2, with no FID or IS on a held-out prompt distribution and no diagnostic of cluster stability. Please report FID/IS on PartiPrompts or another held-out prompt set, and quantify cluster-assignment stability across the calibration and evaluation prompts, for example the fraction of channel or pixel vectors that change group, or the drift in per-group ranges.
- [Section 4.3, Appendix B, Table 2] The number of groups K appears to be selected post hoc in a setting-dependent way. The 8-bit ablation in Table 3(b) shows that K=2 gives the best FID with dimension selection, while the 6-bit ablation in Table A.1 shows that larger K consistently helps, and Table 2 then reports DGQ with K=8 and K=16 for all settings. Because the headline FID 13.15 depends on this choice, the paper should justify K with an explicit validation protocol, for example choosing K on a calibration split and reporting the selected K per bit width, or presenting the full K sweep for every setting and clearly stating the selection rule.
minor comments (4)
- [Section 3.2] The word 'repectively' should be 'respectively'.
- [Section 3.3, Eq. (4)] The notation a^{max}_{i,d} and a^{min}_{i,d} should be defined more precisely; as written, 'the i-th vector in dimension d' is ambiguous about whether i indexes channels or pixels.
- [Section 4.1, Table 2] BOPs are computed as FLOPs multiplied by weight bits and activation bits, but the per-timestep scale parameters in Eq. (7) and the full-precision <start> attention column in Eq. (9) are not reflected in the BOPs numbers; please clarify whether the reported BOPs include all overhead and whether the same formula applies to all methods.
- [Section 4.3, Table 3(c)] The 8/6-bit rows in Table 3(c) were obtained with outlier-preserving group quantization already active, so the isolated effect of attention-aware quantization at 6 bits is not measured; this should be stated in the main text rather than only in the table caption.
Circularity Check
No significant circularity: DGQ's calibration-based scales are measured outputs, and evaluation includes the held-out PartiPrompts set.
full rationale
DGQ is a calibration-based post-training quantization method, not a derivation that returns its own inputs as a 'prediction.' The quantization scales and zero-points in Eqs. (4)-(6) are fitted to activation statistics from 64 MS-COCO captions, but the reported FID, IS, CLIP, and BOPs figures are measured outputs of the quantized network; they are not algebraically forced by the fitted parameters. The attention quantizer in Eq. (8) uses an inference-time dynamic scale, s = max(A[:,1:]), so no calibration parameter is recycled as a predicted result. The central evaluation includes PartiPrompts, which is outside the calibration distribution and is reported separately. The only self-citation (Ryu et al., 2024) appears in background discussion of quantization-aware fine-tuning and is not load-bearing. Concerns about outlier-channel stability across prompts and the post-hoc choice of group count are empirical validation and model-selection issues, not circular reductions: the paper does not define its success metric in terms of its fitted quantities, nor does it invoke an author-supplied uniqueness theorem. The claim of being 'first' to achieve sub-8-bit PTQ without fine-tuning is an external comparison claim, not a circular derivation. Overall, the derivation chain is self-contained and empirically falsifiable.
Assumptions & free parameters
free parameters (2)
- Number of quantization groups K per layer =
8 or 16 in main results; 2 was best in the 8-bit ablation
- Per-group quantization scale s_k and zero-point z_k =
Calibrated values not reported in the paper
assumptions (3)
- domain assumption Activation outliers are concentrated in specific channels or pixels and are stable enough across prompts and timesteps to be captured by calibration on 64 prompts.
- domain assumption Cross-attention scores excluding the <start> token follow a log-scale distribution whose maximum is the right dynamic scale, and the <start> column can be kept in full precision without breaking the efficiency claim.
- standard math Standard quantization operations (clamp, round, log, bit-shift) and full-precision softmax are available in the target inference stack.
Cite this review
Pith. "Pith review of DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models." pith.science (2026). https://pith.science/paper/GPUFSIL5
@misc{pith2026250104304,
author = {Pith},
title = {Pith review of: DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/GPUFSIL5}},
note = {Machine review of arXiv:2501.04304}
}
abstract
Despite the widespread use of text-to-image diffusion models across various tasks, their computational and memory demands limit practical applications. To mitigate this issue, quantization of diffusion models has been explored. It reduces memory usage and computational costs by compressing weights and activations into lower-bit formats. However, existing methods often struggle to preserve both image quality and text-image alignment, particularly in lower-bit($<$ 8bits) quantization. In this paper, we analyze the challenges associated with quantizing text-to-image diffusion models from a distributional perspective. Our analysis reveals that activation outliers play a crucial role in determining image quality. Additionally, we identify distinctive patterns in cross-attention scores, which significantly affects text-image alignment. To address these challenges, we propose Distribution-aware Group Quantization (DGQ), a method that identifies and adaptively handles pixel-wise and channel-wise outliers to preserve image quality. Furthermore, DGQ applies prompt-specific logarithmic quantization scales to maintain text-image alignment. Our method demonstrates remarkable performance on datasets such as MS-COCO and PartiPrompts. We are the first to successfully achieve low-bit quantization of text-to-image diffusion models without requiring additional fine-tuning of weight quantization parameters. Code is available at https://github.com/ugonfor/DGQ.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Understanding and overcoming the challenges of efficient transformer quantization
Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Understanding and overcoming the challenges of efficient transformer quantization. arXiv preprint arXiv:2109.12948,
-
[2]
and calculated the statistics. As shown in Figure 5(b), the maximum values of cross-attention scores vary more dynamically than those of self-attention. According to the statistics of the maximum attention scores, the standard Statistic Value Std of cross-attention 0.826 Std of self-attention 0.334 Mean ratio of each layer’s attention std 3.210 Table A.3:...
work page 2025
-
[7]
Quantizing deep convolutional networks for efficient inference: A whitepaper
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342,
-
[9]
Gligen: Open-set grounded text-to-image generation
11 Published as a conference paper at ICLR 2025 Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. CVPR, 2023b. Zhikai Li and Qingyi Gu. I-vit: Integer-only quantization for efficient vision transformer infer- ence. In Proceedings of the IEEE/CVF ...
work page 2025
-
[10]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Pro- ceedings, Part V 13, pp. 740–755. Springer,
work page 2014
-
[12]
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen
URL https://openreview.net/forum?id=di52zR8xgf. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3,
-
[14]
Deep unsupervised learning using nonequilibrium thermodynamics
12 Published as a conference paper at ICLR 2025 Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing, pp. 2256–2265. PMLR,
2025
-
[16]
Siao Tang, Xin Wang, Hong Chen, Chaoyu Guan, Zewen Wu, Yansong Tang, and Wenwu Zhu. Post-training quantization with progressive calibration and activation relaxing for text-to-image diffusion models. arXiv preprint arXiv:2311.06322,
Show all 28 references
-
[17]
Efficient diffusion models for vision: A sur- vey
Anwaar Ulhaq, Naveed Akhtar, and Ganna Pogrebna. Efficient diffusion models for vision: A sur- vey. arXiv preprint arXiv:2210.09292,
-
[19]
Qdrop: Ran- domly dropping quantization for extremely low-bit post-training quantization
Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu. Qdrop: Ran- domly dropping quantization for extremely low-bit post-training quantization. arXiv preprint arXiv:2203.05740,
-
[20]
Scaling autoregressive models for content- rich text-to-image generation
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5,
-
[21]
Mixdq: Memory-efficient few-step text-to-image diffusion models with metric-decoupled mixed precision quantization
Tianchen Zhao, Xuefei Ning, Tongcheng Fang, Enshu Liu, Guyue Huang, Zinan Lin, Shengen Yan, Guohao Dai, and Yu Wang. Mixdq: Memory-efficient few-step text-to-image diffusion models with metric-decoupled mixed precision quantization. arXiv preprint arXiv:2405.17873,
-
[22]
Binarydm: Towards accurate binarization of diffusion model
Xingyu Zheng, Haotong Qin, Xudong Ma, Mingyuan Zhang, Haojie Hao, Jiakai Wang, Zixiang Zhao, Jinyang Guo, and Xianglong Liu. Binarydm: Towards accurate binarization of diffusion model. arXiv preprint arXiv:2404.05662,
-
[23]
A survey on model compression for large language models
Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. A survey on model compression for large language models. arXiv preprint arXiv:2308.07633,
-
[24]
We analyze the effects of the attention score corresponding to <start> token
13 Published as a conference paper at ICLR 2025 A E FFECTS OF THE ATTENTION SCORE CORRESPONDING TO <S T A R T> TOKEN . We analyze the effects of the attention score corresponding to <start> token. For that, we adjust the attention scores, and compare the sampled images. We cha...
2025
-
[25]
A cat riding a bike
Clamping is used to check whether this can be excluded when determining the quantization scale of the quantizer, and dropping is used to check whether this can be excluded altogether. We confirmed that the <start> token doesn’t change the main contents of the images, but it af...
2025
-
[26]
The evaluation is conducted on 30K samples from the MS-COCO dataset
and the human preference model ImageReward (Xu et al., 2024). The evaluation is conducted on 30K samples from the MS-COCO dataset. As shown in Table A.2, in almost all cases, DGQ significantly outperforms the baseline. With 8-bit activation settings, TFMQ achieves slightly hig...
2024
-
[27]
A photo of a cat and a dog
requires a huge dataset and significant computational cost to obtain a quantized model. In contrast, DGQ (Ours) is a model generated through PTQ(Post-Training Quantization) that does not require a dataset, requires only 64 prompts, and has a minimal computational cost. Specifi...
2023
-
[29]
Figure A.4 shows the visualization of full activation matrix
or channel (case 2), we visual- ized only the values around the indexes where outliers occur in Figure 4(b). Figure A.4 shows the visualization of full activation matrix. Outliers on the specific pixel channel index pixel index (a) Case1: Outliers on the specific pixel pixel i...
2025
-
[2015]
Bitsfusion: 1.99 bits weight quantization of diffusion model
Yang Sui, Yanyu Li, Anil Kag, Yerlan Idelbayev, Junli Cao, Ju Hu, Dhritiman Sagar, Bo Yuan, Sergey Tulyakov, and Jian Ren. Bitsfusion: 1.99 bits weight quantization of diffusion model. arXiv preprint arXiv:2406.04333,
-
[2016]
Post-training quantization on diffusion models
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1972–1981,
1972
-
[2018]
Q-diffusion: Quantizing diffusion models
Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 17535–17545, October 2023a. Yuhang Li, Ru...
-
[2019]
Efficientdm: Efficient quantization- aware fine-tuning of low-bit diffusion models
Yefei He, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Efficientdm: Efficient quantization- aware fine-tuning of low-bit diffusion models. arXiv preprint arXiv:2310.03270,
-
[2020]
Glide: Towards photorealistic image generation and editing with text-guided diffusion models
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741,
-
[2021]
Vision transformers need registers
Timoth´ee Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588,
-
[2022]
Quest: Low-bit diffusion model quantization via efficient selective finetuning
Haoxuan Wang, Yuzhang Shang, Zhihang Yuan, Junyi Wu, and Yan Yan. Quest: Low-bit diffusion model quantization via efficient selective finetuning. arXiv preprint arXiv:2402.03666,
-
[2023]
Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee
URL https: //arxiv.org/abs/2305.15798. Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization. Advances in Neural Information Pro...
-
[2024]
Learned step size quantization
Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmen- dra S Modha. Learned step size quantization. arXiv preprint arXiv:1902.08153,
1902 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.