{"id":"d625732b-0a9b-4529-ba0e-cefea46bb2b9","arxiv_id":"2501.04304","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DGQ quantizes text-to-image diffusion models to 4-8 bits without fine-tuning by preserving activation outliers and applying prompt-specific log quantization to cross-attention scores.","lead":"Compressing image-generation models to use less memory and compute usually ruins image quality or ignores the text prompt. This paper presents DGQ, a method that handles extreme values in the network separately and adapts the compression to each prompt, cutting computation by over 90% while keeping generated images and text alignment close to the original model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's core assumption that outlier channels/pixels and their range clusters are stable across prompts is not quantitatively established; if it fails, DGQ's per-group scales misassign at inference.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing premise I would stress: DGQ's grouping structure is learned from 64 calibration prompts and must transfer to unseen prompts and timesteps. The method is otherwise coherent, the ablations show each component helps, and the qualitative analysis of outliers is suggestive. However, the paper never quantifies cross-prompt stability of outlier locations or range clusters, and it does not report image-quality metrics on a held-out prompt distribution; only CLIP is reported for PartiPrompts. Because the central claim of preserving image quality at low bits depends on correct group assignment at inference, this is the most serious soft spot. The proposed test would settle it directly. I agree with the reader's CONDITIONAL verdict and would not change it; the concern supports the same conditions rather than moving to rejection.","tokens_in":16252,"tokens_out":6541,"duration_ms":70372,"concrete_test":"Quantize SD v1.4 with DGQ using five disjoint 64-prompt calibration sets drawn from different distributions (e.g., MS-COCO, PartiPrompts, DrawBench, random captions). For each calibration, record the chosen dimension d* and K-means centroids per layer/timestep, then compute agreement of group assignments on a fixed set of 1,000 held-out prompts using adjusted Rand index or centroid match rate. Generate 30K images with identical seeds for one calibration set per distribution and compute FID and CLIP on MS-COCO. If d* flips across calibration sets, group assignments disagree substantially, or FID/CLIP varies beyond seed noise, the outlier-stability premise fails and the calibration-transfer claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DGQ preserves image quality at low bits without weight fine-tuning rests on transferability of the per-layer grouping computed during calibration. Equations (4)-(6) select the grouping dimension d* and derive group-wise scales and zero-points from activation ranges observed on 64 MS-COCO prompts; Equation (7) makes these per-timestep but not per-prompt. At inference, every channel/pixel vector must be assigned to one of the K calibration clusters (apparently by its current range), so if unseen prompts or timesteps shift which channels or pixels are outliers, or change which range cluster a vector falls into, the assigned scale is wrong and the very outliers the method protects are clipped or coarsely quantized. The paper's support for stability is qualitative ('this pattern persists' in Section 3.2) plus a CLIP-only evaluation on PartiPrompts; no FID/IS is reported on a held-out prompt distribution, and the group count K is selected post hoc (Tables 3 and A.1 show different optimal K for 8-bit versus 6-bit). Thus the headline evidence (FID 13.15, CLIP drop 0.001) is conditional on a stability property that is asserted, not measured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes DGQ, a post-training quantization method for text-to-image diffusion models. It makes two distributional observations: activation outliers appear on specific channels or pixels and are critical for image quality, while cross-attention scores have a distinct peak for the <start> token and prompt-dependent ranges. The method selects a grouping dimension via the Dd metric, uses K-means to group channels or pixels per layer and per timestep with per-group scales, and applies a logarithmic quantizer with a prompt-specific dynamic scale to attention scores while keeping the <start> token in full precision. Experiments on Stable Diffusion v1.4 and SDXL Turbo report improved FID and CLIP scores over Q-Diffusion and TFMQ-DM, including FID 13.15 versus 14.44 for full precision, and the paper claims the first sub-8-bit activation quantization of text-to-image diffusion models without fine-tuning weight quantization parameters.","tokens_in":16394,"tokens_out":6106,"duration_ms":59565,"significance":"If the reported results are reproducible, DGQ is a valuable PTQ contribution: it avoids fine-tuning, the components are simple and hardware-oriented, and the paper includes component ablations, 30K-sample evaluation, and code release. The main significance hinges on two points that need strengthening: a fair comparison with baselines as originally configured, and evidence that the calibration-derived grouping transfers to unseen prompts. The outlier-stability concern raised by the stress-test is real; it is currently addressed with qualitative observations rather than quantitative diagnostics.","major_comments":[{"comment":"The claim that DGQ outperforms Q-Diffusion and TFMQ-DM is weakened by the baseline protocol. Section 4.1 states that the attention-score quantizer of both baselines is forced to match the activation bit width (8 or 6 bits), whereas Q-Diffusion and TFMQ-DM were originally designed to keep attention scores at 16 bits. Because text-image alignment is the quantity most affected by attention quantization, this changes the baseline methods rather than merely equalizing the comparison. Please rerun the baselines with their original 16-bit attention quantizer and report both settings, and ideally compare DGQ against a variant that uses the same attention quantizer as the baseline, so that the benefit of the attention-aware component is not conflated with the baseline handicap.","section":"Section 4.1, Table 2"},{"comment":"The transferability of the Dd-based dimension choice and the K-means grouping from 64 MS-COCO calibration prompts to unseen prompts is load-bearing but is only supported qualitatively. Equation (7) makes the scales per-timestep, not per-prompt, and group assignment at inference must rely on the calibration clusters; if the outlier channels or pixels shift across prompts, the per-group scales will be misassigned. The only held-out evidence is the CLIP score on PartiPrompts in Table 2, with no FID or IS on a held-out prompt distribution and no diagnostic of cluster stability. Please report FID/IS on PartiPrompts or another held-out prompt set, and quantify cluster-assignment stability across the calibration and evaluation prompts, for example the fraction of channel or pixel vectors that change group, or the drift in per-group ranges.","section":"Section 3.3, Eqs. (4)-(7)"},{"comment":"The number of groups K appears to be selected post hoc in a setting-dependent way. The 8-bit ablation in Table 3(b) shows that K=2 gives the best FID with dimension selection, while the 6-bit ablation in Table A.1 shows that larger K consistently helps, and Table 2 then reports DGQ with K=8 and K=16 for all settings. Because the headline FID 13.15 depends on this choice, the paper should justify K with an explicit validation protocol, for example choosing K on a calibration split and reporting the selected K per bit width, or presenting the full K sweep for every setting and clearly stating the selection rule.","section":"Section 4.3, Appendix B, Table 2"}],"minor_comments":[{"comment":"The word 'repectively' should be 'respectively'.","section":"Section 3.2"},{"comment":"The notation a^{max}_{i,d} and a^{min}_{i,d} should be defined more precisely; as written, 'the i-th vector in dimension d' is ambiguous about whether i indexes channels or pixels.","section":"Section 3.3, Eq. (4)"},{"comment":"BOPs are computed as FLOPs multiplied by weight bits and activation bits, but the per-timestep scale parameters in Eq. (7) and the full-precision <start> attention column in Eq. (9) are not reflected in the BOPs numbers; please clarify whether the reported BOPs include all overhead and whether the same formula applies to all methods.","section":"Section 4.1, Table 2"},{"comment":"The 8/6-bit rows in Table 3(c) were obtained with outlier-preserving group quantization already active, so the isolated effect of attention-aware quantization at 6 bits is not measured; this should be stated in the main text rather than only in the table caption.","section":"Section 4.3, Table 3(c)"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the manuscript is already published as an ICLR 2025 conference paper, so a journal submission should clarify what is new beyond the conference version. The main technical risk is the calibration-transfer issue raised in major comment 2; it is fixable with additional experiments, but it currently blocks the strong generalization claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read DGQ. The genuinely new thing here is the distributional analysis of cross-attention scores: the <start> token forms its own peak and the remaining scores have a prompt-dependent range, unlike the usual log-normal self-attention story. The attention-aware quantizer follows directly from that observation — log scale, separate <start> path, dynamic scale from the input — and the ablation shows each piece earns its keep. The outlier-preserving group quantization is less novel in spirit, but the pixel-vs-channel dimension selection plus K-means grouping is a sensible adaptation, and the component ablation is more careful than most papers in this area. Code is released.\n\nThe soft spots are real but fixable. First, the baselines are run with their attention quantizers forced to the activation bit width; the authors disclose this, but the reported margins over Q-Diff and TFMQ-DM depend on that choice. I'd want both the original 16-bit attention setting and the matched setting tabulated. Second, there are no error bars or repeated seeds on the FID/CLIP numbers, and the 'FID below full precision' result could easily be sampling noise. Third, and most substantively, the stability of outlier locations and group assignments across prompts is asserted, not measured. The grouping and scales are calibrated on 64 COCO prompts; the paper reports only CLIP on PartiPrompts, no FID, so we don't know whether the per-group scales transfer to a genuinely different prompt distribution. That's the load-bearing assumption, and it needs a direct test — held-out FID, sensitivity to K, and ideally a measure of group-assignment drift across prompts and timesteps.\n\nThe 'first low-bit PTQ without fine-tuning' claim also needs scoping against PTQD, which is PTQ for diffusion models and appears in their own references.\n\nNone of this sinks the paper. The method is coherent, the analysis is a real contribution, and the issues are addressable with additional experiments. I'd send it to review and ask for those measurements in the revision.","headline":"Solid, well-ablated quantization paper with a genuinely new cross-attention observation; the headline numbers need error bars and the outlier-stability assumption needs measuring, but this deserves a serious referee.","tokens_in":17024,"tokens_out":3207,"would_cite":true,"duration_ms":31533,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A post-training quantization scheme for text-to-image diffusion models preserves image quality and text alignment at bit widths down to 4-6 bits without fine-tuning weight quantization parameters.","keywords":["post-training quantization","text-to-image diffusion","activation outliers","cross-attention quantization","logarithmic quantization","group quantization","low-bit quantization","Stable Diffusion"],"falsifier":"Take a trained DGQ model and compare, across 100 diverse prompts, the per-layer indices of the channels and pixels that the $D_d$ metric flags as outliers; if those indices vary across prompts more than they vary across timesteps for one prompt, the grouping premise fails. A sharper test: calibrate DGQ on 64 captions from a domain with systematically different outlier structure (for example, medical or satellite imagery) and evaluate on MS-COCO; if FID or CLIP degrades far beyond the MS-COCO-calibrated model, the transferability of the calibrated groups is falsified, and the reported gains are calibration-set-specific.","tokens_in":15953,"feed_emoji":"⚡","tokens_out":11922,"duration_ms":100607,"temperature":0.7,"pith_summary":"This paper claims that the two failures of low-bit quantization in text-to-image diffusion models—degraded image quality and broken text–image alignment—have separable distributional causes and can be fixed without retraining. The first cause is activation outliers that concentrate in a few channels or pixels and carry image structure; the second is cross-attention scores whose <start> token forms a near-one peak and whose remaining values follow a prompt-dependent logarithmic distribution. DGQ handles the first with group-wise quantizers whose scales are computed along the channel or pixel dimension where activation ranges vary most, and the second with dynamic logarithmic quantization that keeps the <start> token in full precision. The paper reports that at 8-bit weights and activations on MS-COCO this preserves image quality (FID 13.15 vs 14.44 for full precision) and text alignment (CLIP drop 0.001) while cutting bit operations by 93.7%, and that it remains usable down to W4A6 where the baselines collapse. If correct, it is the first sub-8-bit post-training quantization of text-to-image diffusion models that requires no fine-tuning of weight quantization parameters.","feed_headline":"8-bit diffusion quantization beats full precision on FID","feed_subtitle":"DGQ cuts bit operations by 93.7% and holds CLIP text alignment steady without any fine-tuning.","key_machinery":"Two distributional diagnostics carry the argument. First, the activation metric\n$$D_d = \\left(\\max_i $a^{{\\max}}$_{i,d} - \\min_i $a^{{\\max}}$_{i,d}\\right) + \\left(\\max_i $a^{{\\min}}$_{i,d} - \\min_i $a^{{\\min}}$_{i,d}\\right)$$\nmeasures how much the per-vector activation ranges spread along dimension $d \\in \\{\\text{channel},\\text{pixel}\\}$; the dimension with larger $D_d$ is selected, and K-means clustering on that dimension splits activations into $K$ groups, each with its own scale $s_k$ and zero-point $z_k$, so outlier values keep a scale matched to their range while the bulk of values get finer quantization. Second, for cross-attention, the paper isolates the <start> token's attention scores, applies a logarithmic quantizer $A^q = \\mathrm{clamp}(\\lfloor -\\log_2(A/s)\\rceil, 0, 2^b-1)$ with a dynamic scale $s = \\max(A[:,1:])$ computed at inference time from the non-start scores, and leaves the start-token scores in full precision. The per-group activation scales are stored for each timestep, adding roughly 2.29 MB (about 0.1% of UNet memory) at 25 steps with 16 groups.","core_discovery":"The core discovery is that a text-to-image diffusion model can be aggressively quantized in a post-training, hardware-friendly way if two distributional facts are respected. First, activation outliers are not random: they sit on a small number of identifiable channels or pixels, so a metric $D_d$ that measures how much the per-channel or per-pixel min/max ranges vary can pick the right grouping dimension, and K-means on that dimension yields per-group scales that keep the outliers exact while shrinking overall quantization error. Second, cross-attention scores are not log-normal like self-attention: the <start> token forms its own peak near 1.0 and drives background attention, while the remaining scores concentrate or disperse depending on the prompt, so the paper applies a logarithmic quantizer whose scale is set per prompt to the maximum of the non-<start> scores and leaves the <start> token in full precision. On MS-COCO with Stable Diffusion v1.4, the method records FID 13.15 at W8A8—below the full-precision 14.44—and a CLIP score within 0.001 of full precision, with a 93.7% reduction in bit operations. The authors state this is the first time low-bit (<8 bit activation) quantization of text-to-image diffusion models has been achieved without additional fine-tuning of weight quantization parameters.","pith_inferences":["A natural extension: if outlier locations stay stable across broader prompt distributions, the same $D_d$-and-K-means recipe could be applied to video or audio diffusion backbones with only a small recalibration set.","A natural extension: isolating semantically special tokens before quantizing attention (as done for <start>) may also help LLM quantization, where CLS or separator tokens attract disproportionate attention mass.","A testable extension: replacing K-means with a per-layer outlier budget could reduce sensitivity to the 64-prompt calibration set at 4-bit activations while preserving DGQ's group-wise format.","A testable extension: since DGQ only quantizes activations, pairing it with a weight-only sub-4-bit quantizer is a direct next experiment that the paper does not run."],"forward_implications":["At W8A8 on MS-COCO, DGQ achieves FID 13.15 (full precision: 14.44) and CLIP 0.297 (full precision: 0.298), while reducing bit operations by 93.7% from 694 TBOPs to 43.4 TBOPs.","At W4A6, where Q-Diffusion and TFMQ-DM produce FID above 200 and CLIP below 0.16, DGQ reaches FID 43.66 and CLIP 0.274, keeping generation usable in a setting the baselines cannot handle.","The method generalizes to PartiPrompts, a dataset of prompts unlike the MS-COCO calibration captions, indicating the calibrated scales and groupings are not overfit to the 64 calibration prompts.","On SDXL Turbo with only 4 denoising steps, DGQ at W8A8 keeps FID 22.46 versus 21.25 full precision, whereas TFMQ-DM degrades to 111.69, so the approach transfers to a different architecture and sampling procedure.","The overhead of storing timestep- and group-dependent quantization parameters is about 2.29 MB (about 0.1% of UNet memory) for 25 steps and 16 groups, making the scheme practical for edge deployment."],"supporting_citations":[{"why":"Q-Diffusion; supplies the timestep-aware activation calibration and is the primary baseline DGQ outperforms.","marker":"Li et al., 2023a"},{"why":"TFMQ-DM; provides the temporal-feature-maintenance baseline and the main comparison point in every quantitative table.","marker":"Huang et al., 2024"},{"why":"BRECQ; supplies the block-reconstruction weight quantization used to set weights in both baselines and DGQ.","marker":"Li et al., 2021"},{"why":"AdaRound; supplies adaptive rounding for weight quantization in the evaluated low-bit settings.","marker":"Nagel et al., 2020"},{"why":"MS-COCO; provides the 64 calibration captions and the main 30K-prompt evaluation set.","marker":"Lin et al., 2014"},{"why":"PartiPrompts; provides the out-of-distribution prompt set used to test text-image alignment generalization.","marker":"Yu et al., 2022"},{"why":"FQ-ViT; documents the log-normal self-attention score distribution that DGQ contrasts with cross-attention's two-peak pattern.","marker":"Lin et al., 2022"},{"why":"Vision Transformers need registers; supports the paper's claim that a few outlier tokens carry disproportionate image structure.","marker":"Darcet et al., 2023"},{"why":"Defines the Stable Diffusion v1.4 architecture that DGQ quantizes and evaluates.","marker":"Rombach et al., 2022"}],"fun_headline_variants":["DGQ: 8-bit diffusion, FID 13.15 vs 14.44 full precision","First low-bit diffusion quantization without weight fine-tuning","Distribution-aware quantization cuts diffusion bit ops by 93.7%","8-bit diffusion: better FID and CLIP, no retraining","Quantize diffusion to 8-bit and beat full precision on FID"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that activation outliers stay on the same few channels or pixels across unseen prompts and timesteps, so the groupings and scales calibrated from 64 MS-COCO captions remain valid at inference; if the outliers move, the per-group scales miss the very values they were designed to preserve.","fun_headline_variants_meta":{"raw":{"variants":["DGQ: 8-bit diffusion, FID 13.15 vs 14.44 full precision","First low-bit diffusion quantization without weight fine-tuning","Distribution-aware quantization cuts diffusion bit ops by 93.7%","8-bit diffusion: better FID and CLIP, no retraining","Quantize diffusion to 8-bit and beat full precision on FID"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":3048,"prompt_tokens":1047,"completion_tokens":2001,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":1904}},"tokens_in":663,"tokens_out":2001,"duration_ms":16425,"temperature":1.0,"reasoning_tokens":1904,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:36:54.005577+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained DGQ model and compare, across 100 diverse prompts, the per-layer indices of the channels and pixels that the $D_d$ metric flags as outliers; if those indices vary across prompts more than they vary across timesteps for one prompt, the grouping premise fails. A sharper test: calibrate DGQ on 64 captions from a domain with systematically different outlier structure (for example, medical or satellite imagery) and evaluate on MS-COCO; if FID or CLIP degrades far beyond the MS-COCO-calibrated model, the transferability of the calibrated groups is falsified, and the reported gains are calibration-set-specific.","supporting_citations":[{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"MS-COCO; provides the 64 calibration captions and the main 30K-prompt evaluation set."}],"review_version":1}