{"id":"bebfadb3-16f8-4679-a5de-01bebc416d0b","arxiv_id":"2501.00124","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"PQD calibrates diffusion-model quantization on time steps drawn from a tuned normal distribution, reporting competitive 8-bit FID on 64x64 ImageNet but much worse 4-bit FID and no quantitative text-to-image results.","lead":"The paper proposes PQD, a way to compress pretrained image diffusion models to 8-bit or 4-bit arithmetic without retraining, by choosing a small set of noisy images at selected denoising steps to calibrate the quantization. It claims this keeps image quality close to the full-precision model and extends to 512x512 text-to-image generation for the first time.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's own 4-bit rows contradict the central 'few FID change' claim: W4A32 FID is 69.14 vs 21.63 full-precision, a 3.2x degradation, and IS drops from 14.88 to 4.40; W4A8 is 75.32. The paper's strongest claim is therefore unsupported by its own evidence.","rationale":"Read in good faith, PQD contributes a calibration-set construction for training-free PTQ of diffusion models. The strongest advertised outcome is the abstract: 8-bit or 4-bit quantization with comparable performance and only a few FID change. For that claim to hold, the 4-bit rows of Table 1 need to show small degradation. They show the opposite: W4A32 FID is 47.5 points worse than full precision and IS collapses from 14.88 to 4.40; W4A8 is similarly degraded. This is an internal inconsistency in the evidence, not a disagreement with an external consensus or a question of taste. The 8-bit result is a credible positive, and I would not dismiss it; but the abstract promises 4-bit. The calibration time-step distribution (N(0.4,0.4), no sensitivity analysis, 'skew normal' vs 'normal' mismatch in Algorithm 1) is a second genuine concern, but it is secondary: even if the calibration design were exactly as described, the reported 4-bit numbers would still contradict the abstract. A reproduction of the W4A32 row is the single check that would settle whether Table 1 contains an error or the central claim is overstated. Until then the rejection stands.","tokens_in":8050,"tokens_out":8796,"duration_ms":82290,"concrete_test":"Reproduce the W4A32 row of Table 1: run PQD on the same ImageNet 64x64 DDPM/DDIM-250 setup with N=5120, mu=0.4, sigma=0.4, and QDrop 4-bit weight quantization, then compute FID/IS with a standard reference implementation. If FID is about 69, the 'few FID change' claim for 4-bit is false and the abstract must be revised. If FID is about 25 or better, inspect whether Table 1 mislabels the rows or used a different sampling/calibration protocol; either way the discrepancy must be resolved before the central claim can be accepted. Also verify the full-precision FID 21.63 under the same sampler, since all comparisons depend on this baseline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim asserts direct quantization to 8-bit or 4-bit while maintaining comparable performance and achieving a few FID change on ImageNet. The paper's own Table 1 reports, for ImageNet 64x64 with DDIM 250 steps: full precision W32A32 FID 21.63 / IS 14.88; PQD W4A32 FID 69.14 / IS 4.40; PQD W4A8 FID 75.32 / IS 4.48. The 4-bit FID is roughly 3.2-3.5 times worse than full precision and IS drops by about 70%. This is neither 'a few FID change' nor 'comparable performance.' Section 4.2's claim of 'few increase in FID and imperceptible distortions' is contradicted by the table it accompanies. Since the abstract explicitly lists 4-bit quantization, the central claim fails on the paper's own reported numbers, independent of any external comparison. The 8-bit row (FID 22.29 vs 21.63) is plausible and should be credited, but it does not rescue the 4-bit claim. The only way the central claim survives is if the table entries are mislabeled or computed under a different protocol, which is exactly what needs to be checked.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PQD, a post-training quantization framework for diffusion models. Its main idea is to construct a calibration dataset by running the full-precision model for time steps sampled from a Gaussian distribution (µ=0.4, σ=0.4), then applying the existing QDrop quantizer. The authors claim that this allows direct 8-bit and 4-bit quantization of diffusion models with 'a few FID change' on ImageNet and extends for the first time to 512×512 text-guided generation. Experiments cover ImageNet 64×64 with DDIM 250 steps, and Stable Diffusion for text-to-image. The 8-bit result (FID 22.29 vs 21.63) is plausible; the 4-bit rows report FID 69.14 (W4A32) and 75.32 (W4A8), roughly 3–3.5 times the full-precision FID, against the paper's own text.","tokens_in":8340,"tokens_out":4723,"duration_ms":37237,"significance":"If the claims held, PQD would be a practically valuable training-free method for reducing diffusion model inference cost, and the 8-bit ImageNet result is indeed encouraging. The proposed calibration-time-step distribution is a simple and interesting idea, and the paper explicitly builds on an external method (QDrop), making the empirical recipe easy to reproduce in principle. However, the paper's own Table 1 invalidates the central 4-bit claim: FID degrades from 21.63 to 69.14–75.32 and IS drops from 14.88 to about 4.4–4.5. That is not 'comparable performance' nor 'a few FID change'. As the abstract and Section 4.2 explicitly rest this claim on 4-bit results, the central contribution is unsupported. The text-to-image extension is also reported only qualitatively, with no FID/CLIP numbers. Thus the paper does not currently establish a significant improvement over the state of the art.","major_comments":[{"comment":"The central claim that PQD maintains comparable performance under 4-bit quantization is contradicted by Table 1: full-precision FID is 21.63 and IS is 14.88, while PQD W4A32 gives FID 69.14 and IS 4.40, and PQD W4A8 gives FID 75.32 and IS 4.48. These are roughly 3.2–3.5 times worse FID and about 70% lower IS, not 'a few increase in FID' or 'imperceptible distortions' as Section 4.2 states. The abstract explicitly promises 4-bit quantization 'while maintaining comparable performance'; this must be corrected or the 4-bit claims removed. As written, the central claim fails on the manuscript's own reported numbers.","section":"Abstract; Section 4.2, Table 1"},{"comment":"The calibration distribution N(µ=0.4, σ=0.4) is hand-tuned on the target benchmark with no validation split, sensitivity analysis, or transfer study. Since the entire method consists of choosing this time-step distribution, the paper should justify that µ and σ are not overfit to ImageNet, and should report how FID varies with µ and σ or with alternative calibration distributions (e.g., uniform over steps). Without this, the method's main free parameters are unvalidated.","section":"Section 3.4; Algorithm 1"},{"comment":"No error bars, random seeds, or multiple runs are reported. The claimed 8-bit improvement over PTQ4DM (FID 22.29 vs 23.96) and the small gap to full precision (22.29 vs 21.63) could be within run-to-run noise; a single run is not sufficient to support the state-of-the-art comparison, especially for generative metrics like FID.","section":"Section 4.2"},{"comment":"The text-to-image extension provides only qualitative samples. No quantitative metrics (FID, CLIP score, or other) are given for MS-COCO, although Section 1 claims 'state-of-the-art results on 8-bits models for 512x512 high-resolution text-guided image generation.' A quantitative comparison with full-precision Stable Diffusion and with baselines (e.g., PTQ4DM and PTQD) is required to support this claim.","section":"Section 4.3; Figure 3"}],"minor_comments":[{"comment":"The phrase 'skew normal distribution N(µ, σ)' is contradictory; N typically denotes the normal distribution. If a skew-normal is intended, define it and give its parameters.","section":"Algorithm 1, line 2"},{"comment":"The notation q(xt-1 | xt) appears to be the forward transition q(xt | xt-1); the subscripts are inconsistent with the text.","section":"Equation (1)"},{"comment":"The quantization/dequantization formula uses the zero point z in a nonstandard way; check against the standard QDrop/PTQ formulation to avoid ambiguity.","section":"Equation (3)"},{"comment":"The unit 'Bops' should be spelled 'BOPS' or 'GOPs'; also the parenthetical about 'INT8 attention act-to-act matmuls' in the PTQ4DM row is unclear.","section":"Table 1 caption"},{"comment":"The paper compares only against PTQ4DM; PTQD (He et al., 2024) is discussed in related work but is not included in the experiments.","section":"Section 4.2"},{"comment":"The claim of 'first' text-guided 512x512 post-training quantization should be verified against recent literature; PTQD or other works may already cover text-to-image.","section":"Section 1; Section 4.3"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early version: there is no code release, no seeds, and the experimental section is brief. The strongest advertised claim (4-bit with comparable performance) is internally contradicted by Table 1, so the paper cannot be fixed by local revisions alone. The 8-bit idea may be salvageable as a separate contribution, but the current framing and evidence would need to change substantially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper has a load-bearing internal contradiction. The abstract promises direct 8-bit or 4-bit quantization with \"a few FID change\" and \"comparable performance.\" Their own Table 1 shows W4A32 FID 69.14 and W4A8 FID 75.32 against 21.63 for full precision, with IS dropping from 14.88 to about 4.4. That is not a few FID change; it is a threefold degradation. Section 4.2 claims \"few increase in FID and imperceptible distortions\" right next to that table. Either the table is mislabeled or the claim is wrong, and the paper does not explain which.\n\nWhat is actually new: the specific combination of a Gaussian time-step sampling distribution for calibration data and applying it to latent-space text-guided generation with conditional and unconditional features. That is a small variant of PTQ4DM, which already samples calibration across all time frames, and the paper acknowledges this. The 8-bit ImageNet row (FID 22.29 vs 21.63) is a genuinely decent result and should be credited. The qualitative 512x512 text-to-image samples look reasonable, but the paper gives no quantitative evaluation for that section, so the \"first time\" and state-of-the-art claims are unverified.\n\nSoft spots beyond the central contradiction: the calibration parameters μ=0.4, σ=0.4 are tuned on the target benchmarks with no validation split, no sensitivity analysis, and no transfer study. The distribution is assumed to generalize without evidence. There are no error bars or seeds for any reported number. No code is released, so nothing is reproducible. The novelty is thin, and the writing has some sloppy moments (e.g., repeated definitions).\n\nWho this is for: someone working on efficient diffusion inference might read the 8-bit part and the text-to-image extension, but only to mine ideas, not to trust the claims. The paper does not deserve acceptance as-is. It could become a workshop-level paper if the 4-bit claim is fixed or removed, the text-to-image metrics are added, and the calibration parameters are justified or ablated.\n\nFor peer review: I would still send it to referees rather than desk-reject, because the 8-bit result and the latent-space extension are worth checking, and the contradiction needs an official record. But that referee should be told to focus on Table 1 and the missing validation.","headline":"The 8-bit ImageNet result is plausible, but the 4-bit claim is dead on arrival: Table 1 shows roughly 3x FID degradation, contradicting the abstract.","tokens_in":8896,"tokens_out":1380,"would_cite":false,"duration_ms":14956,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PQD claims that sampling calibration data along the denoising trajectory lets post-training quantization compress diffusion models to 8-bit or 4-bit weights while keeping image quality close to full precision, including for 512x512…","keywords":["post-training quantization","diffusion models","time-aware calibration","training-free quantization","text-to-image generation","latent diffusion models","QDrop","image generation"],"falsifier":"Run the method on the same ImageNet DDPM with the calibration time-step distribution replaced by a uniform distribution over [0, T] or by a Gaussian with a different center (for example, mu = 0.7), keeping all other settings fixed. If the quantized model's FID remains essentially unchanged, the specific hand-tuned distribution is not what carries the method; if the FID degrades sharply, the method's success depends on that distribution in a way the paper does not characterize.","tokens_in":7772,"feed_emoji":"🎨","tokens_out":7841,"duration_ms":72139,"temperature":0.7,"pith_summary":"This paper claims that the main obstacle to post-training quantization of diffusion models is the time-varying activation distribution, and that a calibration set sampled along the denoising trajectory fixes it. The proposed PQD method runs the full-precision model for a random number of steps, with the sampling time drawn from a fixed normal distribution N(0.4, 0.4), and uses the recorded intermediate activations to calibrate quantizers with QDrop. On 64x64 ImageNet DDPM, it reports 8-bit weight-and-activation quantization with FID 22.29 versus 21.63 for the full-precision model, and it reports the first post-training quantization of 512x512 Stable Diffusion text-to-image generation. If the claim holds, low-bit diffusion inference on commodity hardware would not require retraining or access to the original training data.","feed_headline":"Quantized diffusion models keep image quality at 8 bits","feed_subtitle":"A time-aware calibration set preserves fidelity and extends quantization to 512x512 text-to-image generation.","key_machinery":"The central mechanism is Algorithm 1, the time-aware calibration dataset sampler. Instead of using static images, PQD draws a denoising time step from a normal distribution, rolls the full-precision model forward from pure noise to that step, and records the intermediate latent or input tensor together with its time embedding and text conditioning. This yields a calibration set whose activation ranges intentionally cover the denoising trajectory. Quantization itself is done by QDrop, and for high-resolution text-to-image generation the quantization is applied in the latent space of a Stable Diffusion model rather than to pixel-space inputs.","core_discovery":"The paper's core claim is that diffusion models can be quantized in a training-free manner when the calibration data are collected in a time-aware way, because the activation distribution of the noise-estimation network shifts systematically as denoising progresses. PQD constructs a calibration set by starting from random noise, running the full-precision denoiser down to a randomly drawn time step t (sampled from a normal distribution with mean 0.4 and standard deviation 0.4), and storing the intermediate noisy sample, its time embedding, and — for text-guided generation — paired conditional and unconditional text features. Quantization parameters are then fit with QDrop, which randomly keeps some operations in full precision during calibration. The paper reports W8A8 results on ImageNet DDPM with FID 22.29 versus 21.63 for the full-precision model, a W4A8 run with FID 75.32, and successful 8-bit quantization of 512x512 Stable Diffusion text-to-image generation, which it states has not been achieved before.","pith_inferences":["Because the paper fixes the calibration time-step distribution to N(0.4, 0.4) by hand, a natural extension is to estimate the distribution from the target sampler or model, or to make it adaptive, which could improve transfer to faster samplers.","The method's reliance on a fixed quantizer per layer suggests a testable extension: time-conditioned quantization scales that vary with the denoising phase could capture the activation shift more tightly.","The success on latent diffusion points toward applying the same latent-space calibration to larger models, such as 1024x1024 text-to-image or video diffusion models, which the paper does not examine.","If the calibration distribution's match to production activations is the active ingredient, then PQD should combine naturally with sampling-acceleration methods, and the combined quantization-plus-acceleration error is untested in the paper."],"forward_implications":["If PQD's claims hold, pretrained diffusion models can be deployed at 8-bit weight-and-activation precision with only a small FID penalty and no fine-tuning.","The same time-aware calibration recipe transfers from pixel-space DDPMs to latent diffusion models, so 512x512 text-to-image generation can run quantized.","Because calibration requires only running the full-precision model for a few denoising steps per sample, the overhead of building the calibration set is small relative to the cost of generation.","The paper's 4-bit weight results show that the method supports weight bit-widths below 8, though the reported FID movement is larger than at 8-bit."],"supporting_citations":[{"why":"Supplies the first post-training quantization method for diffusion models that PQD extends, and serves as the comparison baseline.","marker":"Shang et al. [2022]"},{"why":"QDrop is the quantization routine applied on top of PQD's calibration data to fit quantization parameters.","marker":"Wei et al. [2022]"},{"why":"The DDPM pretrained on ImageNet is the unconditional generation model being quantized at 64x64.","marker":"Ho et al. [2020]"},{"why":"The Stable Diffusion / latent diffusion model that PQD quantizes for 512x512 text-to-image generation.","marker":"Rombach et al. [2021]"},{"why":"DDIM is the 250-step sampler that defines the denoising trajectory used in the unconditional experiments.","marker":"Song et al. [2020]"},{"why":"Provides the FID metric used to report the image-quality results for quantized versus full-precision models.","marker":"Heusel et al. [2017]"},{"why":"ImageNet is the dataset for the unconditional DDPM experiments.","marker":"Deng et al. [2009]"}],"fun_headline_variants":["Time-aware calibration enables training-free diffusion quantization","Quantize diffusion models without retraining via time-aware calibration","Training-free 8-bit diffusion quantization with time-aware samples","Time-step-aware sampling makes diffusion quantization train-free","First training-free quantization for 512x512 text-to-image diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a fixed, hand-tuned Gaussian time-step distribution (mean 0.4, standard deviation 0.4) yields a calibration set whose activation ranges match what the model will encounter across every step of the production denoising trajectory, and the paper offers no sensitivity analysis or transfer study for this distribution.","fun_headline_variants_meta":{"raw":{"variants":["Time-aware calibration enables training-free diffusion quantization","Quantize diffusion models without retraining via time-aware calibration","Training-free 8-bit diffusion quantization with time-aware samples","Time-step-aware sampling makes diffusion quantization train-free","First training-free quantization for 512x512 text-to-image diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001325,"raw_usage":{"total_tokens":5366,"prompt_tokens":891,"completion_tokens":4475,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":4397}},"tokens_in":507,"tokens_out":4475,"duration_ms":32502,"temperature":1.0,"reasoning_tokens":4397,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:58:26.212999+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the method on the same ImageNet DDPM with the calibration time-step distribution replaced by a uniform distribution over [0, T] or by a Gaussian with a different center (for example, mu = 0.7), keeping all other settings fixed. If the quantized model's FID remains essentially unchanged, the specific hand-tuned distribution is not what carries the method; if the FID degrades sharply, the method's success depends on that distribution in a way the paper does not characterize.","supporting_citations":[{"cited_title":"Permutation invariant graph generation via score-based generative modeling","cited_arxiv_id":null,"evidence_quote":"DDIM is the 250-step sampler that defines the denoising trajectory used in the unconditional experiments."}],"review_version":1}