{"id":"94877ecd-8be5-4222-b921-2962772ba88e","arxiv_id":"2412.11549","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MPQ-DM combines kurtosis-based intra-layer mixed-precision weight quantization with time-smoothed relation distillation to keep diffusion models accurate at 2 to 4 bit widths.","lead":"This paper presents a method to shrink diffusion image generators to very low bit widths while keeping output quality. It assigns extra bits to weight channels with extreme outlier values and uses a time-smoothed similarity distillation to stabilize training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The OMQ bit-allocation in Eq. (6) is not actually optimized: the paper substitutes a kurtosis ranking plus a coarse group-count search (k=cout/10, at most five groups), and no ablation validates this against the stated output-error objective; since OMQ carries most of the reported FID gain, this…","rationale":"Good-faith reading: the paper is a legitimate quantization study. The OMQ and TRD ideas are clearly motivated, the experiments span three LDM settings plus Stable Diffusion, and the ablations in Tables 5-7 support the qualitative value of mixed precision, kurtosis-based selection, and relation distillation over direct L2 alignment. The central quantitative claim, however, rests on OMQ being a real optimization rather than a coarse heuristic. The full text of Section 3.2 defines Eq. (6) as an output-error minimization over per-channel bit assignments, then immediately restricts the search to group counts after kurtosis sorting, with k=cout/10 and at most five groups. This is presented as an acceleration, but no experiment measures the gap between this surrogate and the stated objective. Table 6 compares selection functions (random, head-tail, kurtosis) but holds the coarse grid fixed, so it cannot validate the grid. Since Table 5 attributes most of the W3A4 improvement to OMQ (FID from 11.02 to 6.91), a failure of the heuristic would materially shrink the reported gains. The abstract's 58% versus 43% discrepancy is real but secondary; it is a reporting issue, not a validity issue. The proposed test is feasible with the released pipeline and would distinguish a robust method from one that happens to work on the chosen grid. Even if the test shows fragility, the appropriate outcome is CONDITIONAL rather than REJECT, because the gains over baseline are large and the search can be refined.","tokens_in":15978,"tokens_out":10877,"duration_ms":93306,"concrete_test":"On LDM-4 ImageNet 256x256 W3A4 with the same LoRA/TRD setup, compare the published OMQ (k=cout/10, five group counts) against (a) a per-channel greedy swap that, starting from uniform N-bit, exchanges the channel whose +1-bit reduces Eq. (6) error most with the channel whose -1-bit increases it least, and (b) finer group sizes k=cout/20 and k=cout/5. Report FID/sFID on 50k samples and the resulting bit-allocation maps. If the finer/greedy search changes FID by more than ~1.5 or selects substantially different channels, the kurtosis-group-count heuristic is not approximating Eq. (6) and the core OMQ claim needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the claim that the OMQ search in Section 3.2 solves or approximates Eq. (6). Eq. (6) minimizes output quantization error over channel bit assignments, but the implemented procedure only ranks channels by kurtosis and then searches over the number of promoted/demoted groups, with group size k=cout/10 and search region [0, cout//k/2] (at most five groups). This restricts candidate assignments to counts that are multiples of 10% of channels and never re-evaluates whether the kurtosis ranking matches the output-error objective, which also depends on activation magnitudes. The paper's own text admits this is an acceleration heuristic, and Table 5 attributes most of the W3A4 FID improvement to OMQ (FID 11.02 to 6.91), so if the heuristic is fragile the central FID gains shrink. Table 6 shows kurtosis beats random/head-tail selection, but it does not compare to a per-channel or finer-grained search on the Eq. (6) objective, so the approximation is unvalidated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MPQ-DM, a mixed-precision quantization method for diffusion models at extremely low bit-widths (2-4 bit). The method has two main components: (1) Outlier-Driven Mixed Quantization (OMQ), which uses kurtosis to identify outlier-heavy weight channels within a layer and assigns them higher bit-widths while demoting the same number of low-kurtosis channels to lower bit-widths, keeping the average bit-width constant; and (2) Time-Smoothed Relation Distillation (TRD), which distills the quantized model from the full-precision model using smoothed features over consecutive time steps and a cosine-similarity relation space rather than direct numerical alignment. The method is evaluated on ImageNet 256x256 with LDM-4, LSUN-Bedrooms with LDM-4, LSUN-Churches with LDM-8, and Stable Diffusion v1.4 on COCO captions, reporting FID, sFID, IS, Precision, and CLIP Score. The authors report consistent improvements over EfficientDM and PTQ baselines, including large gains at W2A4, and release code on GitHub.","tokens_in":16199,"tokens_out":8842,"duration_ms":72733,"significance":"If the reported results hold, MPQ-DM would be a meaningful step toward practical low-bit deployment of diffusion models: it is one of the few methods that maintains non-collapsed generation quality at W2A4, and it combines two relatively orthogonal ideas (intra-layer mixed precision driven by channel kurtosis and time-smoothed relation distillation). The paper is transparent in providing component ablations (Tables 5-8), studies of the kurtosis selection function, and a public code repository, which are strengths. However, the central algorithmic contribution, OMQ, is currently presented as solving a well-defined output-error minimization while the actual implementation is a kurtosis-based heuristic whose equivalence to the stated objective is not validated. The abstract also overstates the headline number. These issues, together with the absence of variance reporting for several close comparisons, need to be resolved before the empirical claims can be fully accepted.","major_comments":[{"comment":"Eq. (6) defines the bit-allocation problem as minimizing an output-quantization-error objective, but the implemented procedure does not optimize this objective: it ranks channels by kurtosis and then searches only over the number of promoted/demoted groups with group size k=cout/10 and search region [0, cout//k/2], i.e., at most five candidates. This assumes that the kurtosis ordering of weight channels correctly predicts the output-error reduction, although the objective in Eq. (6) also depends on activation magnitudes. Table 6 validates kurtosis against random and head/tail selection, but it does not compare against the Eq. (6) objective itself or against a finer-grained or exhaustive search on a subset of layers. Since Table 5 attributes most of the W3A4 FID improvement to OMQ (FID decreases from 11.02 to 6.91), the central empirical claim rests on this unvalidated heuristic. Please provide evidence that the kurtosis group search approximates the Eq. (6) minimizer well, for example by comparing objective values and FID for a per-channel or finer-grained search on a few layers, or by reporting the gap between the heuristic assignment and a brute-force assignment for small layers.","section":"Section 3.2, Eq. (6)"},{"comment":"The abstract states that \"MPQ-DM achieves a 58% FID decrease under W2A4 setting compared with baseline,\" but Table 1 shows that the 58% decrease (FID 64.45 to 27.11) is achieved by MPQ-DM+, which uses additional channel bit allocation and a larger model size (101.6 MB). The main MPQ-DM at 96.8 MB yields a 43% decrease (FID 64.45 to 36.59). Section 4.2 correctly reports the 27.86 decrease for MPQ-DM. The abstract should attribute the 58% figure to MPQ-DM+ or use the 43% figure for MPQ-DM, to avoid overstating the main method's result.","section":"Abstract / Section 4.2, Table 1"},{"comment":"The paper reports no variance or multiple-seed results, while several claimed improvements are small. Examples include LSUN-Churches W3A6 (EfficientDM 9.53 vs MPQ-DM 9.28 in Table 3) and the distillation metric study in Table 7, where relation distillation improves FID from 9.12 to 9.10 on a 10k-sample evaluation. These differences may be within sampling noise, particularly for the 10k-sample evaluations in Tables 6-7. Please report mean and standard deviation over at least three independent fine-tuning runs for the main comparisons, or otherwise demonstrate that the observed margins are statistically significant.","section":"Section 4.2, Tables 1-3, 6-7"}],"minor_comments":[{"comment":"The abbreviation is introduced as \"Outlier-Driven Mixed Quantization (OMD)\" in the introduction but the rest of the paper uses \"OMQ\"; Table 5 also uses \"+TSD\" instead of \"+TRD\". Please unify the abbreviations throughout.","section":"Section 3.2 and Table 5"},{"comment":"The expression \"||X_f W_f^T, Q(\\hat{X}_f) Q(\\hat{W}_f | [c_1, \\cdots, c_n])^T||_2\" appears to be missing a minus sign between the two terms; the intended objective is presumably the norm of the difference between the full-precision output and the quantized output.","section":"Eq. (6)"},{"comment":"Eq. (11) uses \"Ltarget\", but the task loss was defined as \"Ltask\" in Eq. (7). Please use a consistent symbol and define it before first use.","section":"Eq. (11)"},{"comment":"The statement that MPQ-DM+ adds only \"0.6% increase in model size compared with FP model\" is inconsistent with Table 1: MPQ-DM+ is 101.6 MB versus 96.8 MB for MPQ-DM, and 101.6 MB relative to the full-precision size 1529.7 MB is about 0.3%. Please clarify what the reported percentage refers to.","section":"Section 4.1 and Table 1"},{"comment":"The first row \"PTQD\" is misaligned and its values do not match Table 1 (for PTQ-D at W3A4, Table 1 lists IS 10.86, FID 286.57, sFID 273.16, Precision 0.02, whereas Table 5 lists 237.16 and 0.05). Please correct the formatting and the numbers.","section":"Table 5"},{"comment":"Eq. (9) sums t=0 to N, which involves N+1 time steps, while the text says \"N consecutive steps\". Make the indexing consistent, e.g., define N as the number of smoothing steps minus one or sum t=1 to N.","section":"Section 3.3, Eq. (9)"},{"comment":"Figure 2 contains garbled text (\"??????|| ??\") that should be replaced with the proper equation rendering so the figure is readable.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a vision/compression venue and addresses a timely problem. The main risk is that the OMQ heuristic is the primary driver of the reported quality gains but is not validated against the stated optimization objective; asking for such validation is essential before acceptance. The abstract overstates the main result, and the lack of variance reporting makes some close comparisons difficult to interpret. I recommend major revision rather than rejection because the central direction is promising and the issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper is worth reading. It reports large, consistent gains over EfficientDM and HAWQ-V3 on extremely low-bit diffusion quantization (W2A4, W3A4), across three LDM benchmarks and Stable Diffusion. The core idea — allocating more bits to outlier-heavy weight channels within a layer using kurtosis, and distilling smoothed cross-time feature relations — is simple and, on the evidence, effective. The ablations are decent: Table 6 shows kurtosis beats random/head-tail selection, Table 5 shows each component helps, and the comparison set is reasonable.\n\nThe main soft spot is the gap between the stated optimization and what is implemented. Eq. (6) presents OMQ as a minimization of output quantization error over channel bit assignments. What they actually do is rank channels by kurtosis, then search over a small count of promoted/demoted groups (k=cout/10, region [0, cout//k/2], so at most five candidates). That is a heuristic, not the arg min. The paper does disclose the acceleration step, but it never validates that the kurtosis-ranking plus coarse count search approximates Eq. (6), and since Table 5 attributes most of the FID improvement to OMQ (11.02 to 6.91), this is the load-bearing component. I do not think the empirical results collapse — the kurtosis ablation is real — but the framing oversells the principled nature of the bit allocation.\n\nTwo smaller issues. The abstract's headline '58% FID decrease' is for MPQ-DM+, the variant with an extra 10% channels promoted to 2-bit, which costs an extra ~4.8 MB; the main method's number is 43%. That is a meaningful difference and should be stated in the abstract. Second, no variance or seed counts are reported anywhere, so it's hard to know how stable the W3A4 'better than FP' result is. Also, the smoothing factor in Eq. (5) is essentially SmoothQuant's scaling, and while related work is cited, SmoothQuant itself is not; minor.\n\nWho this is for: anyone working on quantized diffusion or efficient generation. It deserves a serious referee; the empirical contribution is substantial and the method is reproducible enough to check. My recommendation: send it out, and ask the reviewers to push on the OMQ search validation and the abstract's variant attribution. The authors have a good paper; they just need to be more honest about what is optimized and what is heuristically chosen.","headline":"Solid empirical paper on low-bit diffusion quantization with an overstated optimization claim and a headline number that belongs to a different variant.","tokens_in":16804,"tokens_out":3320,"would_cite":true,"duration_ms":29953,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MPQ-DM keeps 2–4-bit quantized diffusion models at full-precision generation quality by shifting bits toward outlier-heavy weight channels and distilling time-smoothed feature relations.","keywords":["diffusion models","model quantization","mixed-precision quantization","extremely low-bit quantization","kurtosis","outlier channels","relation distillation","image generation"],"falsifier":"On a small diffusion layer, exhaustively evaluate the output reconstruction error of every equal-count bit promotion/demotion assignment and compare the error-minimizing set with the highest-kurtosis set. If even one layer exists where a low-kurtosis channel contributes more output error than a high-kurtosis channel, the ranking assumption behind OMQ is violated; equivalently, replacing the kurtosis ranking with measured per-channel quantization error in the search and testing whether MPQ-DM's FID gains persist would settle the claim.","tokens_in":15690,"feed_emoji":"🎨","tokens_out":11768,"duration_ms":89839,"temperature":0.7,"pith_summary":"The paper sets out to make extremely low-bit (2–4 bit) quantization of diffusion models practically usable, arguing that the two main obstacles are outlier-heavy weight channels that are destroyed when all channels share one bit-width, and discretized activations that vary unstably across denoising time steps. Its method, MPQ-DM, addresses both: an intra-layer mixed-precision scheme uses kurtosis to find the most outlier-heavy channels, gives them an extra bit, and takes a bit from the same number of calmest channels so the average bit-width is unchanged; a time-smoothed relation distillation matches the quantized model's feature-similarity distributions to the full-precision model's over consecutive time steps. If the claims hold, 2–4-bit diffusion models become viable on constrained hardware without giving up image quality: on ImageNet LDM-4, W3A4 MPQ-DM reaches FID 6.72 versus 11.28 for the full-precision model, and W2A4 improves from 64.45 (EfficientDM) to 36.59.","feed_headline":"Outlier-aware bit shifts cut W2A4 FID by 43%","feed_subtitle":"MPQ-DM redistributes bits within each layer and distills time-smoothed features, beating EfficientDM at 2-bit weights.","key_machinery":"The carrying mechanism is the intra-layer mixed-precision assignment Q̂(Ŵf |[c1,...,cn]) of Eq. (6): a per-channel bit-width vector in which an equal number of channels move to N+1 and N-1 bits, preserving the layer's average bit-width. The paper does not solve the full optimization; it uses kurtosis κ as a saliency ranking and searches over the number of swapped channels in a restricted region, which is what makes the assignment tractable. The second mechanism is the time-smoothed relation distillation loss of Eq. (10), L_dis = Σ_i D_kl(S^i_f || S^i_q), where S^i are cosine-similarity distributions of the time-smoothed features from Eq. (9); it transfers discrete and continuous features into one similarity space so that numerical mismatch does not turn distillation into negative optimization.","core_discovery":"On its own terms, the paper's central discovery is that the failure of low-bit diffusion quantization is not primarily a capacity problem but a distribution-mismatch problem, and that both components of the mismatch can be corrected within a fixed average bit budget. For weights, MPQ-DM pre-scales each channel by a smooth factor, ranks channels by kurtosis κ, and assigns an extra bit to the most outlier-heavy channels while demoting the same number of least-outlier channels, selecting how many to swap by minimizing the layer output error in a restricted search; this keeps the layer average at N bits. For activations and the denoising trajectory, it replaces direct numerical alignment of discrete quantized features with continuous full-precision features by a relation distillation: features from N consecutive time steps are summed, reshaped, and converted into cosine-similarity distributions, which are matched with KL divergence. In the paper's experiments the combination makes W3A4 LDM-4 FID lower than the full-precision model on ImageNet (6.72 against 11.28) and brings W2A4 FID to 36.59 versus 64.45 for the EfficientDM baseline, with similar or larger gains on LSUN datasets and Stable Diffusion text-to-image.","pith_inferences":["We infer that the kurtosis ranking is one of several possible saliency measures; replacing it with a measured per-channel output-error or Hessian-based sensitivity within the same search could be more reliable on layers where long tails are not the only source of quantization loss.","We infer that the equal +1/-1 swap constraint could be relaxed: allowing asymmetric promotion/demotion counts or occasional 2-bit jumps would trade a small model-size change for potentially larger FID gains, since the MPQ-DM+ results already show that relaxing the zero-overhead constraint helps.","We infer that because the relation distillation operates on cosine-similarity distributions of features, the same smoothing-and-relation recipe might transfer to other iterative generative models with discrete latents, such as discrete diffusion or autoregressive image models, where direct feature alignment is equally mismatched.","We infer that the time-smoothed target uses a simple sum of N consecutive features; an exponential moving average or variance-weighted smoothing could control which time steps influence the distillation most, a testable variant the paper does not consider."],"forward_implications":["At W3A4, a fully quantized LDM-4 on ImageNet scores FID 6.72, below the full-precision 11.28, so generation quality need not be traded away at this bit budget.","At W2A4, where existing PTQ diffusion methods collapse (FID above 200 on ImageNet), MPQ-DM keeps FID at 36.59, making 2-bit weight deployment a realistic target for latent diffusion.","The bit redistribution preserves the average bit-width per layer, so the gains come with essentially no model-size increase; the MPQ-DM+ variant adds only 0.6% size while cutting W2A4 FID to 27.11.","The gains transfer across tasks: LSUN-Bedrooms and LSUN-Churches show consistent FID and sFID improvements (e.g., sFID 21.58 at W2A4 on Bedrooms), and Stable Diffusion v1.4 text-to-image CLIP score rises by over 0.3 at W3A4 and W2A6, with MPQ-DM+ adding 1.79 at W2A6."],"supporting_citations":[{"why":"Supplies the EfficientDM quantization-aware low-rank fine-tuning baseline and the primary comparison whose W2A4 FID MPQ-DM cuts from 64.45 to 36.59.","marker":"He et al. 2023"},{"why":"Provides PTQ-D, a post-training diffusion quantization baseline that collapses at 2-bit activations, framing the failure mode MPQ-DM targets.","marker":"He et al. 2024"},{"why":"Gives TFMQ-DM, a temporal-feature-aware diffusion quantization baseline and comparison.","marker":"Huang et al. 2024"},{"why":"Gives QuEST, a low-bit diffusion quantization method that also keeps the final projection layer at 8-bit, a design MPQ-DM retains.","marker":"Wang et al. 2024b"},{"why":"Supplies HAWQ-v3, the layer-wise mixed-precision baseline MPQ-DM outperforms, motivating intra-layer allocation.","marker":"Yao et al. 2021"},{"why":"Defines kurtosis as tailedness, the statistical basis for using κ to rank outlier-salient channels.","marker":"DeCarlo 1997"},{"why":"Formulates the DDPM Markov denoising process whose consecutive time-step features TRD smooths and distills.","marker":"Ho, Jain, and Abbeel 2020"},{"why":"Supports the claim that heavily discretized intermediate features are numerically unstable and resist direct alignment.","marker":"Martinez et al. 2020"},{"why":"Introduces latent diffusion models and Stable Diffusion, the architectures on which MPQ-DM is evaluated.","marker":"Rombach et al. 2022"}],"fun_headline_variants":["Outlier-driven bit reassignment cuts low-bit diffusion FID","Time-smoothed distillation improves quantized diffusion models","Mixed-precision per-layer bits fix 2-bit diffusion collapse","Kurtosis-guided mixed quantization beats low-bit diffusion","Relation distillation bridges full and low-bit diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the channels with the highest kurtosis are the channels whose quantization bit-width matters most, so that promoting the most outlier-heavy channels and demoting the most ordinary ones in equal numbers, with a short search over how many to swap, lands close to the bit assignment that truly minimizes output error; if a dataset or layer violates this ranking, the OMQ gains can shrink or reverse.","fun_headline_variants_meta":{"raw":{"variants":["Outlier-driven bit reassignment cuts low-bit diffusion FID","Time-smoothed distillation improves quantized diffusion models","Mixed-precision per-layer bits fix 2-bit diffusion collapse","Kurtosis-guided mixed quantization beats low-bit diffusion","Relation distillation bridges full and low-bit diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000367,"raw_usage":{"total_tokens":2056,"prompt_tokens":1112,"completion_tokens":944,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":728,"completion_tokens_details":{"reasoning_tokens":866}},"tokens_in":728,"tokens_out":944,"duration_ms":8813,"temperature":1.0,"reasoning_tokens":866,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:49:18.421843+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small diffusion layer, exhaustively evaluate the output reconstruction error of every equal-count bit promotion/demotion assignment and compare the error-minimizing set with the highest-kurtosis set. If even one layer exists where a low-kurtosis channel contributes more output error than a high-kurtosis channel, the ranking assumption behind OMQ is violated; equivalently, replacing the kurtosis ranking with measured per-channel quantization error in the search and testing whether MPQ-DM's FID gains persist would settle the claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives TFMQ-DM, a temporal-feature-aware diffusion quantization baseline and comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies HAWQ-v3, the layer-wise mixed-precision baseline MPQ-DM outperforms, motivating intra-layer allocation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines kurtosis as tailedness, the statistical basis for using κ to rank outlier-salient channels."}],"review_version":1}