{"id":"4e5814bd-24f0-4e86-a922-08758d178e0f","arxiv_id":"2412.00144","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A mixed-precision quantization scheme for diffusion models allocates bits per layer using timestep-aggregated network orthogonality (CKA) scores.","lead":"MPQ-Diff assigns different bit precisions to different layers of a diffusion model based on a cross-layer similarity metric, aiming to keep image quality while shrinking the model. The paper reports FID gains over uniform quantization on ImageNet and LSUN, but the most dramatic numbers compare different model sizes and one sFID result gets worse.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing assumption is that ORM (Eq. 5) ranks per-layer sensitivity to quantization error in diffusion U-Nets, yet the paper neither validates this nor fixes the sign of the metric; without a sensitivity-rank check, the bit allocation is unprincipled.","rationale":"The paper's strongest evidence is Table 1, where mixed precision at the same model size improves FID by 2.85 (PTQD) and 2.57 (EfficientDM) on ImageNet. That is real, but it does not identify why the method works. The chosen mechanism, time-aggregated ORM, is the load-bearing part of the central claim. The paper references OMPQ for the claim that orthogonality correlates with quantization robustness, but diffusion U-Nets have skip connections and time-varying activations, and the paper itself contains contradictory statements about whether high ORM or low ORM marks an important layer (Section 3.3 vs. the Introduction and Algorithm 1). Without a direct comparison of ORM and theta to measured per-layer quantization error, the allocation could be following noise, and the same-size improvements could come from the LP's model-size constraint or from the chosen 4-bit and 8-bit base, not from ORM. The abstract's dramatic FID reductions are also presented as comparisons to fixed precision without making clear that the mixed-precision point is roughly 16.5% larger than the 2-bit baseline, so the headline overstates the result. I would reject in current form; a revision that validates ORM against empirical sensitivity, fixes the sign and unit reporting, and addresses the sFID regressions could become viable.","tokens_in":12805,"tokens_out":5938,"duration_ms":52801,"concrete_test":"For LDM-4 on ImageNet with 20 steps, isolate each layer's sensitivity by quantizing only that layer to 4-bit weights while keeping all other layers full-precision, then record the FID or output MSE drop per layer. Compute theta_i from Algorithm 1 and test the Spearman rank correlation between the empirical sensitivity ranks and theta_i; also test the reversed ranking, since the paper contradicts itself on whether high or low ORM means important. If the correlation is weak (|rho| < 0.5), or if allocating bits by reversed theta does not clearly degrade FID relative to allocating by theta at the same model size, the ORM proxy is not validated and the central allocation claim fails.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim requires that the ORM of Eq. 5 track how much each layer's quantization hurts the output. That link is inherited from OMPQ, a classification-network method, and is never tested here. The paper is also internally inconsistent about the sign: Section 3.3 says larger bit-widths should go to layers with stronger orthogonality (lower ORM), while the Introduction says higher ORM indicates higher importance, and Algorithm 1's theta = e^{-rho} is compatible only with low ORM meaning important. Because the LP of Eqs. 7-8 consumes theta as the importance score, an unvalidated or sign-flipped theta makes the entire allocation scheme unfalsifiable in its current form. The empirical section does not close this gap: Table 1's same-size FID gains are positive but accompanied by a large sFID regression for Ours(EfficientDM) on ImageNet (11.46 vs 7.82), and the abstract's headline gains (65.73 to 15.39 and 52.66 to 14.93) are not same-size comparisons; they come from Figure 7, where the mixed-precision point is at a larger model size than the 2-bit fixed baseline that produced 65.73 and 52.66 (e.g., 109 to 127 MB for Churches, a 16.5% increase, while the text elsewhere says 10%). The mechanistic claim, not just the reporting, needs a direct test.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MPQ-Diff, a post-training mixed-precision quantization scheme for diffusion models. It computes a per-layer 'network orthogonality metric' (ORM), defined in Eq. (5) and equivalent to linear CKA, at sampled timesteps; aggregates these scores using an exponential weighting; and feeds them into a linear programming problem inherited from OMPQ to select bit widths per layer under a model-size budget. Experiments on ImageNet, LSUN-Churches, and LSUN-Bedrooms with LDM-4/LDM-8 show same-size FID improvements over PTQD and EfficientDM in most settings, and larger-model-size versions show large FID drops from a fixed 2-bit baseline. The paper also proposes uniform timestep sampling to reduce profiling cost and reports an ablation of sampling fractions.","tokens_in":13111,"tokens_out":6930,"duration_ms":57116,"significance":"If the central assumption were validated, the work would be a useful first mixed-precision post-training quantization scheme for diffusion models, and the same-size FID gains in Table 1 are plausible evidence that non-uniform bit allocation can help. The integration with two existing fixed-precision methods (PTQD and EfficientDM) is a strength. However, the paper does not release code or trained bit allocations, and its significance hinges on the untested claim that ORM ranks per-layer sensitivity to quantization error in diffusion U-Nets; as submitted, the evidence is insufficient to establish that, and the headline numbers are not equal-size comparisons. The paper is a reasonable starting point, but the core mechanism needs direct validation before the contribution can be assessed.","major_comments":[{"comment":"The central claim rests on the assumption that ORM, defined in Eq. (5) and equal to linear CKA, ranks the sensitivity of layers to quantization error. This assumption is inherited from OMPQ, which was designed for classification networks, and is not validated for diffusion U-Nets with skip connections. The paper provides no experiment comparing the ORM ranking with per-layer quantization error, nor an ablation replacing the ORM scores with random or reversed importance scores. Such a sensitivity-rank check or randomized baseline is necessary to establish that the bit allocation, not just the added flexibility of mixed precision, produces the observed FID gains.","section":"§3.3, Eq. (5); Algorithm 1"},{"comment":"The headline improvements (FID 65.73→15.39 and 52.66→14.93) are not same-size comparisons: they compare a fixed 2-bit model at 109 MB with a mixed-precision model at 127 MB (16.5% larger), as correctly stated in Section 5.3. The abstract reports these numbers 'compared to their fixed precision quantization' without mentioning the size difference, and the Introduction says 'only 10% increase in model size,' contradicting the 16.5% figure. The equal-size comparisons in Table 1 are the relevant evidence; the Figure 7 comparisons should be relabeled or removed from the abstract.","section":"Abstract; Introduction (p.1); §5.3, Figure 7"},{"comment":"On ImageNet, Ours(EfficientDM) improves FID from 8.64 to 6.07 but sFID degrades sharply from 7.82 to 11.46. The text in Section 5.1 says sFID does 'not show a similar performance' but does not quantify this regression. Given the paper's claim of 'consistent improvements' over fixed-precision quantization, the sFID degradation on a main benchmark is a substantive caveat that should be analyzed rather than dismissed.","section":"Table 1; §5.1"},{"comment":"The text is internally inconsistent about the sign of the importance score. The Introduction states that a layer with higher aggregated ORM has higher relative importance, while Section 3.3 says larger bit-widths should go to layers with stronger orthogonality (i.e., lower ORM), and Algorithm 1's θ_i = e^{-ρ_i} implements the latter. This contradiction needs to be resolved: either the prose or the algorithm is wrong, and the paper should state explicitly that low ORM corresponds to high importance.","section":"Introduction (p.1); §3.3; Algorithm 1"}],"minor_comments":[{"comment":"The quantization formula uses ⌊·⌋ but calls it 'the round operation'; use round() or define the rounding explicitly.","section":"§3.2"},{"comment":"The legend 'Ours Uniform quantization 2-bit 4-bit' is unclear; distinguish points for fixed 2-bit, fixed 4-bit, and mixed precision with model sizes labeled.","section":"Figure 7"},{"comment":"The text says 'only a slight increase in model size' and elsewhere '10% increase,' while the actual 16.5% figure appears only in one place; the inconsistency should be fixed throughout.","section":"§5.3"},{"comment":"The term 'network orthogonality metric' is potentially confusing because it is a correlation measure; consider renaming to 'cross-layer correlation' or defining the direction of orthogonality clearly.","section":"§3.3"},{"comment":"The paper does not report wall-clock time or memory overhead of the profiling step (including the 50-timestep sampling), which is relevant for the claimed efficiency.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable first step but the missing validation of the core metric and the misleading headline comparisons make it unsuitable for acceptance in current form. I would require the authors to add a per-layer sensitivity analysis and a random/reversed allocation baseline, to correct the sign inconsistency, and to redo all comparisons at equal model size or explicitly report size as a trade-off axis. If the ORM ranking is shown to be no better than random, the method's novelty collapses."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first thing to know: MPQ-Diff is a real, useful extension of OMPQ to diffusion models, and the same-budget FID results in Table 1 are consistently positive — all six rows improve on FID at matched model size. The abstract's headline numbers, however, are not the same-budget story, and the text contradicts itself about how the metric works. Send it to review; expect major revision.\n\nWhat's new: the first mixed-precision quantization framework for diffusion models (as far as the cited literature goes), built on OMPQ's ORM — which is linear CKA — adapted to the time-varying behavior of denoising. The observation that per-layer ORM varies strongly across timesteps is real, and the exponential weighting that emphasizes low-orthogonality, high-importance timesteps is a reasonable aggregation. The uniform timestep sampling is practical, and the sampling ablation shows 1/4 sampling shifts theta values by only ~3%. Credit is given where it's due: OMPQ, BRECQ, Q-Diffusion, PTQD, EfficientDM are all cited cleanly, with no self-citation issue. The evaluation is also clean in one respect — the allocation is computed from activations and evaluated on held-out FID, not fitted to the test metric.\n\nSoft spots, in proportion. The biggest is the framing: the abstract and contribution list sell 65.73→15.39 as an improvement over fixed precision, which reads as same-budget. It isn't — that's the 2-bit model at 109 MB versus mixed precision at 127 MB, 16.5% bigger, and Section 5.3 says 16.5% while the contributions say 10%. Same-budget gains are real but modest in two rows, and sFID degrades on both ImageNet rows (EfficientDM: 7.82→11.46); 'small room for improvement' doesn't explain making sFID worse. Second, the mechanistic claim: that ORM tracks quantization sensitivity is inherited from OMPQ without validation here. A direct test — compare the ORM ranking to per-layer quantization-error impact — is missing; without it the ORM-specific story is unproven, though the uniform-baseline comparison does test the scheme as a whole, so calling it 'unfalsifiable' is too strong. The stress-test note also overstates the sign problem: Algorithm 1 and Section 4.1 consistently give more bits to low-ORM (more orthogonal) layers; only the Introduction says the opposite, in one sentence. That's a drafting error, not a load-bearing ambiguity. Third, no error bars, no code, and the ablation measures theta deviation, not FID, so the quality impact of sampling 1/4 of the timesteps is unknown.\n\nWho this is for: people working on diffusion-model compression. The timestep-aggregation idea is the worthwhile part. Off-the-shelf publication as written? No — the headline claim is misleading and the mechanism needs the missing validation. But it deserves serious refereeing with the expectation of major revision.","headline":"Timestep-aware ORM bit allocation for diffusion models: same-budget FID wins in all six Table 1 rows, but the headline numbers are a different-size comparison and the ORM-sensitivity link is never tested; worth a serious referee.","tokens_in":13690,"tokens_out":13500,"would_cite":true,"duration_ms":92980,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MPQ-Diff: per-layer bit allocation beats uniform quantization for diffusion models","keywords":["mixed precision quantization","diffusion models","post-training quantization","network orthogonality","FID","timestep aggregation","linear programming","U-Net"],"falsifier":"Compare MPQ-Diff's bit allocations against allocations chosen by directly measured per-layer output error (e.g., reconstruction MSE or Hessian trace) on the same LDM-4 model; if ORM-ranked layers do not match error-ranked layers, the FID gains would be shown to come from the LPP flexibility rather than the metric. A simpler control: shuffle the ORM scores before solving the LPP; if FID stays similar, ORM is not the driver.","tokens_in":12559,"feed_emoji":"📉","tokens_out":4020,"duration_ms":33101,"temperature":0.7,"pith_summary":"Diffusion models are slow because every denoising step runs a large U-Net, and low-bit quantization speeds them up but degrades image quality. MPQ-Diff asks whether giving different layers different bit-widths—rather than one uniform width—can recover that quality at the same memory budget. The paper's answer is yes. It treats a layer's average cross-correlation with all other layers, called the network orthogonality metric (equivalent to linear CKA), as a proxy for how much that layer matters, aggregates this across sampled denoising timesteps, and solves a small linear program to allocate 2–8 bit weights per layer. On ImageNet and LSUN benchmarks, the resulting mixed-precision models improve FID by 1.6–2.85 points at the same model size, and by about 50 points when a small extra memory allowance is granted.","feed_headline":"Per-layer bitwidths beat uniform quantization for diffusion models","feed_subtitle":"At equal size, FID drops from 9.83 to 6.98 on ImageNet and from 65.73 to 15.39 on LSUN-Churches.","key_machinery":"The load-bearing object is the ORthogonality Metric (ORM), the Frobenius inner product of two layers' activations normalized by their norms, which equals the linear Centered Kernel Alignment (CKA) similarity. The paper aggregates ORM over time by zero-normalizing each layer's row sum $\\gamma_i^{(t)}$ across timesteps, exponentiating the negative z-score to form weights $\\theta_i$, and feeding these into a linear program whose objective (Equation 7) assigns higher bit-widths to more independent layers under a model-size constraint (Equation 8), with uniform sampling of timesteps (Section 4.3) to control profiling cost.","core_discovery":"The paper claims that a per-layer mixed-precision bit allocation, chosen by maximizing a time-aggregated orthogonality objective under a memory constraint, consistently outperforms uniform 4-bit quantization of diffusion models at the same or slightly larger model size. The central quantity is the ORM of Equation 5, which gives a value in $[0,1]$ measuring how much two layers' outputs correlate; a layer's weight is its summed ORM against all other layers, normalized and exponentially transformed so that high-importance (low-correlation) layers receive more bits. The allocation is obtained by solving a linear programming problem, and to keep profiling costs low the ORM is computed on a uniformly sampled subset of denoising steps rather than all of them.","pith_inferences":["Because ORM equals linear CKA, the method implicitly assumes representational similarity to other layers tracks quantizability; this assumption is untested for U-Nets with skip connections, so a direct comparison against per-layer reconstruction-error or Hessian-based sensitivity scores would be the natural next experiment.","The same time-aggregation and linear-programming recipe could transfer to other iterative generative models that reuse a single network across steps, such as flow matching or consistency models.","The stability of the allocation under uniform sampling suggests that a timestep-dependent allocation, where bit-widths change mid-sampling, might yield further quality gains at the same memory cost.","If ORM rankings were shuffled before solving the LPP and FID stayed similar, that would indicate the gains come from the optimization flexibility rather than the orthogonality proxy itself; this is a testable control experiment."],"forward_implications":["Because MPQ-Diff wraps any fixed-precision post-training quantization method, both PTQD and EfficientDM can inherit the FID improvement without additional training.","The bit-allocation procedure is one-shot: a single forward pass over a batch gives the ORM matrices, then the linear program runs in negligible time, so different memory budgets can be explored cheaply.","The FID-versus-model-size curve becomes a smooth, tunable knob rather than a few discrete uniform-width points, as shown by the ablation in Figure 7.","With only a ~16% increase in model size over 2-bit uniform quantization, mixed precision reaches FID 15.39 on LSUN-Churches, while 4-bit uniform needs 29% more memory to reach FID 9.75.","Sampling one quarter of the denoising timesteps keeps the allocation within about 3% of using all timesteps, so the method scales to models with hundreds of sampling steps."],"supporting_citations":[{"why":"Supplies the ORM metric and the linear-programming objective (Equations 7–8) that MPQ-Diff adapts to diffusion models.","marker":"[19]"},{"why":"PTQD is one of the two fixed-precision quantization baselines that MPQ-Diff is built on and compared against.","marker":"[8]"},{"why":"EfficientDM is the other fixed-precision quantization baseline used to demonstrate the method's compatibility and gains.","marker":"[9]"},{"why":"Q-Diffusion provides the observation that activations of a layer are similar across neighboring timesteps, supporting the uniform sampling strategy.","marker":"[15]"},{"why":"Establishes CKA, which the paper notes is identical to the ORM metric, tying the metric to a known representation-similarity measure.","marker":"[13]"},{"why":"Latent diffusion models are the architecture evaluated (LDM-4, LDM-8) on ImageNet and LSUN datasets.","marker":"[23]"},{"why":"BRECQ is the prior mixed-precision quantization framework for classification networks that motivates the mixed-precision approach and provides background for block-wise sensitivity.","marker":"[16]"}],"fun_headline_variants":["Mixed precision per layer slashes diffusion FID","Adaptive bit-widths for diffusion models improve FID","MPQ-Diff: per-layer bits boost diffusion quality","Orthogonality-guided bit allocation for diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The allocation is only as good as the orthogonality metric's ability to rank layers by how much quantization hurts them; the paper adopts ORM from classification-network work and does not validate that it tracks actual per-layer quantization error in U-Nets with skip connections.","fun_headline_variants_meta":{"raw":{"variants":["Mixed precision per layer slashes diffusion FID","Adaptive bit-widths for diffusion models improve FID","MPQ-Diff: per-layer bits boost diffusion quality","Orthogonality-guided bit allocation for diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0007,"raw_usage":{"total_tokens":3139,"prompt_tokens":905,"completion_tokens":2234,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":2172}},"tokens_in":521,"tokens_out":2234,"duration_ms":15413,"temperature":1.0,"reasoning_tokens":2172,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:16:27.064751+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare MPQ-Diff's bit allocations against allocations chosen by directly measured per-layer output error (e.g., reconstruction MSE or Hessian trace) on the same LDM-4 model; if ORM-ranked layers do not match error-ranked layers, the FID gains would be shown to come from the LPP flexibility rather than the metric. A simpler control: shuffle the ORM scores before solving the LPP; if FID stays similar, ORM is not the driver.","supporting_citations":[{"cited_title":"Ompq: Orthogonal mixed precision quantization, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the ORM metric and the linear-programming objective (Equations 7–8) that MPQ-Diff adapts to diffusion models."},{"cited_title":"Ptqd: Accurate post-training quantization for diffusion models, 2023","cited_arxiv_id":null,"evidence_quote":"PTQD is one of the two fixed-precision quantization baselines that MPQ-Diff is built on and compared against."},{"cited_title":"E fficientdm: E fficient quantization-aware fine- tuning of low-bit diffusion models, 2024","cited_arxiv_id":null,"evidence_quote":"EfficientDM is the other fixed-precision quantization baseline used to demonstrate the method's compatibility and gains."},{"cited_title":"Q-diffusion: Quantizing diffusion models, 2023","cited_arxiv_id":null,"evidence_quote":"Q-Diffusion provides the observation that activations of a layer are similar across neighboring timesteps, supporting the uniform sampling strategy."},{"cited_title":"Similarity of neural network representa- tions revisited","cited_arxiv_id":null,"evidence_quote":"Establishes CKA, which the paper notes is identical to the ORM metric, tying the metric to a known representation-similarity measure."},{"cited_title":"High-resolution image syn- thesis with latent diffusion models, 2022","cited_arxiv_id":null,"evidence_quote":"Latent diffusion models are the architecture evaluated (LDM-4, LDM-8) on ImageNet and LSUN datasets."},{"cited_title":"Brecq: Pushing the limit of post-training quantization by block reconstruc- tion, 2021","cited_arxiv_id":null,"evidence_quote":"BRECQ is the prior mixed-precision quantization framework for classification networks that motivates the mixed-precision approach and provides background for block-wise sensitivity."}],"review_version":1}