Pith. sign in

REVIEW 3 major objections 7 minor 56 references

MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MPQ-DM keeps 2–4-bit quantized diffusion models at full-precision generation quality by shifting bits toward outlier-heavy weight channels and distilling time-smoothed feature relations.

desk verdict Solid empirical paper on low-bit diffusion quantization with an overstated optimization claim and a headline number that belongs to a different variant. read the letter →

arxiv 2412.11549 v1 pith:HKUEIFF2 submitted 2024-12-16 cs.CV

Weilun Feng , Haotong Qin , Chuanguang Yang , Zhulin An , Libo Huang , Boyu Diao , Fei Wang , Renshuai Tao
show 2 more authors
Yongjun Xu Michele Magno
This is my paper · ORCID
classification cs.CV
keywords diffusionmodelsmodelquantizationmixed-precisionextremelylow-bitkurtosisoutlierchannelsrelationdistillationimagegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to make extremely low-bit (2–4 bit) quantization of diffusion models practically usable, arguing that the two main obstacles are outlier-heavy weight channels that are destroyed when all channels share one bit-width, and discretized activations that vary unstably across denoising time steps. Its method, MPQ-DM, addresses both: an intra-layer mixed-precision scheme uses kurtosis to find the most outlier-heavy channels, gives them an extra bit, and takes a bit from the same number of calmest channels so the average bit-width is unchanged; a time-smoothed relation distillation matches the quantized model's feature-similarity distributions to the full-precision model's over consecutive time steps. If the claims hold, 2–4-bit diffusion models become viable on constrained hardware without giving up image quality: on ImageNet LDM-4, W3A4 MPQ-DM reaches FID 6.72 versus 11.28 for the full-precision model, and W2A4 improves from 64.45 (EfficientDM) to 36.59.

What carries the argument

The carrying mechanism is the intra-layer mixed-precision assignment Q̂(Ŵf |[c1,...,cn]) of Eq. (6): a per-channel bit-width vector in which an equal number of channels move to N+1 and N-1 bits, preserving the layer's average bit-width. The paper does not solve the full optimization; it uses kurtosis κ as a saliency ranking and searches over the number of swapped channels in a restricted region, which is what makes the assignment tractable. The second mechanism is the time-smoothed relation distillation loss of Eq. (10), L_dis = Σ_i D_kl(S^i_f || S^i_q), where S^i are cosine-similarity distributions of the time-smoothed features from Eq. (9); it transfers discrete and continuous features into one similarity space so that numerical mismatch does not turn distillation into negative optimization.

What would settle it

On a small diffusion layer, exhaustively evaluate the output reconstruction error of every equal-count bit promotion/demotion assignment and compare the error-minimizing set with the highest-kurtosis set. If even one layer exists where a low-kurtosis channel contributes more output error than a high-kurtosis channel, the ranking assumption behind OMQ is violated; equivalently, replacing the kurtosis ranking with measured per-channel quantization error in the search and testing whether MPQ-DM's FID gains persist would settle the claim.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the failure of low-bit diffusion quantization is not primarily a capacity problem but a distribution-mismatch problem, and that both components of the mismatch can be corrected within a fixed average bit budget. For weights, MPQ-DM pre-scales each channel by a smooth factor, ranks channels by kurtosis κ, and assigns an extra bit to the most outlier-heavy channels while demoting the same number of least-outlier channels, selecting how many to swap by minimizing the layer output error in a restricted search; this keeps the layer average at N bits. For activations and the denoising trajectory, it replaces direct numerical alignment of discrete quantized features with continuous full-precision features by a relation distillation: features from N consecutive time steps are summed, reshaped, and converted into cosine-similarity distributions, which are matched with KL divergence. In the paper's experiments the combination makes W3A4 LDM-4 FID lower than the full-precision model on ImageNet (6.72 against 11.28) and brings W2A4 FID to 36.59 versus 64.45 for the EfficientDM baseline, with similar or larger gains on LSUN datasets and Stable Diffusion text-to-image.

Load-bearing premise

The load-bearing premise is that the channels with the highest kurtosis are the channels whose quantization bit-width matters most, so that promoting the most outlier-heavy channels and demoting the most ordinary ones in equal numbers, with a short search over how many to swap, lands close to the bit assignment that truly minimizes output error; if a dataset or layer violates this ranking, the OMQ gains can shrink or reverse.

Editorial extensions

If this is right

  • At W3A4, a fully quantized LDM-4 on ImageNet scores FID 6.72, below the full-precision 11.28, so generation quality need not be traded away at this bit budget.
  • At W2A4, where existing PTQ diffusion methods collapse (FID above 200 on ImageNet), MPQ-DM keeps FID at 36.59, making 2-bit weight deployment a realistic target for latent diffusion.
  • The bit redistribution preserves the average bit-width per layer, so the gains come with essentially no model-size increase; the MPQ-DM+ variant adds only 0.6% size while cutting W2A4 FID to 27.11.
  • The gains transfer across tasks: LSUN-Bedrooms and LSUN-Churches show consistent FID and sFID improvements (e.g., sFID 21.58 at W2A4 on Bedrooms), and Stable Diffusion v1.4 text-to-image CLIP score rises by over 0.3 at W3A4 and W2A6, with MPQ-DM+ adding 1.79 at W2A6.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the kurtosis ranking is one of several possible saliency measures; replacing it with a measured per-channel output-error or Hessian-based sensitivity within the same search could be more reliable on layers where long tails are not the only source of quantization loss.
  • We infer that the equal +1/-1 swap constraint could be relaxed: allowing asymmetric promotion/demotion counts or occasional 2-bit jumps would trade a small model-size change for potentially larger FID gains, since the MPQ-DM+ results already show that relaxing the zero-overhead constraint helps.
  • We infer that because the relation distillation operates on cosine-similarity distributions of features, the same smoothing-and-relation recipe might transfer to other iterative generative models with discrete latents, such as discrete diffusion or autoregressive image models, where direct feature alignment is equally mismatched.
  • We infer that the time-smoothed target uses a simple sum of N consecutive features; an exponential moving average or variance-weighted smoothing could control which time steps influence the distillation most, a testable variant the paper does not consider.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes MPQ-DM, a mixed-precision quantization method for diffusion models at extremely low bit-widths (2-4 bit). The method has two main components: (1) Outlier-Driven Mixed Quantization (OMQ), which uses kurtosis to identify outlier-heavy weight channels within a layer and assigns them higher bit-widths while demoting the same number of low-kurtosis channels to lower bit-widths, keeping the average bit-width constant; and (2) Time-Smoothed Relation Distillation (TRD), which distills the quantized model from the full-precision model using smoothed features over consecutive time steps and a cosine-similarity relation space rather than direct numerical alignment. The method is evaluated on ImageNet 256x256 with LDM-4, LSUN-Bedrooms with LDM-4, LSUN-Churches with LDM-8, and Stable Diffusion v1.4 on COCO captions, reporting FID, sFID, IS, Precision, and CLIP Score. The authors report consistent improvements over EfficientDM and PTQ baselines, including large gains at W2A4, and release code on GitHub.

Significance. If the reported results hold, MPQ-DM would be a meaningful step toward practical low-bit deployment of diffusion models: it is one of the few methods that maintains non-collapsed generation quality at W2A4, and it combines two relatively orthogonal ideas (intra-layer mixed precision driven by channel kurtosis and time-smoothed relation distillation). The paper is transparent in providing component ablations (Tables 5-8), studies of the kurtosis selection function, and a public code repository, which are strengths. However, the central algorithmic contribution, OMQ, is currently presented as solving a well-defined output-error minimization while the actual implementation is a kurtosis-based heuristic whose equivalence to the stated objective is not validated. The abstract also overstates the headline number. These issues, together with the absence of variance reporting for several close comparisons, need to be resolved before the empirical claims can be fully accepted.

major comments (3)
  1. [Section 3.2, Eq. (6)] Eq. (6) defines the bit-allocation problem as minimizing an output-quantization-error objective, but the implemented procedure does not optimize this objective: it ranks channels by kurtosis and then searches only over the number of promoted/demoted groups with group size k=cout/10 and search region [0, cout//k/2], i.e., at most five candidates. This assumes that the kurtosis ordering of weight channels correctly predicts the output-error reduction, although the objective in Eq. (6) also depends on activation magnitudes. Table 6 validates kurtosis against random and head/tail selection, but it does not compare against the Eq. (6) objective itself or against a finer-grained or exhaustive search on a subset of layers. Since Table 5 attributes most of the W3A4 FID improvement to OMQ (FID decreases from 11.02 to 6.91), the central empirical claim rests on this unvalidated heuristic. Please provide evidence that the kurtosis group search approximates the Eq. (6) minimizer well, for example by comparing objective values and FID for a per-channel or finer-grained search on a few layers, or by reporting the gap between the heuristic assignment and a brute-force assignment for small layers.
  2. [Abstract / Section 4.2, Table 1] The abstract states that "MPQ-DM achieves a 58% FID decrease under W2A4 setting compared with baseline," but Table 1 shows that the 58% decrease (FID 64.45 to 27.11) is achieved by MPQ-DM+, which uses additional channel bit allocation and a larger model size (101.6 MB). The main MPQ-DM at 96.8 MB yields a 43% decrease (FID 64.45 to 36.59). Section 4.2 correctly reports the 27.86 decrease for MPQ-DM. The abstract should attribute the 58% figure to MPQ-DM+ or use the 43% figure for MPQ-DM, to avoid overstating the main method's result.
  3. [Section 4.2, Tables 1-3, 6-7] The paper reports no variance or multiple-seed results, while several claimed improvements are small. Examples include LSUN-Churches W3A6 (EfficientDM 9.53 vs MPQ-DM 9.28 in Table 3) and the distillation metric study in Table 7, where relation distillation improves FID from 9.12 to 9.10 on a 10k-sample evaluation. These differences may be within sampling noise, particularly for the 10k-sample evaluations in Tables 6-7. Please report mean and standard deviation over at least three independent fine-tuning runs for the main comparisons, or otherwise demonstrate that the observed margins are statistically significant.
minor comments (7)
  1. [Section 3.2 and Table 5] The abbreviation is introduced as "Outlier-Driven Mixed Quantization (OMD)" in the introduction but the rest of the paper uses "OMQ"; Table 5 also uses "+TSD" instead of "+TRD". Please unify the abbreviations throughout.
  2. [Eq. (6)] The expression "||X_f W_f^T, Q(\hat{X}_f) Q(\hat{W}_f | [c_1, \cdots, c_n])^T||_2" appears to be missing a minus sign between the two terms; the intended objective is presumably the norm of the difference between the full-precision output and the quantized output.
  3. [Eq. (11)] Eq. (11) uses "Ltarget", but the task loss was defined as "Ltask" in Eq. (7). Please use a consistent symbol and define it before first use.
  4. [Section 4.1 and Table 1] The statement that MPQ-DM+ adds only "0.6% increase in model size compared with FP model" is inconsistent with Table 1: MPQ-DM+ is 101.6 MB versus 96.8 MB for MPQ-DM, and 101.6 MB relative to the full-precision size 1529.7 MB is about 0.3%. Please clarify what the reported percentage refers to.
  5. [Table 5] The first row "PTQD" is misaligned and its values do not match Table 1 (for PTQ-D at W3A4, Table 1 lists IS 10.86, FID 286.57, sFID 273.16, Precision 0.02, whereas Table 5 lists 237.16 and 0.05). Please correct the formatting and the numbers.
  6. [Section 3.3, Eq. (9)] Eq. (9) sums t=0 to N, which involves N+1 time steps, while the text says "N consecutive steps". Make the indexing consistent, e.g., define N as the number of smoothing steps minus one or sum t=1 to N.
  7. [Figure 2] Figure 2 contains garbled text ("??????|| ??") that should be replaced with the proper equation rendering so the figure is readable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central FID/IS/CLIP results are evaluated against external benchmarks and independent baselines; the OMQ heuristic and TRD objective are not fitted to the reported metrics.

full rationale

MPQ-DM's central claims are validated on external generation metrics (FID, sFID, IS, CLIP Score) under fixed bit-widths, and neither proposed component reduces to those metrics by construction. The OMQ bit allocation is determined by a kurtosis-based channel ranking plus a small group-count search guided by the quantization-error objective in Eq. (6); it is not fit to FID, and Tables 5 and 6 compare it against the EfficientDM baseline and random/head-tail alternatives. The time-smoothed relation distillation in Eqs. (8)-(10) is a training objective whose effect is measured on held-out generation quality, with no identity between the loss and the reported scores. Self-citations, such as Feng et al. 2024, appear only as context for common alternative practices and do not carry the derivation. The weaker points noted in review—the coarse k=cout/10 search approximating Eq. (6) and the abstract attributing the 58% FID reduction to MPQ-DM when Table 1 shows that figure for MPQ-DM+—are correctness, consistency, or rigor concerns, not circularity. The claimed derivation chain is therefore self-contained with respect to its external benchmarks.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on two hyperparameters (alpha, N) plus the channel-group fraction k set by hand or by small ablations, and on several domain assumptions about kurtosis, channel independence, time-step feature similarity, and calibration representativeness. The OMQ search heuristic is the most fragile load-bearing input. No new entities are introduced.

free parameters (4)
  • distillation loss weight alpha = 100 for LDM, 1 for Stable Diffusion
    Set by hand in Section 7.3; no ablation or sensitivity analysis is reported.
  • search group size k = cout/10 (10% of output channels)
    Set empirically in Section 3.2 to speed up the OMQ search; no sensitivity study.
  • time smoothing window N = 1
    Chosen from ablation Table 8 based on sFID on the same task; used as a hyperparameter for all main results.
  • additional 2-bit channel proportion for MPQ-DM+ = 10% of channels
    Used only in the MPQ-DM+ variant; increases model size by 4.8 MB and is the source of the abstract's 58% FID decrease claim.
assumptions (5)
  • domain assumption Kurtosis of a weight channel is a reliable proxy for the quantization difficulty (outlier salience) of that channel.
    Section 3.2 uses kurtosis to rank channels and allocate bits; if false, the bit reallocation could hurt rather than help.
  • ad hoc to paper The optimal bit assignment in Eq. (6) is well approximated by promoting the top-kurtosis channels and demoting the bottom-kurtosis channels in equal numbers, with the count search restricted to [0, cout//k/2].
    The paper explicitly constrains the search region in Section 3.2 and sets k=cout/10; this is a heuristic, not a solved optimization.
  • domain assumption Intermediate features of the diffusion model at consecutive time steps are similar enough that summing N consecutive features gives a stable distillation target.
    Section 3.3 and Fig. 4 justify this empirically, but it is assumed for all tasks and time steps; if the trajectory changes rapidly, the smoothed target may be incorrect.
  • domain assumption Channel-wise quantization of weights is independent, so bit-widths can be redistributed within a layer without cross-channel interference.
    Section 3.2 states this; for convolution and linear layers with per-channel quantization this is standard, but it is an assumption.
  • domain assumption The calibration data used to compute max|X| in the smooth factor of Eq. (5) is representative of the test distribution.
    The smoothing factor delta_i in Eq. (5) depends on calibration statistics; if the calibration set is not representative, the scaling may not generalize.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models." pith.science (2026). https://pith.science/paper/HKUEIFF2

@misc{pith2026241211549,
  author       = {Pith},
  title        = {Pith review of: MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HKUEIFF2}},
  note         = {Machine review of arXiv:2412.11549}
}
abstract

Diffusion models have received wide attention in generation tasks. However, the expensive computation cost prevents the application of diffusion models in resource-constrained scenarios. Quantization emerges as a practical solution that significantly saves storage and computation by reducing the bit-width of parameters. However, the existing quantization methods for diffusion models still cause severe degradation in performance, especially under extremely low bit-widths (2-4 bit). The primary decrease in performance comes from the significant discretization of activation values at low bit quantization. Too few activation candidates are unfriendly for outlier significant weight channel quantization, and the discretized features prevent stable learning over different time steps of the diffusion model. This paper presents MPQ-DM, a Mixed-Precision Quantization method for Diffusion Models. The proposed MPQ-DM mainly relies on two techniques:(1) To mitigate the quantization error caused by outlier severe weight channels, we propose an Outlier-Driven Mixed Quantization (OMQ) technique that uses $Kurtosis$ to quantify outlier salient channels and apply optimized intra-layer mixed-precision bit-width allocation to recover accuracy performance within target efficiency.(2) To robustly learn representations crossing time steps, we construct a Time-Smoothed Relation Distillation (TRD) scheme between the quantized diffusion model and its full-precision counterpart, transferring discrete and continuous latent to a unified relation space to reduce the representation inconsistency. Comprehensive experiments demonstrate that MPQ-DM achieves significant accuracy gains under extremely low bit-widths compared with SOTA quantization methods. MPQ-DM achieves a 58\% FID decrease under W2A4 setting compared with baseline, while all other methods even collapse.

Figures

Figures reproduced from arXiv: 2412.11549 by the authors.

Figure 1
Figure 1. Visualization of samples generated by baseline and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of proposed MPQ-DM, consisting of Outlier-Driven Mixed Quantization to apply intra-layer mixed quan [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Weight distribution of LDM-4 ImageNet 256x256. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (16 more)
Figure 5
Figure 5. Figure 5: Visualization of different activation maps of (Left) [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: More visualization of different channel weight distribution. Upper row with lower [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: More visualization of different layer weight distribution. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Randomly generated samples W3A6 Stable Diffusion model with prompt “A cozy cabin nestled in a snowy forest [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Randomly generated samples W3A6 Stable Diffusion model with prompt “a magical fairy tale castle on a hilltop [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Randomly generated samples W3A6 Stable Diffusion model with prompt “A digital illustration of the Babel tower, [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Visualization of samples generated by W3A4 LDM model on ImageNet 256 [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Visualization of samples generated by W3A6 LDM model on ImageNet 256 [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Visualization of samples generated by W2A4 LDM model on LSUN-Churchs 256 [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Visualization of samples generated by W2A6 LDM model on LSUN-Churchs 256 [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Visualization of samples generated by W3A4 LDM model on LSUN-Churchs 256 [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Visualization of samples generated by W3A6 LDM model on LSUN-Churchs 256 [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]
Figure 17
Figure 17. Figure 17: Visualization of samples generated by W2A4 LDM model on LSUN-Bedrooms 256 [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Visualization of samples generated by W2A6 LDM model on LSUN-Bedrooms 256 [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Visualization of samples generated by W3A4 LDM model on LSUN-Bedrooms 256 [PITH_FULL_IMAGE:figures/full_fig_p017_19.png]
Figure 20
Figure 20. Figure 20: Visualization of samples generated by W3A6 LDM model on LSUN-Bedrooms 256 [PITH_FULL_IMAGE:figures/full_fig_p018_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 28 canonical work pages

  1. [1]

    Chen, Q.; Diao, B.; Yang, Y.; and Xu, Y. 2024. SCP: A Structure Combination Pruning Method via Structured Sparse for Deep Convolutional Neural Networks. In International Conference on Pattern Recognition, 238--253. Springer

    work page 2024

  2. [2]

    T.; Mittal, S.; Emani, M.; Vishwanath, V.; and Somani, A

    Chitty-Venkata, K. T.; Mittal, S.; Emani, M.; Vishwanath, V.; and Somani, A. K. 2023. A survey of techniques for optimizing transformer inference. Journal of Systems Architecture, 102990

    work page 2023

  3. [3]

    Dai, L.; Gong, L.; An, Z.; Xu, Y.; and Diao, B. 2024. Sketch-fusion: A gradient compression method with multi-layer fusion for communication-efficient distributed training. Journal of Parallel and Distributed Computing, 185: 104811

    work page 2024

  4. [4]

    DeCarlo, L. T. 1997. On the meaning and use of kurtosis. Psychological methods, 2(3): 292

    work page 1997

  5. [5]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee

    2009

  6. [6]

    Dhariwal, P.; and Nichol, A. 2021. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 8780--8794

    2021

  7. [7]

    Ding, Y.; Feng, W.; Chen, C.; Guo, J.; and Liu, X. 2024. Reg-PTQ: Regression-specialized Post-training Quantization for Fully Quantized Object Detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16174--16184

    work page 2024

  8. [8]

    W.; and Keutzer, K

    Dong, Z.; Yao, Z.; Arfeen, D.; Gholami, A.; Mahoney, M. W.; and Keutzer, K. 2020. Hawq-v2: Hessian aware trace-weighted quantization of neural networks. Advances in neural information processing systems, 33: 18518--18529

    work page 2020

Show all 56 references
  1. [9]

    W.; and Keutzer, K

    Dong, Z.; Yao, Z.; Gholami, A.; Mahoney, M. W.; and Keutzer, K. 2019. Hawq: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE/CVF international conference on computer vision, 293--302

    2019

  2. [10]

    K.; McKinstry, J

    Esser, S. K.; McKinstry, J. L.; Bablani, D.; Appuswamy, R.; and Modha, D. S. 2019. Learned step size quantization. arXiv preprint arXiv:1902.08153

    2019 arXiv

  3. [11]

    Feng, W.; Yang, C.; An, Z.; Huang, L.; Diao, B.; Wang, F.; and Xu, Y. 2024. Relational diffusion distillation for efficient image generation. In Proceedings of the 32nd ACM International Conference on Multimedia, 205--213

    2024

  4. [12]

    W.; and Keutzer, K

    Gholami, A.; Kim, S.; Dong, Z.; Yao, Z.; Mahoney, M. W.; and Keutzer, K. 2022. A survey of quantization methods for efficient neural network inference. In Low-Power Computer Vision, 291--326. Chapman and Hall/CRC

    2022

  5. [13]

    He, Y.; Liu, J.; Wu, W.; Zhou, H.; and Zhuang, B. 2023. Efficientdm: Efficient quantization-aware fine-tuning of low-bit diffusion models. arXiv preprint arXiv:2310.03270

    2023 arXiv

  6. [14]

    He, Y.; Liu, L.; Liu, J.; Wu, W.; Zhou, H.; and Zhuang, B. 2024. Ptqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Processing Systems, 36

    2024

  7. [15]

    L.; and Choi, Y

    Hessel, J.; Holtzman, A.; Forbes, M.; Bras, R. L.; and Choi, Y. 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    2021 arXiv

  8. [16]

    Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

    2017

  9. [17]

    Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840--6851

    2020

  10. [18]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  11. [19]

    Huang, Y.; Gong, R.; Liu, J.; Chen, T.; and Liu, X. 2024. Tfmq-dm: Temporal feature maintenance quantization for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7362--7371

    2024

  12. [20]

    Hubara, I.; Nahshan, Y.; Hanani, Y.; Banner, R.; and Soudry, D. 2020. Improving post training neural quantization: Layer-wise calibration and integer programming. arXiv preprint arXiv:2006.10518

    2020 arXiv

  13. [21]

    Jacob, B.; Kligys, S.; Chen, B.; Zhu, M.; Tang, M.; Howard, A.; Adam, H.; and Kalenichenko, D. 2018. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, ...

    2018

  14. [22]

    Krishnamoorthi, R. 1806. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv 2018. arXiv preprint arXiv:1806.08342

    2018 arXiv

  15. [23]

    Li, X.; Liu, Y.; Lian, L.; Yang, H.; Dong, Z.; Kang, D.; Zhang, S.; and Keutzer, K. 2023. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 17535--17545

    2023

  16. [24]

    Li, Y.; Xu, S.; Cao, X.; Sun, X.; and Zhang, B. 2024. Q-dm: An efficient low-bit quantized diffusion model. Advances in Neural Information Processing Systems, 36

    2024

  17. [25]

    Liu, H.; Diao, B.; Chen, W.; and Xu, Y. 2024 a . A resource-aware workload scheduling method for unbalanced GEMMs on GPUs. The Computer Journal, bxae110

    2024

  18. [26]

    Liu, J.; Niu, L.; Yuan, Z.; Yang, D.; Wang, X.; and Liu, W. 2023 a . Pd-quant: Post-training quantization based on prediction difference metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24427--24437

    2023

  19. [27]

    Liu, Z.; Shen, Z.; Savvides, M.; and Cheng, K.-T. 2020. Reactnet: Towards precise binary neural network with generalized activation functions. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XIV 16, 143--159. Springer

    2020

  20. [28]

    Liu, Z.; Zhang, F.; He, J.; Wang, J.; Wang, Z.; and Cheng, L. 2023 b . Text-guided mask-free local image retouching. In 2023 IEEE International Conference on Multimedia and Expo (ICME), 2783--2788. IEEE

    2023

  21. [29]

    Liu, Z.; Zhao, C.; Fedorov, I.; Soran, B.; Choudhary, D.; Krishnamoorthi, R.; Chandra, V.; Tian, Y.; and Blankevoort, T. 2024 b . SpinQuant--LLM quantization with learned rotations. arXiv preprint arXiv:2405.16406

    2024 arXiv

  22. [30]

    Martinez, B.; Yang, J.; Bulat, A.; and Tzimiropoulos, G. 2020. Training binary neural networks with real-to-binary convolutions. arXiv preprint arXiv:2003.11535

    2020 arXiv

  23. [31]

    Mei, K.; and Patel, V. 2023. Vidm: Video implicit diffusion models. In Proceedings of the AAAI conference on artificial intelligence, volume 37, 9117--9125

    2023

  24. [32]

    Nash, C.; Menick, J.; Dieleman, S.; and Battaglia, P. W. 2021. Generating images with sparse representations. arXiv preprint arXiv:2103.03841

    2021 arXiv

  25. [33]

    Pilipovi \'c , R.; Buli \'c , P.; and Risojevi \'c , V. 2018. Compression of convolutional neural networks: A short survey. In 2018 17th International Symposium INFOTEH-JAHORINA (INFOTEH), 1--6. IEEE

    2018

  26. [34]

    Qin, H.; Ding, Y.; Zhang, M.; Yan, Q.; Liu, A.; Dang, Q.; Liu, Z.; and Liu, X. 2022. Bibert: Accurate fully binarized bert. arXiv preprint arXiv:2203.06390

    2022 arXiv

  27. [35]

    Qin, H.; Gong, R.; Liu, X.; Shen, M.; Wei, Z.; Yu, F.; and Song, J. 2020. Forward and backward information retention for accurate binary neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2250--2259

    2020

  28. [36]

    Qin, H.; Zhang, M.; Ding, Y.; Li, A.; Cai, Z.; Liu, Z.; Yu, F.; and Liu, X. 2023. Bibench: Benchmarking and analyzing network binarization. In International Conference on Machine Learning, 28351--28388. PMLR

    2023

  29. [37]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684--10695

    2022

  30. [38]

    Salimans, T.; Goodfellow, I.; Zaremba, W.; Cheung, V.; Radford, A.; and Chen, X. 2016. Improved techniques for training gans. Advances in neural information processing systems, 29

    2016

  31. [39]

    Shang, Y.; Yuan, Z.; Xie, B.; Wu, B.; and Yan, Y. 2023. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 1972--1981

    2023

  32. [40]

    Song, J.; Meng, C.; and Ermon, S. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502

    2020 arXiv

  33. [41]

    P.; Kumar, A.; Ermon, S.; and Poole, B

    Song, Y.; Sohl-Dickstein, J.; Kingma, D. P.; Kumar, A.; Ermon, S.; and Poole, B. 2020. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456

    2020 arXiv

  34. [42]

    Wang, C.; Wang, Z.; Xu, X.; Tang, Y.; Zhou, J.; and Lu, J. 2024 a . Towards Accurate Post-training Quantization for Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16026--16035

    2024

  35. [43]

    Wang, H.; Shang, Y.; Yuan, Z.; Wu, J.; and Yan, Y. 2024 b . Quest: Low-bit diffusion model quantization via efficient selective finetuning. arXiv preprint arXiv:2402.03666

    2024 arXiv

  36. [44]

    Wang, P.; Chen, Q.; He, X.; and Cheng, J. 2020. Towards accurate post-training network quantization via bit-split and stitching. In International Conference on Machine Learning, 9847--9856. PMLR

    2020

  37. [45]

    Wei, X.; Gong, R.; Li, Y.; Liu, X.; and Yu, F. 2022. Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization. arXiv preprint arXiv:2203.05740

    2022 arXiv

  38. [46]

    Wu, J.; Wang, H.; Shang, Y.; Shah, M.; and Yan, Y. 2024. PTQ4DiT: Post-training Quantization for Diffusion Transformers. arXiv preprint arXiv:2405.16005

    2024 arXiv

  39. [47]

    Yang, C.; An, Z.; Huang, L.; Bi, J.; Yu, X.; Yang, H.; Diao, B.; and Xu, Y. 2024 a . CLIP-KD: An Empirical Study of CLIP Model Distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15952--15962

    2024

  40. [48]

    Yang, C.; An, Z.; Zhou, H.; Zhuang, F.; Xu, Y.; and Zhang, Q. 2023. Online knowledge distillation via mutual contrastive learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8): 10212--10227

    2023

  41. [49]

    Yang, C.; Zhou, H.; An, Z.; Jiang, X.; Xu, Y.; and Zhang, Q. 2022. Cross-image relational knowledge distillation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12319--12328

    2022

  42. [50]

    Yang, Y.; Cheng, D.; Fang, C.; Wang, Y.; Jiao, C.; Cheng, L.; and Wang, N. 2024 b . Diffusion-based Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detection

    2024

  43. [51]

    Yao, Z.; Dong, Z.; Zheng, Z.; Gholami, A.; Yu, J.; Tan, E.; Wang, L.; Huang, Q.; Wang, Y.; Mahoney, M.; et al. 2021. Hawq-v3: Dyadic neural network quantization. In International Conference on Machine Learning, 11875--11886. PMLR

    2021

  44. [52]

    Yu, F.; Seff, A.; Zhang, Y.; Song, S.; Funkhouser, T.; and Xiao, J. 2015. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365

    2015 arXiv

  45. [53]

    Zhao, T.; Fang, T.; Liu, E.; Rui, W.; Soedarmadji, W.; Li, S.; Lin, Z.; Dai, G.; Yan, S.; Yang, H.; et al. 2024. ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation. arXiv preprint arXiv:2406.02540

    2024 arXiv

  46. [54]

    Zheng, X.; Qin, H.; Ma, X.; Zhang, M.; Hao, H.; Wang, J.; Zhao, Z.; Guo, J.; and Liu, X. 2024. Binarydm: Towards accurate binarization of diffusion model. arXiv preprint arXiv:2404.05662

    2024 arXiv

  47. [55]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.