Pith. sign in

REVIEW 4 major objections 4 minor 46 references

Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read 4-bit floating-point quantization with mixup-sign formats and timestep-routed LoRAs beats 4-bit integer fine-tuning for diffusion models.

desk verdict Useful novel quantization idea, but the ImageNet numbers that carry the 4-bit claim are internally inconsistent and the paper's explanation does not hold. read the letter →

arxiv 2505.21591 v1 pith:EJ4UHSHL submitted 2025-05-27 cs.LG cs.AI

Maosen Zhao , Pengtao Chen , Chong Yu , Yan Wen , Xudong Tan , Tao Chen This is my paper
classification cs.LGcs.AI
keywords 4-bitquantizationfloating-pointdiffusionmodelcompressionmixup-signtimestep-awareLoRAdenoising-factorlossalignmentpost-trainingfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that fully quantized 4-bit diffusion models—both weights and activations at 4 bits—work better in floating-point format than in integer format, once three specific mismatches are removed. It identifies those mismatches as: signed FP quantization squanders precision on the skewed activation layers created by the SiLU nonlinearity; a single LoRA adapter cannot cover the distinct tasks the denoiser performs at early versus late timesteps; and the standard MSE loss on predicted noise does not reflect how much quantization error actually degrades the output at each step. To remove them it introduces MSFP (unsigned FP quantization with a zero point, mixed with signed FP), TALoRA (multiple LoRA adapters selected by a timestep-aware router), and DFA (weighting the fine-tuning loss by the DDIM denoising coefficient $\gamma_t$). Reported 4-bit results beat the compared integer-quantization fine-tuning baselines on every dataset: CIFAR-10 lands at FID 6.02 versus 4.26 for full precision and 38.40 for the best 4-bit integer baseline, and LSUN-Church improves from 18.40 to 8.81. The paper's stated contribution is that this is the first time 4-bit FP quantization has achieved superior performance for diffusion models using fine-tuning rather than retraining.

What carries the argument

The load-bearing identity is the denoising factor $\gamma_t = (1-\alpha_t)/\sqrt{\alpha_t(1-\alpha_t)}$, which the paper derives as the coefficient of the predicted noise $\varepsilon_\theta$ in the DDIM update $x_{t-1} = (1/\sqrt{\alpha_t})(x_t - ((1-\alpha_t)/\sqrt{1-\alpha_t})\varepsilon_\theta) + \sigma_t \delta$. It carries the loss-alignment argument: multiplying the raw noise-prediction MSE by $\gamma_t$ makes the fine-tuning loss proportional to how strongly that step's error moves the output image, which the paper treats as the real quantization damage. The other two mechanisms are a quantizer and a router. The MSFP quantizer searches both signed FP formats and unsigned FP formats (sign bit cleared, zero point added) and assigns each activation layer the better one, since the asymmetric layers want unsigned formats while normal layers keep signed FP. The TALoRA router is a small shared network—a timestep embedding plus an MLP—that outputs a distribution over LoRA adapters for each timestep, with a straight-through estimator converting it to a one-hot selection, so each layer is fine-tuned by exactly one active adapter per step.

What would settle it

Fix MSFP and TALoRA, and compare DFA-weighted fine-tuning against uniformly weighted fine-tuning by FID on 50k generated samples from a dataset such as CIFAR-10; the alignment claim predicts a clear, reproducible FID win for the weighted loss, and statistically indistinguishable FID would mean the $\gamma_t$ reweighting is not doing the work claimed. A complementary check targets the asymmetry story: forcing unsigned-FP-with-zero-point onto a layer with genuinely symmetric activations should measurably worsen activation quantization error, and if it does not, the SiLU-asymmetry explanation for MSFP's gains is incomplete.

Watch

Extended reading notes

Core claim

The central discovery is that the poor showing of low-bit FP quantization in diffusion models is not a failure of FP formats themselves but the sum of three fixable causes. Layers whose activations are skewed by SiLU—the Anomalous-Activation-Distribution Layers, where essentially all negative values are squeezed into the narrow band [−0.278, 0)—lose almost all sub-zero precision under signed FP quantization; giving those layers an unsigned FP format with an added zero point frees the wasted sign bit to become exponent or mantissa precision and cuts activation quantization error in over 95% of such layers. The denoising process is itself a multi-task procedure—outlines first, details later—so a single LoRA shared across all timesteps underfits; a router that reads the timestep and dispatches one LoRA per step reflects that structure, and the learned allocation indeed settles into the two-phase pattern, with two adapters sufficing in most experiments. Finally, the per-step fine-tuning loss tracks real degradation only when reweighted by the denoising factor $\gamma_t = (1-\alpha_t)/\sqrt{\alpha_t(1-\alpha_t)}$, the coefficient with which predicted noise enters the DDIM update, so that steps where prediction error moves the output more are weighted more. In the CelebA ablation the three components together take the 4-bit model from FID 16.02 to 7.69 against a full-precision 6.49, and the full method reports 4-bit results that beat the compared baselines on CIFAR-10, LSUN-Bedroom, LSUN-Church, and ImageNet.

Load-bearing premise

The method rests on the assumption that the damage quantization does to the predicted noise at a given timestep is faithfully captured by the DDIM coefficient $\gamma_t$, so that reweighting the per-step MSE by $\gamma_t$ makes the loss proportional to real degradation—yet the paper checks this only against a proxy, $\mathrm{MSE}(x_{t-1}, \hat{x}_{t-1})$, computed from the same DDIM update that defines $\gamma_t$, and never against end-to-end FID or at steps where $\gamma_t$ is near zero.

Editorial extensions

If this is right

  • Fully 4-bit (W4/A4) diffusion models become deployable via lightweight fine-tuning rather than retraining: CIFAR-10 lands at FID 6.02 versus the full-precision 4.26, and LSUN-Church at 8.81 versus 4.06, with Inception Score roughly preserved.
  • The mixup-sign search extends the FP-quantization toolbox from large language models to convolutional diffusion models, making unsigned FP with a zero point a first-class candidate format inside the standard search-based quantization pipeline.
  • Reweighting the fine-tuning loss by the DDIM coefficient makes the loss curve track measured performance degradation across timesteps rather than run opposite to it, so fine-tuning capacity is spent where quantization damage is real.
  • On ImageNet conditional generation the method reports 4-bit sFID 7.42 versus 7.67 for the full-precision model and 14.42 for the best compared 4-bit fine-tuning baseline, with IS 190.74 versus 139.45, under identical layer-wise quantization settings.
  • Two LoRAs suffice: the router's learned allocation concentrates on a two-stage pattern consistent with the outlines-first, details-later account of denoising, and a four-LoRA hub adds no gain over two in most datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The DFA recipe is not tied to FP formats or to LoRA: any quantized iterative generative solver could reweight its per-step loss by the sensitivity of the output to that step's prediction error, and applying the same $\gamma_t$ weighting to an integer-quantized diffusion fine-tuner would test how much of the reported FP advantage is due to the alignment rather than the format.
  • Because the learned router converges to a stable two-stage allocation across datasets, it could likely be frozen after training and replaced by a fixed outline/detail schedule, removing routing overhead at inference; the h=4 experiments, where the router effectively uses two LoRAs, support this.
  • The paper's own observation that FID improves as bit-width decreases on ImageNet—while sFID and IS degrade—implies FID alone is unreliable for ranking quantized generative models, so earlier integer-quantization comparisons built on FID may deserve re-examination.
  • The mixup-sign principle suggests a broader quantizer design where the search decides not only exponent and mantissa splits but whether the format carries a sign at all, per layer and per tensor; the paper applies this only to activations, leaving weights and per-channel sign choices as natural extensions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a fully quantized 4-bit floating-point (FP) scheme for diffusion models, combining three components: mixup-sign FP quantization (MSFP), timestep-aware LoRA (TALoRA), and denoising-factor loss alignment (DFA). MSFP searches per layer between signed and unsigned FP formats, adding a zero point for activation layers whose SiLU-induced distributions are asymmetric. TALoRA trains a small router that selects one of several LoRA adapters per denoising timestep. DFA reweights the noise-prediction MSE by the DDIM denoising coefficient gamma_t. The paper reports DDIM and LDM experiments on CIFAR-10, CelebA, LSUN-Bedroom, LSUN-Church, and ImageNet, claiming state-of-the-art 4-bit results that beat integer-quantization fine-tuning baselines such as EfficientDM and QuEST.

Significance. If the central empirical claim holds, the paper would be a useful step for deploying diffusion models at 4-bit precision. The main positive contributions are (i) identifying the asymmetric-activation failure of signed FP quantization in AALs and offering a simple zero-point fix, (ii) treating fine-tuning as a timestep-dependent multi-task problem, and (iii) providing a broad evaluation across two diffusion architectures and several datasets. The per-layer FP-format search is straightforward and the method is plausible. However, the ImageNet FID anomaly and the partly circular validation of DFA mean that the headline result is not yet fully established; the paper also does not state a code-release plan, which limits reproducibility.

major comments (4)
  1. [Section 5.2, Table 3; Appendix F, Table 10] The ImageNet results contain an internally contradictory FID trend that the paper acknowledges but does not explain: the 4-bit h=2 model reports FID 6.50 versus 11.69 for the full-precision model and 10.10 for its own 6-bit model, and the same inversion appears in the PLMS and DPM-Solver results of Table 10 (e.g., 4-bit FID 7.27 versus FP 11.71). Because sFID and IS are computed from the same generated samples, any protocol issue that corrupts FID can also corrupt those metrics. The paper needs to identify the cause (e.g., class-conditioning handling, LoRA modules left in full precision at inference, or FID implementation) and either report corrected numbers or provide an independent, defensible reason for selecting sFID and IS over FID.
  2. [Section 3.2, Observation 3; Eq. (3), (4), (9); Figure 3] The validation of the denoising-factor loss alignment is circular. The 'performance deterioration metric' in Figure 3 is defined as MSE(xt-1, xhat-1) using the DDIM update equation, and the proposed reweighting multiplies the per-step loss by exactly the coefficient gamma_t that appears in that same equation. The observed alignment therefore follows in part from the definitions and does not independently demonstrate that the reweighted loss tracks end-to-end image degradation. The authors should validate DFA against an independent measure (e.g., FID of quantized versus full-precision generation, or error in the final x0 estimate), and report behavior at timesteps where gamma_t is near zero.
  3. [Section 5.2, Table 2 and following text] The claim that 6-bit quantization achieves 'nearly identical performance to full precision' is contradicted by Table 2: LSUN-Bedroom FID is 8.42 versus 3.02 and LSUN-Church is 6.24 versus 4.06. This overstatement matters because it supports the argument that FP quantization is inherently well suited to diffusion models and makes the anomalous ImageNet FID improvement in the 4-bit setting harder to dismiss. Please rephrase the claim and discuss the actual gaps.
  4. [Section 5.1, Baselines paragraph; Appendix G] The baseline standardization is not fully specified. The main text says EfficientDM is 'standardized in a consistent manner' and that other standardized results are 'procured from DilateQuant,' but the exact layer sets, calibration sizes, fine-tuning schedules, and sampling protocols are not reported in one place. Without this information, the central comparison to EfficientDM and QuEST cannot be independently reproduced. Please provide a complete protocol table for every baseline and dataset, including which layers are quantized in each method.
minor comments (4)
  1. [Appendix B and C] There are several typos and OCR-style errors, including 'f ormat', 'abd', 'linsapce', and 'f omrat'; these should be corrected.
  2. [Table 4] The row labels '%' and '!' are unexplained; please add a legend defining which modules are active in each row.
  3. [Figure 3] The caption says 'Two loss'; it should read 'Two losses' or 'Comparison of two loss functions.'
  4. [Overall] The paper does not mention code or model release; adding a reproducibility statement with the exact random seeds, calibration sets, and evaluation scripts would strengthen the submission.

Circularity Check

2 steps flagged · score 3.0 of 10

Localized definitional circularity in the DFA validation and the MSFP per-layer MSE evidence; the central external benchmark claims remain independent.

  1. self definitional [Section 3.2 (Eqs. 3-4 and the performance-gap proxy) and Section 4.3 / Figure 3]
    "γt = 1√αt · 1 − αt√1 − αt . (4) According to Equation 3, γt indicates the impact of the prediction noise under timestep t. ... We input the previous output image xt from the full-precision model and calculate the performance gap between the denoised image xt−1 from the full-precision model and the denoised image bxt−1 from the quantized model, measured by M SE(xt−1, bxt−1). ... By introducing γt, which accurately reflects the utilization of the predicted noise at each time step, we achieve a preliminary alignment between the loss and the actual quantization error, as shown in Figure 3."

    The denoising factor γt in Eq. 4 is exactly the coefficient multiplying the predicted-noise error in the DDIM update Eq. 3. For identical xt and injected noise, xt−1 − x̂t−1 = γt(εθ − ε̂θ), so the validation proxy MSE(xt−1, x̂t−1) equals γt²·MSE(εθ, ε̂θ). The 'performance deterioration metric' is therefore a deterministic rescaling of the same per-step noise MSE that DFA reweights; the consistent trend in Figure 3 follows from the definitions of Eqs. 3 and 4 rather than from an independent measurement of end-to-end image-quality degradation. The DFA ablation in Table 4 is combined with the other modules, so no isolated end-to-end test of the gamma weighting is provided.

  2. fitted input called prediction [Section 4.1 / Figure 4 and Appendix B, Algorithm 1]
    "We implement the proposed MSFP strategy using a search-based method [2, 25], wherein the quantization parameters are determined by minimizing the MSE between the distributions before and after quantization. ... As illustrated in Figure 4, unsigned FP quantization with a zero point significantly improves representation in over 95% of AALs, compared to traditional signed FP quantization."

    Algorithm 1 selects, per Anomalous-Activation-Distribution Layer, the sign/zero-point variant that minimizes the per-layer activation MSE. The 'over 95% of AALs' improvement is then reported using that same per-layer MSE objective. Thus the quantitative evidence for MSFP's representational advantage is the direct output of the optimization criterion being minimized, not an independent evaluation of the format choice. The end-to-end FID gains in Tables 2-4 are external to this per-layer metric and retain independent content.

full rationale

The paper's headline claim is an empirical state-of-the-art claim against external baselines (EfficientDM, QuEST, Q-Diffusion, EDA-DM) using FID, sFID and IS on CIFAR-10, CelebA, LSUN and ImageNet. Those comparisons are self-contained and are not derived from the paper's own assumptions, so the central result is not circular. The circularity found is confined to two internal validations. First, the DFA alignment is validated in Figure 3 against MSE(xt−1, x̂t−1), a proxy that is a deterministic function of the same DDIM coefficient γt used to define the reweighted loss; the agreement is therefore partly built into the definitions rather than established by an independent image-quality measurement. Second, the 'over 95% of AALs' advantage of MSFP in Figure 4 is measured with the same per-layer activation MSE that Algorithm 1 minimizes when choosing between signed and unsigned+zero-point formats, so that statistic restates the optimization objective. Neither of these issues infects the external FID/sFID benchmarks, which is why the score is not higher. The paper also explicitly flags an evaluation anomaly on ImageNet - 'as the bit-width of the quantized model decreases, the FID score improves, which contradicts the expected trend' - and excludes FID from the ImageNet discussion; that is a serious evaluation-validity concern but it is not circularity. There is no load-bearing self-citation chain or imported uniqueness theorem: the search-based FP initialization is credited to prior external work [2, 25], and the final benchmark numbers are compared against independently published baselines.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a modest set of fitted quantization parameters (per-layer format, maxval, zero point) and standard PTQ assumptions, plus a loss-reweighting heuristic and an empirical assumption about SiLU activation bounds. No new physical entities are introduced. The most fragile load-bearing premise is that the gamma_t reweighting matches real quantization damage.

free parameters (6)
  • Per-layer FP format (ExMy) selection = 4-bit: one of E3M0, E2M1, E1M2, E0M3; similar discrete sets for 6/8 bit
    Chosen per layer by MSE search over calibration data (Algorithm 1, Appendix B, Table 6). The model's final quality depends on these format choices, making them effective free parameters of the method.
  • Per-layer maxval threshold (bias) = Searched inside [0.8*maxval0, 2*maxval0] for weights and linspace(0, maxval0, 100) for activations
    Sets the scale/bias of the FP quantizer. maxval0 is measured from 2000 random forward passes; the final threshold is fitted per layer to minimize quantization MSE.
  • Zero point zp for unsigned FP in AALs = Searched over linspace(-0.3, 0, 6), typically near -0.278
    Offset that shifts the unsigned FP grid to recover sub-zero activations caused by SiLU. Fitted per layer during initialization; the search range assumes the SiLU lower bound.
  • TALoRA rank = 32
    Rank of each LoRA expert, chosen without a reported sensitivity study; used across all layers and timesteps.
  • LoRA hub size h and training hyperparameters = h=2 (main), lr=1e-4, epochs=160 (DDIM) or 320 (LDM), batch 16/8/4
    Number of LoRA experts and optimization settings were selected by the authors. The paper shows h=2 beats h=4 in most cases but does not report other choices.
  • Calibration set size and construction = 2000 random images for maxval0; 256 samples (DDIM) or 128 samples (LDM) for calibration
    Quantizer initialization depends on these calibration samples; the exact composition is taken from Q-Diffusion [19].
assumptions (5)
  • domain assumption The DDIM/DDPM update (Eq. 3) and its coefficient gamma_t correctly measure the impact of predicted noise on the next latent, so weighting the fine-tuning loss by gamma_t produces a loss aligned with true quantization damage.
    Invoked in Section 4.3. The paper's validation compares against a proxy MSE(x_{t-1}, x_hat_{t-1}) that is derived from the same equation, so the alignment is partly assumed.
  • domain assumption The SiLU activation compresses negative activations into a fixed range around [-0.278, 0) for AALs, and this range transfers across datasets and models.
    Observed on CelebA (Figure 1) and used to set the zero-point search range; no evidence that the same bound holds on LSUN, ImageNet, or Stable Diffusion.
  • domain assumption Per-layer MSE-based search on calibration images generalizes to test-time activations and leads to better end-to-end FID.
    Standard PTQ assumption used in Algorithm 1; not directly verified per layer against final image quality.
  • domain assumption The denoising process separates into outline generation followed by detail refinement, so two timestep-aware LoRA experts are sufficient.
    Motivates TALoRA; the paper supports it with router-allocation patterns after training (Figure 7, Appendix E.2), which is a post hoc justification.
  • domain assumption On ImageNet, FID is an unreliable metric for comparing quantized models, while sFID and IS are reliable.
    The paper discards the FID number when it improves with lower bit-width (Table 3) and gives no technical explanation for why FID is unreliable in this setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning." pith.science (2026). https://pith.science/paper/EJ4UHSHL

@misc{pith2026250521591,
  author       = {Pith},
  title        = {Pith review of: Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJ4UHSHL}},
  note         = {Machine review of arXiv:2505.21591}
}
read the original abstract

Model quantization reduces the bit-width of weights and activations, improving memory efficiency and inference speed in diffusion models. However, achieving 4-bit quantization remains challenging. Existing methods, primarily based on integer quantization and post-training quantization fine-tuning, struggle with inconsistent performance. Inspired by the success of floating-point (FP) quantization in large language models, we explore low-bit FP quantization for diffusion models and identify key challenges: the failure of signed FP quantization to handle asymmetric activation distributions, the insufficient consideration of temporal complexity in the denoising process during fine-tuning, and the misalignment between fine-tuning loss and quantization error. To address these challenges, we propose the mixup-sign floating-point quantization (MSFP) framework, first introducing unsigned FP quantization in model quantization, along with timestep-aware LoRA (TALoRA) and denoising-factor loss alignment (DFA), which ensure precise and stable fine-tuning. Extensive experiments show that we are the first to achieve superior performance in 4-bit FP quantization for diffusion models, outperforming existing PTQ fine-tuning methods in 4-bit INT quantization.

Figures

Figures reproduced from arXiv: 2505.21591 by the authors.

Figure 1
Figure 1. The activation distributions in NALs and AALs, results on the CelebA dataset. (a) The paradigm of NALs with symmetric [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Effect of bit-width reduction on activation representation capacity in AALs (blue) and NALs (orange) under signed FP quantization, evaluated on CelebA dataset. 0 20 40 60 80 100 Denoising Step 10 3 10 2 10 1 Loss Original MSE Loss Aligned MSE Loss Performance Deterioration Metric 10 6 10 5 10 4 Performance Deterioration [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. The pipeline of our proposed method. UNets are applied to the Mixup-Sign Floating-Point Quantization (MSFP), where dis [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figures from the paper (7 more)
Figure 6
Figure 6. Figure 6: A visual comparison of generation results using our [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Distribution of LoRA allocations over timesteps ob [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The weight distribution of certain layers in the DDIM model on CelebA dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Distribution of LoRA allocations over timesteps ob [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Visualization of random samples from 4-bit LDM-4 on LSUN-Bedroom across different LoRA Hub sizes [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Visualization of random samples from quantized LDM-4 on ImageNet. The size of LoRA Hub is 2. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Comparison of text-to-image outputs from 6-bit quantized and full-precision Stable Diffusion models. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 23 canonical work pages

  1. [1]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L ´eonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. 6

    arXiv 2013

  2. [2]

    Low-Bitwidth Floating Point Quantization for Efficient High-Quality Diffusion Models

    Cheng Chen, Christina Giannoula, and Andreas Moshovos. Low-bitwidth floating point quantization for efficient high- quality diffusion models. arXiv preprint arXiv:2408.06995,

    work page Pith review arXiv

  3. [3]

    Qncd: Quantization noise correction for diffusion models

    Huanpeng Chu, Wei Wu, Chengjie Zang, and Kun Yuan. Qncd: Quantization noise correction for diffusion models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 10995–11003, 2024. 2

    work page 2024

  4. [4]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 7, 4

    work page 2009

  5. [5]

    Qlora: efficient finetuning of quantized llms (2023)

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: efficient finetuning of quantized llms (2023). arXiv preprint arXiv:2305.14314 , 52:3982–3992,

    arXiv 2023

  6. [6]

    Learned step size quantization

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. In International Conference on Learning Representations, 2019. 1, 2

    work page 2019

  7. [7]

    Mixture-of-loras: An efficient multitask tun- ing method for large language models

    Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. Mixture-of-loras: An efficient multitask tun- ing method for large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC- COLING 2024), pages 11371–11380, 2024. 3

    work page 2024

  8. [8]

    Efficientdm: Efficient quantization-aware fine-tuning of low-bit diffusion models

    Yefei He, Jing Liu, Weijia Wu, Hong Zhou, and Bo- han Zhuang. Efficientdm: Efficient quantization-aware fine-tuning of low-bit diffusion models. arXiv preprint arXiv:2310.03270, 2023. 1, 2, 7, 8, 4

    arXiv 2023

Show all 46 references
  1. [9]

    Ptqd: Accurate post-training quantization for diffusion models

    Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, and Bohan Zhuang. Ptqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

    2024

  2. [10]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 7

    2017

  3. [11]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  4. [12]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3

    2021 arXiv

  5. [13]

    An empirical study of llama3 quantization: From llms to mllms

    Wei Huang, Xingyu Zheng, Xudong Ma, Haotong Qin, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, and Michele Magno. An empirical study of llama3 quantization: From llms to mllms. Visual Intelligence, 2(1): 36, 2024. 1

    2024

  6. [14]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceed- ings of the IEEE conference on computer vision ...

    2018

  7. [15]

    Quantizing deep convolu- tional networks for efficient inference: A whitepaper

    Raghuraman Krishnamoorthi. Quantizing deep convolu- tional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342, 2018. 1

    2018 arXiv

  8. [16]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 7

    2009

  9. [17]

    Fp8 quanti- zation: The power of the exponent

    Andrey Kuzmin, Mart Van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. Fp8 quanti- zation: The power of the exponent. Advances in Neural In- formation Processing Systems, 35:14651–14662, 2022. 2

    2022

  10. [18]

    Contemporary ad- vances in neural network quantization: A survey

    Min Li, Zihao Huang, Lin Chen, Junxing Ren, Miao Jiang, Fengfa Li, Jitao Fu, and Chenghua Gao. Contemporary ad- vances in neural network quantization: A survey. In2024 In- ternational Joint Conference on Neural Networks (IJCNN) , pages 1–10. IEEE, 2024. 1

  11. [19]

    Q-diffusion: Quantizing diffusion models

    Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 17535–17545, 2023. 1, 2, 7

    2023

  12. [20]

    Q-dm: An efficient low-bit quantized dif- fusion model

    Yanjing Li, Sheng Xu, Xianbin Cao, Xiao Sun, and Baochang Zhang. Q-dm: An efficient low-bit quantized dif- fusion model. Advances in Neural Information Processing Systems, 36, 2024. 1, 2

    2024

  13. [21]

    Pruning and quantization for deep neural network acceleration: A survey

    Tailin Liang, John Glossner, Lei Wang, Shaobo Shi, and Xi- aotong Zhang. Pruning and quantization for deep neural network acceleration: A survey. Neurocomputing, 461:370– 403, 2021. 1

    2021

  14. [22]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  15. [23]

    Improving neural network efficiency via post-training quan- tization with adaptive floating-point

    Fangxin Liu, Wenbo Zhao, Zhezhi He, Yanzhi Wang, Zongwu Wang, Changzhi Dai, Xiaoyao Liang, and Li Jiang. Improving neural network efficiency via post-training quan- tization with adaptive floating-point. In Proceedings of the IEEE/CVF international conference on computer visio...

    2021

  16. [24]

    Pseudo numerical methods for diffusion models on manifolds.arXiv preprint arXiv:2202.09778, 2022

    Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds.arXiv preprint arXiv:2202.09778, 2022. 4 9

    2022 arXiv

  17. [25]

    Llm-fp4: 4-bit floating-point quan- tized transformers

    Shih-yang Liu, Zechun Liu, Xijie Huang, Pingcheng Dong, and Kwang-Ting Cheng. Llm-fp4: 4-bit floating-point quan- tized transformers. arXiv preprint arXiv:2310.16836, 2023. 2, 4, 1

    2023 arXiv

  18. [26]

    Dilatequant: Accu- rate and efficient diffusion quantization via weight dilation

    Xuewen Liu, Zhikai Li, and Qingyi Gu. Dilatequant: Accu- rate and efficient diffusion quantization via weight dilation. arXiv preprint arXiv:2409.14307, 2024. 7

    2024 arXiv

  19. [27]

    En- hanced distribution alignment for post-training quantization of diffusion models

    Xuewen Liu, Zhikai Li, Junrui Xiao, and Qingyi Gu. En- hanced distribution alignment for post-training quantization of diffusion models. arXiv preprint arXiv:2401.04585, 2024. 2

    2024 arXiv

  20. [28]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730–3738, 2015. 7, 4

    2015

  21. [29]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787,

  22. [30]

    Fp8 formats for deep learning

    Paulius Micikevicius, Dusan Stosic, Neil Burgess, Mar- ius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sang- won Ha, Alexander Heinecke, Patrick Judd, John Kamalu, et al. Fp8 formats for deep learning. arXiv preprint arXiv:2209.05433, 2022. 1, 2

    2022 arXiv

  23. [31]

    Up or down? adap- tive rounding for post-training quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Chris- tos Louizos, and Tijmen Blankevoort. Up or down? adap- tive rounding for post-training quantization. In International Conference on Machine Learning, pages 7197–7206. PMLR,

  24. [32]

    Blackwell platform sets new llm inference records in mlperf inference v4.1, 2024

    NVIDIA. Blackwell platform sets new llm inference records in mlperf inference v4.1, 2024. Available at: https://developer.nvidia.com/blog/nvidia-blackwell- platform-sets-new-llm-inference-records-in-mlperf- inference-v4-1, Accessed: 2024-11-14. 2

    2024

  25. [33]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 7

    2022

  26. [34]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 7

    2016

  27. [35]

    Post-training quantization on diffusion models

    Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1972–1981, 2023. 1, 2

    1972

  28. [36]

    Temporal dynamic quantization for dif- fusion models

    Junhyuk So, Jungwon Lee, Daehyun Ahn, Hyungjun Kim, and Eunhyeok Park. Temporal dynamic quantization for dif- fusion models. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  29. [37]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 7

    2010 arXiv

  30. [38]

    Tmpq-dm: Joint timestep reduction and quantization precision selection for efficient diffusion mod- els

    Haojun Sun, Chen Tang, Zhi Wang, Yuan Meng, Xinzhu Ma, Wenwu Zhu, et al. Tmpq-dm: Joint timestep reduction and quantization precision selection for efficient diffusion mod- els. arXiv preprint arXiv:2404.09532, 2024. 2

    2024 arXiv

  31. [39]

    Fp8 ver- sus int8 for efficient deep learning inference

    Mart van Baalen, Andrey Kuzmin, Suparna S Nair, Yuwei Ren, Eric Mahurin, Chirag Patel, Sundar Subramanian, Sanghyuk Lee, Markus Nagel, Joseph Soriaga, et al. Fp8 ver- sus int8 for efficient deep learning inference. arXiv preprint arXiv:2303.17951, 2023. 2

    2023 arXiv

  32. [40]

    Diffusion models gener- ate images like painters: an analytical theory of outline first, details later

    Binxu Wang and John J Vastola. Diffusion models gener- ate images like painters: an analytical theory of outline first, details later. arXiv preprint arXiv:2303.02490, 2023. 2, 8, 4

    2023 arXiv

  33. [41]

    Towards accurate data- free quantization for diffusion models

    Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, and Jiwen Lu. Towards accurate data- free quantization for diffusion models. arXiv preprint arXiv:2305.18723, 2(5), 2023. 7, 2

    2023 arXiv

  34. [42]

    Quest: Low-bit diffusion model quantization via efficient selective finetuning

    Haoxuan Wang, Yuzhang Shang, Zhihang Yuan, Junyi Wu, Junchi Yan, and Yan Yan. Quest: Low-bit diffusion model quantization via efficient selective finetuning. arXiv preprint arXiv:2402.03666, 2024. 1, 2, 7, 4

    2024 arXiv

  35. [43]

    Fp4-quantization: Lossless 4bit quantization for large lan- guage models

    Jie Wang, Huanxi Liu, Dawei Feng, Jie Ding, and Bo Ding. Fp4-quantization: Lossless 4bit quantization for large lan- guage models. In 2024 IEEE International Conference on Joint Cloud Computing (JCC), pages 61–67. IEEE, 2024. 2

    2024

  36. [44]

    Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop

    Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. 1, 7

    2015 arXiv

  37. [45]

    Integer or floating point? new outlooks for low-bit quantization on large language models

    Yijia Zhang, Lingran Zhao, Shijie Cao, Sicheng Zhang, Wenqiang Wang, Ting Cao, Fan Yang, Mao Yang, Shang- hang Zhang, and Ningyi Xu. Integer or floating point? new outlooks for low-bit quantization on large language models. In 2024 IEEE International Conference on Multimedia a...

    2024

  38. [46]

    The input channels of the router match the channel count of the timestep embedding in the diffusion model

    The selection of different TALoRAs at each timestep is managed by a router, which is implemented as a linear layer. The input channels of the router match the channel count of the timestep embedding in the diffusion model. Adam optimizers are assigned to both the TALoRAs and t...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.