HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations

Boris van Breugel; Marco Federici; Markus Nagel; Paul Whatmough; Riccardo Del Chiaro

REVIEW 3 major objections 4 minor 15 references

HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that HadaNorm, a mean-centering extension of channel scaling and the Hadamard transform, makes 4-bit diffusion transformer quantization match full precision.

desk verdict Simple exact centering trick that improves W4A4 quantization of DiTs, but the paper's own W6A6 appendix undermines its SQNR-as-proxy story and the eval is too thin to trust the headline number. read the letter →

arxiv 2506.09932 v2 pith:TESJRXNN submitted 2025-06-11 cs.CV cs.AI

Marco Federici , Riccardo Del Chiaro , Boris van Breugel , Paul Whatmough , Markus Nagel This is my paper · ORCID

classification cs.CVcs.AI

keywords Post-trainingquantizationdiffusiontransformersHadamardtransformactivationchannelscalingoutliermitigationPixArt-SigmaCLIPscore

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Post-training quantization can shrink diffusion models, but 4-bit activations usually destroy image quality because of outlier channels. This paper claims that a simple linear preprocessing step — computing each token's mean after scaling its channels and applying a Hadamard transform, then subtracting that mean before quantization — removes enough outliers to make 4-bit weights and activations viable. The paper reports a CLIP score of 31.69 on PixArt-Sigma at W4A4, essentially matching the full-precision 31.66, and shows the extra centering step consistently lowers quantization error across transformer components. If correct, this offers a cheap, calibration-light way to deploy image-generation transformers on memory-constrained devices.

What carries the argument

The load-bearing object is the identity in Equation (1): with input $X$, weight $W$, bias $b$, channel scales $\sigma$, and Hadamard matrix $H$, the paper rewrites $XW + b$ as $(\tilde{X})(\tilde{W}) + \tilde{b}$, where $\tilde{X} = X \,\mathrm{diag}(\sigma^{-1})H - \mu$, $\tilde{W} = H^\top \mathrm{diag}(\sigma)W$, and $\tilde{b} = b + \mu \tilde{W}$. Here $\mu$ is the per-token mean of the scaled-and-rotated activations, computed dynamically; $\sigma$ is fixed from calibration as $\sigma_i = \max(|X_i|)^\alpha / \max(|W_i|)^{1-\alpha}$. The Hadamard transform spreads outlier energy across channels, the channel scaling reduces scale mismatches, and the dynamic centering removes the mean differences that remain after mixing. This makes both activations and weights more quantizable while leaving the mathematical function unchanged.

What would settle it

A held-out evaluation of W4A4 PixArt-Sigma on all COCO 2024 captions with several noise seeds would falsify the empirical claim if HadaNorm's CLIP score falls below SDCB's 31.17 or if its SQNR advantage does not translate into a CLIP advantage. A sharper check already exists inside the paper: at W6A6, HadaNorm has the highest SQNR (3.05 dB) but the lowest CLIP score (31.82) among quantized methods, so reproducing that inversion at 4 bits would directly contradict the mechanism.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a Hadamard transform, although effective at mixing channels to reduce heavy-tailed outliers, leaves channels with substantially different means, which wastes a per-token quantization grid. HadaNorm fixes this by first rescaling each channel with static calibration-based scales, applying the Hadamard transform, and then dynamically subtracting the per-token mean of the transformed vector. Because the whole operation is linear, the layer output can be preserved exactly: the inverse scaling and rotation are absorbed into the weights and the centering is compensated by an added bias term $b + \mu \tilde{W}$. On PixArt-Sigma, this raises W4A4 quantization to a CLIP score of 31.69, matching the full-precision 31.66, and it improves SQNR on nearly every quantized activation in the transformer, with the textual-token quantizer as the lone exception.

Load-bearing premise

The argument assumes that reducing quantization noise in the latent space (higher SQNR) is a reliable proxy for better generated images; the paper's own 6-bit results show a case where higher SQNR coincides with a lower CLIP score.

Editorial extensions

If this is right

PixArt-Sigma can be run with 4-bit weights and activations at a CLIP score of 31.69, slightly above the full-precision 31.66, so no fine-tuning is needed for this aggressive compression level.
The per-token mean is computed on the fly and the inverse of the transform is folded into the weights, so the extra inference cost is just one added token and a bias correction.
The largest SQNR gains appear in the image activations feeding attention and feed-forward blocks and in the output-projection layer, indicating where outlier reduction matters most.
Centering and the Hadamard transform are complementary: the paper's per-component analysis shows the combination, not either transform alone, accounts for the improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors leave implicit that the centering step's benefit depends on how much channel means differ; in architectures where channels are already zero-mean, HadaNorm should reduce to a scaled Hadamard transform, so a useful test is to measure per-channel mean spread before deciding where to insert it.
The paper's own W6A6 results show SQNR and CLIP score moving in opposite directions (HadaNorm has the highest SQNR but the lowest CLIP score among quantized methods), so a fair evaluation of the method should report both quantities rather than SQNR alone.
Because the transformation is linear and output-preserving, it can likely be composed with other quantization tools, such as low-rank outlier absorption or time-dependent transforms, to reach 3-bit weights or to cover video diffusion transformers without retraining.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know about arXiv:2506.09932. First, the core idea is genuinely simple and new in this literature: take the SmoothQuant-style channel scaling, apply a Hadamard transform, and then subtract a per-token mean computed after the transform. The identity in Eq. (1) is exact, the bias can be absorbed into appended tokens, and the overhead is negligible. The ablation (Fig. 4) shows centering and Hadamard each contribute, and dynamic centering on top of HT gives the best SQNR. That part is solid and worth borrowing.

Second, the headline W4A4 result (CLIP 31.69 on PixArt-Sigma, matching full precision 31.66) is the main reason to care, but the paper's own Appendix A, Table 2 breaks the argument. At W6A6, HadaNorm has the highest SQNR (3.05 dB) of all transforms and the lowest CLIP score among quantized methods (31.82), below No Transform (32.39). The authors use SQNR gains throughout as evidence of quality, and their own numbers show the proxy fails at the bitwidth where they report it. Maybe the SQNR-CLIP relationship is nonmonotonic, or the W6A6 CLIP differences are noise, but the paper doesn't address that. Combined with no error bars or multiple seeds, the 0.52 CLIP advantage over SDCB at W4A4 is indistinguishable from evaluation noise. Also, alpha in Eq. (3) is tuned on a calibration set and never reported, so we can't know how much selection went into the result.

What the paper does well: clear problem statement, an exact derivation, the right baselines cited, and the per-component SQNR analysis is informative. The comparison set is reasonable (SmoothQuant, QuaRot, SDCB). I'd have liked AffineQuant or a static-bias comparison, since this is essentially a dynamic affine per-token correction, but that's a minor omission.

Who should read this: anyone doing PTQ for diffusion transformers or outlier-mitigating rotations. It's a five-minute read and the idea can be plugged into existing frameworks. For a referee: I'd send it out, but with a request for major revision. The authors need to report alpha, add seeds/error bars, and either explain the W6A6 contradiction or reframe the claims so they don't rest on SQNR as a proxy. As is, the evidence supports the method as a heuristic that helps at W4A4, not as a demonstrated SOTA.

Referee Report

3 major / 4 minor

Summary. The paper proposes HadaNorm, a linear post-training quantization transformation for diffusion transformers. The transformation combines per-channel scaling, a Hadamard rotation, and dynamic per-token mean centering before activation quantization, with inverse operations folded into the weights and bias. Equation (1) is an exact identity, so the method does not approximate the layer output. Experiments on PixArt-Sigma at W4A4 report the highest SQNR and CLIP score among the compared transforms, with CLIP 31.69 versus the full-precision 31.66, and improved visual outputs in Figure 3. At W6A6 (Appendix A), however, HadaNorm attains the highest SQNR but its CLIP score is lower than several baselines, which undermines the paper's implicit claim that SQNR improvements translate into better generation quality.

Significance. If the W4A4 result is robust, HadaNorm is a valuable practical contribution: it is an exact, cheap, and simple transformation that enables 4-bit activation quantization of a diffusion transformer with essentially no loss in CLIP score. The derivation in Eq. (1) is exact and parameter-free apart from the calibration-based scale, the inverse transforms are fused into weights, and the dynamic centering adds only one extra token to the sequence. The paper also compares against relevant baselines (SmoothQuant, QuaRot, SDCB). The main weakness is that the empirical support is narrow: one model, one dataset subset, no error bars or multiple seeds, and an internal inconsistency between SQNR and CLIP at W6A6. The significance of the method is therefore conditional on resolving these empirical issues.

major comments (3)

[Appendix A, Table 2] The W6A6 results contradict the use of SQNR as a proxy for generation quality. HadaNorm achieves the highest SQNR (3.05 dB) yet its CLIP score (31.82) is below No Transform (32.39), Dynamic Centering (31.87), and QuaRot (31.85), and is essentially tied with SDCB (31.81); CLIP IQA is also flat across all quantized methods while No Transform is highest (0.92). Because Section 4.1 and the abstract present SQNR improvements as evidence that HadaNorm consistently outperforms existing methods, the paper must either explain why the highest SQNR does not yield the best CLIP score at W6A6, or rely on direct generation-quality evidence for the W4A4 claim rather than on SQNR.
[Section 4, Table 1] All end-to-end results are reported for a single model (PixArt-Sigma), a single COCO subset, and without error bars or multiple seeds. The headline W4A4 CLIP gap (31.69 for HadaNorm vs 31.17 for SDCB) is 0.52 points, which could be evaluation noise; the W6A6 table shows a 0.57-point gap in the opposite direction between No Transform and HadaNorm. Please report standard deviations, bootstrap confidence intervals, or results over at least three seeds, and ideally include a second DiT architecture or dataset to support the generality claim.
[Section 3.2, Eq. (3)] The exponent alpha in Eq. (3) is tuned on a disjoint calibration set, but its value is never reported and no sensitivity analysis is provided. Since alpha directly controls the channel scaling that is central to HadaNorm, the reader cannot rule out that the empirical gains depend on a particular choice of alpha or on selection of alpha with respect to the evaluation metric. Please report the chosen alpha values (per layer or a summary) and a sensitivity curve showing CLIP/SQNR as a function of alpha.

minor comments (4)

[Figures 2 and 4] The legend entry 'HadaNorm (no scale)' is never defined in the text; please clarify which component is removed relative to full HadaNorm and why this ablation is included.
[Throughout] The model name is spelled inconsistently as 'Pixart-Sigma' and 'PixArt-Sigma'; please standardize, and also fix the typo 'litteraure' in Section 4.
[Appendix A, Table 2] The fact that No Transform at W6A6 yields CLIP 32.39, above the full-precision Original 31.66, is unexpected and should be commented on, as it suggests metric noise or prompt/sample variation that may affect the comparisons.
[Section 3.1] The phrase 'per-token quantization' is used before the dynamic per-token grid is defined; please define the term explicitly or use a different phrase to avoid confusion with per-channel or per-tensor quantization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HadaNorm's transform is an exact algebraic identity, and the evaluation uses a disjoint calibration/evaluation split.

full rationale

The paper's derivation chain is self-contained. Equation (1) is an exact algebraic identity: XW + b is rewritten using a per-channel scale, a Hadamard transform, and a per-token mean that is absorbed into the bias term, so HadaNorm does not reduce to a fitted quantity renamed as a prediction. The only tunable hyperparameter is alpha in Eq. (3), which is determined on a disjoint calibration set; although the alpha values are not reported, the evaluation uses a separate subset of COCO 2024, so the headline CLIP result is not forced by construction. The paper cites SmoothQuant only for the standard scale formula and compares against external baselines; there are no load-bearing self-citations or imported uniqueness claims. The W6A6 results in Appendix A (highest SQNR of 3.05 dB but lowest CLIP score among quantized methods, 31.82) do undermine the paper's implicit assumption that higher SQNR implies better generation quality, and that is a legitimate correctness risk, but an inconsistent proxy is not circular derivation. No step in the paper reduces, by its own equations or citations, to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method rests on an exact algebraic identity (Eq. 1) and standard Hadamard orthogonality. The only tuned quantity is alpha, fit on a calibration set; the paper does not disclose its value. No new physical or architectural entities are introduced.

free parameters (2)

alpha (scaling exponent) = not reported
In Eq. 3, sigma_i = max(|X_i|)^alpha / max(|W_i|)^{1-alpha}. alpha is tuned on a disjoint calibration set (Sec. 4) and is not disclosed, so the exact method cannot be reproduced or sensitivity assessed.
Calibration dataset statistics = empirically computed
The scales sigma_i are computed from max activations and weights over a calibration set (Eq. 3). The paper does not report the calibration set size or captions used, which affects the resulting sigma.

assumptions (4)

standard math Hadamard transform is orthogonal, H^T H = I
Used in Eq. 1 to fold the transform into weights without changing the model output.
domain assumption Channel-wise activation statistics are approximately stationary across the calibration and test distributions
The static scales sigma_i are derived from a calibration set and applied to test-time activations; if the distribution shifts during the 20 denoising steps used for evaluation, the scales may be suboptimal. The paper does not analyze this shift.
domain assumption CLIP score and CLIP IQA reflect perceptual quality of generated images
Used as evaluation metrics; the paper presents CLIP scores as the main performance measure, but does not validate against human ratings or other IQA metrics.
standard math Central limit theorem motivates that Hadamard mixing yields approximately Gaussian channel distributions
Section 3.1 uses this intuitive motivation to justify HT; it is not a rigorous theorem for correlated channels, but serves as motivation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations." pith.science (2026). https://pith.science/paper/TESJRXNN

@misc{pith2026250609932,
  author       = {Pith},
  title        = {Pith review of: HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TESJRXNN}},
  note         = {Machine review of arXiv:2506.09932}
}

read the original abstract

Diffusion models represent the cutting edge in image generation, but their high memory and computational demands hinder deployment on resource-constrained devices. Post-Training Quantization (PTQ) offers a promising solution by reducing the bitwidth of matrix operations. However, standard PTQ methods struggle with outliers, and achieving higher compression often requires transforming model weights and activations before quantization. In this work, we propose HadaNorm, a novel linear transformation that extends existing approaches by both normalizing channels activations and applying Hadamard transforms to effectively mitigate outliers and enable aggressive activation quantization. We demonstrate that HadaNorm consistently reduces quantization error across the various components of transformer blocks, outperforming state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2506.09932 by the authors.

**Figure 1.** HadaNorm reduces quantization error. We take an illustrative setting of four channels with different distributions (top-left). Normalization (bottom-left) improves quantization, but it does not mix channels and hence cannot get rid of heavy tails. Hadamard transform (HT) (top-right) suffers when the channels have different means. HadaNorm (bottom-right) achieves better whitening, by both normalizing and applying the… view at source ↗

**Figure 2.** HadaNorm reduces quantization error of all quantizers. Effect of activation quantization for various components of the DiT architecture without (left) and with (right) HadaNorm transformations (indicated in purple). Activation Quantizers (circles) are colored by the corresponding impact on the SQNR. The dynamic centering is not applied on the textual quantizer. and assume vectors µ,σ ∈ R d . We can write: XW + b = (… view at source ↗

**Figure 3.** Visualization of the denoised images for the W4A4 quantized model starting from the same noise input and the prompt "An adorable cat attempts to hide in a purse to steal the persons identity". A1_i A2_i IP_i O1_i O2_i OP_i TX Quantized Activation No Transform Dynamic Centering SmoothQuant QuaRot SDCB HadaNorm (no scale) HadaNorm Transformation -0.77 -0.43 -1.9 1.8 4.4 0.66 2.2 -0.58 1.2 0.79 2.2 5.5 0.71 -0.43 -0.39… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: HadaNorm’s gain is mostly due to the combination of centering and the Hadamard transform. Visualization of the SQNR resulting from the quantization of specific activations in the DiT architecture using various transformations. Results. We see ( [PITH_FULL_IMAGE:figure…

**Figure 5.** Figure 5: Additional mage generations for the W4A4 Pixart-Sigma model with several transforms and COCO prompts. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Image generations for the W6A6 Pixart-Sigma model with several transforms and COCO prompts. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 8 canonical work pages

[1]

L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J

Ashkboos, S., Mohtashami, A., Croci, M. L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J. Quarot: Outlier-free 4-bit inference in rotated llms. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neura...

work page 2024
[2]

Pixart-  : Weak-to-strong training of diffusion transformer for 4k text-to-image generation

Chen, J., Ge, C., Xie, E., Wu, Y., Yao, L., Ren, X., Wang, Z., Luo, P., Lu, H., and Li, Z. Pixart-  : Weak-to-strong training of diffusion transformer for 4k text-to-image generation. In Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., and Varol, G. (eds.), Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, Septemb...

doi:10.1007/97 2024
[3]

L., and Choi, Y

Hessel, J., Holtzman, A., Forbes, M., Bras, R. L., and Choi, Y. Clipscore: A reference-free evaluation metric for image captioning. In Moens, M., Huang, X., Specia, L., and Yih, S. W. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 20...

doi:10.18653/v1/2021.emnlp-main.595 2021
[4]

Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models

Li, M., Lin, Y., Zhang, Z., Cai, T., Li, X., Guo, J., Xie, E., Meng, C., Zhu, J., and Han, S. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. CoRR, abs/2411.05007, 2024. doi:10.48550/ARXIV.2411.05007. URL https://doi.org/10.48550/arXiv.2411.05007

work page doi:10.48550/arxiv.2411.05007 2024
[5]

Duquant: Distributing outliers via dual transformation makes stronger quantized llms

Lin, H., Xu, H., Wu, Y., Cui, J., Zhang, Y., Mou, L., Song, L., Sun, Z., and Wei, Y. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural ...

work page 2024
[6]

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (eds.), Computer Vision -- ECCV 2014, pp.\ 740--755, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10602-1

work page 2014
[7]

Spinquant: LLM quantization with learned rotations

Liu, Z., Zhao, C., Fedorov, I., Soran, B., Choudhary, D., Krishnamoorthi, R., Chandra, V., Tian, Y., and Blankevoort, T. Spinquant: LLM quantization with learned rotations. CoRR, abs/2405.16406, 2024. doi:10.48550/ARXIV.2405.16406. URL https://doi.org/10.48550/arXiv.2405.16406

arXiv doi:10.48550/arxiv.2405.16406 2024
[8]

Affinequant: Affine transformation quantization for large language models

Ma, Y., Li, H., Zheng, X., Ling, F., Xiao, X., Wang, R., Wen, S., Chao, F., and Ji, R. Affinequant: Affine transformation quantization for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=of2rhALq8l

work page 2024

Show all 15 references

[9]

W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Meila, M. and Zhang, T. (eds.), Proceedings of the 3...

2021
[10]

Omniquant: Omnidirectionally calibrated quantization for large language models

Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y., and Luo, P. Omniquant: Omnidirectionally calibrated quantization for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, M...

2024
[11]

TR-DQ: time-rotation diffusion quantization

Shao, Y., Lin, D., Zeng, F., Yan, M., Zhang, M., Chen, S., Fan, Y., Yan, Z., Wang, H., Guo, J., Wang, Y., Qin, H., and Tang, H. TR-DQ: time-rotation diffusion quantization. CoRR, abs/2503.06564, 2025. doi:10.48550/ARXIV.2503.06564. URL https://doi.org/10.48550/arXiv.2503.06564

2025 doi arXiv
[12]

Wang, J., Chan, K. C. K., and Loy, C. C. Exploring CLIP for assessing the look and feel of images. In Williams, B., Chen, Y., and Neville, J. (eds.), Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Art...

2023 doi
[13]

Smoothquant: Accurate and efficient post-training quantization for large language models

Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S. Smoothquant: Accurate and efficient post-training quantization for large language models. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Mac...

2023
[14]

Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation

Zhao, T., Fang, T., Huang, H., Wan, R., Soedarmadji, W., Liu, E., Li, S., Lin, Z., Dai, G., Yan, S., Yang, H., Ning, X., and Wang, Y. Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation. In The Thirteenth International Conferen...

2025
[15]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J

Ashkboos, S., Mohtashami, A., Croci, M. L., Li, B., Cameron, P., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J. Quarot: Outlier-free 4-bit inference in rotated llms. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neura...

work page 2024

[2] [2]

Pixart- \( \) : Weak-to-strong training of diffusion transformer for 4k text-to-image generation

Chen, J., Ge, C., Xie, E., Wu, Y., Yao, L., Ren, X., Wang, Z., Luo, P., Lu, H., and Li, Z. Pixart- \( \) : Weak-to-strong training of diffusion transformer for 4k text-to-image generation. In Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., and Varol, G. (eds.), Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, Septemb...

doi:10.1007/97 2024

[3] [3]

L., and Choi, Y

Hessel, J., Holtzman, A., Forbes, M., Bras, R. L., and Choi, Y. Clipscore: A reference-free evaluation metric for image captioning. In Moens, M., Huang, X., Specia, L., and Yih, S. W. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 20...

doi:10.18653/v1/2021.emnlp-main.595 2021

[4] [4]

Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models

Li, M., Lin, Y., Zhang, Z., Cai, T., Li, X., Guo, J., Xie, E., Meng, C., Zhu, J., and Han, S. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. CoRR, abs/2411.05007, 2024. doi:10.48550/ARXIV.2411.05007. URL https://doi.org/10.48550/arXiv.2411.05007

work page doi:10.48550/arxiv.2411.05007 2024

[5] [5]

Duquant: Distributing outliers via dual transformation makes stronger quantized llms

Lin, H., Xu, H., Wu, Y., Cui, J., Zhang, Y., Mou, L., Song, L., Sun, Z., and Wei, Y. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural ...

work page 2024

[6] [6]

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (eds.), Computer Vision -- ECCV 2014, pp.\ 740--755, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10602-1

work page 2014

[7] [7]

Spinquant: LLM quantization with learned rotations

Liu, Z., Zhao, C., Fedorov, I., Soran, B., Choudhary, D., Krishnamoorthi, R., Chandra, V., Tian, Y., and Blankevoort, T. Spinquant: LLM quantization with learned rotations. CoRR, abs/2405.16406, 2024. doi:10.48550/ARXIV.2405.16406. URL https://doi.org/10.48550/arXiv.2405.16406

arXiv doi:10.48550/arxiv.2405.16406 2024

[8] [8]

Affinequant: Affine transformation quantization for large language models

Ma, Y., Li, H., Zheng, X., Ling, F., Xiao, X., Wang, R., Wen, S., Chao, F., and Ji, R. Affinequant: Affine transformation quantization for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=of2rhALq8l

work page 2024