Pith. sign in

REVIEW 3 major objections 4 minor 58 references

Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that pricing each weight's information content during quantization, through a quadratic rate estimate added to the layer-wise loss, lets post-trained networks be stored 20-40 percent smaller than NNCodec at equal accuracy.

desk verdict A genuine extension of OPTQ/OBS with a public implementation, but the headline 20-40% bitrate gain over NNCodec is not yet established because Figure 3's comparison points come from unequal sweeps without a documented extraction rule. read the letter →

arxiv 2505.18758 v1 pith:VSET767Z submitted 2025-05-24 cs.LG

Alexander Conzelmann , Robert Bamler This is my paper
classification cs.LG
keywords post-trainingquantizationentropycodingrate-distortionoptimizationOptimalBrainSurgeonDeepCABACneuralnetworkcompressionNNCodeclayer-wiseloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a post-training compression framework, CERWU, that prices each quantized neural-network weight by the number of bits an entropy coder would actually spend on it, and claims that this pricing belongs inside the quantization objective, not after it. Adding a quadratic estimate of the bit rate to the standard layer-wise squared-error loss, and then applying Optimal Brain Surgeon weight updates to the modified objective, yields compressed networks whose measured bit rates are 20 to 40 percent lower than those of the NNCodec standard at the same accuracy. The paper's case is that quantization and entropy coding should be optimized jointly: quantizing first and compressing afterward, as the round-to-nearest-plus-entropy-coding baseline does, throws away compressibility. If correct, storage-limited edge devices get the same accuracy from files one-fifth to two-fifths smaller, with no slowdown in decompression.

What carries the argument

The load-bearing object is the entropy-regularized layer-wise quadratic loss $\mathcal{L}'_\lambda(\widehat W)=\|WX-\widehat WX\|_2^2+\frac{\lambda\gamma}{2}\|\widehat W\|_2^2$, with $\gamma=1/(\ln(2)\operatorname{Var}(\{W_{ij}\}))$, which approximates the true rate term $\lambda R(\widehat W)$ by the log-width of a Gaussian fit to the layer's weights. Completing the square turns this into a quadratic form with Hessian $H' = 2XX^T + \lambda\gamma I$ and regularized target $W' = W H (H')^{-1}$. The paper applies the Optimal Brain Surgeon (OBS) procedure to this quadratic form: after each weight is rounded, the remaining unquantized weights in the same row are shifted to compensate by the inverse-Hessian-weighted rounding error, and the resulting quadratic cost feeds the grid search. The same curvature thus shapes both which grid point is chosen and how the row adjusts, which is how the rate enters the weight updates rather than only the final entropy coding.

What would settle it

Record the per-weight negative log-probabilities DeepCABAC assigns to CERWU's quantized output and compare $\lambda R(\widehat W)$ against $(\lambda\gamma/2)\|\widehat W\|_2^2$ layer by layer; if the remainder is not small at the settings where the 20-40 percent gain is reported, then the quadratic approximation is doing work the derivation assumes away. A second check is to re-estimate $\gamma$ from the current weights after every row update instead of fixing it from the original layer; if the gain over the $\gamma=0$ variant disappears, the reported edge is an artifact of the frozen $\gamma$.

Watch

Extended reading notes

Core claim

The central claim is that the layer-wise rate-distortion objective $\|\widehat W X - W X\|_2^2 + \lambda R(\widehat W)$, with $R(\widehat W)=-\log_2 P_{\widehat W}(\widehat W)$ for an autoregressive entropy model, can be approximately minimized by replacing the entropy-model rate in the weight-update step with a Gaussian quadratic penalty $(\lambda\gamma/2)\|\widehat W\|_2^2$ whose curvature is $\gamma = 1/(\ln(2)\operatorname{Var}(\{W_{ij}\}))$. That replacement makes the loss quadratic, so the Optimal Brain Surgeon update for each rounded weight has a closed form, while the quantization step itself still consults the true entropy model through a grid search. The paper claims that this procedure, interleaved with the OBS updates, produces more favorable rate-distortion curves than NNCodec on ResNet, VGG, and MobileNet models at ImageNet and CIFAR-10 scale, with 20 to 40 percent lower bit rate at equal accuracy and the same decoder and decompression speed.

Load-bearing premise

The argument stands on the assumption that the actual number of bits the entropy coder needs for a weight stays close to the simple Gaussian-shaped estimate built from the layer's variance; if that gap grows during quantization, the compensating weight updates are no longer moving toward the true compression goal.

Editorial extensions

If this is right

  • At the 20-40 percent bit-rate gain claimed at equal accuracy, a network that once needed 100 MB on an edge device fits in roughly 60-80 MB, with no retraining and with the same entropy decoder as NNCodec.
  • Because the decoder is unchanged, decompression time is the same for CERWU files as for NNCodec files; the compression gain is entirely an encoder-side improvement.
  • The quantizer accepts arbitrary grid sizes, not only powers of two, so users can keep integer-arithmetic inference kernels while letting entropy coding absorb the cost of finer grids.
  • The calibration forward pass is paid once; changing the rate-distortion trade-off parameter or grid size for the same network reuses the Hessian and makes subsequent runs considerably cheaper.
  • The ablation without the $\gamma$ term shows that unrestricted OBS updates drift weights toward larger, harder-to-compress values, so the entropy-regularized update is what keeps the updated weights on the compressible distribution the entropy model expects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: nothing in the derivation is specific to DeepCABAC, so the same quadratic-rate regularization should transfer to other autoregressive entropy models as long as their marginal probability mass is roughly Gaussian; the paper demonstrates the mechanism only with DeepCABAC.
  • Editorial inference: on heavy-tailed weight distributions, a single global variance severely underestimates the true rate of small weights, so the 'small by construction' remainder can grow; a testable fix is to fit $\gamma$ from the empirical entropy curve rather than from the variance.
  • Editorial inference: the paper reports weaker results on a small language model, which points to a natural extension: combine this rate-constrained OBS update with an entropy model designed for large-activation outliers rather than with a Gaussian approximation.
  • Editorial inference: the same rate-pricing principle could be applied to pruning masks, since a pruned weight is just a grid point at zero; the paper notes the information-theoretic formulation subsumes pruning, but does not run that experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes CERWU, a post-training neural-network compression method that combines rate-aware quantization with entropy coding. The authors extend the layer-wise squared-error loss of AdaRound/OPTQ by adding a quadratic rate estimate, then derive entropy-regularized Optimal Brain Surgeon (OBS) weight updates. The method is evaluated on ImageNet and CIFAR-10 CNNs against NNCodec and RTN+EC, with the central claim that CERWU and CERWU-gamma=0 achieve a 20-40% bit-rate reduction over NNCodec at the same accuracy. Appendix C provides a detailed derivation of the update equations and verifies Equation 8.

Significance. If the empirical claims are correct, the paper is a valuable practical contribution to model-storage compression: it gives a clean extension of OBS to rate-constrained quantization, works with arbitrary grids and entropy models, and reports very low bit rates. The derivation in Appendix C is self-contained and verifies the key algebraic claims, and the release of code is a plus. However, the headline 20-40% advantage over NNCodec rests on Figure 3, whose extraction rule is underspecified and which is based on unequal hyperparameter sweeps. The theoretical assumption that the non-quadratic rate remainder is small by construction is also not empirically supported. These issues make the central empirical claim not yet fully established.

major comments (3)
  1. [Section 5.1, Figure 3, Appendix A] The headline claim of 20-40% bit-rate reduction over NNCodec is based on the 'minimum bitrate while retaining 99%/95% of original accuracy' shown in Figure 3, but the paper does not specify how this value is extracted from the Pareto front. If it is simply the lowest-rate measured sweep point whose accuracy is above the threshold, the unmatched sweep grids (a dense grid-size and lambda sweep for CERWU versus a qp sweep in (-38,-4) for NNCodec) can systematically inflate NNCodec's bitrate, because NNCodec may have no point near the threshold. Please state the extraction rule (e.g., interpolation, nearest point, threshold crossing), report the sensitivity of the reported percentages to sweep resolution, or evaluate the comparison on matched rate points.
  2. [Appendix A (MobileNet preprocessing)] For MobileNet models, grouped convolutions are pre-processed into separate single-filter convolutions before applying CERWU. The manuscript does not state whether NNCodec is given the same pre-processed representation or the original grouped convolutions. If NNCodec is not given the same representation, the MobileNet comparison is not on equal footing; if it is, this should be stated explicitly.
  3. [Section 4, after Eq. (10)] The derivation of the entropy-regularized weight updates assumes that the non-quadratic remainder L''_lambda(W_hat) is 'small by construction,' but no evidence is given that this holds for the DeepCABAC entropy model, which is not the Gaussian distribution used to set gamma. If DeepCABAC's probabilities deviate substantially from the Gaussian fit, the updates are not optimal for the true rate-distortion objective. The authors should either justify this assumption empirically (e.g., by measuring the magnitude of the remainder or the accuracy of the Gaussian fit) or rephrase the theoretical claim. This does not invalidate the empirical results for CERWU-gamma=0, but it affects the explanation of why the full method works.
minor comments (4)
  1. [Section 5.1] The phrase 'slight but significant edge' is not supported by confidence intervals or repeated runs; please add error bars or statistical tests for the differences shown in Figure 3.
  2. [Appendix C.5] The phrase 'To proof equivalence' should read 'To prove equivalence'.
  3. [Algorithm 1, line 10] The notation P.autoregressiveUpdate(...) is not defined in the main text; please clarify the interface to the entropy model.
  4. [Figures 1 and 2] The inset panels are very small and the line styles are hard to distinguish in the printed version; consider larger insets or separate panels.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the rate term is part of the optimized objective, and the headline compression gain is an empirical comparison against the external NNCodec baseline.

full rationale

The derivation chain is self-contained rather than circular. The rate term in the objective (Eq. 6) is defined as R(Ŵ) = -log2 P_Ŵ(Ŵ) and the same entropy model P is used both in the quantization rule (Eq. 12) and in the final entropy coding, which is the standard rate–distortion construction, not a fit to the reported bit rate: the bit rates are measured after actual coding with DeepCABAC, not taken from the model. The quadratic rate estimate λγ||Ŵ||²/2 in Eq. 7 is derived from the weight variance (γ = 1/(ln 2 · Var({W_ij}))), not fitted to target accuracy or to the final bit rate, and the method's hyperparameters λ and grid size k are swept to form a Pareto front rather than tuned to a pre-specified result. The OBS update derivation is explicitly attributed to Hassibi et al. and Frantar et al. and is carried out for the quadratic part L'_λ; the statement 'assuming that the non-quadratic remainder L''_λ(Ŵ) is small by construction' is an approximation assumption, not a definition that makes the conclusion true by construction. The self-citations [4] and [57] appear only as background on ANS and non-uniform quantization grids and are not load-bearing. The 20–40% improvement over NNCodec is an empirical comparison against an external baseline; concerns about unequal sweep granularity in Fig. 3 would be experimental fairness issues, not circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The method relies on standard information theory, the OBS update rule, and a domain assumption that the layer-wise loss is a good proxy for accuracy. The Gaussian rate approximation is an ad hoc modeling choice, and lambda, grid size, and scan order are swept hyperparameters that shape the reported Pareto fronts.

free parameters (4)
  • lambda (rate-distortion trade-off) = swept 10^-8 to 10^-1 in log space
    Controls the strength of the rate term; final rate-distortion curves select the best Pareto point over the sweep.
  • Grid size k = sweep over subset of {4,6,8,12,16,32,48,64,128,256,512,1024}
    The discrete reconstruction grid size is a user choice swept to build rate-distortion curves.
  • Scan order = row-major or column-major, best selected per method
    For autoregressive entropy models, the order affects the achievable rate; the paper selects the better version per method.
  • gamma (quadratic rate scaling) = 1 / (ln(2) * Var({W_ij}))
    Ad hoc scaling that makes the rate term quadratic; it is computed from the unquantized weights rather than tuned to test accuracy, but it is a modeling choice.
assumptions (6)
  • standard math The source coding theorem and existence of near-optimal entropy coders such as arithmetic coding and ANS.
    Section 3.2 uses these to equate bit rate with negative log probability.
  • standard math The Optimal Brain Surgeon update equations from Hassibi et al. apply to the quadratic loss L'.
    Section 4 and Appendix C reuse OBS; the derivation is acknowledged as a combination of Hassibi and Frantar.
  • domain assumption The layer-wise loss ||W X - W_hat X||^2 is a good proxy for final model accuracy.
    Used throughout Section 4 and Section 5; OPTQ and AdaRound rely on the same assumption.
  • domain assumption The non-quadratic remainder L'' is small by construction, so minimizing the quadratic approximation L' is a valid surrogate for the true rate-distortion objective.
    Stated in Section 4 after Equation 10; this is the weakest load-bearing assumption.
  • domain assumption The calibration set X (40,000 ImageNet or 64,000 CIFAR-10 samples) is representative for computing layer-wise Hessians.
    Appendix A; if the calibration set is not representative, the W' initialization and updates are biased.
  • domain assumption DeepCABAC provides a good autoregressive entropy model for the quantized weights.
    Used as the sole entropy model; the rate term and quantization step rely on its probability estimates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding." pith.science (2026). https://pith.science/paper/VSET767Z

@misc{pith2026250518758,
  author       = {Pith},
  title        = {Pith review of: Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VSET767Z}},
  note         = {Machine review of arXiv:2505.18758}
}
read the original abstract

The ever-growing size of neural networks poses serious challenges on resource-constrained devices, such as embedded sensors. Compression algorithms that reduce their size can mitigate these problems, provided that model performance stays close to the original. We propose a novel post-training compression framework that combines rate-aware quantization with entropy coding by (1) extending the well-known layer-wise loss by a quadratic rate estimation, and (2) providing locally exact solutions to this modified objective following the Optimal Brain Surgeon (OBS) method. Our method allows for very fast decoding and is compatible with arbitrary quantization grids. We verify our results empirically by testing on various computer-vision networks, achieving a 20-40\% decrease in bit rate at the same performance as the popular compression algorithm NNCodec. Our code is available at https://github.com/Conzel/cerwu.

Figures

Figures reproduced from arXiv: 2505.18758 by the authors.

Figure 1
Figure 1. Performance of our compression methods on various networks trained and evaluated [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Performance of compression methods for CIFAR10-trained networks, analogous to [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Minimum bits per weight achieved at 99% (left) and 95% (right) of the original test accuracy [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Run times for compressing ResNets of differing sizes. Left: run times for the first run; [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Rate-distortion performance of our methods on the small language model Pythia-70M. We [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 53 canonical work pages

  1. [1]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs, October 2024

    work page 2024

  2. [2]

    GPTVQ: The blessing of dimensionality for LLM quantization

    Mart Van Baalen, Andrey Kuzmin, Markus Nagel, Peter Couperus, Artem Bolshakov, Cedric Bastoul, Eric Mahurin, Tijmen Blankevoort, and Paul Whatmough. GPTVQ: The blessing of dimensionality for LLM quantization. InWorkshop on Efficient Systems for Foundation Models II, International Conference on Machine Learning (ICML), 2024

    work page 2024

  3. [3]

    ONNX: Open neural network exchange, 2019

    Junjie Bai, Fang Lu, Ke Zhang, et al. ONNX: Open neural network exchange, 2019

    work page 2019

  4. [4]

    Understanding entropy coding with asymmetric numeral systems (ans): a statistician’s perspective.arXiv preprint arXiv:2201.01741, 2022

    Robert Bamler. Understanding entropy coding with asymmetric numeral systems (ans): a statistician’s perspective.arXiv preprint arXiv:2201.01741, 2022

    arXiv 2022

  5. [5]

    Bronstein, and Avi Mendelson

    Chaim Baskin, Brian Chmiel, Evgenii Zheltonozhskii, Ron Banner, Alex M. Bronstein, and Avi Mendelson. CAT: Compression-Aware Training for bandwidth reduction.Journal of Machine Learning Research, 22(269):1–20, 2021

    work page 2021

  6. [6]

    NNCodec: An Open Source Software Implementation of the Neural Network Coding ISO/IEC Standard

    Daniel Becking, Paul Haase, Heiner Kirchhoffer, Karsten Müller, Wojciech Samek, and Detlev Marpe. NNCodec: An Open Source Software Implementation of the Neural Network Coding ISO/IEC Standard. InWorkshop Neural Compression: From Information Theory to Applications, International Conference on Machine Learning (ICML), July 2023

    work page 2023

  7. [7]

    EfficientQAT: Efficient Quantization-Aware Training for Large Language Models, October 2024

    Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. EfficientQAT: Efficient Quantization-Aware Training for Large Language Models, October 2024

    work page 2024

  8. [8]

    Bronstein, and Avi Mendelson

    Brian Chmiel, Chaim Baskin, Ron Banner, Evgenii Zheltonozhskii, Yevgeny Yermolin, Alex Karbachevsky, Alex M. Bronstein, and Avi Mendelson. Feature Map Transform Coding for Energy-Efficient CNN Inference, September 2019

    work page 2019

Show all 58 references
  1. [9]

    Universal Deep Neural Network Compres- sion.IEEE Journal of Selected Topics in Signal Processing, 14(4):715–726, May 2020

    Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. Universal Deep Neural Network Compres- sion.IEEE Journal of Selected Topics in Signal Processing, 14(4):715–726, May 2020

    2020

  2. [10]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  3. [11]

    GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale.Neural Information Processing Systems (NeurIPS), January 2022

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. GPT3.int8(): 8-bit Matrix Multiplication for Transformers at Scale.Neural Information Processing Systems (NeurIPS), January 2022

    2022

  4. [12]

    QLoRA: Efficient Finetuning of Quantized LLMs, May 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient Finetuning of Quantized LLMs, May 2023

    2023

  5. [13]

    Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh

    Tim Dettmers, Ruslan A. Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression. InThe Twelfth International Conferen...

    2023

  6. [14]

    The case for 4-bit precision: K-bit Inference Scaling Laws

    Tim Dettmers and Luke Zettlemoyer. The case for 4-bit precision: K-bit Inference Scaling Laws. InProceedings of the 40th International Conference on Machine Learning, pages 7750–7774. PMLR, July 2023

    2023

  7. [15]

    STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs, August 2024

    Peijie Dong, Lujun Li, Dayou Du, Yuhan Chen, Zhenheng Tang, Qiang Wang, Wei Xue, Wenhan Luo, Qifeng Liu, Yike Guo, and Xiaowen Chu. STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs, August 2024

    2024

  8. [16]

    The use of asymmetric numeral systems as an accurate replacement for huffman coding

    Jarek Duda, Khalid Tahboub, Neeraj J Gadgil, and Edward J Delp. The use of asymmetric numeral systems as an accurate replacement for huffman coding. In2015 Picture Coding Symposium (PCS), pages 65–69. IEEE, 2015

    2015

  9. [17]

    Extreme Compression of Large Language Models via Additive Quantization, September 2024

    Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, and Dan Alistarh. Extreme Compression of Large Language Models via Additive Quantization, September 2024

    2024

  10. [18]

    Optimal Brain Compression: A Framework for Accurate Post- Training Quantization and Pruning

    Elias Frantar and Dan Alistarh. Optimal Brain Compression: A Framework for Accurate Post- Training Quantization and Pruning. InAdvances in Neural Information Processing Systems (NeurIPS), volume 35, pages 4475–4488, December 2022

    2022

  11. [19]

    SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot, March 2023

    Elias Frantar and Dan Alistarh. SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot, March 2023

    2023

  12. [20]

    OPTQ: Accurate quan- tization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quan- tization for generative pre-trained transformers. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  13. [21]

    Compression Scaling Laws:Unifying Sparsity and Quantization, February 2025

    Elias Frantar, Utku Evci, Wonpyo Park, Neil Houlsby, and Dan Alistarh. Compression Scaling Laws:Unifying Sparsity and Quantization, February 2025

    2025

  14. [22]

    MiniLLM: Knowledge Distillation of Large Language Models

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. MiniLLM: Knowledge Distillation of Large Language Models. InThe Twelfth International Conference on Learning Representations, October 2023

    2023

  15. [23]

    Song Han, Huizi Mao, and William J. Dally. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding, February 2016

    2016

  16. [24]

    NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks, October 2024

    Yongchang Hao, Yanshuai Cao, and Lili Mou. NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks, October 2024

    2024

  17. [25]

    Hassibi, D.G

    B. Hassibi, D.G. Stork, and G.J. Wolff. Optimal Brain Surgeon and general network pruning. InIEEE International Conference on Neural Networks, pages 293–299 vol.1, March 1993

    1993

  18. [26]

    Deep Residual Learning for Image Recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, June 2016

    2016

  19. [27]

    Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experi- ences

    Fred Hohman, Mary Beth Kery, Donghao Ren, and Dominik Moritz. Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experi- ences. InProceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–18, May 2024

    2024

  20. [28]

    Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization, July 2024

    2024

  21. [29]

    Le, and Hartwig Adam

    Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V . Le, and Hartwig Adam. Searching for MobileNetV3. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 13...

    2019

  22. [30]

    Accurate Post Training Quantization With Small Calibration Sets

    Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate Post Training Quantization With Small Calibration Sets. InProceedings of the 38th International Conference on Machine Learning, pages 4466–4475. PMLR, July 2021. 11

    2021

  23. [31]

    Mahoney, and Kurt Keutzer

    Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, and Kurt Keutzer. SqueezeLLM: Dense-and-Sparse Quantization, February 2024

    2024

  24. [32]

    Aksu, Miska M

    Heiner Kirchhoffer, Paul Haase, Wojciech Samek, Karsten Müller, Hamed Rezazadegan- Tavakoli, Francesco Cricri, Emre B. Aksu, Miska M. Hannuksela, Wei Jiang, Wei Wang, Shan Liu, Swayambhoo Jain, Shahab Hamidi-Rad, Fabien Racapé, and Werner Bailer. Overview of the Neural Network...

    2022

  25. [33]

    Adaptive weight compression for memory-efficient neural networks

    Jong Hwan Ko, Duckhwan Kim, Taesik Na, Jaeha Kung, and Saibal Mukhopadhyay. Adaptive weight compression for memory-efficient neural networks. InDesign, Automation & Test in Europe Conference & Exhibition (DATE), 2017, pages 199–204, March 2017

    2017

  26. [34]

    Cifar-10 (canadian institute for advanced research)

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research)

  27. [35]

    Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channels

    Mounssif Krouka, Anis Elgabli, Chaouki Ben Issaid, and Mehdi Bennis. Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channels. In2021 IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PI...

    2021

  28. [36]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention, September 2023

    2023

  29. [37]

    Memory Efficient Optimizers with 4-bit States.Advances in Neural Information Processing Systems, 36:15136–15171, December 2023

    Bingrui Li, Jianfei Chen, and Jun Zhu. Memory Efficient Optimizers with 4-bit States.Advances in Neural Information Processing Systems, 36:15136–15171, December 2023

    2023

  30. [38]

    PENNI: Pruned Kernel Sharing for Efficient CNN Inference

    Shiyu Li, Edward Hanson, Hai Li, and Yiran Chen. PENNI: Pruned Kernel Sharing for Efficient CNN Inference. InInternational Conference on Machine Learning (ICML), pages 5863–5873. PMLR, November 2020

    2020

  31. [39]

    AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration, April 2024

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration, April 2024

    2024

  32. [40]

    Cambridge university press, 2003

    David JC MacKay and David JC Mac Kay.Information theory, inference and learning algo- rithms. Cambridge university press, 2003

    2003

  33. [41]

    Range encoding: an algorithm for removing redundancy from a digitised message

    G Nigel N Martin. Range encoding: an algorithm for removing redundancy from a digitised message. InProc. Institution of Electronic and Radio Engineers International Conference on Video and Data Recording, page 48, 1979

    1979

  34. [42]

    Up or Down? Adaptive Rounding for Post-Training Quantization

    Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or Down? Adaptive Rounding for Post-Training Quantization. InInternational Conference on Machine Learning (ICML), pages 7197–7206, November 2020

    2020

  35. [43]

    A White Paper on Neural Network Quantization, June 2021

    Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and Tijmen Blankevoort. A White Paper on Neural Network Quantization, June 2021

    2021

  36. [44]

    Kübler, Jiaji Huang, Matthäus Kleindessner, Jun Huan, V olkan Cevher, Yida Wang, and George Karypis

    Youngsuk Park, Kailash Budhathoki, Liangfu Chen, Jonas M. Kübler, Jiaji Huang, Matthäus Kleindessner, Jun Huan, V olkan Cevher, Yida Wang, and George Karypis. Inference Optimiza- tion of Foundation Models on AI Accelerators. InProceedings of the 30th ACM SIGKDD Conference on K...

    2024

  37. [45]

    PhD thesis, Stanford University CA, 1976

    Richard Clark Pasco.Source coding algorithms for fast data compression. PhD thesis, Stanford University CA, 1976

    1976

  38. [46]

    PyTorch: An Imperative Style, High-Performance Deep Learning Library, December 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fan...

    2019

  39. [47]

    Accurate LoRA-Finetuning Quantization of LLMs via Information Retention, May 2024

    Haotong Qin, Xudong Ma, Xingyu Zheng, Xiaoyang Li, Yang Zhang, Shouda Liu, Jie Luo, Xianglong Liu, and Michele Magno. Accurate LoRA-Finetuning Quantization of LLMs via Information Retention, May 2024

    2024

  40. [48]

    Arithmetic coding.IBM Journal of research and development, 23(2):149–162, 1979

    Jorma Rissanen and Glen G Langdon. Arithmetic coding.IBM Journal of research and development, 23(2):149–162, 1979

    1979

  41. [49]

    Kavya Saravanan and Abbas Z. Kouzani. Advancements in On-Device Deep Neural Networks. Information, 14(8):470, August 2023

    2023

  42. [50]

    C. E. Shannon. A mathematical theory of communication.The Bell System Technical Journal, 27(3):379–423, July 1948

    1948

  43. [51]

    FlexGen: High-Throughput Generative Infer- ence of Large Language Models with a Single GPU

    Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher Re, Ion Stoica, and Ce Zhang. FlexGen: High-Throughput Generative Infer- ence of Large Language Models with a Single GPU. InProceedings of the 40th International Conference ...

    2023

  44. [52]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. InInternational Conference on Learning Representations (ICLR), 2015

    2015

  45. [53]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A Simple and Effective Pruning Approach for Large Language Models, May 2024

    2024

  46. [54]

    QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks, June 2024

    Albert Tseng, Jerry Chee, Qingyao Sun, V olodymyr Kuleshov, and Christopher De Sa. QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks, June 2024

    2024

  47. [55]

    DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks.IEEE Journal of Selected Topics in Signal Processing, 14(4):700–714, May 2020

    Simon Wiedemann, Heiner Kirchoffer, Stefan Matlage, Paul Haase, Arturo Marban, Talmaj Marinc, David Neumann, Tung Nguyen, Ahmed Osman, Detlev Marpe, Heiko Schwarz, Thomas Wiegand, and Wojciech Samek. DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks.IEEE Jo...

    2020

  48. [56]

    Compact and computationally efficient representation of deep neural networks.IEEE Transactions on Neural Networks and Learning Systems, 31(3):772–785, 2020

    Simon Wiedemann, Klaus-Robert Müller, and Wojciech Samek. Compact and computationally efficient representation of deep neural networks.IEEE Transactions on Neural Networks and Learning Systems, 31(3):772–785, 2020

    2020

  49. [57]

    Variational bayesian quantization

    Yibo Yang, Robert Bamler, and Stephan Mandt. Variational bayesian quantization. InInterna- tional Conference on Machine Learning, pages 10670–10680. PMLR, 2020

    2020

  50. [58]

    optimally compensate

    J. Ziv. On universal quantization.IEEE Transactions on Information Theory, 31(3):344–347, May 1985. 13 A Experimental Details Evaluation details.For the tested networks, we report the accuracy, which is the top-1 accuracy and the average number of bits required to encode one w...

    1985

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.