Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Quantizing only pointwise convolutions to ternary weights lets a 1.25x MobileNetV2 match float16 accuracy at 23.9x lower energy.

desk verdict PROM's recipe is simple and the accuracy results look credible, but the headline 23.9x energy claim rests on an op-count model that probably overstates real hardware savings. read the letter →

arxiv 2505.03254 v2 pith:37ABDDUQ submitted 2025-05-06 cs.CV cs.LG

Lukas Meiner , Jens Mehnert , Alexandru Paul Condurache This is my paper · ORCID
classification cs.CVcs.LG
keywords quantizationternaryweightsmixed-precisiondepthwise-separableCNNsMobileNetV2energy-efficientinferenceint8additionsImage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PROM (Prioritize Reduction of Multiplications Over Lower Bit-Widths) is a mixed-precision quantization recipe for depthwise-separable CNNs: it quantizes only the pointwise 1x1 convolutions, the layers that dominate energy cost, to ternary weights, while keeping all other weights and activations at 8 bits. Because multiplying by a ternary weight is adding, subtracting, or skipping, those expensive layers become int8 additions, and the paper argues this removes more energy than shrinking every layer to 2 or 4 bits would. On ImageNet, a 1.25x MobileNetV2 quantized this way scores 71.58% top-1 accuracy, effectively matching the 71.78% of a float16 1.0x MobileNetV2, with an estimated 23.9x reduction in per-forward-pass energy and 2.7x reduction in storage. The paper claims this fixed, hardware-friendly scheme advances the Pareto frontier of accuracy versus energy for quantized CNNs on ImageNet.

What carries the argument

The load-bearing mechanism is the ternary pointwise convolution implemented as an int8-addition kernel. With weights restricted to $\{-1,0,1\}$, a $1\times1$ convolution needs no multipliers: each output is a sum of the input channels whose weight is $+1$ minus the channels whose weight is $-1$, followed by a per-channel scale. The scale comes from channel-wise absmean quantization, $\alpha_i = \frac{1}{C_{\mathrm{in}}}\sum_j |W_{i,j}|$, with rounding and clamping to the ternary set; depthwise convolutions and activations use 8-bit absmax quantization so the bulk of arithmetic stays native int8. The straight-through estimator lets gradients pass through the round function during quantization-aware training, and the paper's cost analysis explains why this trades a little capacity in cheap layers for a large reduction in expensive ones.

What would settle it

Run the PROM-quantized MobileNetV2 on an edge accelerator and measure per-layer energy with hardware counters; if the ternary pointwise layers cost substantially more than an int8-add-only kernel because scaling, batch-norm folding, or memory traffic dominates, the claimed 23.9x reduction over float16 will not reproduce.

Watch

Extended reading notes

Core claim

The central discovery is that the efficiency of quantizing a depthwise-separable CNN is governed by where the cost sits, not only by bit-width. In MobileNetV2, pointwise convolutions account for about 90% of estimated inference energy, depthwise convolutions for about 9.5%, and multiplications dominate additions even at low bit-widths (in an int2 model, multiplications cost about 9.5x more than all additions combined). PROM therefore assigns ternary weights to all pointwise convolutions, using channel-wise absmean quantization, and 8-bit weights to depthwise convolutions and the linear layer, with 8-bit activations. At inference, a ternary-weight 1x1 convolution with int8 activations becomes a sum of selected input channels, eliminating multiplications in the most expensive layers. With quantization-aware training, a 1.25x MobileNetV2 reaches 71.58% top-1 on ImageNet, close to the 71.78% of the float16 1.0x baseline, while using 2.7x less storage and an estimated 23.9x less energy per forward pass; wider and deeper variants exceed the float16 baseline's accuracy while remaining far cheaper.

Load-bearing premise

The headline energy saving rests on the assumption that a real kernel for a ternary-weight pointwise convolution runs at the cost of int8 additions alone, with sub-8-bit baselines costing half as much per bit; if actual hardware pays overheads for scaling, packing, or memory movement, the 23.9x figure shrinks even though the accuracy result stands.

Editorial extensions

If this is right

  • A 1.25x MobileNetV2 with PROM reaches 71.58% top-1 on ImageNet, essentially matching the 71.78% of a float16 1.0x MobileNetV2, with 23.9x less estimated energy and 2.7x less storage.
  • Scaling the recipe restores accuracy lost to ternary weights: the 2.0x MobileNetV2 scores 74.81% top-1, 3.03 points above the float16 1.0x baseline, while still using 1.4x less storage and 11.9x less energy on 7nm.
  • PROM beats learned mixed-precision quantization on the paper's energy metric: at comparable top-1 accuracy it uses 36% less energy on 7nm than the best learned mixed-precision MobileNetV2 result.
  • Because all arithmetic except the final per-channel scales is int8, PROM runs with native int8 instructions on commodity CPUs and GPUs, avoiding the packing and unpacking overhead of sub-8-bit schemes.
  • Width and depth scaling transfer the recipe to RegNetX and ResNeXt, where PROM models use up to 4.8x and 5.6x less storage, respectively, at comparable accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same cost-geometry argument should transfer to other depthwise-separable families, such as EfficientNet or MobileNetV3, where pointwise layers dominate similarly; applying PROM there and measuring accuracy-versus-energy is a direct test.
  • Editorial inference: the paper's observation that wider PROM models learn many zero-valued ternary weights suggests that combining this recipe with structured pruning or zero-skipping hardware could compound the savings beyond the reported 23.9x.
  • Editorial inference: since the energy accounting excludes memory transfer but PROM also cuts storage by up to 5.6x, a memory-aware energy model would likely widen the gap to the float16 baseline while possibly narrowing the gap to very compact 2-bit models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes PROM, a quantization-aware training recipe for depthwise-separable CNNs in which pointwise 1x1 convolutions use ternary weights ({-1,0,1} via channel-wise absmean quantization), while depthwise convolutions and linear layers use 8-bit weights, and activations are quantized to 8-bit tensor-wise. The motivation is a cost analysis showing that pointwise convolutions dominate energy in MobileNetV2, RegNet, and ResNeXt, and that multiplications dominate over additions. During inference, ternary pointwise convolutions are presented as int8 addition operations, eliminating the need for multiplications in those layers. Experiments on ImageNet report top-1 accuracies for MobileNetV2 width multipliers from 0.75x to 2.0x, RegNetX, and ResNeXt, with the headline result that a 1.25x MobileNetV2 reaches 71.58% top-1, comparable to the 71.78% of a float16 1.0x MobileNetV2, while estimated inference energy drops 23.9x and storage drops 2.7x. The paper asserts that this advances the accuracy-energy Pareto frontier for quantized CNNs on ImageNet.

Significance. The contribution is a simple and practical quantization recipe that is more hardware-friendly than sub-8-bit schemes and avoids learned per-layer bit assignments. Strengths include a clear cost analysis, an ablation study over key design choices with three seeds, a breadth of architectures, and an explicit discussion of hardware instruction support. The pseudocode and training modifications are described in enough detail to reproduce the accuracy results if the table inconsistency discussed below is resolved. However, the headline energy reductions are analytical estimates from operation counts and external energy tables, not measured inference, and the cost model for ternary pointwise convolutions is idealized. The paper would be a solid empirical contribution if the energy claims are either validated at the kernel level or appropriately reframed; as written, the central quantitative claim is not fully established.

major comments (2)
  1. [Section 3.1 and Section 4, Table 1] The energy model counts one int8 ADD per input channel for each ternary pointwise convolution, but the reduction over C_in signed 8-bit activations requires a widening accumulator (int16 or int32) for correct results. The cited Horowitz/Zhang tables report energy for byte-wide ADD and MUL instructions, not for widening accumulation trees, so the cost of this accumulation is undercounted. In addition, the inference path in Figure 12 performs per-channel scale_weight and per-tensor scale_x dequantization after every pointwise convolution; even if these are folded into BatchNorm parameters, they are part of the convolution kernel's post-processing on real hardware and are not captured by the op counts. Because the 23.9x energy claim and the Pareto-frontier comparisons in Figure 1 and Tables 1-4 rest on this model, the quantitative energy advantage of PROM is not yet established. Please provide a kernel-level implementation or a careful sensitivity analysis (e.g., int16 accumulator, scaling overhead, SIMD throughput) rather than only the idealized ADD/MUL op counts.
  2. [Table 5 vs. Table 3] The ablation row for ResNeXt-50 with 8-bit depthwise weights, cosine decay, weight-decay reset, no PReLU, and per-tensor pointwise quantization reports 75.71 +/- 0.04 top-1, yet Table 3 lists the same PROM ResNeXt-50(32x4d) configuration as 76.71. Since Section 3.2 applies PReLU only to MobileNetV2 and RegNetX, the 75.71 row appears to be the configuration used for Table 3. The unexplained 1.0-point gap undermines the cross-architecture claims and must be reconciled by correcting one of the tables or by stating the exact configuration and seeds used for the main ResNeXt experiments.
minor comments (5)
  1. [Section 6, Related Work] The word 'adpoted' in the first paragraph should be 'adopted'.
  2. [Table 1] The PROM-1.5xMobileNetV2 row appears twice with identical entries; the duplicate should be removed.
  3. [Tables 1-4] The W/A notation '(1.58/8)/8' should be defined explicitly in a table note; as written, it is easy to misread as a three-component bit-width, and the distinction between pointwise and non-pointwise weights is not visible from the column header.
  4. [Section 4, Energy] The claim that 'memory transfer typically scales proportionately with a reduction in model size' is not self-evident for activation memory traffic, which is often the dominant memory cost in mobile inference kernels; please qualify this statement.
  5. [Figure 12 and Section 3.1] The inference pseudocode dequantizes by division by scale_weight and scale_x after the convolution, while the main text says the convolution output is scaled by alpha afterwards; please specify the exact inference arithmetic (including the handling of BatchNorm folding) unambiguously.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: PROM's accuracy is measured on ImageNet and its energy savings are computed from an external op-count model, not fitted to or forced by the paper's own outputs.

full rationale

The paper does not claim to derive accuracy from an efficiency model or vice versa. Section 3 defines the ternary/8-bit quantization scheme, and Section 5 reports ImageNet top-1 accuracies from actual quantization-aware training runs; these numbers are not fitted to, nor forced by, the energy estimates. The 23.9x energy claim in Section 4 is an operation-count estimate using external per-operation energy tables from Horowitz and Zhang, with the paper's own explicitly stated conventions, including excluding elementwise operations and memory transfer and conservatively assuming halved energy for sub-8-bit methods. The reduction of ternary pointwise convolutions to int8 additions follows directly from the weight set {-1,0,1} and is a structural property, not a parameter fitted to the target claim. The Pareto-frontier comparison is against externally reported baselines and published numbers, not against a quantity defined by PROM. The only self-citations, for example reference [33] and the pruning survey references [49-51], appear in related-work context and are not load-bearing for the main claim. Concerns about the validity of the energy model, such as wider accumulation registers or omitted rescaling costs, are correctness and validity risks, not circularity. Therefore no circular step is identifiable.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The efficiency claims depend on external energy tables, an assumed halving rule for sub-8-bit arithmetic, and the exclusion of elementwise operations; no fitted parameters are used to obtain the accuracy results. The bit-width assignment is a hand-chosen design. No new physical entities are introduced.

free parameters (3)
  • sub-8-bit energy scaling assumption = energy halves per bit step (int8 to int4 to int2)
    Section 4 states 'the energy cost halves when going from int8 to int4, and from int4 to int2'. This arbitrary halving rule changes the relative energy ranking of PROM versus sub-8-bit competitors.
  • bit-width assignment (pointwise=ternary, others=8-bit) = 1.58-bit / 8-bit
    The core recipe is hand-chosen based on a cost analysis, not derived from an optimization objective; a different split would change the accuracy/efficiency trade-off.
  • epsilon in quantization = 1e-5
    Numerical stabilizer in Equations (3), (5), and (7); not load-bearing.
assumptions (4)
  • domain assumption Operation-level energy tables from Horowitz [20] and PokeBNN [55] provide valid relative costs for int8 ADD and MUL on modern nodes.
    Section 4: all energy results are computed from these tables; if they are inaccurate, the 23.9x claim changes.
  • ad hoc to paper Sub-8-bit energy halves with each bit reduction and packing/unpacking overhead is negligible.
    Section 4: 'we make the assumption that the energy cost halves'. This understates competitor energy, making PROM's advantage conservative, but it is unmeasured.
  • domain assumption Elementwise operations (BatchNorm, scaling) can be excluded from energy comparisons without changing relative rankings.
    Section 4 and Figure 2 exclude BatchNorm and elementwise ops; if these are significant, absolute energy numbers shift.
  • domain assumption Straight-through estimator and standard QAT suffice to train ternary weights in these architectures.
    Section 3.2 relies on STE for gradient flow, a standard but heuristic technique.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs." pith.science (2026). https://pith.science/paper/37ABDDUQ

@misc{pith2026250503254,
  author       = {Pith},
  title        = {Pith review of: PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37ABDDUQ}},
  note         = {Machine review of arXiv:2505.03254}
}
read the original abstract

Convolutional neural networks (CNNs) are crucial for computer vision tasks on resource-constrained devices. Quantization effectively compresses these models, reducing storage size and energy cost. However, in modern depthwise-separable architectures, the computational cost is distributed unevenly across its components, with pointwise operations being the most expensive. By applying a general quantization scheme to this imbalanced cost distribution, existing quantization approaches fail to fully exploit potential efficiency gains. To this end, we introduce PROM, a straightforward approach for quantizing modern depthwise-separable convolutional networks by selectively using two distinct bit-widths. Specifically, pointwise convolutions are quantized to ternary weights, while the remaining modules use 8-bit weights, which is achieved through a simple quantization-aware training procedure. Additionally, by quantizing activations to 8-bit, our method transforms pointwise convolutions with ternary weights into int8 additions, which enjoy broad support across hardware platforms and effectively eliminates the need for expensive multiplications. Applying PROM to MobileNetV2 reduces the model's energy cost by more than an order of magnitude (23.9x) and its storage size by 2.7x compared to the float16 baseline while retaining similar classification performance on ImageNet. Our method advances the Pareto frontier for energy consumption vs. top-1 accuracy for quantized convolutional models on ImageNet. PROM addresses the challenges of quantizing depthwise-separable convolutional networks to both ternary and 8-bit weights, offering a simple way to reduce energy cost and storage size.

Figures

Figures reproduced from arXiv: 2505.03254 by the authors.

Figure 1
Figure 1. Comparison of quantized CNNs on ImageNet in terms of the trade-off between accuracy and energy consumption per forward pass in microjoules. The shape of a marker represents the underlying model architecture, while its color represents the quantization method used. quantization can reduce floating-point weights and activations to lower bit-width integer formats, reducing both model size and com￾putational cost. Quant… view at source ↗
Figure 3
Figure 3. The energy consumption per inference pass of a MobileNetV2 model, quantized to different bit-widths (int8, int4, int2 and our proposed method), with the float16 model as a baseline for comparison. We visualize the cost of additions separately from the cost of multiplications. We find that multiplications consume the most energy by far, which is especially notable for the int8, int4 and int2 models. to integer weight… view at source ↗
Figure 2
Figure 2. The distribution of energy cost per forward pass for different model architectures. Pointwise convolutions dominate the energy consumption, while depthwise convolutions and the linear layer require very little energy. Cost is Shifted Towards Pointwise Convolutions. For all of these models, the cost between different operations is not evenly dis￾tributed. Taking the MobileNetV2 model as an example, the depth￾wise con… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparison between a regular depthwise-separable block structure and our quantized version. We quantize pointwise convolutions to ternary weights using channel-wise absmean quantization. Depthwise convolutions are quantized to 8-bit integers using channel-wise absmax q…
Figure 5
Figure 5. Figure 5: Accuracy-resource trade-off for models quantized with PROM compared to a float16 baseline MobileNetV2 architecture. Each curve shows Top-1 accuracy as we sweep the width multiplier (denoted in gray) for the model from 0.75× (float16 models) or 1.0 × (PROM models) to 2.…
Figure 6
Figure 6. Figure 6: Comparison with binary models in the trade-off between task performance and model size. In terms of the trade-off between model size and top-1 accuracy, we analyze the performance of our method in contrast to binary CNNs in [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Influence of MobileNetV2’s width scaling factor on the distribution of zero weights in the ternary convolutions. After training, the 2.0×MobileNetV2 model contains more weights which are equal to zero. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 2…
Figure 8
Figure 8. Figure 8: Distribution of ternary weight values in the pointwise convolutions of a 1.0×MobileNetV2 after training. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Pointwise Convolution Index 0.0 0.2 0.4 0.6 0.8 1.0 Fraction of Weight …
Figure 9
Figure 9. Figure 9: Distribution of ternary weight values in the pointwise convolutions of a 2.0×MobileNetV2 after training [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Pseudocode for the quantization process of pointwise and depthwise convolution weights. """ Quantize the activations to 8-bit via tensor-wise absmax quantization """ def quantize_activation(x, eps = 1e-5): """ Args: x (Tensor): The input to be quantized. Expects shape…
Figure 11
Figure 11. Figure 11: Pseudocode for the quantization of activations [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Pseudocode for a quantized convolution module [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Integrating Pruning with Quantization for Efficient Deep Neural Networks Compression

    cs.NE 2025-09 conditional novelty 4.0 of 10

    Simultaneous or sequential integration of geometric-median filter pruning with 4-bit additive-power-of-two quantization compresses ResNet and VGG models on CIFAR-10 by about 15x with modest accuracy loss.

Reference graph

Works this paper leans on

61 extracted references · 55 canonical work pages · cited by 1 Pith paper

  1. [1]

    NEON data types

    ARM. NEON data types. URL https://developer.arm.com/ documentation/dui0473/m/neon-programming/neon-data-types. Ac- cessed: 08 Aug 2024

    work page 2024

  2. [2]

    Introduction to Armv8-M architecture

    ARM. Introduction to Armv8-M architecture. URL https://developer.arm.com/documentation/107656/0101/ Introduction-to-Armv8-M-architecture. Accessed: 08 Aug 2024

    work page 2024

  3. [3]

    Bethge, C

    J. Bethge, C. Bartz, H. Yang, Y . Chen, and C. Meinel. MeliusNet: An Improved Network Architecture for Binary Neural Networks. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1438–1447, 2021

    work page 2021

  4. [4]

    Chauhan, U

    A. Chauhan, U. Tiwari, and V . N R. Post Training Mixed Precision Quantization of Neural Networks using First-Order Information. 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 1335–1344, 2023

    work page 2023

  5. [5]

    W. Chen, P. Wang, and J. Cheng. Towards Mixed-Precision Quantiza- tion of Neural Networks via Constrained Optimization. In IEEE/CVF International Conference on Computer Vision (ICCV) , pages 5330– 5339, 2021

    work page 2021

  6. [6]

    F. Chollet. Xception: Deep Learning with Depthwise Separable Con- volutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1800–1807, 2017

    work page 2017

  7. [7]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An Image is Worth 16x16 Words: Trans- formers for Image Recognition at Scale. In International Conference on Learning Representations, 2021

    work page 2021

  8. [8]

    Elkerdawy, M

    S. Elkerdawy, M. Elhoushi, H. Zhang, and N. Ray. Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022

    work page 2022

Show all 61 references
  1. [9]

    S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy, and D. S. Modha. Learned Step Size Quantization. In International Conference on Learning Representations, 2020

    2020

  2. [10]

    Ferrarini, M

    B. Ferrarini, M. J. Milford, K. D. McDonald-Maier, and S. Ehsan. Bi- nary neural networks for memory-efficient and effective visual place recognition in changing environments. IEEE Transactions on Robotics, 38(4):2617–2631, 2022

    2022

  3. [11]

    A. Fog. Instruction tables: Lists of instruction latencies, through- puts and micro-operation breakdowns for Intel, AMD and VIA CPUs. Copenhagen University College of Engineering, page 469, 2022

    2022

  4. [12]

    X. Gao, Y . Zhao, L. Dudziak, R. Mullins, and C.-z. Xu. Dynamic Chan- nel Pruning: Feature Boosting and Suppression. In International Con- ference on Learning Representations, 2019

    2019

  5. [13]

    R. Gong, X. Liu, S. Jiang, T.-H. Li, P. Hu, J. Lin, F. Yu, and J. Yan. Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4851–4860, 2019

    2019

  6. [14]

    N. Guo, J. Bethge, H. Guo, C. Meinel, and H. Yang. Towards Optimization-Friendly Binary Neural Network. Transactions on Ma- chine Learning Research, 2023. ISSN 2835-8856

    2023

  7. [15]

    S. Han, H. Mao, and W. J. Dally. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huff- man Coding. In Y . Bengio and Y . LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4...

    2016

  8. [16]

    K. He, X. Zhang, S. Ren, and J. Sun. Delving Deep into Recti- fiers: Surpassing Human-Level Performance on ImageNet Classifica- tion. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1026–1034, 2015

    2015

  9. [17]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  10. [18]

    He and L

    Y . He and L. Xiao. Structured Pruning for Deep Convolutional Neu- ral Networks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2900–2919, 2024

    2024

  11. [19]

    Hinton, O

    G. Hinton, O. Vinyals, and J. Dean. Distilling the Knowledge in a Neu- ral Network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  12. [20]

    Horowitz

    M. Horowitz. Computing’s energy problem (and what we can do about it). In 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC). IEEE, Feb. 2014

    2014

  13. [21]

    Howard, M

    A. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. MobileNets: Efficient Convolutional Neu- ral Networks for Mobile Vision Applications. Apr. 2017

    2017

  14. [22]

    Hubara, M

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio. Binarized Neural Networks. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Pro- cessing Systems, volume 29. Curran Associates, Inc., 2016

    2016

  15. [23]

    Intel Intrinsics Guide

    Intel. Intel Intrinsics Guide. https://www.intel.com/content/www/us/ en/docs/intrinsics-guide/index.html. Accessed: 08 Aug 2024

    2024

  16. [24]

    Ioffe and C

    S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. InInternational Con- ference on Machine Learning, pages 448–456, 2015

    2015

  17. [25]

    H.-B. Kim, E. Park, and S. Yoo. BASQ: Branch-wise Activation- clipping Search Quantization for Sub-4-bit Neural Networks. In Eu- ropean Conference on Computer Vision (ECCV), 2022

    2022

  18. [26]

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf. Pruning Filters for Efficient ConvNets. InInternational Conference on Learning Representations, 2017

    2017

  19. [27]

    Y . Li, R. Gong, X. Tan, Y . Yang, P. Hu, Q. Zhang, F. Yu, W. Wang, and S. Gu. BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction. In International Conference on Learning Repre- sentations, 2021

    2021

  20. [28]

    Z. Liu, B. Wu, W. Luo, X. Yang, W. Liu, and K.-T. Cheng. Bi-real net: Enhancing the performance of 1-bit cnns with improved represen- tational capability and advanced training algorithm. In Proceedings of the European Conference on Computer Vision (ECCV), pages 722–737, 2018

    2018

  21. [29]

    Z. Liu, Z. Shen, M. Savvides, and K.-T. Cheng. ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions. In European Conference on Computer Vision (ECCV), 2020

    2020

  22. [30]

    S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, and F. Wei. The Era of 1-bit LLMs: All Large Lan- guage Models are in 1.58 Bits. arXiv preprint arXiv:2402.17764, 2024

    2024 arXiv

  23. [31]

    S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, and F. Wei. The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits. Supplementary Mate- rial. 2024. URL https://github.com/microsoft/unilm/blob/master/bitnet/ The-Era-of-1-bit-LLMs__Trainin...

    2024

  24. [32]

    Martinez, J

    B. Martinez, J. Yang, A. Bulat, and G. Tzimiropoulos. Training binary neural networks with real-to-binary convolutions. InInternational Con- ference on Learning Representations, 2020

    2020

  25. [33]

    Meiner, J

    L. Meiner, J. Mehnert, and A. Condurache. Data-Free Dynamic Compression of CNNs for Tractable Efficiency. In Proceedings of the 20th International Joint Conference on Computer Vision, Imag- ing and Computer Graphics Theory and Applications , pages 196–208. SCITEPRESS - Science...

    2025

  26. [34]

    Park and S

    E. Park and S. Yoo. PROFIT: A Novel Training Method for sub-4- bit MobileNet Models. In European Conference on Computer Vision (ECCV), pages 430–446, Berlin, Heidelberg, 2020. Springer-Verlag

    2020

  27. [35]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. PyTorch: An Imperative Style, High- Pe...

    2019

  28. [36]

    Radosavovic, R

    I. Radosavovic, R. P. Kosaraju, R. Girshick, K. He, and P. Dollár. Designing Network Design Spaces. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10425– 10433, 2020

    2020

  29. [37]

    Rastegari, V

    M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. In European Conference on Computer Vision (ECCV), 2016

    2016

  30. [38]

    A. J. Redfern, L. Zhu, and M. K. Newquist. BCNN: A Binary CNN With All Matrix Ops Quantized To 1 Bit Precision. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 4599–4607, 2021

    2021

  31. [39]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. Berg, and L. Fei- Fei. ImageNet Large Scale Visual Recognition Challenge.International Journal of Computer Vision, 115(3):211–252, Dec. 2015

    2015

  32. [40]

    Sandler, A

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. Mo- bileNetV2: Inverted Residuals and Linear Bottlenecks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4510–4520, 2018

    2018

  33. [41]

    X. Sun, N. Wang, C.-Y . Chen, J. Ni, A. Agrawal, X. Cui, S. Venkatara- mani, K. El Maghraoui, V . V . Srinivasan, and K. Gopalakrishnan. Ultra-Low Precision 4-bit Training of Deep Neural Networks. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advanc...

    2020

  34. [42]

    Tan and Q

    M. Tan and Q. Le. EfficientNetV2: Smaller Models and Faster Train- ing. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10096–10106. PMLR, 18–24 Jul 2021

    2021

  35. [43]

    TorchVision: PyTorch’s Computer Vision library

    TorchVision maintainers and contributors. TorchVision: PyTorch’s Computer Vision library. https://github.com/pytorch/vision, 2016

    2016

  36. [44]

    Touvron, M

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jé- gou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning , pages 10347–10357. PMLR, 2021

    2021

  37. [45]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is All you Need. In I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural In- formation Process...

  38. [46]

    H. Wang, S. Ma, L. Dong, S. Huang, H. Wang, L. Ma, F. Yang, R. Wang, Y . Wu, and F. Wei. BitNet: Scaling 1-bit Transformers for Large Lan- guage Models. arXiv preprint arXiv:2310.11453, 2023

    2023 arXiv

  39. [47]

    K. Wang, Z. Liu, Y . Lin, J. Lin, and S. Han. Hardware-Centric AutoML for Mixed-Precision Quantization. International Journal of Computer Vision, 128:2035–2048, 2020

    2020

  40. [48]

    W. Wen, C. Wu, Y . Wang, Y . Chen, and H. Li. Learning Structured Sparsity in Deep Neural Networks. In D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Infor- mation Processing Systems 29: Annual Conference on Neural Informa- tion P...

    2016

  41. [49]

    Wimmer, J

    P. Wimmer, J. Mehnert, and A. Condurache. COPS: Controlled Prun- ing Before Training Starts. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2021

    2021

  42. [50]

    Wimmer, J

    P. Wimmer, J. Mehnert, and A. Condurache. Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12527–12537, June 2022

    2022

  43. [51]

    Wimmer, J

    P. Wimmer, J. Mehnert, and A. P. Condurache. Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network: A Survey. Artificial Intelligence Review, 56(12):14257–14295, 2023

    2023

  44. [52]

    J. Won, J. Si, S. Son, T. J. Ham, and J. W. Lee. ULPPACK: Fast Sub- 8-bit Matrix Multiply on Commodity SIMD Hardware. In Proceedings of Machine Learning and Systems, volume 4, pages 52–63, 2022

    2022

  45. [53]

    S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He. Aggregated Resid- ual Transformations for Deep Neural Networks. In 2017 IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR) , pages 5987–5995, 2017

    2017

  46. [54]

    Yang and Q

    L. Yang and Q. Jin. FracBits: Mixed Precision Quantization via Frac- tional Bit-Widths. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12):10612–10620, 2021

    2021

  47. [55]

    Zhang, Z

    Y . Zhang, Z. Zhang, and Ł. Lew. PokeBNN: A Binary Pursuit of Lightweight Accuracy. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12465–12475, 2022

    2022

  48. [56]

    Zhang, A

    Y . Zhang, A. Garg, Y . Cao, L. Lew, B. Ghorbani, Z. Zhang, and O. Firat. Binarized Neural Machine Translation. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  49. [57]

    F. Zhu, R. Gong, F. Yu, X. Liu, Y . Wang, Z. Li, X. Yang, and J. Yan. Towards Unified INT8 Training for Convolutional Neural Network. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 1966–1976, 2020

    2020

  50. [58]

    R.-J. Zhu, Y . Zhang, E. Sifferman, T. Sheaves, Y . Wang, D. Richmond, P. Zhou, and J. K. Eshraghian. Scalable MatMul-free Language Model- ing. arXiv preprint arXiv:2406.02528, 2024

    2024 arXiv

  51. [59]

    S. Zhu, L. H. K. Duong, and W. Liu. XOR-Net: An Efficient Computa- tion Pipeline for Binary Neural Network Inference on Edge Devices. In 2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS), pages 124–131, 2020. Supplementary Material A Distribu...

    2020

  52. [61]

    "" if weight.shape[2:] == (1,1): # Pointwise convolution

    This balance of positive and negative weights leads to stable acti- vations with less variability in their magnitude. In part, this behavior may be explained through the usage of BatchNorm [24] directly after pointwise convolutions, which encourages its inputs to be centered, ...

  53. [2017]

    URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

    2017

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.