Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DeepHoyer claims that training with a scale-invariant \ell^1/\ell^2 ratio regularizer produces sparser neural networks than previous pruning methods at the same accuracy.

desk verdict A clean scale-invariant regularizer with a correct main derivation, but overstated SOTA gains and a test-set-tuned pruning threshold make the empirical comparison unconvincing. read the letter →

arxiv 1908.09979 v2 pith:4Q5SPD5W submitted 2019-08-27 cs.LG stat.ML

Huanrui Yang , Wei Wen , Hai Li This is my paper · ORCID
classification cs.LGstat.ML
keywords sparseneuralnetworksweightpruningstructuralscale-invariantregularizerHoyermeasureHoyer-Squaregroupsparsitymodelcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a family of sparsity-inducing regularizers, DeepHoyer, built on the ratio of the \$ell^{1}$ and \$ell^{2}$ norms. The central claim is that these regularizers are differentiable almost everywhere and scale-invariant, so they combine the easy optimization of \$ell^{1}$ regularization with the scale-free behavior of \$ell^{0}$ while avoiding \$ell^{0}$'s unusable gradients. Applied during training, the Hoyer-Square regularizer pushes small weights to zero while protecting large ones, yielding networks that are sparser than previous pruning methods at the same accuracy. The same construction extended to groups of weights gives structural pruning via the Group-HS regularizer. If the claim holds, DeepHoyer offers a simple drop-in way to make trained networks dramatically smaller and faster without specialized optimization machinery.

What carries the argument

The load-bearing object is the Hoyer-Square regularizer $\mathrm{HS}(W)=(\sum_i |w_i|)^2/\sum_i w_i^2$, a squared \$ell^{1}$-to-\$ell^{2}$ ratio for a layer's weights, together with its group analogue $\mathrm{GH}(W)=(\sum_g \|w^{(g)}\|_2)^2/\|W\|_2^2$ for structural pruning. The ratio form is what makes the penalty scale-invariant, while the square aligns its range $[1,N]$ with the \$ell^{0}$ norm's range. The gradient of HS contains an adaptive threshold $\sum_i w_i^2/\sum_i |w_i|$: weights below this threshold are pushed toward zero, weights above it are pushed away, which the paper identifies as the mechanism for producing sparsity without shrinking all weights uniformly.

What would settle it

Re-run the LeNet-300-100 and LeNet-5 element-wise comparisons with one validation split used to select the threshold/std ratio for every method, and report sparsity at exactly matched accuracy; if Hoyer-Square no longer keeps a 38%/63% sparsity advantage over the next-best method, the central empirical claim fails. A second check: during training, record the threshold $\sum_i w_i^2/\sum_i |w_i|$ and the sign of each weight's gradient under HS; if weights above the threshold are not protected (or if the threshold stops adapting as weights approach zero), the proposed mechanism is not what drives the sparsity.

Watch

Extended reading notes

Core claim

The paper's central claim is that sparsity in deep networks can be induced by a regularizer of the form $\mathrm{HS}(W)=(\sum_i |w_i|)^2/\sum_i w_i^2$, the square of the Hoyer measure, applied layer-wise. This Hoyer-Square regularizer is scale-invariant, so multiplying all weights by a constant leaves the penalty unchanged, and it is differentiable except at zero, so it can be minimized with ordinary stochastic gradient descent. Its gradient moves a weight toward zero exactly when $|w_j| < \sum_i w_i^2 / \sum_i |w_i|$ and away from zero otherwise; the paper argues this adaptive trimming threshold is what lets Hoyer-Square shrink small weights to zero while preserving large weights. On LeNet-300-100 the regularizer reduces nonzero weights by 54.5x and on LeNet-5 by 122x without accuracy loss, beating prior element-wise pruning methods by 38% and 63% respectively, and the group extension Group-HS achieves structural pruning whose accuracy-FLOPs tradeoffs sit above the Pareto frontier of earlier methods on ResNet models.

Load-bearing premise

The reported sparsity margins rely on choosing the pruning threshold (as a multiple of each layer's weight standard deviation) on the test set to maximize sparsity without accuracy loss; if the same search were granted to every baseline, the claimed advantage could shrink.

Editorial extensions

If this is right

  • Training with Hoyer-Square can replace \ell^1 regularization in existing pruning pipelines with no change to the optimizer, while giving higher sparsity at matched accuracy.
  • The scale-invariance of the penalty means the regularization strength is automatically comparable across layers of very different weight scales, which should reduce per-layer hyperparameter tuning.
  • Group-HS offers a gradient-based route to hardware-friendly structured sparsity at ImageNet scale, where Bayesian and \ell^0 methods are impractical.
  • The 21.3x AlexNet compression at unchanged top-5 error suggests DeepHoyer can be combined with weight quantization and encoding schemes to lower deployed model size further.
  • Because the regularizer has minima along axes, training with it can be seen as a continuous relaxation of \ell^0 pruning that avoids the discrete search of iterative magnitude pruning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If scale-invariance is the active ingredient, other homogeneous penalty ratios should show similar pruning behavior; comparing HS against, say, an \ell^1/\ell^p ratio with $p>2$ would isolate the role of the denominator.
  • The adaptive threshold in the HS gradient is a directly observable quantity; tracking it during training could yield a principled pruning schedule and might eliminate the separate threshold-selection step.
  • The paper frames HS as a differentiable approximation to \ell^0, but does not characterize the loss landscape; a formal analysis of its stationary points and sublevel sets would predict which layers get pruned and how sparsity scales with depth.
  • The same regularizer could be applied to activations, gradients, or attention maps, not just weights, if the goal is sparse computation rather than sparse storage.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes DeepHoyer, a family of sparsity-inducing regularizers for neural network pruning. The Hoyer-Square (HS) regularizer, defined as the squared ratio of the L1 norm to the L2 norm of a layer's weights, is shown to be scale-invariant, almost everywhere differentiable, monotone in the Hoyer measure, with the same range and minima structure as the L0 norm, and its gradient induces an adaptive trimming threshold. A Group-HS variant replaces the outer L1 norm with a sum of group L2 norms for structural pruning. The authors report element-wise pruning results on LeNet-300-100, LeNet-5, and AlexNet, and structural pruning results on LeNet and ResNet models, claiming state-of-the-art sparsity at equal accuracy, and they release code. The core derivations of the HS regularizer's properties are presented in the main text, with gradient derivations and experimental details in the appendices.

Significance. If the empirical claims survive a controlled comparison, this is a valuable contribution: a simple, differentiable, scale-invariant sparsity regularizer that can be plugged into standard SGD training for both element-wise and structural pruning. The main-text derivation of the HS gradient (Eq. 4) is correct, the scale-invariance and trimming-threshold properties are established cleanly from definitions, and the paper provides a public implementation and experiments across MNIST, CIFAR-10, and ImageNet. The structural pruning results on ResNet and the Pareto-frontier comparisons broaden the applicability beyond earlier element-wise methods. The credibility of the central 'even sparser than previous works' claim is, however, currently limited by the test-set threshold-selection procedure and by an algebraic error in the appendix derivation of the Group-HS gradient.

major comments (2)
  1. [Appendix A, Eq. (10)] The final line of the Group-HS gradient derivation replaces the sum of group L2 norms, sum_g ||w^(g)||_2, with the element-wise L1 norm, sum_i |w_i|. These two quantities are not equal for non-overlapping groups covering the weight matrix, so Eq. (10) does not follow from the preceding line. The correct final expression should contain the factor 2 w_j (sum_g ||w^(g)||_2) / (||w^(g_hat)||_2 (sum_i w_i^2)^2) before the bracket. This is more than a typo: a reader implementing the printed formula would obtain a different descent direction. The authors should correct the derivation and re-check whether the experimental implementation used automatic differentiation, which would presumably be unaffected, but the mathematical presentation should be accurate.
  2. [Section 5.1 and Appendix B.1] The pruning threshold is selected on the test set: Appendix B.1 states that 'the threshold/std ratio is chosen to achieve the highest sparsity without accuracy loss,' while the baseline numbers in Tables 1-3 are taken from prior work under their own threshold protocols. This makes the reported sparsity margins a joint result of the regularizer and a test-set threshold search, and the comparison is not controlled. To support the claim that DeepHoyer produces 'even sparser neural network models than previous works,' the authors should provide a comparison in which all methods are evaluated under the same threshold-selection procedure (for example, a per-layer threshold sweep with accuracy held fixed), or at least demonstrate that the advantage is robust across a range of thresholds. Without this, the improvements in Tables 1-3 may not be attributable to the regularizer.
minor comments (4)
  1. [Abstract and Section 5.1] The reported improvement percentages (38% and 63%) do not obviously match the numbers in Tables 1 and 2. From the tables, the best-prior-to-Hoyer-Square ratios are 6.4k/4.6k = 1.39 on LeNet-300-100 and 5.4k/3.5k = 1.54 on LeNet-5, which correspond to approximately 28% and 35% reductions in nonzero weights, or 39% and 54% increases in compression ratio depending on the definition. Please state the exact metric and recompute the headline percentages.
  2. [Section 3] There is a typo in the sentence beginning 'Unders the assumption'; it should read 'Under the assumption.'
  3. [References] The reference for 'Tran, Clayton Webster, et al.' has a malformed author list and appears to be missing the article title; please correct it.
  4. [Section 5.1] The statement that 'the value of the Hoyer-Square regularizer is proportional to the number of non-zero elements in the weight' is imprecise. HS(W) ranges in [1, N] and is N only when all magnitudes in the layer are equal, so it is not generally proportional to the L0 count. Consider rephrasing the observation about layer-wise compression.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DeepHoyer regularizer properties and gradients are derived from definitions, and the empirical claims rest on external benchmarks.

full rationale

The paper's formal claims are derived directly from the definition HS(W) = (sum |w_i|)^2 / sum w_i^2 and standard calculus. Scale-invariance follows from the homogeneity of the l1 and l2 norms; the trimming condition (|w_j| < sum w_i^2 / sum |w_i|) is a rearrangement of the gradient in Eq. (4), and Eq. (9) derives that gradient from the quotient rule. The Group-HS regularizer is likewise defined in Eq. (6) and differentiated in Eq. (10). These are mathematical consequences of the stated formulas, not circular inputs. The central empirical claim—that DeepHoyer achieves sparser models than prior work—is a measured outcome on external benchmarks (MNIST, ImageNet, CIFAR-10) against externally published baselines, not a prediction obtained from a fitted parameter. The authors' comparison with the original Hoyer regularizer is an ablation, and self-citations to Wen et al. (2016, 2017) are background references for group-lasso-style regularization, not load-bearing uniqueness arguments. One experimental-protocol caveat appears in Appendix B.1: the threshold/std ratio is 'chosen to achieve the highest sparsity without accuracy loss' on the test set, which raises a possible benchmarking-fairness concern but does not make the derivation equivalent to its inputs. No self-definitional step, fitted-input-as-prediction, or self-citation chain forces the reported results.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on the empirical behavior of the regularizer under SGD, which is not theoretically guaranteed. The only new mathematical content is the gradient calculation and scale-invariance, both correct for Hoyer-Square but with an error for Group-HS in the appendix. No new physical or conceptual entities are introduced.

free parameters (4)
  • Regularization strength alpha (HS) = LeNet-300-100: 0.0002; LeNet-5: 0.0001; AlexNet: 1e-6
    Chosen by hand to reach the best result (Appendix B.1).
  • Pruning threshold/std ratio = MNIST HS: 0.03; Group-HS LeNet-300-100: 0.8; Group-HS LeNet-5: 0.008; AlexNet: conv 1e-4, FC 0.4*std
    Selected to achieve the highest sparsity without accuracy loss (Appendix B.1 and B.2).
  • Group-HS regularization strengths alpha_n, alpha_c = ResNet-50: 1e-5 to 4e-5; LeNet-300-100: 0.002; LeNet-5: 0.1
    Tuned to explore the accuracy-FLOPs tradeoff (Appendix B.2, Tables 8-10).
  • L2 weight decay beta = 1e-4 for AlexNet; standard values elsewhere
    Standard regularizer, not a novel parameter, but part of the objective in Equation (7).
assumptions (4)
  • standard math Cauchy-Schwarz and gradient calculus are used to derive the range and gradient of the Hoyer-Square regularizer.
    Used in Section 4.1 and Appendix A to establish scale-invariance and the trimming threshold.
  • domain assumption Minimizing the Hoyer-Square regularizer alongside the task loss via SGD will produce sparse weights without degrading accuracy.
    This is the core empirical premise of the method, not proven theoretically; the paper relies on experiments.
  • domain assumption The gradient-based trimming effect analyzed in isolation also governs the joint optimization with the data loss.
    Section 4.1 derives the threshold from the regularizer gradient alone; the interaction with the task loss is not analyzed.
  • domain assumption Groups in Group-HS cover all elements without overlap so the second equality in Equation (6) holds.
    The paper notes this holds in their experiments; otherwise the more general first form must be used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures." pith.science (2026). https://pith.science/paper/4Q5SPD5W

@misc{pith2026190809979,
  author       = {Pith},
  title        = {Pith review of: DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4Q5SPD5W}},
  note         = {Machine review of arXiv:1908.09979}
}
read the original abstract

In seeking for sparse and efficient neural network models, many previous works investigated on enforcing L1 or L0 regularizers to encourage weight sparsity during training. The L0 regularizer measures the parameter sparsity directly and is invariant to the scaling of parameter values, but it cannot provide useful gradients, and therefore requires complex optimization techniques. The L1 regularizer is almost everywhere differentiable and can be easily optimized with gradient descent. Yet it is not scale-invariant, causing the same shrinking rate to all parameters, which is inefficient in increasing sparsity. Inspired by the Hoyer measure (the ratio between L1 and L2 norms) used in traditional compressed sensing problems, we present DeepHoyer, a set of sparsity-inducing regularizers that are both differentiable almost everywhere and scale-invariant. Our experiments show that enforcing DeepHoyer regularizers can produce even sparser neural network models than previous works, under the same accuracy level. We also show that DeepHoyer can be applied to both element-wise and structural pruning.

Figures

Figures reproduced from arXiv: 1908.09979 by the authors.

Figure 1
Figure 1. Comparing the `1 and the Hoyer regularizer of a 2-D vector. Their contours are shown in the left 2 subplots (darker color corresponds to a lower value). The right 2 subplots compare their negative gradients. Salzmann, 2016); Liu et al. (2017) and Gordon et al. (2018) (MorphNet) enforce sparsity-inducing regularization on the scaling parameters within Batch Normalization layers to remove the corre￾sponding channels i… view at source ↗
Figure 2
Figure 2. Minimization path of Hoyer-Square regularizer during gradient descent, with [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparisons of accuracy-#FLOPs tradeoff on ImageNet and CIFAR-10, black dash lines [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Histogram of nonzero weight elements of each layer in the LeNet-300-100 model. From top to bottom corresponds to layer FC1, FC2, FC3 respectively. The original pretrained model is shown in column 1, column 2 shows the model achieved after HS regularization, column 3 sh…
Figure 5
Figure 5. Figure 5: Histogram of nonzero weight elements of each layer in the LeNet-5 model. From top to bottom corresponds to layer CONV1, CONV2, FC1, FC2 respectively. The original pretrained model is shown in column 1, column 2 shows the model achieved after HS regularization, column 3…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 13 citations worldwide. Full citation record

  1. Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity

    cs.LG 2025-06 conditional novelty 4.0 of 10

    Applying single-shot SNIP pruning with dataset-size-dependent sparsity reduces overfitting and improves episodic return of offline RL algorithms on 5k to 100k sample D4RL continuous control tasks.

Reference graph

Works this paper leans on

24 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [3]

    For all the tables, the results of previous works are listed on the top, and are ordered based on publication year

    Table 8 shows the result of pruning ResNet-50 model on ImageNet, Table 9 shows the result of pruning ResNet-56 model on CIFAR-10 and Table 10 shows the result of pruning ResNet-110 model on CIFAR-10. For all the tables, the results of previous works are listed on the top, and are ordered based on publication year. Results achieved with the Group-HS regula...

    work page

  2. [5]

    Soft filter pruning for accelerating deep convolutional neural networks

    Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang. Soft filter pruning for accelerating deep convolutional neural networks. arXiv preprint arXiv:1808.06866, 2018a. Yihui He, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision , pp. 1389–1397,

    arXiv

  3. [6]

    ILSVRC2012

    B.2 I MAGE NET AND CIFAR-10 EXPERIMENTS The ImageNet dataset is a large-scale color-image dataset containing 1.2 million images of 1000 categories (Russakovsky et al., 2015), which has long been utilized as an important bench- mark on image classification problems. In this paper, we use the “ILSVRC2012” version of the dataset, which can be found at http://...

    work page 2013

  4. [8]

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf

    URL https://openreview.net/forum?id=B1VZqjAcYX. Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710,

    arXiv

  5. [9]

    Bayesian compression for deep learning

    Christos Louizos, Karen Ullrich, and Max Welling. Bayesian compression for deep learning. In Advances in Neural Information Processing Systems , pp. 3288–3298, 2017a. Christos Louizos, Max Welling, and Diederik P Kingma. Learning sparse neural networks through l_0 regularization. arXiv preprint arXiv:1712.01312, 2017b. Jian-Hao Luo and Jianxin Wu. Autopru...

    arXiv

  6. [10]

    Deep supervised learning for hyperspectral data classification through convolutional neural net- works

    Konstantinos Makantasis, Konstantinos Karantzalos, Anastasios Doulamis, and Nikolaos Doulamis. Deep supervised learning for hyperspectral data classification through convolutional neural net- works. In 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) , pp. 4959–4962. IEEE,

    work page 2015

  7. [12]

    Faster cnns with direct sparse convolutions and guided pruning

    Jongsoo Park, Sheng Li, Wei Wen, Ping Tak Peter Tang, Hai Li, Yiran Chen, and Pradeep Dubey. Faster cnns with direct sparse convolutions and guided pruning. arXiv preprint arXiv:1608.01409,

    arXiv

  8. [13]

    Karen Simonyan and Andrew Zisserman

    doi: 10.1007/s11263-015-0816-y. Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

    arXiv

Show all 24 references
  1. [14]

    Leveraging filter correlations for deep model compression

    Pravendra Singh, Vinay Kumar Verma, Piyush Rai, and Vinay P Namboodiri. Leveraging filter correlations for deep model compression. arXiv preprint arXiv:1811.10559,

    arXiv

  2. [15]

    Reconstruction of jointly sparse vectors via manifold optimization

    Armenak Petrosyan Tran, Clayton Webster, et al. Reconstruction of jointly sparse vectors via manifold optimization. arXiv preprint arXiv:1811.08778,

    arXiv

  3. [17]

    Learning intrinsic sparse structures within long short-term memory

    Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, and Hai Li. Learning intrinsic sparse structures within long short-term memory. arXiv preprint arXiv:1709.05027,

    arXiv

  4. [18]

    Model selection and estimation in regression with grouped variables

    11 Published as a conference paper at ICLR 2020 Ming Yuan and Yi Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 68(1):49–67,

    2020

  5. [19]

    an elementwj in the weight matrixW

    12 Published as a conference paper at ICLR 2020 A D ERIVATION OF DEEP HOYER REGULARIZERS ’ GRADIENTS In this section we provide detailed derivation of the gradient of the Hoyer-Square regularizer and the Group-GS regularizer w.r.t. an elementwj in the weight matrixW . The grad...

    2020

  6. [20]

    All the MNIST experiments are done with a single TITAN XP GPU

    with learning rate 0.001 is used throughout the training process. All the MNIST experiments are done with a single TITAN XP GPU. Both the LeNet-300-100 model and the LeNet-5 model are firstly pretrained without the sparsity- inducing regularizer, where they achieve the testing ...

    2020

  7. [22]

    torchvision

    to evaluate the structural pruning performance on ResNet-56 and ResNet-110 models. The CIFAR-10 dataset can be directly accessed through the dataset API provided in the “torchvision” python package. Standard preprocessing, 14 Published as a conference paper at ICLR 2020 Figure...

    2020

  8. [23]

    Since most of the weight elements will be zero in the end, we only plot the histogram of nonzero weight elements for better observation

    C A DDITIONAL EXPERIMENT RESULTS C.1 W EIGHT DISTRIBUTION AT DIFFERENT STAGES Here we demonstrate how will the weight distribution change in each layer at different stages of our element-wise pruning process. Since most of the weight elements will be zero in the end, we only p...

    2020

  9. [25]

    Model Base acc Acc gain #FLOPs reduction Pruning-A (Li et al.,

    73.54% 91.50% 2.96× Group-HS 1e-5 76.43% 93.07% 1.89× Group-HS 2e-5 75.20% 92.52% 3.09× Group-HS 3e-5 73.19% 91.36% 4.68× Group-HS 4e-5 71.08% 90.21% 5.48× 17 Published as a conference paper at ICLR 2020 Table 9: Structural pruning result of the ResNet-56 model on CIFAR-10. Mo...

    2020

  10. [2009]

    Adam: A method for stochastic optimization

    9 Published as a conference paper at ICLR 2020 Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

    2020 arXiv

  11. [2014]

    Blind deconvolution using a normalized sparsity measure

    Dilip Krishnan, Terence Tay, and Rob Fergus. Blind deconvolution using a normalized sparsity measure. In CVPR 2011, pp. 233–240. IEEE,

    2011

  12. [2015]

    Playing atari with deep reinforcement learning

    10 Published as a conference paper at ICLR 2020 V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602,

    2020 arXiv

  13. [2016]

    Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

    Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015a. Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections ...

    arXiv

  14. [2017]

    Xiaohan Ding, Guiguang Ding, Yuchen Guo, and Jungong Han

    URL http://arxiv.org/abs/ 1711.02017. Xiaohan Ding, Guiguang Ding, Yuchen Guo, and Jungong Han. Centripetal sgd for pruning very deep convolutional networks with complicated structure. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4943–4953,

    arXiv

  15. [2018]

    Learning structured sparsity in deep neural networks

    Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In Advances in neural information processing systems , pp. 2074–2082,

  16. [2019]

    A method for finding structured sparse solutions to nonnegative least squares problems with applications

    Ernie Esser, Yifei Lou, and Jack Xin. A method for finding structured sparse solutions to nonnegative least squares problems with applications. SIAM Journal on Imaging Sciences , 6(4):2010–2046,

    2010

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.