Pith. sign in

REVIEW 4 major objections 5 minor 44 references

Information Consistent Pruning: How to Efficiently Search for Sparse Networks?

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Flow proximity, not full retraining, preserves pruned-network accuracy

desk verdict The flow-based stopping criterion for IMP is a novel idea that could save real compute, but the current proof only gives lower bounds and the experiments lack the accuracy-based early-stopping control needed to isolate the flow metric's effect. read the letter →

arxiv 2501.15592 v1 pith:TOXJ65IJ submitted 2025-01-26 cs.LG cs.ITcs.NEmath.IT

classification cs.LGcs.ITcs.NEmath.IT
keywords iterativemagnitudepruninglotterytickethypothesisinformationflowgradientstoppingcriterionsparseneuralnetworkstrainingefficiencynetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Information Consistent Pruning (InCoP) claims that iterative magnitude pruning can stop retraining long before a network recovers its original accuracy, as long as the network's layer-wise information flow or gradient flow has returned within a small tolerance of the fully trained dense network. The paper argues that accuracy-based stopping is too coarse because it only inspects the final output layer, while flow-based stopping tracks how each layer relates to its neighbors or to the gradient. Across Fashion-MNIST, MNIST, and CIFAR-10 with several ResNet and VGG architectures, InCoP reaches the same final sparsity and accuracy as the SAP baseline while training for far fewer epochs; for InCoP-IF with $(p,q)=(0.5,1)$ on one ResNet18 setting, the paper reports 13% of SAP's execution time. If the flow-accuracy correspondence holds, this turns the most time-consuming part of lottery-ticket pruning into a cheap early-stopping check.

What carries the argument

The load-bearing object is the flow vector $\Phi$, either the information-flow connectivity matrix $\Delta_w(f^{(l)}, f^{(l+1)}) = \mathbb{E}_{(X,Y)\sim D}[f_i^{(l)} f_j^{(l+1)}]$ between consecutive layers, or the gradient flow $g_i = \partial \ell / \partial w^{(i)} \odot m^{(i)}$ per layer. The stopping rule compares these to the corresponding quantities $\Phi^*$ of the fully trained dense network and halts training at epoch $e$ once $\|\Phi_e - \Phi^*\| \le \epsilon$. This turns the stopping criterion from a global accuracy comparison into a layer-wise proximity check, and it is what lets Algorithm 1 cut training epochs while keeping the same sparsity schedule as SAP.

What would settle it

A concrete falsifier: run InCoP on MNIST with ResNet18 and a small $\epsilon$, log $\|\Phi_e - \Phi^*\|$ and test accuracy at the end of every retraining epoch, and check whether training ever stops while the accuracy gap to the dense network remains larger than the gap the accuracy-based stopping rule would allow; if such an epoch exists, the flow-accuracy premise fails.

Watch

Extended reading notes

Core claim

The paper's discovery claim is that the expensive retrain-to-full-accuracy step in iterative magnitude pruning is unnecessary: a pruned network can be considered ready for the next pruning round once its information flow, the expected product of activations in consecutive layers, or its gradient flow, the masked gradients per layer, is within $\epsilon$ of the corresponding flow of the fully trained dense network. InCoP implements this by computing $\Phi^*$ once on the optimal dense network and then stopping each retraining phase at the first epoch where $\|\Phi_e - \Phi^*\| \le \epsilon$. The authors report that this reaches the same final accuracy as SAP and LTH on Fashion-MNIST, MNIST, and CIFAR-10 with VGG16, ResNet18, and ResNet50 while using substantially fewer epochs, and in one configuration uses 13% of SAP's execution time. The supplementary material argues, using a reverse Jensen inequality and Lipschitz continuity of activations, that small connectivity distance implies small accuracy distance.

Load-bearing premise

The load-bearing premise is that a network whose layer-wise information or gradient flow is close to the optimal dense network's flow will also have test accuracy close to the optimal dense network's accuracy; if flow closeness and accuracy closeness can diverge, stopping early on flow alone may leave the pruned network underperforming.

Editorial extensions

If this is right

  • The retraining phase of an iterative magnitude pruning loop, usually a fixed number of epochs per iteration, can be replaced by an adaptive number of epochs determined by flow proximity to the dense reference.
  • Because InCoP inherits SAP's pruning rates, the final sparsity for a given $(p,q)$ is the same as SAP; only the stopping point changes.
  • The information-flow and gradient-flow stopping rules are independent of the pruning method, so they could be attached to other magnitude- or sensitivity-based pruning procedures.
  • The choice of $\epsilon$ controls the tradeoff: larger values stop earlier and behave more conservatively, while very small values risk stopping before the network has recovered enough for the next pruning step.
  • InCoP-IF is more sensitive to aggressive pruning because connectivity between consecutive layers drops on the order of $M_l^2$ as neurons are removed, while InCoP-GF stays stable across pruning ratios.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the same flow-matching stopping rule could be attached to any iterative pruning schedule with a reference dense network, including continual-learning and transfer settings.
  • The supplementary proof bounds the accuracy gap from below, not above, so a rigorous guarantee for the stopping rule would need a separate argument showing small flow distance forces high accuracy; until then the method rests on the empirical flow-accuracy correspondence.
  • The reported 13% runtime figure is for one configuration; on very wide or deep networks the per-epoch cost of computing connectivity could outweigh the epoch savings, so the speedup has to be re-measured per architecture.
  • A testable extension is whether one $\epsilon$ value transfers across datasets and architectures or must be retuned, since the paper's ablation shows the tradeoff shifts with $\epsilon$.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes InCoP, an early-stopping criterion for iterative magnitude pruning (IMP). After fully training a dense reference network, InCoP stores its layer-wise information flow (correlation between consecutive-layer activations) and gradient flow; during each pruning iteration it retrains only until the corresponding flow distance to the reference is below a threshold epsilon, then uses SAP's adaptive pruning ratio to remove weights. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 with VGG16, ResNet18, and ResNet50 report the same final accuracy as SAP, while using substantially fewer training epochs. A supplementary section (Section 6.1) attempts to prove that small flow distance implies small accuracy gap.

Significance. If the empirical claims hold, InCoP would be a practically useful technique that reduces the dominant training cost of IMP while preserving final sparsity and accuracy. The paper has several strengths: the method is simple, it is evaluated across several dataset/architecture combinations, the code is publicly available, and the authors report wall-clock comparisons rather than only epoch counts. However, the theoretical support in Section 6.1 of the supplementary material is not valid as written, and the experiments do not isolate the flow criterion from generic early stopping, so the central claim is not yet convincingly established.

major comments (4)
  1. [Supplementary Material, Section 6.1, Eqs. (5)-(13)] The proof of the Analytical Hypothesis is logically inverted. Eq. (6) applies a reverse Jensen bound to obtain a lower bound on |Acc_e - Acc*| without justifying that the required convexity and positivity conditions hold for the expression Y * |l(w_e) - l(w*)|. Eq. (8) treats lambda_min as if it were the smallest eigenvalue while calling it the maximum eigenvalue, and the second-order Taylor argument requires a positive-definite Hessian. More importantly, Eqs. (8) and (12) are both lower bounds on the accuracy gap and on the weight distance, respectively, and Eq. (13) is asserted without derivation and with an inequality direction that is not justified: taking eC >= C_sigma makes the left-hand side larger than E[C_sigma ||w_e - w*||^2], so it cannot provide the stated upper connection. The final conclusion |Acc_e - Acc*| >= epsilon - L*C_sigma is only a lower bound and becomes trivial as epsilon -> 0; it cannot establish the claimed implication 'small flow gap implies small accuracy gap', which would require an upper bound such as |Acc_e - Acc*| <= C * flow_gap. The theoretical claim in the abstract and Section 2.3 is therefore unsupported by the presented mathematics.
  2. [Section 3.1 and 3.2, Figs. 3-5] The experimental section compares InCoP only with SAP and LTH, both of which train a fixed number of epochs per iteration. There is no baseline in which the same per-iteration training budget is terminated by a conventional accuracy-based or validation-loss-based early-stopping rule. Without such a control, the observed epoch savings cannot be attributed specifically to the information/gradient-flow criterion, because any sensible validation-based early-stopping heuristic might yield similar savings. Please add such a baseline and report the resulting accuracy and training time, so that the role of the flow metrics is isolated.
  3. [Algorithm 1, lines 12-13] The pseudocode resets the remaining weights to w_t, the weights at the beginning of the iteration, rather than to the trained weights w_e: 'Reset the remaining weights to w_t by w_{t+1} = m_t ⊙ w_t'. If this is taken literally, the retraining epochs do not update the weights that are carried into the next iteration, which is inconsistent with the described IMP procedure and with the accuracy results reported in Figs. 3 and 4. Please either correct the pseudocode to w_{t+1} = m_t ⊙ w_e, or explain the intended rewind step and why it still produces the reported accuracies.
  4. [Section 3 and Supplementary Section 6.2] The main experiments rely on the threshold epsilon as the stopping tolerance, but the paper does not report how epsilon was selected for the reported runs; the ablation study in Supplementary Section 6.2 only shows qualitative behavior for various values. Please specify the chosen epsilon values for each dataset/architecture combination and the selection rule (for example, using a validation set). This is necessary for reproducibility because epsilon directly controls the trade-off between epoch savings and final accuracy.
minor comments (5)
  1. [Section 3.2] The text refers to 'Table 5' twice when describing the execution-time comparison, but the comparison is presented in Figure 5.
  2. [Supplementary Section 6.2] The hyperparameter study refers to the two variants as 'CIAP' and 'GIAP'; these should be InCoP-IF and InCoP-GF for consistency with the main text.
  3. [Algorithm 1] The pseudocode does not explicitly define how w_t is initialized for t = 1; the reader must infer that w_0 equals the trained reference w*.
  4. [Supplementary Material, Eq. (8)] Equation (8) is missing an absolute value or a sign condition on Y * lambda_min; as written the right-hand side can be negative, making the lower bound vacuous or invalid.
  5. [Section 3.2] The complexity expression O(T * E * N * L * M_l * M_{l+1}) should be written as a sum over layers; with the displayed notation it appears to depend on a layer index l outside the O, which is not a well-formed single bound.

Circularity Check

1 steps flagged · score 4.0 of 10

The empirical stopping-criterion method is not circular, but the supplementary proof of the flow-accuracy correspondence rests on an unproven assertion (Eq. 13) that restates the paper's core premise.

  1. other [Supplementary Material, Section 6.1, Equations (8), (12), (13) and the following paragraph]
    "We claim that ∃ eC such that eC ≥ C σ and E(X,Y )∼D[ eC∥we − w∗∥2] ≤ a/2 E(X,Y )∼D[ Y.λmin∥we − w∗∥2], (13) Combining (8) and (12), we conclude that |Acce − Acc∗| ≥ ϵ − LCσ. This shows that as the ∆we gets closer to ∆w∗ another word ϵ → 0, then the accuracy difference between training at epoch e (Acce) and best performance (Acc∗) gets closer to zero."

    The theorem to be proved is exactly that closeness in layer-wise information flow (small |∆we − ∆w∗|) implies closeness in accuracy (small |Acce − Acc∗|). The proof's only bridge between the flow bound (12) and the accuracy bound (8) is the unsupported 'We claim' in (13), which asserts that a flow-distance-scaled weight term is bounded above by the accuracy-gap term. Since (8) and (12) are lower bounds, their combination yields only |Acce − Acc∗| ≥ ϵ − LCσ, which becomes vacuous as ϵ → 0 and cannot establish the required upper bound. The asserted inequality (13) therefore restates, in mathematical form, the flow-implies-accuracy premise that the proof is supposed to establish, making the theoretical derivation circular at this step.

full rationale

The core stopping criterion is not circular in the statistical sense: ε is a user-set hyperparameter (ablated in SM Fig. 7), not fitted to match the accuracies of SAP or LTH, and the central empirical claim is benchmarked against external baselines, so the reported accuracy and epoch savings are independent evidence. The paper's main weakness is in the supporting theory: SM Section 6.1 attempts to prove that small flow distance implies small accuracy gap, but Eq. (8) and Eq. (12) are lower bounds, and Eq. (13) is an unproven assertion that effectively restates the conclusion. Thus the theoretical derivation is circular at that local step, even though the method's empirical demonstration does not depend on the proof. The experiments also lack an accuracy-based early-stopping control, which weakens attribution of the efficiency gain to the flow criterion, but that is a correctness/experimental-design concern rather than circularity. Overall the circularity burden is moderate: one localized circular step in the proof, with the central empirical claim retaining independent content.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

InCoP's own contribution is the stopping threshold epsilon and the premise that flow proximity preserves accuracy. The pruning-rate machinery (PQI with (p,q), gamma, beta) and the flow definitions themselves are inherited from prior work (SAP and the connectivity/gradient-flow literature). The ledger records these inherited free parameters and the unproved flow-accuracy premise.

free parameters (4)
  • epsilon (flow distance threshold) = not reported for main experiments; ablated in SM Fig. 7
    Controls when training stops; the method is sensitive to this value.
  • PQI (p,q) pair = (1.0, 2.0) and (0.5, 1.0)
    Borrowed from SAP [6]; sets pruning aggressiveness; not derived in this paper.
  • SAP hyperparameters gamma and beta = selected per [6], not listed
    Bound the pruning ratio ct in Section 2.1; inherited from prior work.
  • k fine-tuning epochs for reference network = 10
    Determines the quality of the reference weights w* and hence of the flow target; no optimality certificate is given.
assumptions (6)
  • domain assumption The data distribution D is fixed and the expectations in Eq. (2) are finite and can be estimated from samples.
    Required to define connectivity between layers; the paper does not discuss estimator consistency.
  • standard math Activation functions are Lipschitz continuous and bounded.
    Invoked in Lemmas 1 and 2 of SM 6.1; true for many activations but not stated.
  • standard math The loss l(w) is twice differentiable and the second-order Taylor expansion around w* is valid with gradient at w* equal to zero.
    Used in Eq. (7); unrealistic for ReLU networks but standard in such analyses.
  • standard math The reverse Jensen inequality of [39] applies to the accuracy-difference expression, yielding Eq. (6).
    The application appears to have the inequality direction reversed; the paper gives no proof for this step.
  • ad hoc to paper Maintaining IF/GF proximity to the dense network is sufficient to preserve accuracy during pruning.
    This is the load-bearing premise of InCoP. The SM proof does not establish it: Eq. (13) asserts the needed constant without derivation.
  • domain assumption A network fine-tuned for k=10 epochs is a valid optimal reference w* for computing the flow target.
    Used in Section 3; no convergence check is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information Consistent Pruning: How to Efficiently Search for Sparse Networks?." pith.science (2026). https://pith.science/paper/TOXJ65IJ

@misc{pith2026250115592,
  author       = {Pith},
  title        = {Pith review of: Information Consistent Pruning: How to Efficiently Search for Sparse Networks?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TOXJ65IJ}},
  note         = {Machine review of arXiv:2501.15592}
}
read the original abstract

Iterative magnitude pruning methods (IMPs), proven to be successful in reducing the number of insignificant nodes in over-parameterized deep neural networks (DNNs), have been getting an enormous amount of attention with the rapid deployment of DNNs into cutting-edge technologies with computation and memory constraints. Despite IMPs popularity in pruning networks, a fundamental limitation of existing IMP algorithms is the significant training time required for each pruning iteration. Our paper introduces a novel \textit{stopping criterion} for IMPs that monitors information and gradient flows between networks layers and minimizes the training time. Information Consistent Pruning (\ourmethod{}) eliminates the need to retrain the network to its original performance during intermediate steps while maintaining overall performance at the end of the pruning process. Through our experiments, we demonstrate that our algorithm is more efficient than current IMPs across multiple dataset-DNN combinations. We also provide theoretical insights into the core idea of our algorithm alongside mathematical explanations of flow-based IMP. Our code is available at \url{https://github.com/Sekeh-Lab/InfCoP}.

Figures

Figures reproduced from arXiv: 2501.15592 by the authors.

Figure 1
Figure 1. a. Accuracy-based stopping criterion: the overall performances of the optimal and sparse networks are compared b. gradient flow-based: the gradient flow of corre￾sponding layers are compared (similar shades of colors are compared between the two networks) and c. information flow-based: the information flow of corresponding consec￾utive layers are compared (similarly, boxes with similar shades of colors are compared)… view at source ↗
Figure 2
Figure 2. SGD in pruning with accuracy based training vs. flow based training. The red region corresponds to the set S, representing the information based stopping criterion, while the green region corresponds to the set M, symbolizing the accuracy based stop￾ping criterion. The shift and contraction of these areas occur due to dimensionality reduction resulting from the pruning step [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. ResNet18 - MNIST (top), FashionMNIST (middle), CIFAR10 (bottom). The left column shows accuracy at each iteration t, the number of training epochs required in t is shown in middle column, and the right column shows remaining weights of F (L) w in different iterations. Purple line → (p, q) = (1, 2), green → (p, q) = (0.5, 1), and cyan → LTH. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: VGG16 - MNIST (top), FashionMNIST (middle), CIFAR10 (bottom). As for [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Comparison of execution times for a single IMP experiment across different pruning methods [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: ResNet50 with FashionMNIST. As for [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: ResNet18 on FashionMNIST: Comparing InCoP-IF (top), and InCoP-GF (bot￾tom) for various amounts of epsilon for two pairs of (p, q) Increasing ϵ allows for greater distance between the optimal network and the sparse network, resulting in a less optimal network solution. …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 24 canonical work pages

  1. [1]

    arXiv preprint arXiv:1903.11257 (2019)

    Ahmad, S., Scheinkman, L.: How can we be so dense? the benefits of using highly sparse representations. arXiv preprint arXiv:1903.11257 (2019)

  2. [2]

    Andle, J., Sekeh, S.Y.: Theoretical Understanding of the Information Flow on Continual Learning Performance, pp. 86–101. Lecture Notes in Computer Sci- ence,SpringerNatureSwitzerland(2022). https://doi.org/10.1007/978-3-031- 19775-8_6, http://dx.doi.org/10.1007/978-3-031-19775-8_6

  3. [3]

    Advances in Neural Information Processing Systems34, 19637–19651 (2021)

    Chen,T.,Ji,B.,Ding,T.,Fang,B.,Wang,G.,Zhu,Z.,Liang,L.,Shi,Y.,Yi,S.,Tu, X.: Only train once: A one-shot neural network training and pruning framework. Advances in Neural Information Processing Systems34, 19637–19651 (2021)

  4. [4]

    IEEE Signal Processing Magazine29(6), 141–142 (2012)

    Deng, L.: The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine29(6), 141–142 (2012)

  5. [5]

    arXiv preprint arXiv:1907.04840 (2019)

    Dettmers, T., Zettlemoyer, L.: Sparse networks from scratch: Faster training with- out losing performance. arXiv preprint arXiv:1907.04840 (2019)

  6. [6]

    CoRR (2023),http://arxiv.org/abs/2302

    Diao, E., Wang, G., Zhan, J., Yang, Y., Ding, J., Tarokh, V.: Pruning deep neural networks from a sparsity perspective. CoRR (2023),http://arxiv.org/abs/2302. 05601v3

  7. [7]

    In: International conference on machine learning

    Evci, U., Gale, T., Menick, J., Castro, P.S., Elsen, E.: Rigging the lottery: Making all tickets winners. In: International conference on machine learning. pp. 2943–

  8. [8]

    Evci, U., Ioannou, Y.A., Keskin, C., Dauphin, Y.: Gradient flow in sparse neural networksandhowlottery ticketswin.CoRR(2020), http://arxiv.org/abs/2010. 03533v2

Show all 44 references
  1. [9]

    CoRR (2018),http://arxiv.org/abs/1803.03635v5

    Frankle, J., Carbin, M.: The lottery ticket hypothesis: Finding sparse, trainable neural networks. CoRR (2018),http://arxiv.org/abs/1803.03635v5

  2. [10]

    CoRR (2019),http://arxiv.org/abs/1912.05671v4

    Frankle, J., Dziugaite, G.K., Roy, D.M., Carbin, M.: Linear mode connectivity and the lottery ticket hypothesis. CoRR (2019),http://arxiv.org/abs/1912.05671v4

  3. [11]

    CoRR (2019),http://arxiv.org/abs/1903.01611v3

    Frankle, J., Dziugaite, G.K., Roy, D.M., Carbin, M.: Stabilizing the lottery ticket hypothesis. CoRR (2019),http://arxiv.org/abs/1903.01611v3

  4. [12]

    Ganesh, M.R., Blanchard, D., Corso, J.J., Sekeh, S.Y.: Slimming neural networks using adaptive connectivity scores (2021),http://arxiv.org/abs/2006.12463

  5. [13]

    IEEE Transactions on Neural Networks and Learning Systems (2022)

    Ganesh, M.R., Blanchard, D., Corso, J.J., Sekeh, S.Y.: Slimming neural networks using adaptive connectivity scores. IEEE Transactions on Neural Networks and Learning Systems (2022)

  6. [14]

    IEEE transactions on neural networks and learning systems 35(3), 3794–3808 (2024)

    Ganesh, M.R., Blanchard, D., Corso, J.J., Sekeh, S.Y.: Slimming neural networks using adaptive connectivity scores. IEEE transactions on neural networks and learning systems 35(3), 3794–3808 (2024)

  7. [15]

    In: 2020 25th International Conference on Pattern Recognition (ICPR)

    Ganesh, M.R., Corso, J.J., Sekeh, S.Y.: Mint: Deep network compression via mu- tual information-based neuron trimming. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 8251–8258. IEEE Computer Society (2021)

  8. [16]

    CoRR (2020),http://arxiv.org/abs/2010.02623v1

    Haider, M.U., Taj, M.: Comprehensive online network pruning via learnable scaling factors. CoRR (2020),http://arxiv.org/abs/2010.02623v1

  9. [17]

    CoRR (2015),http://arxiv.org/abs/1506.02626v3

    Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. CoRR (2015),http://arxiv.org/abs/1506.02626v3

  10. [18]

    He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

  11. [19]

    IEEE Signal processing magazine29(6), 82–97 (2012)

    Hinton, G., Deng, L., Yu, D., Dahl, G.E., Mohamed, A.r., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T.N., et al.: Deep neural networks for acoustic Title Suppressed Due to Excessive Length 19 modeling in speech recognition: The shared views of four research gr...

  12. [20]

    arXiv preprint arXiv:1607.03250 (2016)

    Hu, H., Peng, R., Tai, Y.W., Tang, C.K.: Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv preprint arXiv:1607.03250 (2016)

  13. [21]

    In: International conference on machine learning

    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. pp. 448–456. pmlr (2015)

  14. [22]

    CoRR (2020), http://arxiv.org/abs/2010.05983v3

    Jin, G., Yi, X., Zhang, L., Zhang, L., Schewe, S., Huang, X.: How does weight correlation affect the generalisation ability of deep neural networks. CoRR (2020), http://arxiv.org/abs/2010.05983v3

  15. [23]

    Un- published manuscript 40(7), 1–9 (2010)

    Krizhevsky, A., Hinton, G.: Convolutional deep belief networks on cifar-10. Un- published manuscript 40(7), 1–9 (2010)

  16. [24]

    Advances in neural information processing systems25 (2012)

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep con- volutional neural networks. Advances in neural information processing systems25 (2012)

  17. [25]

    Advances in neural infor- mation processing systems2 (1989)

    LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. Advances in neural infor- mation processing systems2 (1989)

  18. [26]

    Lee, N., Ajanthan, T., Torr, P.H.S.: SNIP: Single-shot network pruning based on connection sensitivity (2019),http://arxiv.org/abs/1810.02340

  19. [27]

    In: IJCAI

    Lin, S., Ji, R., Li, Y., Wu, Y., Huang, F., Zhang, B.: Accelerating convolutional networks via global & dynamic filter pruning. In: IJCAI. vol. 2, p. 8. Stockholm (2018)

  20. [28]

    In: Proceedings of the IEEE international conference on computer vision

    Luo, J.H., Wu, J., Lin, W.: Thinet: A filter level pruning method for deep neural network compression. In: Proceedings of the IEEE international conference on computer vision. pp. 5058–5066 (2017)

  21. [29]

    In: International Conference on Machine Learning

    Malach, E., Yehudai, G., Shalev-Schwartz, S., Shamir, O.: Proving the lottery ticket hypothesis: Pruning is all you need. In: International Conference on Machine Learning. pp. 6682–6691. PMLR (2020)

  22. [30]

    Advances in neural information processing systems32 (2019)

    Morcos, A., Yu, H., Paganini, M., Tian, Y.: One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers. Advances in neural information processing systems32 (2019)

  23. [31]

    Ad- vances in Neural Information Processing Systems33, 2925–2934 (2020)

    Orseau, L., Hutter, M., Rivasplata, O.: Logarithmic pruning is all you need. Ad- vances in Neural Information Processing Systems33, 2925–2934 (2020)

  24. [32]

    IEEE transactions on neural networks and learning systems 32(2), 604–624 (2020)

    Otter, D.W., Medina, J.R., Kalita, J.K.: A survey of the usages of deep learning for natural language processing. IEEE transactions on neural networks and learning systems 32(2), 604–624 (2020)

  25. [33]

    ACM Computing Surveys (CSUR)51(5), 1–36 (2018)

    Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M.P., Shyu, M.L., Chen, S.C., Iyengar, S.S.: A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys (CSUR)51(5), 1–36 (2018)

  26. [34]

    arXiv preprint arXiv:1409.1556 (2014)

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  27. [35]

    The journal of machine learning research15(1), 1929–1958 (2014)

    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research15(1), 1929–1958 (2014)

  28. [36]

    arXiv preprint arXiv:2102.01670 (2021)

    Tessera, K.a., Hooker, S., Rosman, B.: Keep the gradients flowing: Using gradient flow to study sparse network optimization. arXiv preprint arXiv:2102.01670 (2021)

  29. [37]

    arXiv preprint arXiv:2002.07376 (2020)

    Wang, C., Zhang, G., Grosse, R.: Picking winning tickets before training by pre- serving gradient flow. arXiv preprint arXiv:2002.07376 (2020)

  30. [38]

    Advances in neural information processing systems29 (2016) 20 S

    Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. Advances in neural information processing systems29 (2016) 20 S. Gharatappeh et al

  31. [39]

    In: 2021 IEEE Information Theory Workshop (ITW)

    Wunder, G., Gross, B., Fritschek, R., Schaefer, R.F.: A reverse jensen inequality re- sult with application to mutual information estimation. In: 2021 IEEE Information Theory Workshop (ITW). pp. 1–6. IEEE (2021)

  32. [40]

    arXiv preprint arXiv:1708.07747 (2017)

    Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

  33. [41]

    arXiv preprint arXiv:1909.11957 (2019)

    You, H., Li, C., Xu, P., Fu, Y., Wang, Y., Chen, X., Baraniuk, R.G., Wang, Z., Lin, Y.: Drawing early-bird tickets: Towards more efficient training of deep networks. arXiv preprint arXiv:1909.11957 (2019)

  34. [42]

    Advances in neural information processing systems32 (2019)

    You, Z., Yan, K., Ye, J., Ma, M., Wang, P.: Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks. Advances in neural information processing systems32 (2019)

  35. [43]

    Zhou, Z., Zhou, W., Hong, R., Li, H.: Online filter weakening and pruning for efficientconvnets.In:2018IEEEInternationalConferenceonMultimediaandExpo (ICME). pp. 1–6. IEEE (2018)

  36. [44]

    In: 202025thInternationalConferenceonPatternRecognition(ICPR).pp.3868–3875

    Zullich, M., Medvet, E., Pellegrino, F.A., Ansuini, A.: Speeding-up pruning for artificial neural networks: introducing accelerated iterative magnitude pruning. In: 202025thInternationalConferenceonPatternRecognition(ICPR).pp.3868–3875. IEEE (2021)

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.