Pith. sign in

REVIEW 4 major objections 5 minor 47 references

Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A concave regularizer on a relaxed mask finds better lottery tickets.

desk verdict The empirical recipe is plausible and the experiments are extensive, but the advertised proof that log beats ℓ1 (Theorem 2) is not actually proven, and the paper overclaims against state-of-the-art pruning methods. read the letter →

arxiv 2501.11135 v1 pith:CJFCNBKD submitted 2025-01-19 cs.LG cs.AI

Giulia Fracastoro , Sophie M. Fosson , Andrea Migliorati , Giuseppe C. Calafiore This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0790C26
keywords lotterytickethypothesisneuralnetworkpruningconcaveregularizationsparseoptimizationrelaxedbinarymaskiterativemagnitudematchingtickets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a new way to play the lottery: instead of pruning a fixed percentage of weights by magnitude, it relaxes the binary mask that selects weights into a continuous variable in [0,1]^d and penalizes it with a concave sparsity-promoting regularizer, such as the logarithmic penalty. The claim is that this soft, threshold-based pruning finds subnetworks that train in isolation as well as iterative magnitude pruning (IMP) at moderate sparsities and better at high sparsities, across CIFAR-10, CIFAR-100, and Tiny ImageNet with ResNet-20, VGG-11, and WideResNet-20. The paper also proves, in a convex setting, that a strictly concave regularizer yields a strictly smaller error bound than '1 for recovering the unknown optimal mask. If correct, this gives practitioners a simple, principled alternative to magnitude pruning for sparse trainable networks.

What carries the argument

The machinery is the relaxed binary mask m in [0,1]^d combined with a separable concave regularizer R(m) = 'sum'_i r(m_i), with the logarithmic choice r_'epsilon'(m_i) = log((m_i+'epsilon')/'epsilon')/log((1+'epsilon')/'epsilon'). The mask encodes the network topology, and its concavity makes the penalty lie above the '1 norm on the interior of the cube, so any non-binary recovered mask pays a strictly higher penalty under log than under '1. This extra penalty 'phi'(m*) is exactly what shrinks the error bound in Theorem 2. Training uses projected gradient descent on the cube, which sidesteps the nondifferentiability of '1 at zero, and a small threshold '$\alpha$' converts the relaxed mask into hard zero-one decisions.

What would settle it

Run the mask-recovery experiment of Section IV-C with a loss that violates Assumption 2, for example a small neural network trained with the same 'lambda' and initialization, and measure '||'m*' - 'e_m'||'_2 for log versus '1 regularization: the theory predicts the log error bound to be strictly smaller whenever m* is non-binary, so observing no gap, or a gap that changes sign across sparsity levels, would falsify the claimed mechanism.

Watch

Extended reading notes

Core claim

The central discovery is that concavity of the mask regularizer, not just sparsity of the solution, is what improves ticket search. Given a fixed trained weight vector '$\theta$' and an unknown optimal binary mask 'e_m', minimizing L(m'odot''$\theta$')+'$\lambda$'R(m) over m in [0,1]^d recovers 'e_m' with error bounded by 4'$\lambda$''sqrt'('k'/'gamma') under '1, and by 4'$\lambda$''sqrt'('k')/('gamma' - 'phi'(m')) under any strictly concave R satisfying Assumption 1, where 'phi'(m*) = R(m*) - '||'m*'||'_1 > 0 whenever the recovered mask is not already binary. Thus the logarithmic regularizer provably tightens the recovery bound in the convex regime. The paper claims this advantage carries over to full lottery-ticket training, where the method matches IMP at matching sparsities and outperforms it at extreme sparsities, and to dense-to-sparse pruning, where it retains baseline accuracy with about 77% of weights removed.

Load-bearing premise

The error bounds require the loss to be strongly convex in the mask at the optimal and estimated masks (Assumption 2), which neural-network losses are not, so the theoretical guarantee only covers the convex logistic-regression setting and the experiments rely on the method's empirical behavior.

Editorial extensions

If this is right

  • At matching sparsities the proposed method matches IMP accuracy on all nine dataset and architecture combinations; at extreme sparsities it outperforms IMP, most clearly on ResNet-20.
  • Strictly concave regularization (log) can improve on '1 mask regularization, which the paper attributes to the strictly smaller error bound of Theorem 2.
  • Threshold-based pruning of the relaxed mask beats fixed-percentage pruning of mask magnitudes in the ablation study.
  • On CIFAR-10 with ResNet-56, the method prunes over 77% of parameters while keeping accuracy at or above the dense baseline, surpassing the dense-to-sparse baselines listed in Table II.
  • The method produces sparse models that can be trained in isolation after rewinding, not only sparse models for inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: if the concavity advantage is real and not an artifact of threshold choice, combining the relaxed-mask regularizer with rewinding to an early training iterate could further stabilize high-sparsity tickets; the paper only tests rewinding to the initial weights.
  • My inference: the threshold 'alpha' is left unmodeled by the theory; a natural test is whether the reported gains persist as 'alpha' tends to zero or whether they depend on a carefully tuned threshold.
  • My inference: the projected-gradient training itself may do part of the sparsification work; comparing against an otherwise identical method that uses a convex regularizer or no regularizer on the mask would separate the concavity effect.
  • My inference: because the log regularizer's advantage 'phi'(m*) vanishes as m* approaches a binary vector, the largest relative gains over '1 should appear in early pruning rounds when masks are still soft; experiments could check this directly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a method for finding lottery-ticket subnetworks by optimizing a relaxed binary mask with sparsity-promoting regularization, specifically ℓ1 and a strictly concave logarithmic regularizer. The mask is trained jointly with the weights in a projected-gradient framework, then pruned by thresholding, with an outer loop analogous to iterative magnitude pruning. The authors provide a theoretical analysis in a strongly convex setting (Section IV), claiming in Theorem 2 that replacing ℓ1 with a strictly concave regularizer yields a smaller error bound than the ℓ1 bound of Theorem 1. They support the approach with experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet across three architectures, comparing against IMP and several dense-to-sparse pruning baselines. The central theoretical claim, however, is not established by the proof as written.

Significance. If correct, the theoretical result would give a principled reason to prefer concave regularizers such as the logarithmic penalty for mask-based lottery-ticket search, and the experimental results suggest the method is competitive with or better than IMP at high sparsity. The paper is also one of the first to apply continuous concave regularization to the relaxed-mask formulation of the lottery ticket problem, which is a worthwhile direction. The experimental coverage is broad: nine architecture/dataset combinations and a comparison to many pruning baselines. The strengths are the clear problem formulation and the extensive empirical study. However, the advertised theoretical justification for log over ℓ1 is not supported by the proof in Section IV-B, and the strong convexity assumption used in the theorems does not hold for the deep networks tested. The paper would be of interest after the theoretical derivation is corrected and the scope of the claims is clarified.

major comments (4)
  1. [Section IV-B, Theorem 2] The proof of Theorem 2 stops after Eq. (16) and does not derive the displayed bound (14). Combining Eq. (16) with Assumption 2 and the gradient bound ∥∇L(êm)∥∞ ≤ λ gives (γ/2)∥h∥₂² ≤ 2λ√k∥h∥₂ − λϕ(m⋆), whose correct consequence is ∥h∥₂ ≤ (2λ√k + sqrt(4λ²k − 2γλϕ(m⋆)))/γ, not 4λ√k/(γ − ϕ(m⋆)). The written denominator mixes a dimensionless quantity ϕ(m⋆) with a curvature parameter γ, can be zero or negative, and for any ϕ(m⋆) > 0 it makes the bound larger, not smaller, than Theorem 1's bound. The stated conclusion that a strictly concave regularizer improves the error bound is therefore not established by the presented proof.
  2. [Section IV-B, after Eq. (16)] Even if the proof's algebraic gap were filled, the comparison in Theorem 2 is not supported: the claimed denominator γ − ϕ(m⋆) is dimensionally inconsistent and does not follow from the quadratic inequality that the proof's ingredients imply. The authors should either provide a complete derivation of a correct bound and compare it carefully with Theorem 1, or state a weaker and rigorously proven result. As written, the claim that the log regularizer yields a 'smaller error bound ... of ϕ(m⋆)' is misleading.
  3. [Section IV, Assumption 2] Assumption 2 requires strong convexity of L(m) with respect to the mask at the points m⋆ and êm, but the neural network losses used in Section V are non-convex and do not satisfy this assumption. Consequently Theorems 1 and 2 do not apply to the experimental setting. The paper explicitly says the analysis is 'in the convex framework,' but the introduction and conclusion present the theory as substantiating the method's effectiveness for neural networks. The authors should clearly state that the theoretical results are only motivational for the deep-network experiments, not guarantees, and should discuss what the convex analysis can and cannot establish for the non-convex problem.
  4. [Algorithm 1 and Section V] The joint optimization step in Algorithm 1 is not analyzed for convergence to a stationary point of the non-convex problem (2), yet the introduction claims the approach 'guarantees to find an optimal sparse mask' in the convex case. For the non-convex case used in the experiments, no convergence or local-optimality result is provided. Since the algorithm is central to the empirical claims, a brief discussion of its convergence properties, or a softening of the guarantee language, is needed.
minor comments (5)
  1. [Section V-A] The text states that rϵ(mi) is 'strictly convex'; it should be 'strictly concave' to be consistent with Assumption 1 and the rest of the paper.
  2. [Figure 4 caption] The caption references '(a)' and '(c)' but the figure only has panels (a) and (b); the caption should be corrected to refer to panel (b).
  3. [Equation (16)] The notation '±' in Eq. (16) is nonstandard and ambiguous; the proof should be written without it, using explicit expressions for ∥hS∥1 and ∥hSc∥1.
  4. [Section V-D and Figures 5–9] The experimental results are averaged over three seeds, but the figures do not show error bars or confidence intervals. Adding variability information would strengthen the comparison with IMP and the two regularizers.
  5. [General] No code or detailed hyperparameter search is reported; the paper relies on the public open_lth framework but does not provide the code for the proposed mask-regularization method. Sharing code would improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central claims are derived from stated assumptions rather than from their own conclusions, and the only self-citation is a non-load-bearing background reference.

full rationale

The paper's core derivation is self-contained: Theorem 1 and the attempted Theorem 2 proceed from the definition of m⋆ as a minimizer of (4), Assumption 2 on strong convexity, and the gradient inequality (6); the resulting error bounds are standard consequences of these assumptions and do not smuggle the log-over-ℓ1 conclusion into the premises. The experimental comparison is anchored to external baselines implemented in the same open_lth framework, so the empirical claims do not reduce to the paper's own fitted values. The only self-citation is [28] (Fosson et al.) in a background sentence on concave regularization for compressed sensing; it is not load-bearing for any theorem or experiment, so it does not constitute circularity. Separate concerns raised about the paper, namely that the displayed bound in Theorem 2 does not follow from the proof's inequalities and that Assumption 2 is not satisfied by neural-network losses, are mathematical and scope errors rather than circularity: the paper does not assume the truth of the claimed advantage to prove that advantage. A low circularity score is therefore appropriate.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are introduced. The relaxed mask is a standard optimization construct. The main free parameters are hyperparameters chosen by hand, and the theoretical results depend on strong convexity and gradient-bound assumptions that do not hold for the tested networks.

free parameters (4)
  • lambda (regularization weight) = 3e-6 for l1, 1e-6 for log
    Set by hand in Section V-D; controls the sparsity-accuracy tradeoff. Not derived from theory.
  • epsilon (log regularizer parameter) = 0.1
    Set by hand in Section V-D; defines the concave log penalty shape.
  • alpha (pruning threshold) = Not reported
    Mask values below alpha are set to zero in Algorithm 1. The specific values used to generate each sparsity point are not disclosed, making the results hard to reproduce.
  • mask initialization m0 = 0.5 for all entries
    Set in Algorithm 1 to the midpoint of [0,1]; could affect convergence and final sparsity.
assumptions (5)
  • domain assumption Assumption 1: R(m)=Σ r(m_i) with r continuous, differentiable, concave, non-decreasing, image [0,1].
    Defines the class of regularizers considered; satisfied by l1 and log penalties.
  • ad hoc to paper Assumption 2: L(m⋆) ≥ L(êm) + ∇L(êm)^T(m⋆-êm) + (γ/2)||m⋆-êm||^2.
    Requires strong convexity at two points (Section IV, Eq. 7). This is not satisfied by neural network losses and is needed for both Theorems 1 and 2.
  • ad hoc to paper Gradient bound (6): ||∇L(êm)||∞ ≤ λ.
    Stated as a condition for the error bounds; the authors claim it can always be satisfied by choosing λ, but the experimental λ values are not chosen this way.
  • domain assumption Existence of an optimal binary mask êm that preserves accuracy.
    Section IV assumes such an êm exists; this is essentially the lottery ticket hypothesis itself.
  • ad hoc to paper Convergence of the projected gradient optimization in Algorithm 1 to a good solution of the non-convex problem (2).
    No convergence analysis is provided for the joint optimization over θ and m.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks." pith.science (2026). https://pith.science/paper/CJFCNBKD

@misc{pith2026250111135,
  author       = {Pith},
  title        = {Pith review of: Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CJFCNBKD}},
  note         = {Machine review of arXiv:2501.11135}
}
read the original abstract

The design of sparse neural networks, i.e., of networks with a reduced number of parameters, has been attracting increasing research attention in the last few years. The use of sparse models may significantly reduce the computational and storage footprint in the inference phase. In this context, the lottery ticket hypothesis (LTH) constitutes a breakthrough result, that addresses not only the performance of the inference phase, but also of the training phase. It states that it is possible to extract effective sparse subnetworks, called winning tickets, that can be trained in isolation. The development of effective methods to play the lottery, i.e., to find winning tickets, is still an open problem. In this article, we propose a novel class of methods to play the lottery. The key point is the use of concave regularization to promote the sparsity of a relaxed binary mask, which represents the network topology. We theoretically analyze the effectiveness of the proposed method in the convex framework. Then, we propose extended numerical tests on various datasets and architectures, that show that the proposed method can improve the performance of state-of-the-art algorithms.

Figures

Figures reproduced from arXiv: 2501.11135 by the authors.

Figure 1
Figure 1. Three-stage pipeline for pruning. The initialization of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Eight samples from the MNIST dataset regression. We use the MNIST dataset [36] restricted to the digits 0 and 1. Each image is composed of d = 400 pixels. We consider 200 samples for each digit, 160 for training, and N = 40 for validation test. The corresponding cross-entropy loss function is L(X, y; θ) = X N i=1 yi log(ˆyi) − (1 − yi) log(1 − yˆi) where y ∈ {0, 1} N and yˆ ∈ [0, 1]N respectively are the vectors of … view at source ↗
Figure 4
Figure 4. (a) An 8-sparse mask obtained with logarithmic [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Binary classification on MNIST dataset with logistic [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 5
Figure 5. Figure 5: Test accuracy and sparsity of the subnetworks of ResNet [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Test accuracy and sparsity of the subnetworks of VGG [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Test accuracy and sparsity of the subnetworks of ResNet [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Test accuracy and sparsity of the subnetworks of ResNet [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 42 canonical work pages

  1. [1]

    Optimal brain damage,

    Y . LeCun, J. Denker, and S. Solla, “Optimal brain damage,” in Proc. Adv. Neural Inform. Process. Syst. (NeurIPS) , vol. 2, 1990. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 10

    work page 1990

  2. [2]

    Learning both weights and connections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Proc. Adv. Neural Inform. Process. Syst. (NeurIPS) , 2015, pp. 1135–1143

    work page 2015

  3. [3]

    Learning sparse neural networks through ℓ0 regularization,

    C. Louizos, M. Welling, and D. P. Kingma, “Learning sparse neural networks through ℓ0 regularization,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2018

    work page 2018

  4. [4]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks.” in Proc. Int. Conf. Learn. Represent. (ICLR) , 2019

    work page 2019

  5. [5]

    Designing energy-efficient convolu- tional neural networks using energy-aware pruning,

    T.-J. Yang, Y .-H. Chen, and V . Sze, “Designing energy-efficient convolu- tional neural networks using energy-aware pruning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2017, pp. 5687–5695

    work page 2017

  6. [6]

    Pruning convolutional neural networks for resource efficient inference,

    P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” in Proc. Int. Conf. Learn. Represent. (ICLR) , 2017

    work page 2017

  7. [7]

    What is the state of neural network pruning?

    D. Blalock, J. J. Gonzalez Ortiz, J. Frankle, and J. Guttag, “What is the state of neural network pruning?” in Proc. Mach. Learning Syst. (MLSys , vol. 2, 2020, pp. 129–146

    work page 2020

  8. [8]

    To prune, or not to prune: Exploring the efficacy of pruning for model compression,

    M. H. Zhu and S. Gupta, “To prune, or not to prune: Exploring the efficacy of pruning for model compression,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2018

    work page 2018

Show all 47 references
  1. [9]

    ProxSGD: Training structured neural networks under regularization and constraints,

    Y . Yang, Y . Yuan, A. Chatzimichailidis, R. J. van Sloun, L. Lei, and S. Chatzinotas, “ProxSGD: Training structured neural networks under regularization and constraints,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2020

    2020

  2. [10]

    Adaptive proximal gradient methods for structured neural networks,

    J. Yun, A. C. Lozano, and E. Yang, “Adaptive proximal gradient methods for structured neural networks,” in Proc. Adv. Neural Inform. Process. Syst. (NeurIPS), vol. 34, 2021, pp. 24 365–24 378

    2021

  3. [11]

    Linear mode connectivity and the lottery ticket hypothesis,

    J. Frankle, G. K. Dziugaite, D. Roy, and M. Carbin, “Linear mode connectivity and the lottery ticket hypothesis,” in Proc. Int. Conf. Mach. Learning (ICML), 2020, pp. 3259–3269

    2020

  4. [12]

    Winning the lottery with continuous sparsification,

    P. Savarese, H. Silva, and M. Maire, “Winning the lottery with continuous sparsification,” in Proc. Adv. Neural Inform. Process. Syst. (NeurIPS) , vol. 33, 2020, pp. 11 380–11 390

    2020

  5. [13]

    Learning efficient convolutional networks through network slimming,

    Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, and C. Zhang, “Learning efficient convolutional networks through network slimming,” in 2017 IEEE International Conference on Computer Vision (ICCV) , 2017, pp. 2755–2763

    2017

  6. [14]

    Rethinking the value of network pruning,

    Z. Liu, M. Sun, T. Zhou, G. Huang, and T. Darrell, “Rethinking the value of network pruning,” in Proc. Int. Conf. Learn. Represent. (ICLR) , 2019

    2019

  7. [15]

    SeReNe: Sensitivity-based regularization of neurons for structured sparsity in neural networks,

    E. Tartaglione, A. Bragagnolo, F. Odierna, A. Fiandrotti, and M. Grangetto, “SeReNe: Sensitivity-based regularization of neurons for structured sparsity in neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 12, pp. 7237–7250, 2022

    2022

  8. [16]

    EDropout: Energy-based dropout and pruning of deep neural networks,

    H. Salehinejad and S. Valaee, “EDropout: Energy-based dropout and pruning of deep neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 10, pp. 5279–5292, 2022

    2022

  9. [17]

    DSA: More efficient budgeted pruning via differentiable sparsity allocation,

    X. Ning, T. Zhao, W. Li, P. Lei, Y . Wang, and H. Yang, “DSA: More efficient budgeted pruning via differentiable sparsity allocation,” in Proc. Eur. Conf. Comput. Vis. (ECCV) . Springer, 2020, pp. 592–607

    2020

  10. [18]

    HRank: Filter pruning using high-rank feature map,

    M. Lin, R. Ji, Y . Wang, Y . Zhang, B. Zhang, Y . Tian, and L. Shao, “HRank: Filter pruning using high-rank feature map,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2020, pp. 1529–1538

    2020

  11. [19]

    Network pruning using adaptive exemplar filters,

    M. Lin, R. Ji, S. Li, Y . Wang, Y . Wu, F. Huang, and Q. Ye, “Network pruning using adaptive exemplar filters,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 12, pp. 7357–7366, 2021

    2021

  12. [20]

    Filter sketch for network pruning,

    M. Lin, L. Cao, S. Li, Q. Ye, Y . Tian, J. Liu, Q. Tian, and R. Ji, “Filter sketch for network pruning,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 33, no. 12, pp. 7091–7100, 2021

    2021

  13. [21]

    Carrying out CNN channel pruning in a white box,

    Y . Zhang, M. Lin, C.-W. Lin, J. Chen, Y . Wu, Y . Tian, and R. Ji, “Carrying out CNN channel pruning in a white box,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 10, pp. 7946–7955, 2023

    2023

  14. [22]

    Regression shrinkage and selection via the Lasso,

    R. Tibshirani, “Regression shrinkage and selection via the Lasso,” J. Royal. Statist. Soc. B , vol. 58, pp. 267–288, 1996

    1996

  15. [23]

    Enhancing sparsity by reweighted ℓ1 minimization,

    E. J. Candès, M. B. Wakin, and S. Boyd, “Enhancing sparsity by reweighted ℓ1 minimization,” J. Fourier Anal. Appl. , vol. 14, no. 5- 6, pp. 877–905, 2008

    2008

  16. [24]

    Compressed sensing recovery via nonconvex shrinkage penalties,

    J. Woodworth and R. Chartrand, “Compressed sensing recovery via nonconvex shrinkage penalties,” Inverse Problems, vol. 32, no. 7, pp. 75 004–75 028, 2016

    2016

  17. [25]

    Nearly unbiased variable selection under minimax concave penalty,

    C.-H. Zhang, “Nearly unbiased variable selection under minimax concave penalty,” Ann. Statist., vol. 38, no. 2, pp. 894–942, 2010

    2010

  18. [26]

    Sparsest solutions of underdetermined linear systems via ℓq minimization for 0 < q≤ 1,

    S. Foucart and M.-J. Laui, “Sparsest solutions of underdetermined linear systems via ℓq minimization for 0 < q≤ 1,” Appl. Comput. Harmon. Anal., vol. 26, pp. 395–407, 2009

    2009

  19. [27]

    Sparse regularization via convex analysis,

    I. Selesnick, “Sparse regularization via convex analysis,” IEEE Trans. Signal Process., vol. 65, no. 17, pp. 4481–4494, 2017

    2017

  20. [28]

    Sparse learning with concave regularization: relaxation of the irrepresentable condition,

    V . Cerone, S. M. Fosson, D. Regruto, and A. Salam, “Sparse learning with concave regularization: relaxation of the irrepresentable condition,” in IEEE Conf. Decis. Control (CDC) , 2020, pp. 396–401

    2020

  21. [29]

    A survey on nonconvex regularization-based sparse and low-rank recovery in signal processing, statistics, and machine learning,

    F. Wen, L. Chu, P. Liu, and R. C. Qiu, “A survey on nonconvex regularization-based sparse and low-rank recovery in signal processing, statistics, and machine learning,” IEEE Access, vol. 6, pp. 69 883–69 906, 2018

    2018

  22. [30]

    On the convergence of the iterative shrinkage/thresholding algorithm with a weakly convex penalty,

    I. Bayram, “On the convergence of the iterative shrinkage/thresholding algorithm with a weakly convex penalty,” IEEE Trans. Signal Process. , vol. 64, no. 6, pp. 1597–1608, 2016

    2016

  23. [31]

    A general iterative shrinkage and thresholding algorithm for non-convex regularized opti- mization problems,

    P. Gong, C. Zhang, Z. Lu, J. Huang, and J. Ye, “A general iterative shrinkage and thresholding algorithm for non-convex regularized opti- mization problems,” in Proc. Int. Conf. Mach. Learning (ICML) . PMLR, 2013, pp. 37–45

    2013

  24. [32]

    Minimization of non-smooth, non-convex functionals by iterative thresholding,

    K. Bredies, D. A. Lorenz, and S. Reiterer, “Minimization of non-smooth, non-convex functionals by iterative thresholding,” J. Optim. Theory Appl., vol. 165, no. 1, pp. 78–112, 2015

    2015

  25. [33]

    Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems,

    M. Hong, Z. Q. Luo, and M. Razaviyayn, “Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems,” SIAM J. Optim. , vol. 26, no. 1, pp. 337–364, 2016

    2016

  26. [34]

    What’s hidden in a randomly weighted neural network?

    V . Ramanujan, M. Wortsman, A. Kembhavi, A. Farhadi, and M. Rastegari, “What’s hidden in a randomly weighted neural network?” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2020, pp. 11 890–11 899

    2020

  27. [35]

    Proving the lottery ticket hypothesis: Pruning is all you need,

    E. Malach, G. Yehudai, S. Shalev-Schwartz, and O. Shamir, “Proving the lottery ticket hypothesis: Pruning is all you need,” in Proc. Int. Conf. Mach. Learning (ICML) , vol. 119, 2020, pp. 6682–6691

    2020

  28. [36]

    The MNIST Database of Handwritten Digit Images for Machine Learning Research,

    L. Deng, “The MNIST Database of Handwritten Digit Images for Machine Learning Research,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012

    2012

  29. [37]

    Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks,

    Z. You, K. Yan, J. Ye, M. Ma, and P. Wang, “Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks,” Proc. Adv. Neural Inform. Process. Syst. (NeurIPS) , vol. 32, 2019

    2019

  30. [38]

    Pruning filters for efficient convnets,

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” arXiv preprint arXiv:1608.08710 , 2016

    2016 arXiv

  31. [39]

    Nisp: Pruning networks using neuron importance score propagation,

    R. Yu, A. Li, C.-F. Chen, J.-H. Lai, V . I. Morariu, X. Han, M. Gao, C.-Y . Lin, and L. S. Davis, “Nisp: Pruning networks using neuron importance score propagation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 9194–9203

    2018

  32. [40]

    Discrimination-aware channel pruning for deep neural networks,

    Z. Zhuang, M. Tan, B. Zhuang, J. Liu, Y . Guo, Q. Wu, J. Huang, and J. Zhu, “Discrimination-aware channel pruning for deep neural networks,” Proc. Adv. Neural Inform. Process. Syst. (NeurIPS) , vol. 31, 2018

    2018

  33. [41]

    Scop: Scientific control for reliable neural network pruning,

    Y . Tang, Y . Wang, Y . Xu, D. Tao, C. Xu, C. Xu, and C. Xu, “Scop: Scientific control for reliable neural network pruning,” Proc. Adv. Neural Inform. Process. Syst. (NeurIPS) , vol. 33, pp. 10 936–10 947, 2020

    2020

  34. [42]

    Soft filter pruning for accelerating deep convolutional neural networks,

    Y . He, G. Kang, X. Dong, Y . Fu, and Y . Yang, “Soft filter pruning for accelerating deep convolutional neural networks,” arXiv preprint arXiv:1808.06866, 2018

    2018 arXiv

  35. [43]

    Towards optimal structured cnn pruning via generative adversarial learning,

    S. Lin, R. Ji, C. Yan, B. Zhang, L. Cao, Q. Ye, F. Huang, and D. Doermann, “Towards optimal structured cnn pruning via generative adversarial learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 2790–2799

    2019

  36. [44]

    Filter pruning via geometric median for deep convolutional neural networks acceleration,

    Y . He, P. Liu, Z. Wang, Z. Hu, and Y . Yang, “Filter pruning via geometric median for deep convolutional neural networks acceleration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4340–4349

    2019

  37. [45]

    Amc: Automl for model compression and acceleration on mobile devices,

    Y . He, J. Lin, Z. Liu, H. Wang, L.-J. Li, and S. Han, “Amc: Automl for model compression and acceleration on mobile devices,” in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2018, pp. 784–800

    2018

  38. [46]

    Operation-aware soft channel pruning using differentiable masks,

    M. Kang and B. Han, “Operation-aware soft channel pruning using differentiable masks,” in International Conference on Machine Learning . PMLR, 2020, pp. 5122–5131

    2020

  39. [47]

    Learning filter pruning criteria for deep convolutional neural networks acceleration,

    Y . He, Y . Ding, P. Liu, L. Zhu, H. Zhang, and Y . Yang, “Learning filter pruning criteria for deep convolutional neural networks acceleration,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2020, pp. 2009–2018

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.