Pith. sign in

REVIEW 2 major objections 5 minor 48 references

prunAdag: an adaptive pruning-aware gradient method

T0 review · 2 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pruning-aware variant of Adagrad provably converges with averaged gradient norm bounded by $O(\log(k)/\sqrt{k+1})$.

desk verdict A nice pruning-aware Adagrad variant with a genuinely new parameter split, but Theorem 2.3's Case 1 proof has a false implication that needs fixing before the result is reliable. read the letter →

arxiv 2502.08308 v1 pith:TP6MSFLX submitted 2025-02-12 math.OC

Margherita Porcelli , Giovanni Seraghiti , Philippe L. Toint This is my paper · ORCID
classification math.OC MSC 90C2690C30
keywords prunAdagmodelpruningadaptivegradientmethodsAdagradobjective-function-freeoptimizationconvergencecomplexitysparsity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

prunAdag is a first-order method for optimizing a smooth function while preparing its solution for pruning: at every iteration it labels coordinates as optimisable or decreasable and updates them differently. The paper's central claim is a convergence guarantee: under smoothness, lower-boundedness, and a bounded-iteration assumption, the average squared gradient norm after $k$ steps is bounded by $\lceil n/T\rceil\theta(k)/(k+1)$ with $\theta(k)$ growing only logarithmically, giving a global rate of $O(\log(k)/\sqrt{k+1})$. This matters because earlier relevant/irrelevant pruning schemes lacked a finite-time rate for this kind of adaptive update, and standard Adagrad is not designed to produce pruning-friendly solutions. The paper also reports experiments on least-squares, signal recovery, dictionary learning, and binary classification showing that prunAdag is competitive and that its solutions tolerate thresholding better than Adagrad's.

What carries the argument

The load-bearing mechanism is a double coordinate-wise accumulator. Each coordinate $i$ keeps two weights: $w^O_{i,k}$, which accumulates the squares of past gradient components only on iterations where $i$ is optimisable, and $w^D_{i,k}$, which accumulates the squares of past coordinate magnitudes only on iterations where $i$ is decreasable. Optimisable coordinates take the Adagrad step $-g_{i,k}/w^O_{i,k}$; decreasable coordinates take any step $s^D_{i,k}$ that is no larger in magnitude than $-x_{i,k}/w^D_{i,k}$ and whose gradient product is nonpositive, a trust-region-like condition. The descent lemma (Lemma 2.1) bounds the function decrease, and a log-sum lemma (Lemma 2.2) converts the accumulated sums into logarithmic terms, producing the $\log(k)$ factor in the final rate.

What would settle it

On a smooth problem with known $L$ and a certified bound $\kappa_x$ on the iterates, compute the averaged squared gradient norm at each $k$ and compare it with the right-hand side of (2.22); any violation would refute Theorem 2.3, and monitoring $\max_i|x_{i,k}|$ on a bounded-below problem tests whether AS.4 can fail.

Watch

Extended reading notes

Core claim

The central result is Theorem 2.3. If $f$ is continuously differentiable with $L$-Lipschitz gradient and bounded below, if the iterates $x_k$ satisfy $|x_{i,k}|\le\kappa_x$ for all coordinates and all $k$, and if the relevant set $R_k$ always contains the $T$ largest gradient components in absolute value, then the average squared gradient norm satisfies $$\operatorname*{average}_{j\in\{0,\dots,k\}}\|g_j\|^2 \le \frac{\lceil n/T\rceil\,\$\theta$(k)}{k+1},$$ where $\theta(k)$ is a constant-scale quantity whose slowest growth is $2(\Gamma_0+nL\log(1+(k+1)\kappa_x^2/\varsigma))^2$, so the averaged gradient norm decays as $O(\log(k)/\sqrt{k+1})$. The proof separates each iteration's descent into a negative term from optimisable coordinates and two positive terms bounded by logarithms of accumulated squared gradients and accumulated squared decreasable coordinates; the largest-$T$ selection then converts a bound on optimisable gradients into a bound on the full gradient norm.

Load-bearing premise

The rate in Theorem 2.3 holds only if the iterates stay inside a fixed box ($|x_{i,k}|\le\kappa_x$ for all $i,k$), and the paper does not show that prunAdag's own update rules force this to happen for every smooth bounded-below problem.

Editorial extensions

If this is right

  • A model trained by prunAdag can be thresholded after training to any desired sparsity level; the reported robustness measures $\rho$ and $\omega$ stay small up to roughly 50% sparsity, with Version 3 most reliable in the tested range.
  • The worst-case rate is only a logarithmic factor worse than standard Adagrad's $O(1/\sqrt{k+1})$, so making the method pruning-aware does not change the order of the convergence guarantee.
  • Because the decreasable step is specified only through two abstract conditions, any shrinkage rule satisfying them inherits the same convergence rate, leaving room for alternative pruning strategies.
  • On the tested problems, all four prunAdag versions prune more safely than plain Adagrad, and Version 4 handles very aggressive pruning best.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit generalisation is that the optimisable/decreasable split is not tied to Adagrad specifically; replacing the accumulating weights with exponential moving averages of the Adam or RMSprop type should preserve the structural argument as long as the weights grow fast enough for the logarithmic bound.
  • The stochastic case, which the paper flags as future work, is a natural stress test: with noisy gradients the relevant-set selection becomes random, and one would expect the rate to acquire an additive noise-variance term, so the pruning-robustness measures should be re-examined under mini-batch noise.
  • The experiments show that a smaller target cardinality $T$ yields more components below the sparsity threshold at the price of slower convergence, suggesting a natural adaptive-$T$ strategy that tunes the trade-off during training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes prunAdag, a deterministic adaptive gradient method for unconstrained smooth optimization that classifies coordinates at each iteration into an optimisable set (updated with an Adagrad-like step) and a decreasable set (whose magnitudes are shrunk with a controlled penalty step), thereby extending the relevant/irrelevant pruning approach of Ding et al. and Zimmer et al. Under Assumptions AS.1–AS.4, Theorem 2.3 claims an average squared gradient bound of order O(log(k)/sqrt(k+1)). The proof uses a descent lemma, the log-sum inequality, and a Lambert-W function argument. Numerical experiments on random least-squares, SPARCO problems, dictionary learning, and binary classification compare four prunAdag variants with Frank-Wolfe and Adagrad, focusing on robustness to pruning.

Significance. If Theorem 2.3 is established, the paper provides the first finite-time convergence-rate guarantee for a pruning-aware Adagrad-type method in the deterministic smooth setting, a meaningful extension of earlier relevant/irrelevant heuristics. The numerical study is broad, the four algorithmic variants are clearly specified, and the proof strategy is largely self-contained: the descent lemma is standard, the use of the logarithmic sum bound is appropriate, and the reduction from the full gradient norm to the optimisable-gradient norm via the top-T components is clean. The main caveat is that the proof of the central theorem contains a gap (Major Comment 1), so the claimed rate is not currently demonstrated as written; however, the error is localized and likely repairable without changing the overall scope of the paper.

major comments (2)
  1. [2.1, proof of Theorem 2.3, Case 1] The implication following (2.30) is invalid. The assumption sum_{j=0}^k ||g^O_j||^2 >= max[sigma, sigma/(2e) * Gamma0/(nL)] gives Gamma0 <= 2e nL (sum/sigma), but the proof then claims Gamma0 <= nL log(2 sum/sigma). This does not follow; for example, with Gamma0 = 100 nL and sum = sigma * 100/(2e), the assumption holds while log(2 sum/sigma) is about 3.6, far smaller than Gamma0/(nL) = 100. Because the subsequent inequalities (2.32) and the Lambert-W bound (2.35) depend on this replacement, the constant in the definition of theta(k) is not established by the argument given. Please correct the threshold (for instance, replace the second lower bound by sum >= (sigma/2) exp(Gamma0/(nL)), which makes the logarithmic implication valid at the cost of changing the constant in (2.36) and in theta(k)) or provide a different proof of Case 1.
  2. [2.1, Assumption AS.4] The boundedness assumption AS.4 is load-bearing in (2.27), where it supplies the term n log(1 + (k+1) kappa_x^2/sigma). Without it the right-hand side of (2.28) does not have the form used in the rest of the proof. The algorithm's step rules do not obviously enforce bounded iterates for arbitrary smooth f, and Lemma 2.1 does not give monotone decrease because of the positive terms in (2.17). No sufficient condition on f is provided. Please either prove AS.4 under stated conditions (for example, coercivity plus a suitable descent condition) or present Theorem 2.3 explicitly as conditional on this a priori bound, and comment on whether the numerical test problems satisfy it.
minor comments (5)
  1. [3.4, Eq. (3.4)] The averaged logistic loss is convex, not 'neither linear nor convex' as stated in the text.
  2. [2.1, before AS.4] The phrase 'without loss of generality' for the assumption sigma <= (8nL/3)^2 is imprecise: sigma is an algorithmic parameter and L may not satisfy the inequality. Please state this as an explicit assumption on the parameter range.
  3. [2, Algorithm 2.1, Step 6] In Step 6, the update w^O_i,k = w^O_i,k-1 for i in D_k overwrites the value computed in Step 3. The text should clarify that this is intentional and is what makes identity (2.15) valid.
  4. [3, experimental setup] The paper sets T = n/10; when n is not divisible by 10, please specify how T is rounded, since Theorem 2.3 assumes |R_k| = T.
  5. [2.1, Lemma 2.1] There is a typo: 'boundness' should be 'boundedness' in the paragraph introducing AS.4.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence proof is self-contained and derived from stated assumptions, not from fitted values or load-bearing self-citations.

full rationale

Walking the derivation chain from AS.1–AS.4 to Theorem 2.3, each bound is obtained in-text: (2.17)–(2.18) from Lipschitz smoothness and the step definitions; (2.26)–(2.27) from Lemma 2.2 (an external lemma from [40]) and AS.4; (2.28)–(2.41) from elementary algebra and Lambert-W calculus. No equation defining the target rate is fed back as an input, and no parameter in θ(k) is fitted from experiments. The related-work discussion cites [1,2] and [8,9,10] (including co-authored prior work by Toint) only for inspiration and interpretation; the proof of Theorem 2.3 does not rest on those citations. The numerical comparisons use tuned Frank-Wolfe baselines, but that tuning affects only the empirical comparison, not the theorem. There is therefore no circular step. A separate reviewer concern that the Case 1 implication 'Γ0 ≤ 2e nL Σ/ς ⇒ Γ0 ≤ nL log(2Σ/ς)' is mathematically invalid is a proof-correctness issue, not an input–output equivalence, and is not counted here.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central proof relies only on standard smoothness assumptions plus the explicit boundedness assumption AS.4. The hyperparameters T and sigma are user-chosen, not fitted to hit the convergence claim. No new physical or mathematical entities are introduced.

free parameters (4)
  • T (target number of relevant parameters) = n/10 in most experiments
    User-supplied hyperparameter controlling how many gradient components are classified as relevant; the theorem assumes |R_k| = T.
  • sigma (initial weight constant) = 1/100
    Choice of sigma affects the step-size scale and enters the rate bound; the proof assumes sigma <= (8 n L / 3)^2.
  • Version parameters a_i,k and b_i,k (four versions) = see Table 1
    Hand-chosen bounding sequences define the acceptable set A_k; performance varies across versions.
  • FW baseline hyperparameters (tau1, tau2, beta) = tuned per problem family by trial and error
    Baseline parameters do not affect the prunAdag claim, but they influence the empirical comparisons.
assumptions (5)
  • domain assumption AS.1: f is continuously differentiable
    Standard smoothness required by the descent lemma.
  • domain assumption AS.2: gradient Lipschitz continuous with constant L
    Used in Lemma 2.1 to bound the Taylor remainder.
  • domain assumption AS.3: f is bounded below
    Provides the finite gap Gamma0 used in the rate.
  • domain assumption AS.4: iterates bounded by kappa_x
    Used in (2.27) to bound the decreasable-component sum; not verified or guaranteed by the algorithm.
  • ad hoc to paper Assume without loss of generality sigma <= (8 n L / 3)^2
    Ensures the Lambert W branch is well-defined in (2.34); requires sigma to be small enough relative to the unknown L.

how reviews work

0 comments
Cite this review

Pith. "Pith review of prunAdag: an adaptive pruning-aware gradient method." pith.science (2026). https://pith.science/paper/TP6MSFLX

@misc{pith2026250208308,
  author       = {Pith},
  title        = {Pith review of: prunAdag: an adaptive pruning-aware gradient method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TP6MSFLX}},
  note         = {Machine review of arXiv:2502.08308}
}
abstract

A pruning-aware adaptive gradient method is proposed which classifies the variables in two sets before updating them using different strategies. This technique extends the ``relevant/irrelevant" approach of Ding (2019) and Zimmer et al. (2022) and allows a posteriori sparsification of the solution of model parameter fitting problems. The new method is proved to be convergent with a global rate of decrease of the averaged gradient's norm of the form $\calO(\log(k)/\sqrt{k+1})$. Numerical experiments on several applications show that it is competitive.

Figures

Figures reproduced from arXiv: 2502.08308 by the authors.

Figure 1
Figure 1. Effect of introducing the class of optimisable param [PITH_FULL_IMAGE:figures/full_fig_p017_1.png] view at source ↗
Figure 2
Figure 2. Dynamic of parameters’ classification in the sets [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Norm of the gradient on the left and percentage of pa [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: On top, (a) gradient norm and (b) percentage of comp [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]
Figure 5
Figure 5. Figure 5: On top, (a) gradient norm and (b) percentage of comp [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: MNIST sparse coding step in dictionary learning. V [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: On top, (a) gradient norm decrease and (b) percenta [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: On top, (a) gradient norm decrease and (b) percenta [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 34 canonical work pages

  1. [1]

    Global sparse momentum SGD for pruning very deep neural networks,

    X. Ding, X. Zhou, Y. Guo, J. Han, J. Liu, et al. , “Global sparse momentum SGD for pruning very deep neural networks,” Advances in Neural Information Processing Systems , vol. 32, 2019

    work page 2019

  2. [2]

    Compression-awa re training of neural networks using Frank- Wolfe,

    M. Zimmer, C. Spiegel, and S. Pokutta, “Compression-awa re training of neural networks using Frank- Wolfe,” arXiv preprint arXiv:2205.11921 , 2022

    arXiv 2022

  3. [3]

    Adaptive subgradient methods for online learning and stochastic optimization.,

    J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization.,” Journal of machine learning research , vol. 12, no. 7, 2011

    work page 2011

  4. [4]

    Adaptive bound optimizat ion for online convex optimization,

    H. B. McMahan and M. Streeter, “Adaptive bound optimizat ion for online convex optimization,” arXiv preprint arXiv:1002.4908 , 2010

    arXiv 2010

  5. [5]

    Adam: a method for stochastic optimizatio n,

    D. P. Kingma, “Adam: a method for stochastic optimizatio n,” arXiv preprint arXiv:1412.6980 , 2014

    arXiv 2014

  6. [6]

    Lecture 6.5-rmsprop, course ra: Neural networks for machine learning,

    T. Tieleman and G. Hinton, “Lecture 6.5-rmsprop, course ra: Neural networks for machine learning,” University of Toronto, Technical Report , vol. 6, 2012

    work page 2012

  7. [7]

    ADADELTA: an adaptive learning rate metho d,

    M. D. Zeiler, “ADADELTA: an adaptive learning rate metho d,” arXiv preprint arXiv:1212.5701 , 2012

    arXiv 2012

  8. [8]

    Complexity and performance for two classes of noise-tolerant first-order algorithms

    S. Gratton, S. Jerad, and P. L. Toint, “First-order objec tive-function-free optimization algorithms and their complexity,” arXiv preprint arXiv:2203.01757 , 2022

    work page Pith review arXiv 2022

Show all 48 references
  1. [9]

    Multilevel objective-function-free optimization with an application to neural networks training,

    S. Gratton, A. Kopaniˇ c´ akov´ a, and P. L. Toint, “Multilevel objective-function-free optimization with an application to neural networks training,” SIAM Journal on Optimization , vol. 33, no. 4, pp. 2772–2800, 2023

    2023

  2. [10]

    Complexity of a cl ass of first-order objective-function-free opti- mization algorithms,

    S. Gratton, S. Jerad, and P. L. Toint, “Complexity of a cl ass of first-order objective-function-free opti- mization algorithms,” Optimization Methods and Software , pp. 1–31, 2024

    2024

  3. [11]

    A. R. Conn, N. I. Gould, and P. L. Toint, Trust region methods. SIAM, 2000

    2000

  4. [12]

    Recent advances in trust region algorithm s,

    Y.-x. Yuan, “Recent advances in trust region algorithm s,” Mathematical Programming, vol. 151, pp. 249– 281, 2015

    2015

  5. [13]

    Pruning algorithms-a survey,

    R. Reed, “Pruning algorithms-a survey,” IEEE transactions on Neural Networks , vol. 4, no. 5, pp. 740–747, 1993

    1993

  6. [14]

    Optimal brain damage ,

    Y. LeCun, J. Denker, and S. Solla, “Optimal brain damage ,” Advances in neural information processing systems, vol. 2, 1989

    1989

  7. [15]

    To prune, or not to prune: exploring the efficacy of pruning for model compres- sion,

    M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compres- sion,” arXiv preprint arXiv:1710.01878 , 2017

    2017 arXiv

  8. [16]

    Accelerating very de ep convolutional networks for classification and detection,

    X. Zhang, J. Zou, K. He, and J. Sun, “Accelerating very de ep convolutional networks for classification and detection,” IEEE transactions on pattern analysis and machine intellig ence, vol. 38, no. 10, pp. 1943–1955, 2015

    1943

  9. [17]

    Exploiting linear structure within con- volutional networks for efficient evaluation,

    E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergu s, “Exploiting linear structure within con- volutional networks for efficient evaluation,” Advances in neural information processing systems , vol. 27, 2014

    2014

  10. [18]

    On compressing deep mo dels by low rank and sparse decomposi- tion,

    X. Yu, T. Liu, X. Wang, and D. Tao, “On compressing deep mo dels by low rank and sparse decomposi- tion,” in Proceedings of the IEEE conference on computer vision and pa ttern recognition, pp. 7370–7379, 2017

    2017

  11. [19]

    Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1,

    M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y . Bengio, “Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or -1,” arXiv preprint arXiv:1602.02830, 2016

    2016 arXiv

  12. [20]

    T wo-step quantization for low-bit neural networks,

    P. Wang, Q. Hu, Y. Zhang, C. Zhang, Y. Liu, and J. Cheng, “T wo-step quantization for low-bit neural networks,” in Proceedings of the IEEE Conference on computer vision and pa ttern recognition, pp. 4376– 4384, 2018

    2018

  13. [21]

    Position-based scaled grad ient for model quantization and pruning,

    J. Kim, K. Yoo, and N. Kwak, “Position-based scaled grad ient for model quantization and pruning,” Advances in neural information processing systems , vol. 33, pp. 20415–20426, 2020. Porcelli, Seraghiti, Toint: An adaptive pruning-aware gra dient method 30

    2020

  14. [22]

    Learning both wei ghts and connections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. Dally, “Learning both wei ghts and connections for efficient neural network,” Advances in neural information processing systems , vol. 28, 2015

    2015

  15. [23]

    Exploiting sparsene ss in deep neural networks for large vocabulary speech recognition,

    D. Yu, F. Seide, G. Li, and L. Deng, “Exploiting sparsene ss in deep neural networks for large vocabulary speech recognition,” in 2012 IEEE International conference on acoustics, speech an d signal processing (ICASSP), pp. 4409–4412, IEEE, 2012

    2012

  16. [24]

    Learning spar se neural networks through ℓ0 regularization,

    C. Louizos, M. Welling, and D. P. Kingma, “Learning spar se neural networks through ℓ0 regularization,” arXiv preprint arXiv:1712.01312 , 2017

    2017 arXiv

  17. [25]

    Compression-aware trai ning of deep networks,

    J. M. Alvarez and M. Salzmann, “Compression-aware trai ning of deep networks,” Advances in neural information processing systems , vol. 30, 2017

    2017

  18. [26]

    Network trimming: A data-driven neuron pruning approach towards efficient deep architectures,

    H. Hu, “Network trimming: A data-driven neuron pruning approach towards efficient deep architectures,” arXiv preprint arXiv:1607.03250 , 2016

    2016 arXiv

  19. [27]

    Sparsity in deep learning: Pruning and growth for efficient inference and training in neural network s,

    T. Hoefler, D. Alistarh, T. Ben-Nun, N. Dryden, and A. Pes te, “Sparsity in deep learning: Pruning and growth for efficient inference and training in neural network s,” Journal of Machine Learning Research , vol. 22, no. 241, pp. 1–124, 2021

    2021

  20. [28]

    Dynamic network surgery for efficient DNNs,

    Y. Guo, A. Yao, and Y. Chen, “Dynamic network surgery for efficient DNNs,” Advances in neural infor- mation processing systems , vol. 29, 2016

    2016

  21. [29]

    Scalable training of artificial neural networks with adaptive sparse connectivi ty inspired by network science,

    D. C. Mocanu, E. Mocanu, P. Stone, P. H. Nguyen, M. Gibesc u, and A. Liotta, “Scalable training of artificial neural networks with adaptive sparse connectivi ty inspired by network science,” Nature commu- nications, vol. 9, no. 1, p. 2383, 2018

    2018

  22. [30]

    Soft filter pru ning for accelerating deep convolutional neural networks,

    Y. He, G. Kang, X. Dong, Y. Fu, and Y. Yang, “Soft filter pru ning for accelerating deep convolutional neural networks,” arXiv preprint arXiv:1808.06866 , 2018

    2018 arXiv

  23. [31]

    Deep neural netw ork training with Frank-Wolfe,

    S. Pokutta, C. Spiegel, and M. Zimmer, “Deep neural netw ork training with Frank-Wolfe,” arXiv preprint arXiv:2010.07243, 2020

    2010 arXiv

  24. [32]

    Pruning convolutional neural networks for resource efficient inference,

    P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” arXiv preprint arXiv:1611.06440 , 2016

    2016 arXiv

  25. [33]

    Fas ter gaze prediction with dense networks and Fisher pruning,

    L. Theis, I. Korshunova, A. Tejani, and F. Husz´ ar, “Fas ter gaze prediction with dense networks and Fisher pruning,” arXiv preprint arXiv:1801.05787 , 2018

    2018 arXiv

  26. [34]

    Learn ing pruning-friendly networks via Frank-Wolfe: One-shot, any-sparsity, and no retraining,

    M. Lu, X. Luo, T. Chen, W. Chen, D. Liu, and Z. Wang, “Learn ing pruning-friendly networks via Frank-Wolfe: One-shot, any-sparsity, and no retraining,” in International Conference on Learning Rep- resentations, 2022

    2022

  27. [35]

    Sparse predicti on with the k-support norm,

    A. Argyriou, R. Foygel, and N. Srebro, “Sparse predicti on with the k-support norm,” Advances in Neural Information Processing Systems , vol. 25, 2012

    2012

  28. [36]

    The group k-support norm for learning with structured sparsity,

    N. Rao, M. Dud ´ ık, and Z. Harchaoui, “The group k-support norm for learning with structured sparsity,” in 2017 IEEE International Conference on Acoustics, Speech an d Signal Processing (ICASSP) , pp. 2402– 2406, IEEE, 2017

    2017

  29. [37]

    An algorithm for quadratic programming,

    M. Frank, P. Wolfe, et al. , “An algorithm for quadratic programming,” Naval research logistics quarterly , vol. 3, no. 1-2, pp. 95–110, 1956

    1956

  30. [38]

    Constrained minimizati on methods,

    E. S. Levitin and B. T. Polyak, “Constrained minimizati on methods,” USSR Computational mathematics and mathematical physics , vol. 6, no. 5, pp. 1–50, 1966

    1966

  31. [39]

    Stochasti c Frank-Wolfe methods for nonconvex optimiza- tion,

    S. J. Reddi, S. Sra, B. P´ oczos, and A. Smola, “Stochasti c Frank-Wolfe methods for nonconvex optimiza- tion,” in 2016 54th annual Allerton conference on communication, con trol, and computing (Allerton) , pp. 1244–1251, IEEE, 2016

    2016

  32. [40]

    Wngrad: Learn the learnin g rate in gradient descent,

    X. Wu, R. Ward, and L. Bottou, “Wngrad: Learn the learnin g rate in gradient descent,” arXiv preprint arXiv:1803.02865, 2018

    2018 arXiv

  33. [41]

    On the Lambert W function,

    R. M. Corless, G. H. Gonnet, D. E. Hare, D. J. Jeffrey, and D . E. Knuth, “On the Lambert W function,” Advances in Computational mathematics , vol. 5, pp. 329–359, 1996

    1996

  34. [42]

    Sparco: A testing framework for sparse reconstruction,

    E. van den Berg, M. Friedlander, G. Hennenfent, F. Herrm ann, R. Saab, and O. Yılmaz, “Sparco: A testing framework for sparse reconstruction,” Dept. Comput. Sci., Univ. British Columbia, Vancouver, Tech. Rep. TR-2007-20,[Online]. Available: http://www. c s. ubc. ca/labs/scl/sp...

    2007

  35. [43]

    S2MPJ and CUTEst optimization problems for Matlab, Python and Julia

    S. Gratton and P. L. Toint, “S2MPJ and CUTEst optimization problems for Matlab, Python and Julia.” arXiv:2407.07812, 2024. Porcelli, Seraghiti, Toint: An adaptive pruning-aware gra dient method 31

    2024 arXiv

  36. [44]

    A fast algorit hm for sparse reconstruction based on shrinkage, subspace optimization, and continuation,

    Z. Wen, W. Yin, D. Goldfarb, and Y. Zhang, “A fast algorit hm for sparse reconstruction based on shrinkage, subspace optimization, and continuation,” SIAM Journal on Scientific Computing , vol. 32, no. 4, pp. 1832–1857, 2010

    2010

  37. [45]

    A variable fixing version of the two-block nonlinear constrained Gauss-Seidel algorithm for ℓ 1-regularized least-squares,

    M. Porcelli and F. Rinaldi, “A variable fixing version of the two-block nonlinear constrained Gauss-Seidel algorithm for ℓ 1-regularized least-squares,” Computational Optimization and Applications , vol. 59, no. 3, pp. 565–589, 2014

    2014

  38. [46]

    K-SVD: An algori thm for designing overcomplete dictionaries for sparse representation,

    M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algori thm for designing overcomplete dictionaries for sparse representation,” IEEE Transactions on signal processing , vol. 54, no. 11, pp. 4311–4322, 2006

    2006

  39. [47]

    UCI machine learning repository,

    , “UCI machine learning repository,” 2013

    2013

  40. [48]

    LIBSVM: A library for suppor t vector machines,

    C.-C. Chang and C.-J. Lin, “LIBSVM: A library for suppor t vector machines,” ACM transactions on intelligent systems and technology (TIST) , vol. 2, no. 3, pp. 1–27, 2011. Porcelli, Seraghiti, Toint: An adaptive pruning-aware gra dient method 32 A Detailed numerical results A....

    2011

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.