Pith. sign in

REVIEW 4 major objections 6 minor 32 references

ONG: One-Shot NMF-based Gradient Masking for Efficient Model Sparsification

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One-shot NMF pruning holds accuracy at 95% sparsity

desk verdict The NMF-based scoring idea is new and plausible, but the empirical claims are contradicted by the paper's own tables, and the draft is not reproducible as written. read the letter →

arxiv 2508.12891 v1 pith:3IB75ARZ submitted 2025-08-18 cs.CV

Sankar Behera , Yamuna Prasad This is my paper · ORCID
classification cs.CV
keywords networkpruningone-shotnon-negativematrixfactorizationgradientmaskingsparsetrainingmodelsparsificationResNet56CIFAR-10
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes ONG, a one-shot sparsification method: before any training, each weight is scored by how poorly it is reconstructed by a low-rank nonnegative factorization of its layer's absolute weight matrix, and the worst-reconstructed weights are kept while the rest are zeroed once. From then on, gradients and weights are masked so the zeroed positions stay zero throughout training. The claim is that this cheap, fixed-mask recipe reaches accuracy comparable to or better than stable sparsification baselines on CIFAR-10 and CIFAR-100 with ResNet56 at 80, 90, and 95 percent sparsity, while hitting the requested sparsity almost exactly. If true, it offers a simple, deterministic path to precisely controlled sparse models without iterative pruning cycles or retraining phases.

What carries the argument

The central mechanism is the NMF reconstruction-error score combined with strict gradient and weight masking. NMF decomposes the absolute value of each prunable weight matrix into a product of two nonnegative low-rank matrices, and the absolute difference between the original matrix and its reconstruction gives each weight an importance score: a high residual means the weight is poorly explained by the dominant low-rank factors, so it is considered structurally unique and worth keeping. A binary search over the threshold parameter gamma then sets per-layer cutoffs that achieve a desired global sparsity, and a fixed binary mask is applied once. During training, both gradients and weights are multiplied by this mask at every step, so the pruned positions stay exactly zero and the sparsity level cannot degrade.

What would settle it

Train two ResNet56 models on CIFAR-10 with the same target sparsity, one masked by ONG scores and one by randomly chosen masks of the same per-layer sparsity. If the random-mask model matches ONG's accuracy across 80, 90, and 95 percent sparsity, the NMF scores are not carrying the pruning signal. A weaker check: recompute ONG masks after randomly permuting the initial weights within each layer; if accuracy does not drop, the score values themselves are irrelevant.

Watch

Extended reading notes

Core claim

ONG's central discovery is that NMF reconstruction error on the initial weights can serve as a saliency signal for one-shot pruning, provided training strictly enforces the resulting mask. For every prunable weight matrix, ONG takes absolute values, factors them into two nonnegative low-rank matrices $F$ and $G$, and scores each weight by the per-element absolute difference between the original values and the reconstruction $FG$. Weights with large residual are treated as carrying unique information and are retained. A per-layer threshold, either mean plus gamma times standard deviation or median plus gamma times median absolute deviation, produces a binary mask, and a binary search over gamma matches a user-specified global sparsity. During training, every gradient is multiplied by the mask and every weight tensor is re-masked before the optimizer step. In the paper's experiments, ONG matches or exceeds the accuracy of stable sparsification baselines at target sparsities of 80, 90, and 95 percent on CIFAR-10 and at 80 and 90 percent on CIFAR-100 with ResNet56, while keeping achieved sparsity within about half a percent of the target.

Load-bearing premise

The whole pipeline rests on the assumption that NMF reconstruction-error scores computed from the initial, untrained weights already tell which weights will matter after training; if the initial weights do not carry that information, the fixed mask cannot be saved by the later training.

Editorial extensions

If this is right

  • ONG-M and ONG-S match the best stable baselines at 80 and 90 percent sparsity on CIFAR-10 and stay within about one accuracy point of the top baseline at 95 percent sparsity.
  • At 95 percent sparsity on CIFAR-10, ONG-M reports 91.45 percent top-1 accuracy versus 69.99 percent for soft-threshold reparameterization and 88.55 percent for dynamic sparse training, a gap the paper attributes to the stability of a fixed mask.
  • Achieved sparsity lands close to the target, for example 79.80 percent versus an 80 percent goal and 94.80 percent versus a 95 percent goal, so the automated gamma tuning delivers precise sparsity control.
  • On CIFAR-100, ONG-M at 80 percent sparsity reports 70.29 percent accuracy, matching the best baseline, while the 90 percent result remains competitive.
  • Because masking is applied to both gradients and weights every step, the sparse structure is preserved exactly through training, with no regrowth of pruned weights.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If NMF residual scores really capture structural uniqueness rather than magnitude, the same scoring could be applied to activations or gradients to produce data-dependent masks; the paper does not test that extension.
  • Because the mask is frozen before training and never regrown, ONG is a natural test for the lottery-ticket question: comparing masks chosen from different random initializations would show whether the NMF scores are finding durable structure or just a lucky static subnetwork.
  • The strict masking could be combined with structured pruning by dropping channels whose masks are mostly zero, a post-processing step the paper lists as future direction but does not evaluate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ONG (One-shot NMF-based Gradient Masking), a sparsification method that computes NMF reconstruction-error scores on the initial weight matrices, prunes once via per-layer thresholds controlled by a scaling factor γ, tunes γ by binary search to hit a user-specified global sparsity target, and then trains with strict gradient and weight masking so that the fixed sparse mask is preserved throughout training. Experiments are reported on CIFAR-10 and CIFAR-100 with ResNet56 inside the BIMP framework, and the claim is that ONG achieves accuracy comparable or superior to stable sparsification baselines (GMP, GSM, DPF, DNW, LC, STR, CS, DST) at 80%, 90%, and 95% target sparsity. The writing largely describes the method and tables; however, the paper's own numbers do not support the central comparison claim when sparsity levels are matched.

Significance. If the core claim were established, ONG would offer a simple one-shot alternative to iterative pruning with a clear sparsity-control mechanism and a novel scoring principle based on NMF reconstruction error. The paper's strengths are its explicit mask-construction algorithm (Algorithm 1 and 2), the automated γ-tuning procedure, and the insistence on hard sparsity preservation during training. These contributions are easy to state and, in principle, easy to verify. The weakness is empirical: the central comparative claim is not supported by the matched-sparsity evidence, the key assumption that NMF scores on initial random weights identify trainable subnetworks is not validated against even a random-mask baseline, and critical experimental details are left as unresolved template placeholders, so the results are not independently reproducible in their present form.

major comments (4)
  1. [Section V.A, Tables I and II] The claim that ONG is comparable or superior to established stable sparsifiers is contradicted by the paper's own matched-sparsity comparisons. In Table I, on CIFAR-10 at 80% sparsity, DPF achieves 93.85% accuracy versus 93.51% for ONG-M; at 90%, GMP achieves 92.88% versus 92.49% for ONG-M; at 95%, DPF achieves 92.48% and GMP 92.07% versus 91.45% (ONG-M) and 91.49% (ONG-S). In Table II on CIFAR-100, at 80% DPF and GMP achieve 70.31% and 70.29% versus 70.29% for ONG-M, and at 90% DPF achieves 69.91% versus 69.25% for ONG-M. The favorable comparisons to STR and DST are not matched in achieved sparsity: STR sits at 99.17–99.30% and DST at 97.31–97.41% while ONG sits near 79.80–94.80%, so the large reported margins are artifacts of comparing different problems rather than evidence of superiority.
  2. [Section III.B and Section V.A] The text states that achieved sparsities closely matching targets 'verifies the consistency of the MAD-based masking mechanism,' but this match is guaranteed by construction by the binary-search γ-tuning procedure in Algorithm 2. The binary search explicitly minimizes the difference between achieved and target sparsity, so a close match is not an empirical verification of anything about the masking mechanism or the scoring method. This circular verification step should be removed or reframed as a check on the search implementation, not as evidence for the method's effectiveness.
  3. [Section IV.D and Table I] The experiments are not reproducible because key hyperparameters and result definitions are left as template placeholders. Section IV.D contains unresolved items such as 'k=[e.g., 6]', 'γ search range [e.g., 0.01, 10.0]', 'Initial γ guess [e.g., 1.0 or 1.5]', and 'Random Seed: [Specify if you use a fixed seed...]'. Moreover, the table labels ONG-M and ONG-S are never defined in the text; the reader cannot tell whether these correspond to different thresholding types, different seeds, or different network variants. Without these details and without a clear definition of the two ONG variants, the reported accuracy numbers cannot be independently checked.
  4. [Section III.A and Section V] The load-bearing assumption of the method is that NMF reconstruction error computed once on the initial random weights identifies weights that will remain important after training. The paper motivates this only by the assertion that high reconstruction error means 'unique information,' but it provides no theoretical argument and no control experiment to isolate this factor. In particular, there is no comparison of ONG against a random-mask baseline under the identical fixed-mask training protocol, nor against a magnitude-based one-shot scoring method under the same protocol. Such a comparison is necessary to establish that the NMF scoring, rather than the strong gradient/weight masking or the act of one-shot pruning itself, is responsible for the reported accuracy. Without this isolation, even the matched-sparsity results would not support the claim that NMF scores are a useful saliency signal.
minor comments (6)
  1. [Section V.A] The name 'ONG' is expanded as 'One-shot NMF-based Gradient Masking' in the abstract, but Section V.A refers to 'Orthogonal NMF-based Gradient-masked pruning'; orthogonal NMF is not otherwise defined or used.
  2. [Table II] In the 80% column, the CS row lists achieved sparsity 89.56%, which appears to be a copy-paste error; the achieved sparsity for CS at the 80% target should presumably be near 79.89% like the other GMP-family baselines.
  3. [Section III.A, Eq. (2)] Equation (2) writes Score(w_ij) = |w_abs,ij - w_hat_abs,ij|, but the notation is introduced as W_abs and the score for w_ij; the subscripts should be aligned so that it is clear the score applies to the absolute value of the original element.
  4. [Section IV.B] The list of baselines includes DPF and DNW in the results tables but the baseline list in Section IV.B does not mention DNW; a brief description or citation for DNW should be added.
  5. [Section IV.D] The NMF decomposition in Eq. (1) is stated as a minimization problem, but the paper does not specify which NMF algorithm, initialization, or convergence criterion is used; citing Lee and Seung [31] is not sufficient for reproducibility.
  6. [Abstract and Section I] The abstract claims 'maintaining structural integrity post-pruning,' but the experiments only measure unstructured global sparsity, accuracy, and FLOPs; no structural integrity metric or structured-pruning analysis is reported.

Circularity Check

1 steps flagged · score 4.0 of 10

Achieved-sparsity 'verification' is enforced by the gamma-tuning binary search; central accuracy claim is independent.

  1. fitted input called prediction [Section V.A (Tables I-II); cf. Section III.B.2 and Algorithm 2]
    "Additionally, the achieved sparsities (AS) closely match the targets, verifying the consistency of the MAD-based masking mechanism. ... When a target sparsity S_target is specified, ONG initiates a binary search for an optimal scaling factor γ∗ ... if |S^(t)_achieved − S_target| is within a tolerance ε_sparsity, γ∗ = γ^(t) and search stops."

    Algorithm 2 does not test whether MAD-based masking hits the target; it searches γ until achieved sparsity is within ε of the target, then reports the closest γ. Therefore the Tables I-II AS values (e.g., 79.80 vs 80, 89.88 vs 90, 94.80 vs 95) are outputs of that search, so the close match is guaranteed by construction. Calling this match a 'verification' of the masking mechanism is circular because the mechanism includes the γ-tuner; the fact that the binary search converged is not independent evidence of consistency. The central accuracy claim does not reduce to this fit, but this particular 'verification' is a fitted input presented as a prediction.

full rationale

The main derivation — NMF reconstruction-error scoring on initial weights, one-shot mask creation, and fixed-mask gradient/weight masking — is a heuristic pipeline, not a circular reduction: accuracy is measured after training and was not used to set the mask or the threshold (the threshold is set by sparsity target, not by accuracy). No load-bearing self-citation or imported uniqueness theorem appears; BIMP [23] is an external framework. The only by-construction element is the achieved-sparsity match, which Algorithm 2 forces via binary search over γ. That inflates the paper's claim that AS 'closely matches' targets 'verifying the consistency of the MAD-based masking mechanism,' but it does not by itself make the accuracy results circular. The accuracy comparisons have a separate evidentiary problem (mismatched achieved sparsity for STR/DST), which is a correctness concern outside this pass.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are postulated; the custom MaskedLinear and MaskedConv2d layers are software artifacts, not scientific entities.

free parameters (3)
  • NMF component count k = e.g., 6 (placeholder)
    Controls the rank of the NMF basis and therefore the reconstruction-error scores; no concrete value is fixed in the paper.
  • Threshold scaling gamma = tuned by binary search to hit target sparsity; no fixed numeric value reported
    Sets per-layer thresholds; when a target sparsity is given, gamma is fitted to that target, so achieved sparsity match is by construction.
  • NMF iterations N_iter = 200
    Number of NMF optimization iterations; chosen by hand, not justified.
assumptions (3)
  • ad hoc to paper NMF reconstruction error on initial random weights identifies weights important for later training.
    The whole pruning decision rests on this premise, introduced in Section III.A without theoretical or empirical support beyond the final accuracy numbers.
  • domain assumption Trainable sparse subnetworks exist at initialization (Lottery Ticket Hypothesis).
    The paper cites the Lottery Ticket Hypothesis to justify one-shot pruning at initialization; if this background assumption fails for the tested architectures, the approach has no justification.
  • standard math NMF on absolute weight matrices produces a meaningful low-rank approximation of weight structure.
    The method relies on standard NMF optimization to provide reconstruction errors; this is a routine background fact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ONG: One-Shot NMF-based Gradient Masking for Efficient Model Sparsification." pith.science (2026). https://pith.science/paper/3IB75ARZ

@misc{pith2026250812891,
  author       = {Pith},
  title        = {Pith review of: ONG: One-Shot NMF-based Gradient Masking for Efficient Model Sparsification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3IB75ARZ}},
  note         = {Machine review of arXiv:2508.12891}
}
read the original abstract

Deep Neural Networks (DNNs) have achieved remarkable success but their large size poses deployment challenges. While various pruning techniques exist, many involve complex iterative processes, specialized criteria, or struggle to maintain sparsity effectively during training. We introduce ONG (One-shot NMF-based Gradient Masking), a novel sparsification strategy that identifies salient weight structures using Non-negative Matrix Factorization (NMF) for one-shot pruning at the outset of training. Subsequently, ONG employs a precise gradient masking mechanism to ensure that only unpruned weights are updated, strictly preserving the target sparsity throughout the training phase. We integrate ONG into the BIMP comparative framework and evaluate it on CIFAR-10 and CIFAR-100 with ResNet56, ResNet34, and ResNet18 against established stable sparsification methods. Our experiments demonstrate ONG's ability to achieve comparable or superior performance at various sparsity levels while maintaining structural integrity post-pruning and offering a clear mechanism for targeting desired sparsities.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 20 canonical work pages

  1. [1]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in neural information processing systems (NeurIPS), 2017, pp. 5998–6008

    work page 2017

  2. [2]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” inAdvances in neural information processing systems (NeurIPS), 2012, pp. 1097–1105

    work page 2012

  3. [3]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016, pp. 770–778

    2016

  4. [4]

    Scaling laws for neural language models,

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” inarXiv preprint arXiv:2001.08361, 2020

    arXiv 2001

  5. [5]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    arXiv 2023

  6. [6]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 2704–2713

    2018

  7. [7]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” inarXiv preprint arXiv:1503.02531, NIPS Deep Learning Workshop, 2015

    arXiv 2015

  8. [8]

    Exploiting linear structure within convolutional networks for efficient evaluation,

    E. L. Denton, W. Zaremba, J. Bruna, Y . LeCun, and R. Fergus, “Exploiting linear structure within convolutional networks for efficient evaluation,” inAdvances in neural information processing systems (NeurIPS), 2014, pp. 1269–1277

    work page 2014

Show all 32 references
  1. [9]

    Learning both weights and connections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural network,” inAdvances in neural information processing systems (NeurIPS), 2015, pp. 1135–1143

    2015

  2. [10]

    Pruning filters for efficient convnets,

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” inInternational Conference on Learning Representations (ICLR), 2017

    2017

  3. [11]

    Channel pruning for accelerating very deep neural networks,

    Y . He, X. Zhang, and J. Sun, “Channel pruning for accelerating very deep neural networks,” inProceedings of the IEEE international conference on computer vision (ICCV), 2017, pp. 1389–1397

    2017

  4. [12]

    Learning structured sparsity in deep neural networks,

    W. Wen, C. Wu, Y . Wang, Y . Chen, and H. Li, “Learning structured sparsity in deep neural networks,” inAdvances in neural information processing systems (NeurIPS), 2016, pp. 2074–2082

    2016

  5. [13]

    Dynamic structure pruning for compressing CNNs,

    J.-H. Park, Y . Kim, J. Kim, J.-Y . Choi, and S. Lee, “Dynamic structure pruning for compressing CNNs,” inThe Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI-23), 2023, pp. 9408–9416

    2023

  6. [14]

    Exploring the granularity of sparsity in convolutional neural networks,

    H. Mao, S. Han, J. Pool, W. Li, X. Liu, Y . Wang, and W. J. Dally, “Exploring the granularity of sparsity in convolutional neural networks,” arXiv preprint arXiv:1705.05851

    arXiv

  7. [15]

    To prune, or not to prune: exploring the efficacy of pruning for model compression,

    M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compression,” inarXiv preprint arXiv:1710.01878

    arXiv

  8. [16]

    Dynamic model pruning with feedback,

    T. Lin, L. Barba, M. Jaggi, S. U. Stich, and D. Dmitriev, “Dynamic model pruning with feedback,” inInternational Conference on Learning Representations (ICLR), 2020

    2020

  9. [17]

    Deep rewiring: Training very sparse deep networks,

    G. Bellec, D. Kappel, W. Maass, and R. Legenstein, “Deep rewiring: Training very sparse deep networks,” inInternational Conference on Learning Representations (ICLR), 2018

    2018

  10. [18]

    Soft threshold weight reparameterization for pruning neural networks,

    A. Kusupati, V . Ramanujan, R. Vemulapalli, J. Diffenderfer, L. Kaiser, and S. Singh, “Soft threshold weight reparameterization for pruning neural networks,” inInternational Conference on Machine Learning (ICML). PMLR, 2020, pp. 5531–5541

    2020

  11. [19]

    Winning the lottery with continuous sparsification,

    P. Savarese, H. POSSA, A. Testolina, J. Fleury, R. K. P. Gummadi, P. Frossard, and M. Questembert, “Winning the lottery with continuous sparsification,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019

    2019

  12. [20]

    Do we really need complex pruning algorithms? Dynamic sparse training with static sparsity,

    D. Liu, X. Wang, Y . Zhang, R. Tao, L. Wang, and L. Li, “Do we really need complex pruning algorithms? Dynamic sparse training with static sparsity,”arXiv preprint arXiv:2111.00358, 2021

    2021 arXiv

  13. [21]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks,

    J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” inInternational Conference on Learning Representations (ICLR), 2019

    2019

  14. [22]

    SNIP: Single-shot network pruning based on connection sensitivity,

    N. Lee, T. Ajanthan, and P. H. Torr, “SNIP: Single-shot network pruning based on connection sensitivity,” inInternational Conference on Learning Representations (ICLR), 2019

    2019

  15. [23]

    How I learned to stop worrying and love retraining,

    M. Zimmer, C. Spiegel, and S. Pokutta, “How I learned to stop worrying and love retraining,” inThe Eleventh International Conference on Learning Representations (ICLR), 2023. [Online]. Available: https://openreview.net/forum?id= nF5imFKQI

    2023

  16. [24]

    “Learning-compression

    M. ´A. Carreira-Perpin ´an and Y . Idelbayev, ““Learning-compression” algorithms for neural net pruning,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8532– 8541

    2018

  17. [25]

    Global sparse momentum SGD for pruning very deep neural networks,

    X. Ding, G. Ding, Y . Guo, and J. Han, “Global sparse momentum SGD for pruning very deep neural networks,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019

    2019

  18. [26]

    Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science,

    D. C. Mocanu, E. Mocanu, P. Stone, P. H. Nguyen, M. Gibescu, and A. Liotta, “Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science,” inNature communications, vol. 9, no. 1. Nature Publishing Group, 2018, p. 2383. 7

    2018

  19. [27]

    Picking winning tickets before training by preserving gradient flow,

    C. Wang, G. Zhang, R. M ¨uhlematter, and R. Grosse, “Picking winning tickets before training by preserving gradient flow,”arXiv preprint arXiv:2009.10024

    2009 arXiv

  20. [28]

    Speeding up convolutional neural networks with low rank expansions,

    M. Jaderberg, A. Vedaldi, and A. Zisserman, “Speeding up convolutional neural networks with low rank expansions,” inBritish Machine Vision Conference (BMVC), 2014

    2014

  21. [29]

    SoLA: Leveraging soft activation sparsity and low-rank decomposition for large language model compres- sion,

    X. Huang, Y .-L. Huang, and Z. Wen, “SoLA: Leveraging soft activation sparsity and low-rank decomposition for large language model compres- sion,” inThe Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25), 2025

    2025

  22. [30]

    SVD-LLM: Truncation- aware singular value decomposition for large language model compres- sion,

    X. Wang, Y . Zheng, Z. Wan, and M. Zhang, “SVD-LLM: Truncation- aware singular value decomposition for large language model compres- sion,”arXiv preprint arXiv:2403.07378, 2024

    2024 arXiv

  23. [31]

    Learning the parts of objects by non- negative matrix factorization,

    D. D. Lee and H. S. Seung, “Learning the parts of objects by non- negative matrix factorization,”Nature, vol. 401, no. 6755, pp. 788–791, 1999

    1999

  24. [32]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” University of Toronto, Tech. Rep., 2009

    2009

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.