Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

A Proximal Operator for Inducing 2:4-Sparsity

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that the nonconvex proximal operator of the 2:4-sparsity regularizer can be solved exactly by one quadratic case plus two convex programs, and that masked gradient updates improve existing pruning methods.

desk verdict Useful masked-gradient trick and a clean convex reduction, but the efficient proximal solver rests on an unproven conjecture, so the experiments support a heuristic more than Theorem 7. read the letter →

arxiv 2501.18015 v1 pith:U2GRC4Q3 submitted 2025-01-29 cs.LG

Jonas M Kübler , Yu-Xiang Wang , Shoham Sabach , Navid Ansari , Matthäus Kleindessner , Kailash Budhathoki , Volkan Cevher , George Karypis This is my paper · ORCID
classification cs.LG
keywords 2:4sparsityN:Mstructuredproximaloperatorgradientlargelanguagemodelpruningmaskedupdatesnonconvexoptimizationlocalsquaredloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Structured 2:4 sparsity lets modern GPUs skip matrix work, but deciding which weights to zero usually hurts accuracy. The paper introduces a regularizer whose zero set is exactly the 2:4-sparse patterns and proves that its proximal operator, a nonconvex nonsmooth problem, can be minimized globally by solving three convex subproblems. That makes it practical to run proximal gradient on the per-layer squared loss, so the pruning mask emerges gradually and can exploit correlations between input features. After the mask is fixed, masked gradient updates further lower the local squared loss. On large language models up to 13B parameters the resulting method improves perplexity over Wanda, SparseGPT, and DSnoT, and at 70B it matches them.

What carries the argument

The load-bearing object is the regularizer family $r_{N:M}(w)=\sum_{S\subset[M],|S|=N+1}\prod_{j\in S}|w_j|$, whose null space is exactly the N:M-sparse patterns; for 2:4 it is the sum of the four triple products above. The argument is carried by four steps: Lemma 3 reduces the proximal operator to nonnegative sorted inputs, Lemma 4 classifies any minimizer as 2-sparse, 3-sparse, or dense, Lemma 5 shows $\{w:\nabla^2 f(w)\succeq 0\}$ and $\{w:\nabla^2 g(w)\succeq 0\}$ are convex sets, and Corollary 6 rules out spurious local minima. Algorithm 1 then needs only the trivial quadratic case and the two convex programs (13) and (14). The practical solver replaces interior-point methods with projected gradient descent from $w_0=[0,0,0,0]$ at step size $1/4$, relying on Conjecture 9 that the iterates never leave the convex region.

What would settle it

Run the paper's fast solver (projected gradient descent from $w_0=[0,0,0,0]$ with step $1/4$) on a 2:4 proximal instance whose dense minimizer is known from an interior-point solve; if the iterates ever leave $\{w:\nabla^2 f(w)\succeq 0\}$ or converge to a point with strictly larger objective value, Conjecture 9 is false and the fast implementation lacks the guarantee of Theorem 7.

Watch

Extended reading notes

Core claim

For a cell of four weights define $r_{2:4}(w)=|w_1w_2w_3|+|w_2w_3w_4|+|w_3w_4w_1|+|w_4w_1w_2|$, which is zero exactly when at most two of the four consecutive weights are nonzero. The paper studies the proximal operator $\operatorname{prox}_{\lambda r_{2:4}}(z)=\arg\min_w \tfrac12\|w-z\|^2+\lambda r_{2:4}(w)$ and shows that, after sorting and removing signs, any minimizer falls into one of three regimes: the trivial 2-sparse point $[z_1,z_2,0,0]$, a 3-sparse case with $w_4=0$, or a dense positive case. In the latter two regimes the set of points where the Hessian is positive semidefinite is convex, all local minima have the same objective value, and the search reduces to the convex programs (13) and (14). Theorem 7 states that Algorithm 1, which solves the two convex programs, compares them with the trivial quadratic case, and selects the lowest value, returns an optimal solution to problem (9). In the GPU implementation the convex subproblems are solved by projected gradient descent from the origin with step size $1/4$, a step justified only by Conjecture 9, and 1000 masked-gradient steps follow once the mask is frozen.

Load-bearing premise

All experimental results use a fast gradient solver whose convergence is guaranteed only by an unproven conjecture: starting from all zeros and taking steps of one quarter, the solver never leaves the bowl-shaped region around the true solution and always lands at the global minimum.

Editorial extensions

If this is right

  • Exact 2:4 mask selection under local squared loss becomes tractable: per-cell proximal problems decompose, so cost grows linearly with matrix size.
  • Masked gradient updates are a drop-in refinement: applying them after Wanda or SparseGPT pruning lowers perplexity on C4 and improves mean downstream accuracy by roughly three percentage points on Llama-3.1 8B Instruct.
  • On OpenLlama 3B/7B/13B and Llama-3.1 8B, proximal pruning with masked gradient updates gives lower validation perplexity than Wanda, SparseGPT, and DSnoT; at 70B it matches them.
  • The proximal operator can be used not only for one-shot layer pruning but also within finetuning or pretraining via proximal-gradient or straight-through gradient steps.
  • For 1:M sparsity the proximal operator is a closed-form quadratic, while for general N:M with N>2 the three-regime classification survives but the convexity of the Hessian-positive region no longer follows by the same argument.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Conjecture 9 is ever disproved, the exact guarantee is not lost: the interior-point solver for (13) and (14) still gives Theorem 7's optimal solution, so the practical question is only how much speed, not whether an efficient exact method exists.
  • A clean testable extension is to apply the masked-gradient refinement after 2:4 quantization or after combining sparsity with quantization; the paper's memory-footprint analysis shows the relative gain shrinks as position-index overhead grows, so the benefit there may be smaller.
  • The relative contribution of correlation-awareness versus gradual scheduling could be isolated by varying the off-diagonal strength of the Hessian; the toy experiments suggest the proximal method's advantage grows with feature correlation, which is exactly the regime where Wanda's diagonal assumption fails.
  • Using the same regularizer during pretraining as a smooth structural prior is a natural next step that the paper mentions but does not test; it would let masks emerge during training rather than being imposed post hoc.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces a regularizer r_{2:4} whose null space is exactly the set of 2:4-sparse vectors, derives its proximal operator, and proves (Theorem 7) that the nonconvex proximal problem reduces to one trivial quadratic case and two convex programs. The authors then propose a proximal-gradient algorithm for layerwise squared-loss pruning and a masked-gradient post-processing step, and report experiments on toy problems and LLMs up to 70B parameters. The empirical results are mixed: the proximal method gives small perplexity gains on some models/metrics, while the masked-gradient updates provide larger and more consistent gains when applied on top of Wanda or SparseGPT.

Significance. The theoretical reduction of the 2:4 proximal operator to convex subproblems is a valuable and nontrivial contribution; if the practical solver were certified, it would make the proximal operator tractable at LLM scale. The masked-gradient update is a simple, broadly applicable post-pruning heuristic with clear practical potential. However, the paper's two load-bearing theoretical claims are in tension: the practical solver depends on an explicit unproven conjecture, and one of the stated structural results about local minima is false as stated. The paper is honest about the conjecture, which is to its credit, but the empirical claims are conditional on it. The experimental improvements are modest and inconsistent across the reported metrics, so the contribution is better characterized as a partial advance than as a clear state-of-the-art improvement.

major comments (3)
  1. [Section 3.5, Conjecture 9] All experimental results (Tables 1-3, Figures 2 and 4) are produced by the parallelized gradient-descent solver described in Section 3.5, not by the exact interior-point solver that Theorem 7 requires. The paper explicitly states that Conjecture 9 is unproven and that the correctness of the practical algorithm is conditional on it: "If Conjecture 9 is true, then Theorem 7 holds even with this modification." The statement that the GD solver and the interior-point method "always obtain numerically the same solution" is an empirical claim with no quantitative comparison, no error analysis, and no code provided. Because every reported pruning experiment depends on this unproven conjecture, the main practical claim of the paper is not established. The authors should either prove the conjecture, provide a rigorous and reproducible validation of the GD solver against a certified solver on a large set of cells, or clearly present the GD version as a heuristic and temper the claims accordingly.
  2. [Corollary 6 and its proof] Corollary 6 is false as stated. A concrete counterexample is z=(1,1,1,1) and lambda=0.8. The dense point w=(s,s,s,s) with s=( -1 + sqrt(10.6) )/4.8 ≈ 0.470 is a global minimum with objective ≈ 0.894. The 3-sparse point w=(t,t,t,0) with t=( -1 + sqrt(4.2) )/1.6 ≈ 0.656 satisfies the KKT conditions (its derivative with respect to w4 is positive), it is a local minimum with objective ≈ 0.903, and its full Hessian has a negative eigenvalue, so it does not lie in C4. Thus the set of local minima is not convex and local minima can have different objective values, contradicting both parts of the corollary. The proof incorrectly assumes that every local minimum has a zero gradient and lies in C4; boundary local minima need only satisfy second-order conditions on the critical cone, not on the full space. Although the proof of Theorem 7 enumerates the three support cases and does not require Corollary 6, the corollary is stated as a key structural result and must be corrected or removed.
  3. [Table 1 and Abstract claims] The abstract claims "On models up to 13B we improve over previous state of the art algorithms," but the improvements in Table 1 are small and not consistent across metrics and model sizes. For example, on the 13B model WikiText, prox+GD (17.71) is worse than sp.gpt+GD (17.19); on the 8B model WikiText, the difference is only 0.03 (19.83 vs 19.86); and on the 70B models prox+GD matches or slightly underperforms Wanda+GD on some entries. The empirical evidence supports a more modest statement, such as "on several models and metrics we obtain small perplexity improvements, mostly in-distribution." Please revise the abstract and the introduction to match the actual magnitude and consistency of the results.
minor comments (5)
  1. [Section 3.2] The claim that "as lambda goes to infinity, there always exists lambda* such that when lambda >= lambda* the solution becomes exactly N-sparse" is not proven in the text. A short argument (boundedness of the minimizer plus the fact that the regularizer is positive outside the N-sparse set) would make the termination of Algorithm 2 rigorous.
  2. [Section 3.5, Fact 8 and stopping rule] The statement "Using 1/4 as step size, we are guaranteed to never cross a local minimum" is informal, and the stopping rule "stop when witnessing that the gradient norm increases" is not derived from the stated Lipschitz or convexity properties. Please clarify the exact relationship between the stopping rule and the guarantees assumed in Conjecture 9.
  3. [Section 4.3 and Appendix D] The hyperparameter choices for lambda_0 and beta (including the special 70B settings) are reported in one sentence, and the appendix ablation covers only a single layer. Since the method's practical success depends on these choices, a more systematic sensitivity analysis and a statement of the number of proximal iterations used per model would improve reproducibility.
  4. [Section 2.3] There is a typo: "inputs to the liner layer" should read "inputs to the linear layer." Also, the sentence "Recent work is generally designed for pruning to unstructured sparsity and has then been applied to structured patterns" in the introduction reads awkwardly and could be made more precise.
  5. [Appendix B, proof of Corollary 6] The proof of Corollary 6 uses the equality f(u*) >= f(v*) + (u*-v*)^T grad f(v*) = f(v*), which presumes grad f(v*)=0. This is not justified when v* is a boundary local minimum; the gradient component in the active constraint direction can be strictly positive. This is the source of the error identified in the major comments.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the 2:4 proximal-operator derivation is self-contained, and the unproven Conjecture 9 is a correctness gap rather than a circular step.

full rationale

The paper's central theoretical claim is Theorem 7: Algorithm 1 returns an optimal solution to the 2:4 proximal problem by solving one trivial quadratic case and two convex programs. This derivation does not reduce to fitted values, to the same authors' prior results, or to the definition of the regularizer. The regularizer r_{2:4} is defined independently in Equation (6); Lemma 3 reduces the proximal problem to non-negative sorted inputs; Lemma 5 shows the Hessian-PSD regions C3 and C4 are convex; Corollary 6 rules out spurious local minima under the relevant conditions; and Theorem 7 enumerates the three possible cases and shows each is recovered by one of the convex subproblems. No step in this chain is equivalent to its own input by construction. Section 3.5 replaces the exact convex solvers with projected gradient descent and explicitly states Conjecture 9, admitting that it has not been formally proved. That is an unverified assumption about the gradient solver, not a circular self-reference: the claim being assumed is convergence to the minimizer, not the conclusion of the paper. If Conjecture 9 is false, the experimental results may not reflect the true proximal operator, but that is a correctness risk, not circularity. The paper's self-citation of its own survey (Park et al., 2024) is used only as background on inference optimization and speedup sources, not as a load-bearing uniqueness theorem or as justification of the proximal derivation. Empirical comparisons are made against external baselines such as Wanda and SparseGPT. Accordingly, no circular steps are identified, and the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central theoretical contribution, the convex decomposition of the 2:4 proximal operator, is self-contained and does not rely on fitted parameters. However, the practical algorithm rests on an unproven conjecture and several hand-chosen schedule parameters that affect the empirical results.

free parameters (4)
  • lambda_0 = 0.01 (default), 1e-3 (70B)
    Initial regularization strength in the exponential schedule; chosen by hand, affects when weights become sparse.
  • beta = 1.01 (default), 1.005 (70B)
    Growth rate of the lambda schedule; selected per model family based on validation loss and runtime.
  • masked_gradient_steps = 1000
    Number of gradient updates after the mask is fixed; authors observe convergence by this point.
  • prox_gd_stop_rule = stop when gradient norm increases
    Heuristic stopping criterion for the internal gradient descent that solves the proximal operator; not formally derived.
assumptions (4)
  • ad hoc to paper Conjecture 9: gradient descent with step size 1/4 on the dense 2:4 proximal subproblem converges to the global minimizer and stays in the PSD region; same for the 3-sparse case.
    Section 3.5. Load-bearing for the practical solver and for all reported experiments; the paper states it as a conjecture without proof.
  • ad hoc to paper For the regularizer r_{N:M}, there exists a finite lambda* such that the proximal solution becomes exactly N-sparse for lambda >= lambda*.
    Section 3.2 and used in Algorithm 2 to guarantee termination; stated informally with Figure 1, no proof given.
  • domain assumption The local squared loss (2) is a good proxy for final model perplexity after pruning.
    Section 1 and 4; empirically supported by Figure 4 but not proven in general.
  • domain assumption The Hessian estimated from 2M c4 tokens is sufficient for pruning decisions.
    Section 4.3; authors choose this data budget and do not propagate pruning errors (Appendix D.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Proximal Operator for Inducing 2:4-Sparsity." pith.science (2026). https://pith.science/paper/U2GRC4Q3

@misc{pith2026250118015,
  author       = {Pith},
  title        = {Pith review of: A Proximal Operator for Inducing 2:4-Sparsity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2GRC4Q3}},
  note         = {Machine review of arXiv:2501.18015}
}
read the original abstract

Recent hardware advancements in AI Accelerators and GPUs allow to efficiently compute sparse matrix multiplications, especially when 2 out of 4 consecutive weights are set to zero. However, this so-called 2:4 sparsity usually comes at a decreased accuracy of the model. We derive a regularizer that exploits the local correlation of features to find better sparsity masks in trained models. We minimize the regularizer jointly with a local squared loss by deriving the proximal operator for which we show that it has an efficient solution in the 2:4-sparse case. After optimizing the mask, we use maskedgradient updates to further minimize the local squared loss. We illustrate our method on toy problems and apply it to pruning entire large language models up to 70B parameters. On models up to 13B we improve over previous state of the art algorithms, whilst on 70B models we match their performance.

Figures

Figures reproduced from arXiv: 2501.18015 by the authors.

Figure 1
Figure 1. Illustration of the regularization path, e.g., optimal solution (w1, w2, w3, w4) as a function of λ in (7). From left to right, we are showing the result with (a) easy input z = [1.6, 1.1, 0.8, 0.5] (b) nearly-tied z2:4 with z = [1.6, 1.11, 1.1, 1.09] (c) nearly-tied z1:3 with z = [1.6, 1.59, 1.58, 1.09] (d) All nearly tied with z = [1.6, 1.59, 1.58, 1.57]. The two dashed lines in all figures indicates the two criti… view at source ↗
Figure 2
Figure 2. Toy experiment with generated weights and Hessians. Without correlation (α = 1) all considered pruning methods result in the same mask and loss. As we increase the correlations, our proximal approach leads to the best mask as shown by the smallest loss. features more structurally. Therefore we generate synthetic Hessians and random weights: d=1024 diag = torch.diag(torch.rand(d)) Z = torch.randn(d, d) / float(d)**(1… view at source ↗
Figure 3
Figure 3. Ablation for the regularizer schedule Equation (15) on the down projection of layer 18 of Llama-3.1 8B [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Layerwise local squared loss on an example matrix of openLlama 3bv2. We show the loss relative to the loss WandA (with masked gradient updates) incurrs. As intended by design, the proximal approach of pruning leads to smaller local squared loss. In [PITH_FULL_IMAGE:fi…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Achieving Linear Speedup for Composite Federated Learning

    cs.LG 2026-02 conditional novelty 6.0 of 10

    FedNMap provably achieves communication complexity O(1/(nQ ε⁴)) for nonconvex composite federated learning with nonsmooth regularizers, the first such linear-speedup guarantee.

  2. ProxSparse: Regularized Learning of Semi-Structured Sparsity Masks for Pretrained LLMs

    cs.LG 2025-02 conditional novelty 6.0 of 10

    ProxSparse learns 2:4 semi-structured sparsity masks for pretrained LLMs via regularized proximal-gradient optimization, outperforming heuristic baselines on seven models.

Reference graph

Works this paper leans on

33 extracted references · 22 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    GQA : Training generalized multi-query transformer models from multi-head checkpoints

    Ainslie, J., Lee-Thorp, J., de Jong, M., Zemlyanskiy, Y., Lebron, F., and Sanghai, S. GQA : Training generalized multi-query transformer models from multi-head checkpoints. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    work page 2023

  3. [3]

    L., do Nascimento, M

    Ashkboos, S., Croci, M. L., do Nascimento, M. G., Hoefler, T., and Hensman, J. Slice GPT : Compress large language models by deleting rows and columns. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  4. [4]

    First-order methods in optimization, volume 25

    Beck, A. First-order methods in optimization, volume 25. SIAM, 2017

    work page 2017

  5. [5]

    and Vandenberghe, L

    Boyd, S. and Vandenberghe, L. Convex optimization. Cambridge university press, 2004

    work page 2004

  6. [6]

    T., Mittal, S., Emani, M., Vishwanath, V., and Somani, A

    Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., and Somani, A. K. A survey of techniques for optimizing transformer inference. Journal of Systems Architecture, pp.\ 102990, 2023

    2023

  7. [7]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Dao, T. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv:2307.08691, 2023

    arXiv 2023

  8. [8]

    Llm.int8(): 8-bit matrix multiplication for transformers at scale

    Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. Llm.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems, 2022

    work page 2022

Show all 33 references
  1. [9]

    Pruner-zero: Evolving symbolic pruning metric from scratch for large language models

    Dong, P., Li, L., Tang, Z., Liu, X., Pan, X., Wang, Q., and Chu, X. Pruner-zero: Evolving symbolic pruning metric from scratch for large language models. In Proceedings of the 41st International Conference on Machine Learning, 2024

    2024

  2. [10]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv:2407.21783, 2024

    2024 arXiv

  3. [11]

    Maskllm: Learnable semi-structured sparsity for large language models

    Fang, G., Yin, H., Muralidharan, S., Heinrich, G., Pool, J., Kautz, J., Molchanov, P., and Wang, X. Maskllm: Learnable semi-structured sparsity for large language models. NeurIPS, 2024

    2024

  4. [12]

    and Alistarh, D

    Frantar, E. and Alistarh, D. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, 2023

    2023

  5. [13]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. Gptq: Accurate post-training quantization for generative pre-trained transformers. ICLR, 2023

    2023

  6. [14]

    L., Chen, J., Hoefler, T., and Alistarh, D

    Frantar, E., Castro, R. L., Chen, J., Hoefler, T., and Alistarh, D. Marlin: Mixed-precision auto-regressive parallel inference on large language models. arXiv:2408.11743, 2024

    2024 arXiv

  7. [15]

    and Liu, H

    Geng, X. and Liu, H. Openllama: An open reproduction of llama, May 2023. URL https://github.com/openlm-research/open_llama

    2023

  8. [16]

    and Stork, D

    Hassibi, B. and Stork, D. Second order derivatives for network pruning: Optimal brain surgeon. Advances in neural information processing systems, 1992

    1992

  9. [17]

    Accelerated sparse neural training: A provable and efficient method to find n: m transposable masks

    Hubara, I., Chmiel, B., Island, M., Banner, R., Naor, J., and Soudry, D. Accelerated sparse neural training: A provable and efficient method to find n: m transposable masks. Advances in neural information processing systems, 2021

    2021

  10. [18]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018

    2018

  11. [19]

    Optimal brain damage

    LeCun, Y., Denker, J., and Solla, S. Optimal brain damage. Advances in neural information processing systems, 1989

    1989

  12. [20]

    Llm-pruner: On the structural pruning of large language models

    Ma, X., Fang, G., and Wang, X. Llm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 2023

    2023

  13. [21]

    ALPS : Improved optimization for highly sparse one-shot pruning for large language models

    Meng, X., Behdin, K., Wang, H., and Mazumder, R. ALPS : Improved optimization for highly sparse one-shot pruning for large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  14. [22]

    Pointer sentinel mixture models

    Merity, S., Xiong, C., Bradbury, J., and Socher, R. Pointer sentinel mixture models. arXiv:1609.07843, 2016

    2016 arXiv

  15. [23]

    T., Joshi, R., Chochowski, M., Patwary, M., Shoeybi, M., Catanzaro, B., Kautz, J., and Molchanov, P

    Muralidharan, S., Sreenivas, S. T., Joshi, R., Chochowski, M., Patwary, M., Shoeybi, M., Catanzaro, B., Kautz, J., and Molchanov, P. Compact language models via pruning and knowledge distillation. arXiv:2407.14679, 2024

    2024 arXiv

  16. [24]

    M., Huang, J., Kleindessner, M., Huan, J., Cevher, V., Wang, Y., and Karypis, G

    Park, Y., Budhathoki, K., Chen, L., K\" u bler, J. M., Huang, J., Kleindessner, M., Huan, J., Cevher, V., Wang, Y., and Karypis, G. Inference optimization of foundation models on ai accelerators. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data ...

    2024

  17. [25]

    Accelerating inference with sparsity using the nvidia ampere architecture and nvidia tensorrt, 2021

    Pool, J., Sawarkar, A., and Rodge, J. Accelerating inference with sparsity using the nvidia ampere architecture and nvidia tensorrt, 2021

    2021

  18. [26]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 0 (140): 0 1--67, 2020

    2020

  19. [27]

    Sun, M., Liu, Z., Bair, A., and Kolter, J. Z. A simple and effective pruning approach for large language models. ICLR, 2024

    2024

  20. [28]

    Patient knowledge distillation for bert model compression, 2019

    Sun, S., Cheng, Y., Gan, Z., and Liu, J. Patient knowledge distillation for bert model compression, 2019

    2019

  21. [29]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288, 2023

    2023 arXiv

  22. [30]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 2017

    2017

  23. [31]

    Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

    Wei, X., Zhang, Y., Li, Y., Zhang, X., Gong, R., Guo, J., and Liu, X. Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling. EMNLP, 2023

    2023

  24. [32]

    Sheared LL a MA : Accelerating language model pre-training via structured pruning

    Xia, M., Gao, T., Zeng, Z., and Chen, D. Sheared LL a MA : Accelerating language model pre-training via structured pruning. In The Twelfth International Conference on Learning Representations, 2024

    2024

  25. [33]

    Dynamic sparse no training: Training-free fine-tuning for sparse LLM s

    Zhang, Y., Zhao, L., Lin, M., Yunyun, S., Yao, Y., Han, X., Tanner, J., Liu, S., and Ji, R. Dynamic sparse no training: Training-free fine-tuning for sparse LLM s. In The Twelfth International Conference on Learning Representations, 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.