Pith. sign in

REVIEW 3 major objections 4 minor 63 references

TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TSENOR recasts transposable N:M mask search as an optimal transport problem, yielding a solver that scales to billion-parameter language models and accelerates both forward and backward passes.

desk verdict TSENOR is a real algorithmic contribution for transposable N:M sparsity—arbitrary M and GPU-friendly—but the central empirical claims need code, readable figures, and one pseudocode fix before I'd trust the 1–10% error numbers. read the letter →

arxiv 2505.23949 v1 pith:W4THONZJ submitted 2025-05-29 cs.LG cs.AI

Xiang Meng , Mehdi Makni , Rahul Mazumder This is my paper
classification cs.LGcs.AI
keywords transposableN:MsparsityoptimaltransportentropyregularizationDykstraalgorithmLLMpruningGPUparallelizationstructuredlayer-wise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TSENOR is a solver for transposable N:M sparse masks—binary masks that keep N of every M consecutive weights in both a weight matrix and its transpose, so both forward and backward passes can be accelerated. The paper claims that mask generation can be recast as a capacitated optimal transport problem on each $M\times M$ block, solved quickly with entropy regularization and Dykstra's algorithm followed by a greedy-and-local-search rounding, all implemented with batched tensor operations on GPUs. This is claimed to scale to billion-parameter models, to run up to about two orders of magnitude faster than prior methods, and to produce masks within 1–10% of optimal under the magnitude-preservation objective. If correct, it removes the algorithmic barrier that previously limited transposable N:M sparsity to small patterns like 2:4, and makes arbitrary patterns such as 16:32 practical for large language models.

What carries the argument

The central object is the transposable N:M constraint viewed blockwise: each $M\times M$ submatrix must be a 0–1 matrix with exactly $N$ ones in every row and every column. The paper's key move is to identify the LP relaxation of this block problem with a capacitated optimal transport problem and to solve it by entropy regularization (adding $\tfrac{1}{\tau}H(S)$) using Dykstra's algorithm, whose three projections are simple row-wise normalization, column-wise normalization, and clipping entries at 1. A tensor implementation processes all blocks simultaneously in log space; the fractional output is converted to a binary mask by sorting entries descending and greedily allocating ones while row and column budgets remain, then by a local search that swaps in two off-diagonal entries and removes a conflicting one when the swap score $(|W_{i,j'}| + |W_{i',j}| - |W_{i',j'}|)$ is positive. This machinery is what lets the method handle arbitrary N:M and remain fully GPU-vectorized.

What would settle it

Solve a large set of $M\times M$ blocks (for instance 16:32 patterns extracted from a real LLM) exactly using a network-flow or MILP solver and compare objectives; if TSENOR's average relative error exceeds the claimed 1–10% range, or if there exist blocks where greedy selection saturates rows before all high-magnitude weights are placed and the local search cannot recover, the central claim is falsified.

Watch

Extended reading notes

Core claim

For each $M\times M$ block the paper solves $\max_S \langle S, |W|\rangle$ subject to $S$ being a binary matrix with row and column sums equal to $N$. By invoking the bipartite matching polytope, the binary constraint can be relaxed to $0\le S\le 1$, and the relaxation is exactly a capacitated optimal transport problem: each row ships $N$ units, each column receives $N$ units, and the unit costs are the absolute weights. The paper's discovery is that this formulation, regularized by the entropy term $\tfrac{1}{\tau}H(S)$, can be solved in parallel for millions of blocks by Dykstra's alternating KL projections—row scaling, column scaling, and clipping at 1—and that a carefully vectorized rounding stage (greedy assignment by descending value, then local swap repair) recovers high-quality binary masks. The resulting TSENOR solver plugs into layer-wise pruning routines to produce transposable N:M sparse LLaMA models, and the paper reports that transposable 16:32 masks nearly match their standard N:M counterparts while comfortably beating standard 2:4 models.

Load-bearing premise

The load-bearing premise is that the entropy-regularized fractional solution plus the greedy-and-local-search rounding returns a near-optimal binary mask for every N:M pattern; this is demonstrated empirically on 100 sampled blocks, not proven, and it is the step the speed and quality claims depend on.

Editorial extensions

If this is right

  • Transposable N:M sparsity with large M (for example 16:32) becomes computationally practical, so model compression no longer has to settle for M=4, where transposable constraints hurt accuracy most.
  • Because transposable masks survive transposition, both the forward pass $Y=WX$ and the backward pass $W^\top dL/dY$ can use the same sparse structure, extending N:M acceleration from inference into training and fine-tuning.
  • The solver can be dropped into existing layer-wise pruning pipelines, converting standard N:M pruning into transposable N:M pruning at the cost of a small perplexity gap that shrinks as M grows.
  • For the ALPS-style layerwise reconstruction framework, the paper's modified ADMM updates retain a convergence guarantee whenever each mask update does not decrease the current subproblem objective.
  • Models pruned to transposable 16:32 can outperform models pruned to standard 2:4, indicating that the extra flexibility of larger M more than compensates for the transposable constraint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same optimal-transport view applies to any constraint that decomposes into equal row and column budgets per block, so the method might generalize to other doubly-stochastic-like mask families.
  • Beyond the paper: since the mask solver is cheap, one could use it inside training loops rather than only one-shot pruning, making transposable N:M sparsity a candidate for efficient pretraining or continual learning.
  • Beyond the paper: the quality metric used is absolute-weight preservation; a testable extension is to feed the same fast masks into reconstruction-error objectives to see whether the 1–10% mask error translates into even smaller end-model gaps.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces TSENOR, a GPU-vectorized solver for transposable N:M sparse mask generation in large language models. The mask-selection problem is formulated as a capacitated optimal transport problem per M×M block, solved approximately by entropy regularization and Dykstra's algorithm, followed by a greedy selection and local-search rounding procedure. The solver is integrated into Wanda, SparseGPT, and ALPS, with a convergence theorem for the ALPS variant under an explicit monotonicity assumption. Experiments report relative mask-quality error on 100 sampled blocks, runtime speedups on GPU, and perplexity/zero-shot results on LLaMA3.2-1B/3B/8B for transposable N:M patterns with M up to 32.

Significance. If the empirical quality and scaling claims hold, TSENOR addresses a genuine gap: arbitrary transposable N:M masks are currently either too slow to compute or restricted to M=4, and the paper's 16:32 transposable results being close to standard 16:32 and better than standard 2:4 would be practically valuable for training acceleration. The LP/optimal-transport connection is clean, the LP relaxation is exact by total unimodularity, and the paper is honest about the assumption needed for the ALPS convergence proof. However, the central '1–10% error' claim currently rests on a small, hard-to-read benchmark with no tabulated numbers or code release, so the significance is conditional on the empirical claims being made reproducible.

major comments (3)
  1. [§5.1, Fig. 3] The central quality claim ('1–10% relative error') is not verifiable from the manuscript as written. Figure 3 reports results on only 100 M×M blocks, does not specify which LLaMA3 checkpoint or layers the blocks were sampled from, and is rendered at a scale where the numeric labels are not readable. The values extracted from the text are ambiguous and range from 0.0087% to 43% across different groupings; I could not reconcile this with the '1–10%' claim. Please provide a tabulated version of the relative-error results (with standard deviations across blocks), a precise description of the benchmark (model, layers, block selection, and how the optimal baseline was computed), and either code or a clear protocol for reproducing Figure 3. Since this figure is the only support for the near-optimality of the rounding procedure at large M, the empirical claim is under-supported as presented.
  2. [§4, Appendix A.1.3, Theorem 1] Theorem 1's Assumption 1 requires that the mask returned by TSENOR does not decrease the objective in (10) relative to the previous mask S^(t). Algorithm 2 is a greedy/local-search heuristic with no monotonicity guarantee relative to a previous mask, and the main text does not state that the ALPS D-update compares the new mask against S^(t) and reverts if the objective decreases. The safeguard is mentioned only in the proof appendix. Please make this safeguard an explicit step in the D-update and in any released implementation, and report how often it triggers on the LLaMA experiments; otherwise the convergence result is conditional on a property that is not shown to hold by the described algorithm.
  3. [§3.3, Appendix B.1] The rounding stage has no formal suboptimality bound, and its near-optimality at large M (e.g., 16:32) is asserted only from the 100-block experiment. Because the same rounding is applied inside every ALPS D-update, a single pathological block could in principle degrade downstream results without being visible in a block average. I am not requesting a full approximation guarantee, but the paper should at least report per-pattern worst-case relative error over the 100 blocks (in addition to the mean), and give a sensitivity analysis for the three free parameters that control the quality/speed trade-off: τ = 0.005·max|W|, T = 300 Dykstra iterations, and L = 10 local-search steps.
minor comments (4)
  1. [Abstract, §5.1, Table 3] The speedup claims are stated inconsistently: the abstract says 'up to 100x', Section 5.1 says 'up to 300x' against optimal-solution methods, and Appendix B.2.2/Table 3 reports up to 3000x acceleration. Please harmonize these numbers by specifying the baseline and hardware for each claim.
  2. [Algorithm 1] Line 6 reuses S^(t) to denote the value after the C1/C2 projections; this is easy to misread as the input S^(t) to the iteration. Introducing an intermediate variable (e.g., S̃) would clarify the update and make the equivalence with the log-space code in §A.2 immediate.
  3. [Tables 5–7] The LLM tables report a single run without variance or seed information. If the zero-shot evaluations are deterministic in your setup, please state so explicitly; otherwise provide error bars or multiple-seed statistics.
  4. [§B.1] No code release is mentioned. A public implementation would materially aid reproducibility of both the mask-quality numbers and the GPU runtime claims.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the mask solver's quality is benchmarked against external exact solvers, and the ALPS convergence proof's self-cited portion is supplemented with a separately proven inequality.

full rationale

The central claim—that TSENOR produces high-quality transposable N:M masks—is evaluated against external optimal baselines: network flow and cuPDLP are described as guaranteeing optimal solutions, and the reported relative errors are computed against those exact solutions on sampled LLaMA weight blocks. This makes the 1–10% error claim empirically grounded rather than self-referential. The entropy-regularized optimal transport formulation is derived from standard OT theory (Cuturi 2013, Benamou et al. 2015), and the rounding procedure is a heuristic whose quality is measured relative to the exact objective, not defined in terms of the final model perplexity. The integration with ALPS does cite the authors' prior theorem, and the proof of Theorem 1 states that 'the majority of this proof follows from the convergence analysis in [Meng et al., 2024a, Theorem 1]'. However, the paper isolates the one inequality that is specific to the new D-update—inequality (32)—and proves it in the appendix using Assumption 1. That assumption is explicitly disclosed, and the paper notes it can be enforced by defaulting to the previous mask whenever the new mask would decrease the objective. This is a legitimate extension of a prior result rather than a circular reduction: the convergence guarantee does not assume the conclusion of the theorem, and the solver's suboptimality is handled by a monotonicity safeguard rather than by assuming optimality. No equation is defined in terms of the reported prediction, and no fitted parameter is renamed as a prediction. The paper's own acknowledgment that the rounding has no theoretical suboptimality bound is a limitation, but not a circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The main free parameters are the entropy regularization strength, the iteration count, and the local search steps. The axioms are standard matching polytope theory, the block partition assumption, the magnitude-based pruning proxy, and an ad hoc monotonicity assumption needed for the ALPS convergence guarantee. No new physical or conceptual entities are introduced.

free parameters (4)
  • entropy regularization parameter tau = 0.005 * max|W|
    Chosen by hand to balance solution quality and convergence; no sensitivity analysis across models is provided beyond the default value.
  • maximum Dykstra iterations T = 300
    Fixed iteration count for Algorithm 1; no convergence criterion or adaptive stopping is described.
  • local search steps L = 10
    Number of swap iterations in Algorithm 2; the effect of this parameter on mask quality is not fully explored in the main text.
  • ADMM penalty parameter rho_t = increasing geometric sequence (unspecified)
    ALPS integration uses an adaptive penalty parameter; the specific schedule is left to the ALPS defaults and not stated here.
assumptions (4)
  • standard math The bipartite matching polytope relaxation of problem (2) is exact, so the LP relaxation has an integral optimal solution.
    Invoked in Section 3.1 (bipartite matching polytope theory, Schrijver et al. 2003). Standard result for assignment and transportation polytopes.
  • domain assumption The weight matrix can be partitioned into MxM blocks with no remainder or with padding that preserves the N:M constraint.
    The method operates blockwise; the paper does not specify how non-multiple dimensions are handled.
  • domain assumption Preserving the largest magnitude entries (objective (1)) is a valid proxy for final model quality after pruning.
    This is the standard magnitude-based pruning assumption, used in Wanda and in the paper's mask objective.
  • ad hoc to paper Assumption 1 of Theorem 1: the mask solver does not decrease the ADMM subproblem objective (10) compared to the previous mask.
    Required for the convergence proof; the paper says it can be enforced by comparing objectives and defaulting to the previous mask, and that this safeguard never triggers empirically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks." pith.science (2026). https://pith.science/paper/W4THONZJ

@misc{pith2026250523949,
  author       = {Pith},
  title        = {Pith review of: TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W4THONZJ}},
  note         = {Machine review of arXiv:2505.23949}
}
read the original abstract

Network pruning reduces the computational requirements of large neural networks, with N:M sparsity -- retaining only N out of every M consecutive weights -- offering a compelling balance between compressed model quality and hardware acceleration. However, N:M sparsity only accelerates forward-pass computations, as N:M patterns are not preserved during matrix transposition, limiting efficiency during training where both passes are computationally intensive. While transposable N:M sparsity has been proposed to address this limitation, existing methods for finding transposable N:M sparse masks either fail to scale to large models or are restricted to M=4 which results in suboptimal compression-accuracy trade-off. We introduce an efficient solver for transposable N:M masks that scales to billion-parameter models. We formulate mask generation as optimal transport problems and solve through entropy regularization and Dykstra's algorithm, followed by a rounding procedure. Our tensor-based implementation exploits GPU parallelism, achieving up to 100x speedup with only 1-10% error compared to existing methods. Our approach can be integrated with layer-wise N:M pruning frameworks including Wanda, SparseGPT and ALPS to produce transposable N:M sparse models with arbitrary N:M values. Experiments show that LLaMA3.2-8B with transposable 16:32 sparsity maintains performance close to its standard N:M counterpart and outperforms standard 2:4 sparse model, showing the practical value of our approach.

Figures

Figures reproduced from arXiv: 2505.23949 by the authors.

Figure 1
Figure 1. Efficient GPU-accelerated pipeline for transposable N:M masks generation. The weight [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our proposed rounding procedure for generating a binary mask with transpos [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Solution quality comparison for transposable N:M mask generation. For various N:M [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Upper: perplexity of LLaMA3.2- 8B pruned by ALPS with TSENOR, com￾paring standard and transposable N:M spar￾sity across different N:M patterns. (2) Lower: Computational speedup for for￾ward and backward matrix operations in LLaMA3.2-8B, comparing (transposable) N:M spa…
Figure 5
Figure 5. Figure 5: Perplexity comparison of LLaMA3.2-1B under two pruning ap￾proaches: (1) TSENOR+ALPS followed by fine-tuning, and (2) standard N:M pruning followed by retraining via Bi-NM. We fine-tune transposable N:M sparse models pruned with our approach (TSENOR+ALPS) on C4 and comp…
Figure 6
Figure 6. Figure 6: Solution quality comparison for transposable N:M mask generation. For various N:M [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 30 canonical work pages

  1. [1]

    URL https://huggingface.co/docs/transformers/perplexity

    Perplexity of fixed-length models, 2022. URL https://huggingface.co/docs/transformers/perplexity

    work page 2022

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Progressive gradient flow for robust n: M sparsity training in transformers

    Abhimanyu Rajeshkumar Bambhaniya, Amir Yazdanbakhsh, Suvinay Subramanian, Sheng-Chun Kao, Shivani Agrawal, Utku Evci, and Tushar Krishna. Progressive gradient flow for robust n: M sparsity training in transformers. arXiv preprint arXiv:2402.04744, 2024

    arXiv 2024

  4. [4]

    Careful selection of knowledge to solve open book question answering

    Pratyay Banerjee, Kuntal Kumar Pal, Arindam Mitra, and Chitta Baral. Careful selection of knowledge to solve open book question answering. arXiv preprint arXiv:1907.10738, 2019

    arXiv 1907

  5. [5]

    Iterative bregman projections for regularized transportation problems

    Jean-David Benamou, Guillaume Carlier, Marco Cuturi, Luca Nenna, and Gabriel Peyr \'e . Iterative bregman projections for regularized transportation problems. SIAM Journal on Scientific Computing, 37 0 (2): 0 A1111--A1138, 2015

    work page 2015

  6. [6]

    Fast as CHITA: Neural Network Pruning with Combinatorial Optimization

    Riade Benbaki, Wenyu Chen, Xiang Meng, Hussein Hazimeh, Natalia Ponomareva, Zhe Zhao, and Rahul Mazumder. Fast as chita: Neural network pruning with combinatorial optimization. arXiv preprint arXiv:2302.14623, 2023

    work page Pith review arXiv 2023

  7. [7]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439, 2020

    2020

  8. [8]

    Distributed optimization and statistical learning via the alternating direction method of multipliers

    Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning , 3 0 (1): 0 1--122, 2011

    2011

Show all 63 references
  1. [9]

    The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming

    Lev M Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics, 7 0 (3): 0 200--217, 1967

    1967

  2. [10]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  3. [11]

    Venom: A vectorized n: M format for unleashing the power of sparse tensor cores

    Roberto L Castro, Andrei Ivanov, Diego Andrade, Tal Ben-Nun, Basilio B Fraguela, and Torsten Hoefler. Venom: A vectorized n: M format for unleashing the power of sparse tensor cores. In Proceedings of the International Conference for High Performance Computing, Networking, Sto...

    2023

  4. [12]

    End-to-end autonomous driving: Challenges and frontiers

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  5. [13]

    A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  6. [14]

    Boolq: Exploring the surprising difficulty of natural yes/no questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

    1905 arXiv

  7. [15]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  8. [16]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013

    2013

  9. [17]

    Spqr: A sparse-quantized representation for near-lossless llm weight compression

    Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. Spqr: A sparse-quantized representation for near-lossless llm weight compression. arXiv preprint arXiv:2306.03078, 2023

    2023 arXiv

  10. [18]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  11. [19]

    An algorithm for restricted least squares regression

    Richard L Dykstra. An algorithm for restricted least squares regression. Journal of the American Statistical Association, 78 0 (384): 0 837--842, 1983

    1983

  12. [20]

    Efficient n: M sparse dnn training using algorithm, architecture, and dataflow co-design

    Chao Fang, Wei Sun, Aojun Zhou, and Zhongfeng Wang. Efficient n: M sparse dnn training using algorithm, architecture, and dataflow co-design. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 43 0 (2): 0 506--519, 2023

    2023

  13. [21]

    Sparsegpt: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, pages 10323--10337. PMLR, 2023

    2023

  14. [22]

    A framework for few-shot language model evaluation, 12 2023

    L Gao, J Tow, B Abbasi, S Biderman, S Black, A DiPofi, C Foster, L Golding, J Hsu, A Le Noac’h, et al. A framework for few-shot language model evaluation, 12 2023. URL https://zenodo. org/records/10256836, 7

    2023

  15. [23]

    Knowledge distillation: A survey

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision, 129 0 (6): 0 1789--1819, 2021

    2021

  16. [24]

    Gurobi Optimizer Reference Manual , 2022

    Gurobi . Gurobi Optimizer Reference Manual , 2022. URL https://www.gurobi.com

    2022

  17. [25]

    Learning both weights and connections for efficient neural network

    Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015

    2015

  18. [26]

    Second order derivatives for network pruning: Optimal brain surgeon

    Babak Hassibi and David Stork. Second order derivatives for network pruning: Optimal brain surgeon. Advances in neural information processing systems, 5, 1992

    1992

  19. [27]

    Y. He, X. Zhang, and J. Sun. Channel pruning for accelerating very deep neural networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 1398--1406, Los Alamitos, CA, USA, oct 2017. IEEE Computer Society. doi:10.1109/ICCV.2017.155. URL https://doi.ieeeco...

    2017 doi

  20. [28]

    Accelerating transformer pre-training with 2: 4 sparsity

    Yuezhou Hu, Kang Zhao, Weiyu Huang, Jianfei Chen, and Jun Zhu. Accelerating transformer pre-training with 2: 4 sparsity. arXiv preprint arXiv:2404.01847, 2024

    2024 arXiv

  21. [29]

    Elsa: Exploiting layer-wise n: m sparsity for vision transformer acceleration

    Ning-Chi Huang, Chi-Chih Chang, Wei-Cheng Lin, Endri Taka, Diana Marculescu, and Kai-Chiang Wu. Elsa: Exploiting layer-wise n: m sparsity for vision transformer acceleration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8006--8015, 2024

    2024

  22. [30]

    Accelerated sparse neural training: A provable and efficient method to find n: m transposable masks

    Itay Hubara, Brian Chmiel, Moshe Island, Ron Banner, Joseph Naor, and Daniel Soudry. Accelerated sparse neural training: A provable and efficient method to find n: m transposable masks. Advances in neural information processing systems, 34: 0 21099--21111, 2021 a

    2021

  23. [31]

    Accurate post training quantization with small calibration sets

    Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets. In International Conference on Machine Learning, pages 4466--4475. PMLR, 2021 b

    2021

  24. [32]

    Efficient gpu kernels for n: M-sparse weights in deep learning

    Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, et al. Efficient gpu kernels for n: M-sparse weights in deep learning. Proceedings of Machine Learning and Systems, 5: 0 513--525, 2023 a

    2023

  25. [33]

    Awq: Activation-aware weight quantization for llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv preprint arXiv:2306.00978, 2023 b

    2023 arXiv

  26. [34]

    Tb-stc: Transposable block-wise n: M structured sparse tensor core

    Jun Liu, Shulin Zeng, Junbo Zhao, Li Ding, Zeyu Wang, Jinhao Li, Zhenhua Zhu, Xuefei Ning, Chen Zhang, Yu Wang, et al. Tb-stc: Transposable block-wise n: M structured sparse tensor core. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA), pag...

    2025

  27. [35]

    Haihao Lu and Jinwen Yang. cupdlp. jl: A gpu implementation of restarted primal-dual hybrid gradient for linear programming in julia. arXiv preprint arXiv:2311.12180, 2023

    2023 arXiv

  28. [36]

    Step: learning n: M structured sparsity masks from scratch with precondition

    Yucheng Lu, Shivani Agrawal, Suvinay Subramanian, Oleg Rybakov, Christopher De Sa, and Amir Yazdanbakhsh. Step: learning n: M structured sparsity masks from scratch with precondition. In International Conference on Machine Learning, pages 22812--22824. PMLR, 2023

    2023

  29. [37]

    Nm-spmm: Accelerating matrix multiplication using n: M sparsity with gpgpu

    Cong Ma, Du Wu, Zhelang Deng, Jiang Chen, Xiaowen Huang, Jintao Meng, Wenxi Zhu, Bingqiang Wang, Amelie Chi Zhou, Peng Chen, et al. Nm-spmm: Accelerating matrix multiplication using n: M sparsity with gpgpu. arXiv preprint arXiv:2503.01253, 2025

    2025 arXiv

  30. [38]

    Determining optimal channel partition for 2: 4 fine grained structured sparsity

    Mohit Mahajan, Wen-Mei Hwu, and Rakesh Nagi. Determining optimal channel partition for 2: 4 fine grained structured sparsity. Optimization Letters, 18 0 (9): 0 2079--2090, 2024

    2024

  31. [39]

    A fast and accurate splitting method for optimal transport: Analysis and implementation

    Vien V Mai, Jacob Lindb \"a ck, and Mikael Johansson. A fast and accurate splitting method for optimal transport: Analysis and implementation. arXiv preprint arXiv:2110.11738, 2021

    2021 arXiv

  32. [40]

    The penn treebank: Annotating predicate argument structure

    Mitchell Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. The penn treebank: Annotating predicate argument structure. In Proceedings of the Workshop on Human Language Technology, HLT '94, page 114–119, US...

    1994

  33. [41]

    Alps: Improved optimization for highly sparse one-shot pruning for large language models

    Xiang Meng, Kayhan Behdin, Haoyue Wang, and Rahul Mazumder. Alps: Improved optimization for highly sparse one-shot pruning for large language models. arXiv preprint arXiv:2406.07831, 2024 a

    2024 arXiv

  34. [42]

    Osscar: One-shot structured pruning in vision and language models with combinatorial optimization

    Xiang Meng, Shibal Ibrahim, Kayhan Behdin, Hussein Hazimeh, Natalia Ponomareva, and Rahul Mazumder. Osscar: One-shot structured pruning in vision and language models with combinatorial optimization. arXiv preprint arXiv:2403.12983, 2024 b

    2024 arXiv

  35. [43]

    Pointer sentinel mixture models

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Byj72udxe

    2017

  36. [44]

    Accelerating sparse deep neural networks

    Asit Mishra, Jorge Albericio Latorre, Jeff Pool, Darko Stosic, Dusan Stosic, Ganesh Venkatesh, Chong Yu, and Paulius Micikevicius. Accelerating sparse deep neural networks. arXiv preprint arXiv:2104.08378, 2021

    2021 arXiv

  37. [45]

    Nvidia a100 tensor core gpu architecture

    Nvidia . Nvidia a100 tensor core gpu architecture. , 2020. URL https://www.nvidia.com/content/dam/enzz/ Solutions/Data-Center/nvidia-ampere-architecture-whitepaper.pdf

    2020

  38. [46]

    Accelerating condensed interior-point methods on simd/gpu architectures

    Fran c ois Pacaud, Sungho Shin, Michel Schanen, Daniel Adrian Maldonado, and Mihai Anitescu. Accelerating condensed interior-point methods on simd/gpu architectures. Journal of Optimization Theory and Applications, 202 0 (1): 0 184--203, 2024

    2024

  39. [47]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017

    2017

  40. [48]

    A survey on recognizing textual entailment as an nlp evaluation

    Adam Poliak. A survey on recognizing textual entailment as an nlp evaluation. arXiv preprint arXiv:2010.03061, 2020

    2010 arXiv

  41. [49]

    Channel permutations for n: m sparsity

    Jeff Pool and Chong Yu. Channel permutations for n: m sparsity. Advances in neural information processing systems, 34: 0 13316--13327, 2021

    2021

  42. [50]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492--28518. PMLR, 2023

    2023

  43. [51]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21 0 (1), jan 2020. ISSN 1532-4435

    2020

  44. [52]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64 0 (9): 0 99--106, 2021

    2021

  45. [53]

    Combinatorial optimization: polyhedra and efficiency, volume 24

    Alexander Schrijver et al. Combinatorial optimization: polyhedra and efficiency, volume 24. Springer, 2003

    2003

  46. [54]

    A simple and effective pruning approach for large language models

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695, 2023

    2023 arXiv

  47. [55]

    Dominosearch: Find layer-wise fine-grained n: M sparse schemes from dense neural networks

    Wei Sun, Aojun Zhou, Sander Stuijk, Rob Wijnhoven, Andrew O Nelson, Henk Corporaal, et al. Dominosearch: Find layer-wise fine-grained n: M sparse schemes from dense neural networks. Advances in neural information processing systems, 34: 0 20721--20732, 2021

    2021

  48. [56]

    Manifold regularized dynamic network pruning

    Yehui Tang, Yunhe Wang, Yixing Xu, Yiping Deng, Chao Xu, Dacheng Tao, and Chang Xu. Manifold regularized dynamic network pruning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5018--5028, 2021

    2021

  49. [57]

    Optimal transport: old and new, volume 338

    C \'e dric Villani et al. Optimal transport: old and new, volume 338. Springer, 2008

    2008

  50. [58]

    Learning structured sparsity in deep neural networks

    Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. Advances in neural information processing systems, 29, 2016

    2016

  51. [59]

    Sustainable ai: Environmental implications, challenges and opportunities

    Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4: 0 795--...

    2022

  52. [60]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  53. [61]

    Bi-directional masks for efficient n: M sparse training

    Yuxin Zhang, Yiting Luo, Mingbao Lin, Yunshan Zhong, Jingjing Xie, Fei Chao, and Rongrong Ji. Bi-directional masks for efficient n: M sparse training. In International conference on machine learning, pages 41488--41497. PMLR, 2023

    2023

  54. [62]

    Learning n: m fine-grained structured sparse neural networks from scratch

    Aojun Zhou, Yukun Ma, Junnan Zhu, Jianbo Liu, Zhijie Zhang, Kun Yuan, Wenxiu Sun, and Hongsheng Li. Learning n: m fine-grained structured sparse neural networks from scratch. arXiv preprint arXiv:2102.04010, 2021

    2021 arXiv

  55. [63]

    Sparse tensor core: Algorithm and hardware co-design for vector-wise sparse neural networks on modern gpus

    Maohua Zhu, Tao Zhang, Zhenyu Gu, and Yuan Xie. Sparse tensor core: Algorithm and hardware co-design for vector-wise sparse neural networks on modern gpus. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, pages 359--371, 2019

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.