Pith. sign in

REVIEW 5 major objections 5 minor 39 references

Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Celty co-designs a run-length compressed sparse format, a GPU kernel, and a small SIMT extension to make dual-sparse LLM decoding fast.

desk verdict The kernel contribution is real and reproducible; the headline hardware speedup rests on a proxy that can't model the proposed microarchitecture, and the abstract overstates accuracy. read the letter →

arxiv 2608.01536 v1 pith:HP6MA6UM submitted 2026-08-02 cs.AR cs.LG

Ruokai Yin , Priyadarshini Panda This is my paper
classification cs.ARcs.LG
keywords dualsparsityspMspVGPUkernelRLC-CSCSIMTmicroarchitectureLLMinferenceunstructuredpruningactivation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that dual-sparse LLM decoding—weight matrices pruned offline plus input activations thresholded at runtime—does not have to be slow on GPUs. Its central move is to treat the workload as sparse-matrix-sparse-vector multiplication and to design a storage format, kernel, and tiny hardware change together. The Run-Length Compressed CSC (RLC-CSC) format keeps every non-zero weight paired with its position, so a warp can load compressed columns with fixed-width vectorized loads and skip whole columns whose activations are zero. The paper reports kernel speedups up to 2.8x over cuBLAS and up to 5.3x when the proposed Sparse SIMT Core is added, with end-to-end single-token decode on LLaMA-2-13B up to 2.63x faster at 60% dual-sparsity.

What carries the argument

Run-Length Compressed CSC (RLC-CSC): a sparse column format that stores each non-zero value with a short run-length code giving the number of zeros before it, preserving a one-to-one index-value mapping. It carries the kernel by enabling vectorized compressed column loads and column skipping, and it feeds directly into the proposed Sparse SIMT Core, whose 3-stage pipelined RLC decoder reconstructs absolute row indices in hardware while repurposed local register files act as conflict-free accumulation buffers.

What would settle it

Implement or cycle-accurately simulate the Sparse SIMT Core and measure kernel latency on the 5120x13824 layer at 70% dual-sparsity. If the measured latency with the hardware RLC decoder and register-file accumulation is materially higher than the modified-kernel proxy—for example because concurrent warps collide on register-file ports or the decoder pipeline cannot sustain 8 indices per cycle—the 5.3x over cuBLAS claim would not survive.

Watch

Extended reading notes

Core claim

The paper claims that the spMspV operation arising in dual-sparse single-user LLM decoding can be made efficient by co-designing all three levels: a run-length compressed CSC format, a SIMT kernel, and a microarchitectural extension. The format solves the value-index decoupling problem of bitmap formats: because each non-zero value carries its own compressed position, threads can issue fixed-width vectorized loads without diverging. The kernel exploits both sparsity sources by having each warp inspect one activation and skip the entire weight column when the activation is zero, scattering partial products into shared memory. The Sparse SIMT Core then removes the two remaining software costs—

Load-bearing premise

The headline hardware speedups are estimates, not direct measurements: the authors ran a modified kernel on a real GPU with software index reconstruction and shared-memory scatter writes removed, and assumed the proposed hardware decoder and shared register storage would add no extra stalls or conflicts. If that assumption fails, the microarchitecture-level speedups would overstate what a real chip delivers.

Editorial extensions

If this is right

  • Training-free dual-sparsity becomes a credible latency lever for single-user decoding, not just a compression trick.
  • The kernel-only result already beats dense cuBLAS and sparse baselines from roughly 50% dual-sparsity onward, so software deployments can benefit without hardware changes.
  • On LLaMA-2-13B, end-to-end decode latency drops up to 2.63x at 60% dual-sparsity with WikiText-2 perplexity rising from 4.9 to 7.2.
  • The format's one-to-one index-value pairing removes the need for bitmap index reconstruction, which can inform future sparse GPU kernel designs beyond LLMs.
  • A lightweight SIMT extension (by the paper's estimate under 0.01% additional area) could be added to GPUs to close the remaining reconstruction and bank-conflict overhead.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The hardware-level speedups are estimates, not measurements: they come from running a modified kernel on a real GPU with software reconstruction and shared-memory scatter removed. A faithful RTL/FPGA implementation could reveal extra stalls from shared register-file ports or decoder pipeline bubbles.
  • The same RLC decoder and register-accumulation idea could apply to other moderate-sparsity irregular workloads, such as graph neural network message passing or recommendation ranking, where the sparse vector changes per query.
  • The register-accumulation ratio is a tunable knob between speedup and occupancy; one could imagine an adaptive scheduler that sets it per layer based on measured conflict pressure.
  • The column-skip mechanism depends on activation sparsity surviving thresholding at scale: a testable extension is to measure how Celty's speedup changes when activation sparsity is induced by different thresholding or SVD-based methods.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. Celty targets dual-sparse LLM inference (unstructured pruned weights plus runtime activation sparsity) by formulating the decode-time linear layer as a Sparse Matrix-Sparse Vector (spMspV) operation. The paper proposes a Run-Length Compressed CSC (RLC-CSC) format and a GPU kernel that vectorizes the load of compressed weight columns, skips columns whose activation is zero, and accumulates partial products in shared memory. It then adds a co-designed Sparse SIMT Core with a hardware RLC decoder and register-file-based accumulation. Kernel-only speedups of up to 2.8x over cuBLAS and 2.4x over Flash-LLM are measured on an A5000; hardware-accelerated speedups of up to 5.3x over cuBLAS are estimated by running a modified kernel on the same GPU. End-to-end LLaMA-2-13B decode speedups up to 2.63x are reported, with perplexity increasing from 4.9 (dense) to 7.2 at 60% dual-sparsity.

Significance. If validated, the kernel contribution is a useful step for single-user dual-sparse LLM decoding: the measurements are on real hardware, the code is released, and the comparison includes several relevant baselines. The proposed hardware extension is lightweight and avoids tensor-core overhead, which is attractive for memory-bound N=1 workloads. However, the central co-design claim depends on a software proxy for hardware that does not exist, and the current evidence does not support the 5.3x microarchitecture headline. The significance of the paper therefore rests on the kernel results and on future validation of the hardware estimates rather than on the evidence presented here.

major comments (5)
  1. [§5, 'Hardware simulation and synthesis'] The headline Sparse SIMT Core speedups (abstract; §6.1, Fig. 6) are not measured on the proposed hardware. The modified-kernel proxy removes RLC reconstruction instructions and SMEM scatter, but on an A5000 a CUDA thread cannot write to another thread's register file (Section 4.1, Fig. 7), so the 'direct register-file access' dataflow is not the one proposed. The proxy also cannot model decoder pipeline stalls or register-file port/bank conflicts. I recommend a cycle-accurate simulator (e.g., gem5-accel-sim) or RTL-in-the-loop evaluation, or, failing that, presenting the results explicitly as an upper bound with a sensitivity analysis; the abstract's 5.3x claim should be qualified until then.
  2. [§4.2] The hardware decoder's lane-interleaved input requirement does not match the kernel's contiguous per-thread chunks described in §3.2 and Fig. 2, and it appears to contradict the abstract's claim of operating on the same RLC-CSC format without data layout changes. Since the modified kernel does not implement the interleaved layout, its measured latency is not a faithful estimate of the hardware path. Please specify whether the modified kernel changes the RLC layout; if it does, the format is not identical between the kernel and hardware designs.
  3. [§6.2] The end-to-end LLaMA-2-13B results do not state whether they use the measured SMEM-based kernel or the estimated Sparse SIMT Core. The abstract and conclusion attribute 'up to 2.63x' to Celty as a whole. If this number includes the hardware estimates from Section 5, it rests on the unvalidated proxy and should be separated from the kernel-only measurements; if it is kernel-only, that should be stated explicitly.
  4. [§5, 'Baselines'] The spMM baselines are evaluated with the N dimension padded to 8 to satisfy tensor-core alignment, while Celty targets N=1. This makes the comparison against Coruscant/Flash-LLM an N=8 spMM versus N=1 spMspV comparison, conflating workload size with kernel efficiency. Please report the actual N used for each baseline and, if N=8 is required, present per-output-vector throughput or a version of the baselines adapted to N=1.
  5. [Abstract/§6.2] Calling a perplexity increase from 4.9 to 7.2 'maintaining' (abstract, and 'maintaining ... 5.6' in §6.2) is misleading; this is roughly a 47% relative degradation. Use wording such as 'with a bounded perplexity increase' and rely on the latency-perplexity tradeoff curve (Fig. 8) to convey the tradeoff.
minor comments (5)
  1. [§1, Contribution (1)] The claim 'first GPU kernel designed for dual-sparse LLM inference' needs qualification, since DuoGPT [34] is also a dual-sparsity framework. Please clarify whether the novelty is specifically the kernel/format or the co-design.
  2. [Table 2/Abstract] Dense LLaMA-2-13B WikiText-2 perplexity is reported as 4.88 in Table 2 and 4.9 in the abstract; harmonize the values.
  3. [Figure 3] The axis labels and legend in Figure 3 are difficult to read in the provided version; the x-axis label is rendered ambiguously and the compression-ratio comparison would benefit from clearer annotation.
  4. [Algorithm 1] Algorithm 1 is described as if each thread has its own rlc_arrays, but the relationship between the lane index and the per-thread array chunks is not stated precisely. Add a sentence clarifying the indexing.
  5. [§4.1] Inter-thread register-file sharing is supported by citing [11] (2015). Since this is not a standard feature on current NVIDIA GPUs, please provide more implementation detail or a more recent reference to justify the feasibility assumption.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; kernel claims rest on external benchmarks and the hardware numbers are an acknowledged proxy, not a conclusion derived from its own premises.

full rationale

The Celty kernel speedups are measured against external baselines (cuBLAS, Flash-LLM, Coruscant, DASP, Macko) on an A5000 GPU, and the RLC-CSC kernel behavior is implemented and benchmarked rather than derived from the paper's own claims. The only in-scope concern is the Sparse SIMT Core evaluation in Section 5, where the paper states that a modified kernel with 'RLC reconstruction instructions' removed and 'SMEM-based scatter-accumulation' replaced by 'direct register-file access' has its 'measured latency' used as 'an estimate' of the proposed hardware. This is an acknowledged proxy, and the resulting 5.3x and hardware-assisted end-to-end numbers are unvalidated estimates; however, that is a correctness/evidence limitation, not a circular reduction, because the estimated quantity is not defined in terms of the proxy measurement by construction. DuoGPT [34] is a self-citation used for the dual-sparsity pruning framework and accuracy context, but Celty's throughput claims do not reduce to DuoGPT's outputs; the kernel-vs-baseline measurements stand independently of DuoGPT. No equation sets a predicted quantity equal to an input by construction, so no circular step meets the evidentiary bar.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The results rest on standard GPU execution-model assumptions, the availability of runtime activation sparsity from DuoGPT-style thresholding, the feasibility of inter-thread register sharing, and the validity of the modified-kernel performance proxy. The main designed parameters are the 64-column inner-K tile, the 4-bit RLC width, and the register-accumulation ratio.

free parameters (4)
  • inner-K loop size (Split-K parallelism) = 64
    Chosen from the ablation in Figure 10 as the balance between K-parallelism and atomic merge overhead; all other experiments fix it to 64.
  • RLC-CSC bit-width = 4 bits
    Design choice for index compression; 4-bit RLC-CSC yields about 63% compression efficiency at 50% sparsity, and gaps larger than 15 require zero-padding entries that add overhead.
  • Register-accumulation ratio = 100% for headline speedup, tunable down to 0%
    Ablation in Figure 12 shows speedup varies with the fraction of partial-product accumulation done in registers versus SMEM; the headline Sparse SIMT Core number assumes full register-resident accumulation.
  • Warp tile shape = 256 rows x 64 columns
    Each warp covers 256 rows (32 lanes x 8 weights) and iterates 64 columns; derived from the vectorized load width and the Split-K choice.
assumptions (5)
  • domain assumption Activation sparsity can be induced at runtime by magnitude thresholding with bounded accuracy loss, as in DuoGPT.
    The kernel's speedup depends on the input vector being sparse enough that whole weight columns can be skipped; accuracy depends on the thresholding method from self-cited prior work [34].
  • domain assumption GPU SIMT execution model supports warp-synchronous operations, shared memory, and vectorized 128-bit loads.
    The kernel design relies on warp-level semantics such as shfl_up_sync, LDG.128, and SMEM bank behavior on NVIDIA Ampere GPUs.
  • domain assumption Inter-thread register file sharing (register file virtualization) is feasible in an SIMT core.
    The register-file accumulation buffers are shared among threads within an SM sub-partition; the paper cites [11] for register file virtualization but does not demonstrate it on the target architecture.
  • ad hoc to paper The modified-kernel software proxy accurately estimates Sparse SIMT Core performance.
    Section 5 states that removing RLC reconstruction instructions and SMEM scatter from a real kernel and measuring the modified latency serves as an estimate of the hardware design; this assumes no additional pipeline stalls, port conflicts, or decoder overhead.
  • domain assumption The RLC-CSC gap distribution in LLM weight columns yields good compression at 30-70% sparsity.
    4-bit run-length coding is efficient only if gaps between nonzeros are mostly 15 or less; the paper reports average compression on 4096x4096 layers but does not analyze worst-case columns.
invented entities (2)
  • Celty Sparse SIMT Core with 3-stage pipelined RLC decoder
    purpose: Reconstruct absolute row indices from RLC-CSC in hardware, eliminating software prefix-sum and shuffle instructions.
    No silicon or full-system simulation is provided; performance is estimated by a software-modified kernel and synthesis is reported for the decoder unit only, not the integrated pipeline.
  • Inter-thread shared register-file partial-product buffers
    purpose: Replace SMEM scatter-accumulation with conflict-free register-file writes partitioned per SIMT core.
    Depends on register file virtualization [11] not present in commercial GPUs; the software proxy uses per-thread registers, not actual inter-thread register sharing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference." pith.science (2026). https://pith.science/paper/HP6MA6UM

@misc{pith2026260801536,
  author       = {Pith},
  title        = {Pith review of: Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HP6MA6UM}},
  note         = {Machine review of arXiv:2608.01536}
}
read the original abstract

Large Language Models (LLMs) increasingly rely on sparsity to reduce inference cost, but most prior work targets a single sparsity source-either weight or activation-and optimizes for batched multi-user inference. Dual-sparsity, which combines unstructured weight pruning with runtime activation sparsity, offers a compelling tradeoff among model size, accuracy, and latency for single-user decoding, but formulates as a Sparse Matrix-Sparse Vector (spMspV) workload that existing GPU kernels handle poorly. We propose Celty, a co-designed sparse format, GPU kernel, and SIMT microarchitecture for efficient spMspV in LLM inference. At the kernel level, Celty introduces a Run-Length Compressed CSC (RLC-CSC) format that enables vectorized loading of compressed weight columns and exploits both sparsity sources to skip unnecessary memory accesses, with shared memory used for scattered partial-product accumulation. At the microarchitecture level, the Celty Sparse SIMT Core integrates a pipelined RLC decoder to eliminate software-level index reconstruction and repurposes local register files for conflict-free accumulation-operating directly on the same RLC-CSC format without data layout changes. The Celty GPU kernel achieves up to 2.8x speedup over cuBLAS and 2.4x over Flash-LLM. With the Sparse SIMT Core, speedups reach up to 5.3x over cuBLAS at 70% dual-sparsity.

Figures

Figures reproduced from arXiv: 2608.01536 by the authors.

Figure 1
Figure 1. Illustration of (a) prior spMM kernels. (b) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Celty spMspV kernel design. (a) Offline: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Compression ratio of bitmap vs. RLC-CSC across different sparsity on a 5120×5120 layer. warps, each covering 256 rows, while each warp iterates over 64 columns along K in an inner loop. We use 128-bit vectorized loads [19] for data loading: at FP16 precision, each thread loads 8 consecutive weights (16 bytes), so each 32- thread warp covers 256 rows. To guarantee alignment for vectorized access, we apply reverse off… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: (a) Shared-memory bank conflicts during [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Percentage of RLC-CSC reconstruction la [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of kernel latency across workloads and sparsity. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Celty Sparse SIMT Core microarchitecture. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 9
Figure 9. Figure 9: Comparison of sparse microarchitectural en [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 12
Figure 12. Figure 12: Normalized speedup of Celty Sparse SIMT Core as the register-accumulation portion varies from 0% (pure SMEM) to 100% (pure register) across dual￾sparsity levels. Macko across all configurations, with the dual-sparsity ad￾vantage growing at higher sparsity: Celty achie…
Figure 11
Figure 11. Figure 11: Comparison between Celty and Macko across sparsity levels and layer dimensions. Comparison with Macko across sparsity. Macko [20] is a concurrent spMV kernel that operates on an RLC-CSR for￾mat using SIMT cores. However, its execution flow dedicates each warp to a sin…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 28 canonical work pages

  1. [1]

    Sandhini Agarwal et al. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925(2025)

    arXiv 2025

  2. [2]

    Yu-Hsin Chen et al. 2016. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks.IEEE journal of solid-state circuits52, 1 (2016), 127–138

    work page 2016

  3. [3]

    Yu-Hsin Chen et al. 2019. Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices.IEEE Journal on Emerging and Selected Topics in Circuits and Systems9, 2 (2019), 292–308

    work page 2019

  4. [4]

    Tri Dao et al. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness.Advances in neural information processing systems35 (2022), 16344–16359

    work page 2022

  5. [5]

    Ruibo Fan et al. 2025. Spinfer: Leveraging low-level sparsity for effi- cient large language model inference on gpus. InProceedings of the Twentieth European Conference on Computer Systems. 243–260

    work page 2025

  6. [6]

    Elias Frantar et al. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. InInternational Conference on Machine Learning. PMLR, 10323–10337

    work page 2023

  7. [7]

    Trevor Gale et al. 2020. Sparse GPU Kernels for Deep Learning. In Proceedings of the International Conference for High Performance Com- puting, Networking, Storage and Analysis, SC 2020

    work page 2020

  8. [8]

    Song Han et al. 2016. EIE: Efficient inference engine on compressed deep neural network.ACM SIGARCH Computer Architecture News44, 3 (2016), 243–254

    work page 2016

Show all 39 references
  1. [9]

    Adnan Hoque et al . 2024. Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition.arXiv preprint arXiv:2402.00025(2024)

    2024 arXiv

  2. [10]

    Guyue Huang et al. 2023. Rm-stc: Row-merge dataflow inspired gpu sparse tensor core for energy-efficient sparse acceleration. InPro- ceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture. 338–352

    2023

  3. [11]

    Hyeran Jeon et al. 2015. GPU register file virtualization. InProceedings of the 48th International Symposium on Microarchitecture. 420–432

    2015

  4. [12]

    Haonan Ji et al. 2022. Tilespmspv: A tiled algorithm for sparse matrix- sparse vector multiplication on gpus. InProceedings of the 51st Inter- national Conference on Parallel Processing. 1–11

    2022

  5. [13]

    Donghyeon Joo et al. 2025. Coruscant: Co-Designing GPU Kernel and Sparse Tensor Core to Advocate Unstructured Sparsity in Efficient LLM Inference. InProceedings of the 58th IEEE/ACM International Symposium on Microarchitecture. 232–245

    2025

  6. [14]

    Woosuk Kwon et al. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles

    2023

  7. [15]

    Min Li et al. 2020. Adaptive SpMV/SpMSpV on GPUs for input vectors of varied sparsity.IEEE Transactions on Parallel and Distributed Systems 32, 7 (2020), 1842–1853

    2020

  8. [16]

    Yuanchun Li et al. 2024. Personal llm agents: Insights and survey about the capability, efficiency and security.arXiv preprint arXiv:2401.05459 (2024)

    2024 arXiv

  9. [17]

    James Liu et al. 2025. Training-free activation sparsity in large lan- guage models.The Thirteenth International Conference on Learning Representations(2025)

    2025

  10. [18]

    Yuechen Lu et al . 2023. Dasp: Specific dense matrix multiply- accumulate units accelerated general sparse matrix-vector multiplica- tion. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1–14

    2023

  11. [19]

    2025.CUDA Pro Tip: Increase Per- formance with Vectorized Memory Access

    Justin Luitjens et al . 2025.CUDA Pro Tip: Increase Per- formance with Vectorized Memory Access. NVIDIA Technical Blog. https://developer.nvidia.com/blog/cuda-pro-tip-increase- performance-with-vectorized-memory-access/

    2025

  12. [20]

    Vladimír Macko et al. 2025. MACKO: Sparse Matrix-Vector Multiplica- tion for Low Sparsity.arXiv preprint arXiv:2511.13061(2025)

    2025

  13. [21]

    Xin Men et al. 2024. Shortgpt: Layers in large language models are more redundant than you expect.arXiv preprint arXiv:2403.03853 (2024)

    2024 arXiv

  14. [22]

    Stephen Merity et al. 2016. Pointer sentinel mixture models.arXiv preprint arXiv:1609.07843(2016)

    2016 arXiv

  15. [23]

    Yuyao Niu et al. 2021. Tilespmv: A tiled algorithm for sparse matrix- vector multiplication on gpus. In2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 68–78

    2021

  16. [24]

    NVIDIA Corporation. 2020. NVIDIA A100 Tensor Core GPU Archi- tecture. https://images.nvidia.com/aem-dam/en-zz/Solutions/data- center/nvidia-ampere-architecture-whitepaper.pdf

    2020

  17. [25]

    Fabrizio Sandri et al. 2025. 2SSP: A Two-Stage Framework for Struc- tured Pruning of LLMs.arXiv preprint arXiv:2501.17771(2025)

    2025 arXiv

  18. [26]

    ShareGPT. 2023. ShareGPT. https://sharegpt.com/. https://sharegpt. com/

    2023

  19. [27]

    Aaron Stillmaker et al. 2017. Scaling equations for the accurate predic- tion of CMOS device performance from 180 nm to 7 nm.Integration 58 (2017), 74–81

    2017

  20. [28]

    Mingjie Sun et al. 2023. A simple and effective pruning approach for large language models.arXiv preprint arXiv:2306.11695(2023)

    2023 arXiv

  21. [29]

    Hugo Touvron et al . 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    2023 arXiv

  22. [30]

    Yang Wang et al. 2021. Dual-side sparse tensor core. In2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA). IEEE, 1083–1095

    2021

  23. [31]

    Haojun Xia et al. 2023. Flash-llm: Enabling cost-effective and highly- efficient large generative model inference with unstructured sparsity. arXiv preprint arXiv:2309.10285(2023)

    2023 arXiv

  24. [32]

    Lei Xu et al. 2024. HAM-SpMSpV: An optimized parallel algorithm for masked sparse matrix-sparse vector multiplications on multi-core CPUs. InProceedings of the 33rd International Symposium on High- Performance Parallel and Distributed Computing. 160–173

    2024

  25. [33]

    Carl Yang et al. 2015. Fast sparse matrix and sparse vector multipli- cation algorithm on the GPU. In2015 IEEE International Parallel and Distributed Processing Symposium Workshop. IEEE, 841–847

    2015

  26. [34]

    Ruokai Yin et al. 2025. DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs.arXiv preprint arXiv:2506.20194 (2025)

    2025

  27. [35]

    Susan Zhang et al. 2022. Opt: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068(2022). 9 ICCAD ’26, November 8–12, 2026, San Jose, CA, USA Ruokai Yin and Priyadarshini Panda

    2022 arXiv

  28. [36]

    Zhenyu Zhang et al. 2025. R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference. InThe Thirteenth International Conference on Learning Representations

    2025

  29. [37]

    2022.{SparTA}:{Deep-Learning} Model Sparsity via{Tensor-with-Sparsity-Attribute}

    Ningxin Zheng et al. 2022.{SparTA}:{Deep-Learning} Model Sparsity via{Tensor-with-Sparsity-Attribute}. In16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 213–232

    2022

  30. [38]

    Longguang Zhong et al. 2024. Blockpruner: Fine-grained pruning for large language models.arXiv preprint arXiv:2406.10594(2024)

    2024 arXiv

  31. [39]

    Maohua Zhu et al. 2019. Sparse tensor core: Algorithm and hardware co-design for vector-wise sparse neural networks on modern gpus. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture. 359–371. 10

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.