Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FlashSparse claims that sparse matrix multiplication on tensor cores can be accelerated by cutting the sparse operand granularity to $8\times1$ via the swap-and-transpose identity $A\times B=(B^{\mathsf T}\times A^{\mathsf T})^{\mathsf…

desk verdict A real systems contribution with unusually thorough benchmarks, but the headline SOTA speedup is inflated by comparing FP16 against TF32 baselines; still worth serious refereeing. read the letter →

arxiv 2412.11007 v1 pith:6ILJKVT5 submitted 2024-12-15 cs.DC cs.LG

Jinliang Shi , Shigang Li , Youxuan Xu , Rongtian Fu , Xueying Wang , Tong Wu This is my paper · ORCID
classification cs.DCcs.LG
keywords TensorCoresSparseMatrix-MatrixMultiplicationSDDMMswap-and-transposeMMAGPUkernelsgraphneuralnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlashSparse sets out to show that the main barrier to fast sparse matrix multiplication on tensor cores is not the hardware but the granularity at which the sparse operand is fed to the MMA instruction. Prior tensor-core kernels align the sparse operand with the 16-row left-operand dimension, forcing $16\times1$ nonzero vectors that are mostly zeros. FlashSparse applies the identity $A\times B=(B^{\mathsf T}\times A^{\mathsf T})^{\mathsf T}$ so the sparse operand becomes the right operand of MMA, whose $n$ dimension is 8, allowing $8\times1$ nonzero vectors. This halves the zero-fill per sparse block, reducing MMA invocations by about 43% and data-access cost by 28% to 49%, and the paper reports geometric mean speedups of 5.5x over the prior tensor-core kernel and 3.22x over the prior CUDA-core kernel.

What carries the argument

The load-bearing object is the swap-and-transpose MMA computation, built on the identity $A\times B=(B^{\mathsf T}\times A^{\mathsf T})^{\mathsf T}$. In the kernel, the sparse TC block is loaded row-major and transposed into registers to become the right operand of an m16n8k8 (FP16) or m16n8k4 (TF32) MMA, while the dense block $B$ is loaded column-major and rearranged into an $8\times16$ left operand; the accumulator is transposed before being written back. A memory-efficient thread mapping swaps the columns each thread touches so that four FP16 elements form contiguous $2\times2$ tiles that coalesce into 32-byte transactions, and the ME-BCRS storage format stores only nonzero vectors, eliminating the zero-vector padding used by earlier formats.

What would settle it

Run FlashSparse's SpMM kernel with a fixed sparse matrix and dense widths $N=512$ or $N=1024$, while counting actual DRAM traffic with hardware counters. If execution time or memory traffic grows with $N$ fast enough that FlashSparse becomes slower than the $16\times1$ baseline, the central claim fails for wide dense matrices.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that the $16\times1$ nonzero-vector granularity used by prior tensor-core sparse kernels is the key source of redundant computation and memory traffic: real sparse matrices pack 5.6 to 11.4 times more zeros than nonzeros into those vectors. FlashSparse removes that constraint by transposing and swapping both operands, so the sparse matrix is cut along the $n=8$ dimension of the MMA right operand rather than the $m=16$ left-operand dimension. The resulting $8\times1$ vectors are packed into $8\times8$ sparse TC blocks (or $8\times4$ for TF32), dense enough to use the tensor cores' full arithmetic width, and the paper reports that this translates into a 5.5x geometric-mean speedup over the leading tensor-core kernel, a 3.22x geometric-mean speedup over the leading CUDA-core kernel, and end-to-end graph neural network speedups of 1.57x to 1.79x over the strongest library baseline tested.

Load-bearing premise

The speedup rests on the assumption that the cost of loading the wider $8\times16$ dense operand is outweighed by the savings from halving the sparse operand, so the reported data-access reduction survives once caching and register usage are accounted for.

Editorial extensions

If this is right

  • The $16\times1$ granularity of prior tensor-core kernels is not a hardware law: any MMA whose right-operand dimension is smaller than its left-operand dimension can be exploited the same way.
  • Because the reduction in MMA invocations and data-access cost scales with vector size, switching from $16\times1$ to $8\times1$ attacks both the compute-bound and memory-bound portions of the kernel, which is why the speedup holds across 515 matrices.
  • The same swap-and-transpose machinery accelerates both SpMM and SDDMM, and the SDDMM kernel writes its sparse output in the tiles that the SpMM kernel consumes, so the two operators chain without reformatting.
  • End-to-end graph neural network training inherits the gains, with geometric mean speedups of 1.57x for GCN and 1.79x for AGNN over the leading graph library baseline tested.
  • The ME-BCRS format reduces sparse storage footprint by an average of 11.72% and up to 50% compared with the zero-vector-padding format.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the swap-and-transpose identity is architecture-agnostic, so any matrix unit with a smaller right-operand dimension could adopt the same trick; the $8\times1$ result may transfer beyond NVIDIA GPUs.
  • Editorial inference: the paper counts per-block operand bytes rather than full reuse-aware memory traffic, so for very wide dense matrices (large $N$) the dense left operand may not be amortized; measuring actual DRAM traffic at larger $N$ would test that boundary.
  • Editorial inference: the $8\times1$ granularity suggests a natural extension to adaptive vector sizes chosen per row window based on measured density, which could recover part of the zero-fill that remains even at $8\times1$.
  • Editorial inference: since the technique removes redundant MMA work rather than only overlapping it, the speedup should come with a roughly proportional reduction in tensor-core energy, though the paper does not measure energy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. FlashSparse is a tensor-core kernel design for SpMM and SDDMM that reduces the granularity of the sparse operand from the 16×1 nonzero vectors used by TC-GNN and DTC-SpMM to 8×1. The enabling device is the matrix identity A×B = (B^T×A^T)^T ("swap-and-transpose"): the sparse operand is moved to the n=8 dimension of the MMA instruction so it can be partitioned into half-length vectors, while the dense operand becomes the m=16 left operand. The paper additionally contributes a thread-mapping scheme that reshuffles the dense-B register/global-memory layout to obtain coalesced 32-byte transactions, and ME-BCRS, a storage format that avoids zero-vector padding. Claims include a ~43% reduction in MMA invocations, a 35–49% reduction in operand data-access cost, geometric-mean speedups of 5.5x over DTC-SpMM and 3.22x over RoDe for SpMM on an RTX 4090, and end-to-end GNN speedups of 1.57–1.79x over DGL, all evaluated on 515 matrices on H100 and RTX 4090 with three ablations.

Significance. If the central comparison is re-run precision-matched, the paper's core idea is a genuine, publishable advance: the swap-and-transpose construction is mathematically elementary but non-obviously aligned with MMA's unbalanced operand shapes, and the ablation isolating granularity (8×1 vs 16×1, same implementation) shows a 1.89x geomean for SpMM on H100. The evaluation is unusually broad (515 matrices, two GPU generations, six-plus baselines, ablations for every component), and the authors measure preprocessing overhead (<1%) and validate reduced-precision accuracy in Table 8; these strengths matter because the granularity benefit, the thread-mapping gain (1.18–1.34x), and the ME-BCRS footprint reduction are separately quantified rather than asserted. The main weakness is that the headline "5.5x speedup over DTC-SpMM" mixes FP16 FlashSparse with TF32 baselines, so the abstract's causal phrasing about granularity is not supported by that number, and a TF32-vs-TF32 table is required; the data-access-reduction claim is an operand-count metric, and the capped 100x/≥50 geomeans for TC-GNN are reporting artifacts.

major comments (3)
  1. [Abstract; Table 5; §4.1] The headline speedups are precision-heterogeneous, and the causal claim drawn from them is not supported. Table 3 fixes DTC-SpMM and TC-GNN at TF32, while FlashSparse runs in both TF32 and FP16; §4.1 reports 4888 GFLOPS (FP16) versus 2697 GFLOPS (TF32) geomean on RTX 4090, a 1.81x ratio that mirrors the hardware FP16/TF32 peak ratio. Table 5 never states which FlashSparse precision produced the 5.5x and 3.22x entries, and the Abstract presents these numbers without disclosing the precision mix. The granularity-only ablation (§4.3, Figure 14) gives 1.89x for SpMM on H100, so the 5.5x aggregates the 8×1 scheme, the FP16-vs-TF32 hardware advantage, and the thread-mapping/storage optimizations. Under the natural assumption that the 4888/2697 geomeans and the Table 5 comparison cover the same matrices, the same-precision TF32 speedup over DTC-SpMM would be roughly 5.5 × (2697/4888) ≈ 3.0x, which is still a strong result, but it must be reported, labeled, and used in the Abstract instead of the mixed-precision number. The SDDMM and end-to-end comparisons (Table 6, Figure 16) have the same precision asymmetry against the FP32 CUDA-core baselines; the accuracy validation in Table 8 mitigates but does not remove the need for explicit precision labels on every speedup claim.
  2. [Table 5; Table 6; §4.1] Two result tables report statistics that are artifacts of capping rather than measurements. §4.1 states that TC-GNN's throughput is labeled 0 for matrices with more than 5M nonzeros, yet Table 6 lists "Geo Mean 100.0x" for TC-GNN on both GPUs and Table 5 lists "Mean ≥ 50" for its RTX 4090 column. A geometric mean over zero-valued denominators is undefined, so these entries only indicate that individual ratios were truncated, not that FlashSparse is 100x faster on average. In addition, Figure 11(a)(c) caps the displayed FlashSparse speedup at 8x, and the true median is never given numerically. Please report medians or compute geometric means over matrices with nonzero baseline throughput, and state every truncation threshold in the table captions.
  3. [§3.3; Figure 12; §4.1] The claimed 35–49% data-access reduction is an operand-element count and is not evidence about actual memory traffic. The paper's own definition in §4.1 says the cost is that of "loading data from the memory hierarchy, without distinguishing the data sources (from global memory or caches)", and for FP16 m16n8k8 the per-MMA operand element counts are equal in the two schemes (8×8 + 8×16 = 16×8 + 8×8 = 192), matching the data-access formulas printed in Figures 2 and 6. The reduction in Figure 12 therefore exactly mirrors the MMA-invocation reduction in Figure 1 rather than being an independent data-locality effect; the concern that the enlarged dense-B operand reverses the total operand budget does not materialize at the element-count level, but whether DRAM traffic actually decreases depends on cache or shared-memory reuse of the 8×16 dense-B block, which is not modeled. Since the Section 1 contributions and the Abstract claim reduced data access as a separate benefit, the paper should either provide a reuse-aware traffic model or explicitly qualify the metric as operand-issue counts, and the Section 3.3 statement that data access is "proportionally reduced by 50%" should carry the same qualification.
minor comments (7)
  1. [Tables 5 and 6] The tables use "Mean" for what the text calls "geometric mean" without defining the aggregation or the matrix grouping (small/large) introduced in Figure 11; please state the aggregation rule and the truncation convention directly in each caption.
  2. [§4.3] The granularity-ablation text quotes geometric means only for H100, although Figure 14(c)-(d) shows RTX 4090; please give those numbers in the text, since the headline 5.5x result is on RTX 4090.
  3. [§4.4] The AGNN comparison against TC-GNN is acknowledged to be uneven because TC-GNN lacks the softmax layer, which the paper says "still occupies a certain proportion" of end-to-end time; please report that proportion or re-time TC-GNN with the softmax layer included so the end-to-end comparison is fair.
  4. [Table 7] ME-BCRS is compared with SR-BCRS [26], which is the corresponding author's prior work; the comparison itself is appropriate, but the relationship should be stated explicitly in the text for transparency.
  5. [§3.3; Figures 4 and 5] The register-layout walk-through would be easier to check if the paper stated, for each operand (sparse A, dense B, accumulator C), its layout in global memory, its required MMA layout, and where the transpose occurs (load pattern versus register shuffle); Figure 7 establishes this for B only, and the cost of the A-side layout conversion is never quantified.
  6. [Throughout] Please correct the typos: "Tensor Core unites" (§4), "Transcations" (Figure 7), "Non-coaleasced" (Figure 15), one occurrence of "Flash-Sparse" (§5), and a stray "2" at the end of the Figure 6 caption.
  7. [§1; Artifact] No source-code or artifact link is provided; since the performance claims rest on the kernel implementations, I encourage the authors to release the code in the camera-ready version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the swap-and-transpose derivation is the standard identity A×B=(B^T×A^T)^T, and the performance claims are empirical benchmark results.

full rationale

The central derivation in Section 3.2 is the algebraic identity A×B=(B^T×A^T)^T (Equation 1), used to place the sparse operand on the n=8 side of the MMA instruction. This is self-contained, true by elementary matrix algebra, and does not presuppose any performance conclusion. The reduction in MMA invocations (Figure 1), the operand data-access cost comparison (Figures 2 and 6), and the measured speedups (Tables 5 and 6, Figure 14) are concrete counts and benchmark measurements, not fitted parameters renamed as predictions. The only apparent self-citation is SR-BCRS [26] from the corresponding author's SC22 work; it is used in Section 3.5 and Table 7 as a comparison baseline for the ME-BCRS memory-footprint evaluation. That baseline is measured against 515 matrices and is not load-bearing for the swap-and-transpose claim, so it does not make the argument circular. The skeptic's precision mismatch concern (FlashSparse FP16 headline speedups versus TF32-only baselines such as DTC-SpMM) is a benchmarking-fairness and evidence-quality issue, not a circularity of the derivation; it does not raise the circularity score. No fitted input is relabeled as a prediction, no uniqueness theorem is imported from the authors, and no ansatz is smuggled in via citation. The derivation chain is therefore self-contained.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no fitted constants and no new physical entities. ME-BCRS is a storage format design rather than an invented entity in the sense of a new force or particle. All quantitative claims are empirical, and the only mathematical input is the standard transpose identity.

assumptions (4)
  • standard math A×B = (B^T × A^T)^T for matrix multiplication.
    Used in Section 3.2, Equation 1, to swap the operands so the sparse matrix becomes the MMA right operand and the vector granularity becomes the n=8 dimension.
  • domain assumption NVIDIA MMA/WMMA tensor-core instructions support only fixed operand shapes, with minimum n=8, and have documented fragment layout requirements.
    Table 1 and Section 3.2 rely on m16n8k8/m16n8k4 shapes and the PTX register data layouts cited from NVIDIA documentation [33].
  • domain assumption NVIDIA GPUs have a minimum memory transaction granularity of 32 bytes, making 16-byte thread accesses wasteful.
    Used in Section 3.3 to justify the memory-efficient thread mapping strategy and its claimed 50% reduction in memory transactions.
  • domain assumption The 500+ SuiteSparse matrices and 15 graph matrices used for evaluation are representative of general unstructured sparse workloads.
    The paper generalizes conclusions to sparse matrix multiplications from this benchmark set, as described in Section 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores." pith.science (2026). https://pith.science/paper/6ILJKVT5

@misc{pith2026241211007,
  author       = {Pith},
  title        = {Pith review of: FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ILJKVT5}},
  note         = {Machine review of arXiv:2412.11007}
}
read the original abstract

Sparse Matrix-matrix Multiplication (SpMM) and Sampled Dense-dense Matrix Multiplication (SDDMM) are important sparse operators in scientific computing and deep learning. Tensor Core Units (TCUs) enhance modern accelerators with superior computing power, which is promising to boost the performance of matrix operators to a higher level. However, due to the irregularity of unstructured sparse data, it is difficult to deliver practical speedups on TCUs. To this end, we propose FlashSparse, a novel approach to bridge the gap between sparse workloads and the TCU architecture. Specifically, FlashSparse minimizes the sparse granularity for SpMM and SDDMM on TCUs through a novel swap-and-transpose matrix multiplication strategy. Benefiting from the minimum sparse granularity, the computation redundancy is remarkably reduced while the computing power of TCUs is fully utilized. Besides, FlashSparse is equipped with a memory-efficient thread mapping strategy for coalesced data access and a sparse matrix storage format to save memory footprint. Extensive experimental results on H100 and RTX 4090 GPUs show that FlashSparse sets a new state-of-the-art for sparse matrix multiplications (geometric mean 5.5x speedup over DTC-SpMM and 3.22x speedup over RoDe).

Figures

Figures reproduced from arXiv: 2412.11007 by the authors.

Figure 1
Figure 1. The number of MMA invocations under 16x1 and 8x1 nonzero vector sizes in SpMM. Note that the unit used for IGB-large is ten millions for clear presentation. However, sparse matrices in many real-world applica￾tions [8, 22, 38, 40, 42] exhibit high sparsity ratio (e.g., >99%) and unstructured pattern, which is far different from the structured sparsity natively supported on TCUs. The ma￾jor challenge in utilizing TCU… view at source ↗
Figure 2
Figure 2. SpMM on TCUs with nonzero vector size of 16×1. The operand shape of MMA is m16n8k8. state-of-the-art works. With FP16 precision, MMA requires the matrix shape to be 𝑚16𝑛8𝑘8. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 5
Figure 5. The implementation of SpMM with the swap-and￾transpose MMA computation strategy. 3.3 The implementation of SpMM In this section, we detail the design of our SpMM kernel with the swap-and-transpose MMA computation strategy. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figures from the paper (11 more)
Figure 6
Figure 6. Figure 6: shows that using the 8×1 vector size to partition the sparse matrix only requires 2 MMAs to finish the SpMM in FlashSparse. In contrast, the current SOTA works using the 16×1 vector size requires 4 MMAs as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]
Figure 7
Figure 7. Figure 7: Memory-efficient thread mapping for coalesced data access. The precision of each element is FP16. dense TC block B is column-major. The original sparse ma￾trix A can be pre-stored in row-major format (as illustrated in Section 3.5). However, the TC block B is formed by…
Figure 8
Figure 8. Figure 8: The implementation of SDDMM based on the swap-and-transpose MMA computation strategy. However, the shapes of the sparse TC block A in SpMM are 8×8 and 8×4 for FP16 and TF32, respectively. Therefore, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: Besides, taking the memory space for 𝑅𝑜𝑤𝑃𝑜𝑖𝑛𝑡𝑒𝑟𝑠 as an example, we only need to store M row pointers (M is the number of row windows of a sparse matrix) rather than 2M in the padding-based scheme [26]. This is because we only record information for nonzero vectors. Th…
Figure 10
Figure 10. Figure 10: ME-BCRS storage format. 4 Evaluation Benchmarks and Platforms: We incorporate FlashSparse into the Pytorch framework for performance evaluation. The swap-and-transpose is built into the sparse kernels (both SpMM and SDDMM). Our experimental platform consists of a NVID…
Figure 11
Figure 11. Figure 11: SpMM performance results on H100 and RTX4090 GPU. (a)(c) The speedup distribution of FlashSparse and baselines over cuSPARSE. (b)(d) The measured throughput of the 515 matrices with N=256. The matrices are sorted in ascending order according to the number of nonzero e…
Figure 12
Figure 12. Figure 12: The data access cost of 16×1 and 8×1 vectors in SpMM (N=128) and SDDMM (N=32). The precision is FP16 [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]
Figure 13
Figure 13. Figure 13: SDDMM performance on H100 and RTX4090 GPUs. The matrices are sorted in a manner consistent with [PITH_FULL_IMAGE:figures/full_fig_p009_13.png]
Figure 14
Figure 14. Figure 14: The throughput of FlashSparse under 16×1 and 8×1 vector granularity in SpMM and SDDMM. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_14.png]
Figure 15
Figure 15. Figure 15: The throughput of FlashSparse under non￾coalesced and coalesced data access modes in SpMM. The effectiveness of ME-BCRS storage format. We test 515 matrices to compare the memory footprint of ME￾BCRS with SR-BCRS [26] (the zero vector padding-based method). As shown i…
Figure 16
Figure 16. Figure 16: The end-to-end performance of FlashSparse on H100 and RTX4090 over GCN and AGNN models [PITH_FULL_IMAGE:figures/full_fig_p011_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Fused3S: Fast Sparse Attention on Tensor Cores

    cs.DC 2025-05 conditional novelty 6.0 of 10

    A fused tensor-core sparse attention kernel (SDDMM, softmax, SpMM) that achieves large speedups over prior baselines on H100 and A30.

  2. Acc-SpMM: Accelerating General-purpose Sparse Matrix-Matrix Multiplication with GPU Tensor Cores

    cs.DC 2025-01 conditional novelty 5.0 of 10

    A tensor-core SpMM library using bitmask compression, graph reordering, pipelining, and load balancing reports up to 5.11x speedups over cuSPARSE on modern NVIDIA GPUs.

Reference graph

Works this paper leans on

49 extracted references · 41 canonical work pages · cited by 2 Pith papers

  1. [1]

    Tensor Core

    2023. Tensor Core. . https://www.nvidia.cn/data-center/tensor-cores/

    work page 2023

  2. [2]

    Paul Almasan, José Suárez-Varela, Krzysztof Rusek, Pere Barlet-Ros, and Albert Cabellos-Aparicio. 2022. Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case. Computer Communications 196 (2022), 184–194

    work page 2022

  3. [3]

    Hartwig Anzt, Stanimire Tomov, and Jack J Dongarra. 2015. Acceler- ating the LOBPCG method on GPUs using a blocked sparse matrix vector product.. In SpringSim (HPS). 75–82

    work page 2015

  4. [4]

    David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993–1022

    2003

  5. [5]

    Khac-Hoai Nam Bui, Jiho Cho, and Hongsuk Yi. 2022. Spatial-temporal graph neural network for traffic forecasting: An overview and open research issues. Applied Intelligence 52, 3 (2022), 2763–2774

    work page 2022

  6. [6]

    Zhaodong Chen, Zheng Qu, Liu Liu, Yufei Ding, and Yuan Xie. 2021. Efficient tensor core-based gpu kernels for structured sparsity under reduced precision. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis(SC) . 1–14

    work page 2021

  7. [7]

    Abdul Dakkak, Cheng Li, Jinjun Xiong, Isaac Gelado, and Wen-mei Hwu. 2019. Accelerating reduction and scan using tensor core units. In Proceedings of the ACM International Conference on Supercomputing . 46–57

    work page 2019

  8. [8]

    Timothy A Davis and Yifan Hu. 2011. The University of Florida sparse matrix collection. ACM Transactions on Mathematical Software (TOMS) 38, 1 (2011), 1–25

    2011

Show all 49 references
  1. [9]

    dgl. 2018. DGL. https://docs.dgl.ai

    2018

  2. [10]

    Ruibo Fan, Wei Wang, and Xiaowen Chu. 2024. DTC-SpMM: Bridging the Gap in Accelerating General Sparse Matrix Multiplication with Tensor Cores. In ASPLOS24: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Sys...

    2024

  3. [11]

    Johannes Feldmann, Nathan Youngblood, Maxim Karpov, Helge Gehring, Xuan Li, Maik Stappers, Manuel Le Gallo, Xin Fu, Anton Lukashchuk, Arslan Sajid Raja, et al. 2021. Parallel convolutional pro- cessing using an integrated photonic tensor core. Nature 589, 7840 (2021), 52–58

    2021

  4. [12]

    Boyuan Feng, Yuke Wang, Tong Geng, Ang Li, and Yufei Ding. 2021. Apnn-tc: Accelerating arbitrary precision neural networks on ampere gpu tensor cores. In Proceedings of the international conference for high performance computing, networking, storage and analysis(SC) . 1–13

    2021

  5. [13]

    Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019). 11 PPoPP’25, 2025, Las Vegas, NV, USA Jinliang Shi, Shigang Li, Youxuan Xu, Rongtian Fu, Xueying Wang, and Tong Wu

    2019 arXiv

  6. [14]

    Trevor Gale, Matei Zaharia, Cliff Young, and Erich Elsen. 2020. Sparse gpu kernels for deep learning. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC)

    2020

  7. [15]

    Jaume Gibert, Ernest Valveny, and Horst Bunke. 2012. Graph embed- ding in vector spaces by node attribute statistics. Pattern Recognition 45, 9 (2012), 3072–3083

    2012

  8. [16]

    Changwan Hong, Aravind Sukumaran-Rajam, Israt Nisa, Kunal Singh, and P Sadayappan. 2019. Adaptive sparse tiling for sparse matrix multiplication. In Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming(PPoPP) . 300–314

    2019

  9. [17]

    Guyue Huang, Guohao Dai, Yu Wang, and Huazhong Yang. 2020. {GE-SpMM}: General-purpose sparse matrix-matrix multiplication on gpus for graph neural networks. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC)

    2020

  10. [18]

    Peng Jiang, Changwan Hong, and Gagan Agrawal. 2020. A novel data transformation and execution strategy for accelerating sparse matrix multiplication on GPUs. In Proceedings of the 25th ACM SIGPLAN symposium on principles and practice of parallel programming(PPoPP) . 376–388

    2020

  11. [19]

    Weiwei Jiang and Jiayun Luo. 2022. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications 207 (2022), 117921

    2022

  12. [20]

    Arpandeep Khatua, Vikram Sharma Mailthody, Bhagyashree Taleka, Tengfei Ma, Xiang Song, and Wen-mei Hwu. 2023. Igb: Addressing the gaps in labeling, features, heterogeneity, and size of public graph datasets for deep learning research. In Proceedings of the 29th ACM SIGKDD Conf...

    2023

  13. [21]

    Diksha Khurana, Aditya Koli, Kiran Khatter, and Sukhdev Singh. 2023. Natural language processing: State of the art, current trends and chal- lenges. Multimedia tools and applications 82, 3 (2023), 3713–3744

    2023

  14. [22]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

    2016 arXiv

  15. [23]

    Sanjay Kumar, Abhishek Mallik, Anavi Khetarpal, and BS Panda. 2022. Influence maximization in social networks using graph embedding and graph neural network. Information Sciences 607 (2022), 1617–1636

    2022

  16. [24]

    Jérôme Kunegis and Andreas Lommatzsch. 2009. Learning spectral graph transformations for link prediction. In Proceedings of the 26th Annual International Conference on Machine Learning . 561–568

    2009

  17. [25]

    Andrew S Lan, Andrew E Waters, Christoph Studer, and Richard G Baraniuk. 2014. Sparse factor analysis for learning and content analyt- ics. The Journal of Machine Learning Research 15, 1 (2014), 1959–2008

    2014

  18. [26]

    Shigang Li, Kazuki Osawa, and Torsten Hoefler. 2022. Efficient quan- tized sparse matrix operations on tensor cores. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis (SC). IEEE

    2022

  19. [27]

    Yinglong Li. 2022. Research and application of deep learning in im- age recognition. In 2022 IEEE 2nd International Conference on Power, Electronics and Computer Applications (ICPECA) . IEEE, 994–999

    2022

  20. [28]

    Yuechen Lu and Weifeng Liu. 2023. DASP: Specific Dense Matrix Multiply-Accumulate Units Accelerated General Sparse Matrix-Vector Multiplication. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis . 1–14

    2023

  21. [29]

    Yuyao Niu, Zhengyang Lu, Haonan Ji, Shuhui Song, Zhou Jin, and Weifeng Liu. 2022. TileSpGEMM: A tiled algorithm for parallel sparse general matrix-matrix multiplication on GPUs. In Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming...

    2022

  22. [30]

    NVIDIA. [n. d.]. cuSPARSE. . https://developer.nvidia.cn/cusparse

  23. [31]

    Nvidia. [n. d.]. Warp level matrix multiply-accumulate in- structions. . https://docs.nvidia.com/cuda/parallel-thread- execution/index.html#matrix-shape

  24. [32]

    NVIDIA. 2020. Exploiting NVIDIA Ampere Structured Sparsity with cuSPARSELt . https://developer.nvidia.com/blog/exploiting-ampere- structured-sparsity-with-cusparselt/

    2020

  25. [33]

    NVIDIA. 2023. Matrix Fragments for mma.m16n8k8 . https://docs.nvidia.com/cuda/parallel-thread- execution/index.html#matrix-fragments-for-mma-m16n8k8

    2023

  26. [34]

    Meng Pang, Xiang Fei, Peng Qu, Youhui Zhang, and Zhaolin Li. 2024. A Row Decomposition-based Approach for Sparse Matrix Multiplication on GPUs. In PPoPP24: Proceedings of the 29th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming . 377– 389

    2024

  27. [35]

    Hongwu Peng, Xi Xie, Kaustubh Shivdikar, Md Amit Hasan, Jiahui Zhao, Shaoyi Huang, Omer Khan, David Kaeli, and Caiwen Ding. 2024. Maxk-gnn: Extremely fast gpu kernel design for accelerating graph neural networks training. In Proceedings of the 29th ACM International Conference...

    2024

  28. [36]

    Yinhua Piao, Sangseon Lee, Dohoon Lee, and Sun Kim. 2022. Sparse structure learning via graph neural networks for inductive document classification. In Proceedings of the AAAI Conference on Artificial Intel- ligence(AAAI), Vol. 36. 11165–11173

    2022

  29. [37]

    Jeff Pool, Abhishek Sawarkar, and Jay Rodge. 2021. Accelerating Inference with Sparsity Using the NVIDIA Ampere Architecture and NVIDIA TensorRT

    2021

  30. [38]

    Alessandra Sala, Haitao Zheng, Ben Y Zhao, Sabrina Gaito, and Gian Paolo Rossi. 2010. Brief announcement: revisiting the power-law degree distribution for social graph analysis. In Proceedings of the 29th ACM SIGACT-SIGOPS symposium on Principles of distributed computing. 400–401

    2010

  31. [39]

    Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model.IEEE transactions on neural networks 20, 1 (2008), 61–80

    2008

  32. [40]

    Kiran K Thekumparampil, Chong Wang, Sewoong Oh, and Li-Jia Li

  33. [41]

    Tomasz Tylenda, Ralitsa Angelova, and Srikanta Bedathur. 2009. To- wards time-aware link prediction in evolving social networks. In Pro- ceedings of the 3rd workshop on social network mining and analysis . 1–10

    2009

  34. [42]

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention net- works. arXiv preprint arXiv:1710.10903 (2017)

    2017 arXiv

  35. [43]

    Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. 2019. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 (2019)

    2019 arXiv

  36. [44]

    2021.{GNNAdvisor}: An adaptive and efficient runtime system for{GNN} acceleration on{GPUs}

    Yuke Wang, Boyuan Feng, Gushu Li, Shuangchen Li, Lei Deng, Yuan Xie, and Yufei Ding. 2021.{GNNAdvisor}: An adaptive and efficient runtime system for{GNN} acceleration on{GPUs}. In 15th USENIX symposium on operating systems design and implementation (OSDI)

    2021

  37. [45]

    2023.{TC-GNN}: Bridging Sparse{GNN} Computation and Dense Tensor Cores on{GPUs}

    Yuke Wang, Boyuan Feng, Zheng Wang, Guyue Huang, and Yufei Ding. 2023.{TC-GNN}: Bridging Sparse{GNN} Computation and Dense Tensor Cores on{GPUs}. In 2023 USENIX Annual Technical Conference (USENIX ATC)

    2023

  38. [46]

    Wenchao Wu, Xuanhua Shi, Ligang He, and Hai Jin. 2023. Tur- boMGNN: Improving Concurrent GNN Training Tasks on GPU With Fine-Grained Kernel Fusion. IEEE Transactions on Parallel and Dis- tributed Systems (TPDS) (2023)

    2023

  39. [47]

    Orestis Zachariadis, Nitin Satpute, Juan Gómez-Luna, and Joaquín Olivares. 2020. Accelerating sparse matrix–matrix multiplication with GPU Tensor Cores. Computers & Electrical Engineering 88 (2020), 106848

    2020

  40. [48]

    Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kan- nan, and Viktor Prasanna. 2019. Graphsaint: Graph sampling based 12 PPoPP’25, 2025, Las Vegas, NV, USA inductive learning method. arXiv preprint arXiv:1907.04931 (2019). 13

    2019 arXiv

  41. [2018]

    arXiv preprint arXiv:1803.03735 (2018)

    Attention-based graph neural network for semi-supervised learning. arXiv preprint arXiv:1803.03735 (2018)

    2018 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.