Pith. sign in

REVIEW 3 major objections 4 minor 29 references

DF-GNN: Dynamic Fusion Framework for Attention Graph Neural Networks on GPUs

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fused GPU kernels with adaptive thread scheduling speed attention GNNs up to 7x

desk verdict DF-GNN delivers real, consistent speedups for attention GNN training, but the runtime SMMF/PMF selection rule is underspecified enough that the central dynamic-selection claim is not reproducible as written. read the letter →

arxiv 2411.16127 v1 pith:IXSAUHZY submitted 2024-11-25 cs.LG cs.PF

Jiahui Liu , Zhenkun Cai , Zhiyong Chen , Minjie Wang This is my paper · ORCID
classification cs.LGcs.PF
keywords attentiongraphneuralnetworkskernelfusionGPUcomputingthreadschedulingsharedmemorySDDMMSpMMdynamic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Attention graph neural networks (GAT, AGNN, graph transformers) compute each layer in three steps: an attention score per edge, a normalization over each node's neighbors, and a weighted aggregation of neighbor features. The paper argues that these models train slowly on GPUs because existing fused kernels lock all three steps into one fixed thread-mapping scheme, even though the steps have different ideal layouts. DF-GNN's proposed fix is dynamic bi-level thread scheduling, which lets each step choose its own thread organization across and within thread blocks and picks between two fusion modes depending on whether the graph has super nodes. If the claim is right, attention-GNN training on a single GPU becomes about twice as fast end-to-end, with kernel-level speedups of 1.92x to 7.00x over a non-fused sparse baseline.

What carries the argument

Dynamic Bi-level Thread Scheduling (BTS) is the central mechanism: a fused kernel in which inter-block scheduling (node parallel vs edge parallel) and intra-block scheduling (feature parallel, warp-balanced, or flattened edge-wise) are chosen separately for each operation, with thread reshuffling between stages. Three tailored variants carry the argument: warp-balanced SDDMM spreads a block's edges evenly across warps; redundancy-free Softmax assigns each warp a distinct row and caches normalized weights in shared memory; and vectorized SpMM combines vectorized loads with a shared-memory accumulator. The framework's runtime heuristic selects between SMMF and PMF using the rule that PMF is used only when a node's degree reaches shared-memory capacity divided by bytes per feature and the attention operator is a dot product.

What would settle it

Construct a graph whose maximum degree sits just below and just above the shared-memory threshold with dot-product attention, and time the same layer with SMMF forced and with PMF forced; if the heuristic-selected mode is the slower one on either side of the boundary, the selection rule is wrong in a case the paper does not test.

Watch

Extended reading notes

Core claim

The central claim is that no single thread-scheduling policy is optimal for the whole attention-GNN layer, because the SDDMM step is edge-independent and prefers edge-parallel work distribution, while Softmax and SpMM are row-wise and prefer node-parallel organization. DF-GNN therefore replaces the fixed feature-parallel fusion strategy with dynamic bi-level thread scheduling, rescheduling threads inside each block between operations, and it introduces a runtime choice between two fusion modes: SMMF fuses SDDMM-Softmax-SpMM into one node-parallel kernel with shared-memory intermediates, while PMF uses an edge-parallel SDDMM and fuses only Softmax with SpMM, for graphs whose super nodes would overflow shared memory. The paper reports that this consistently beats existing GNN kernel optimizations on batch and full graphs, with kernel speedups of 1.92x–7.00x against the non-fused sparse baseline and a 2.16x average end-to-end training speedup, and that the same scheduling accelerates the backward pass.

Load-bearing premise

The load-bearing premise is that one static rule — pick PMF only when a node's degree reaches shared-memory capacity divided by bytes per feature and the attention operator is a dot product — always identifies the faster fusion mode; if that heuristic misclassifies a graph, DF-GNN may run the slower mode and the reported average speedups shrink.

Editorial extensions

If this is right

  • On graphs without super nodes, the paper predicts SMMF is optimal: full fusion into a single kernel with shared-memory intermediates maximizes reuse and minimizes kernel launches.
  • On graphs with super nodes and dot-product attention, PMF is predicted to win by avoiding shared-memory overflow and workload imbalance in SDDMM.
  • The backward pass of the attention layer follows the same SDDMM-Softmax-SpMM pattern, so backward fusion contributes to the end-to-end speedup and not just the forward pass.
  • The reported speedups grow with graph degree and with small graph size: high-degree batch graphs gain from warp-level balance, while small full graphs gain most from eliminating kernel launch overhead.
  • The design carries over to at least two GPU architectures and a range of feature dimensions and batch sizes in the ablations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The bi-level scheduling idea is likely applicable to other fused irregular kernels beyond attention GNNs, such as sparse transformer attention with variable sequence lengths; the paper does not test this transfer.
  • The SMMF/PMF heuristic could be replaced by a per-graph cost observation, e.g., a short timing probe of the SDDMM step, which would remove the dependence on a static degree threshold.
  • The 2.16x end-to-end average is for single-GPU training; the paper's claim that the kernels port to distributed training would need communication costs to remain hidden, which is unlikely on small graphs.
  • If the open-sourced kernels are used as a drop-in backend, the practical speedup for a given model will depend on whether numerical results match existing libraries on attention variants beyond GAT, AGNN, and graph transformers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. DF-GNN is a GPU kernel-fusion framework for attention GNNs (GAT, AGNN, and Graph Transformer). It fuses the SDDMM, softmax, and SpMM operators into a single kernel under the SMMF mode, and splits off SDDMM into an edge-parallel kernel under the PMF mode for graphs with super nodes and dot-product attention. The core design is a dynamic bi-level thread scheduler with warp-balanced SDDMM, redundancy-free softmax, and vectorized SpMM, extended to the backward pass and exposed through PyTorch-compatible APIs. The evaluation on batch and full graphs on A10G and H100 reports large kernel-level speedups (up to 7.0x) and end-to-end training speedups (average 2.16x) over DGL, PyG, dgNN, and cuGraph, with ablations attributing the gains to the three scheduling optimizations.

Significance. If the reported numbers are reproducible, this is a practically useful systems contribution: it targets a real performance gap in AT-GNN training on a single GPU, integrates cleanly with PyTorch, is open source, and extends fusion to the backward pass more systematically than many prior GNN kernel-fusion works. The paper's strengths are its concrete decomposition of the AT-GNN workload into SDDMM/softmax/SpMM, the explicit bi-level scheduling design, and the evaluation across two GPU architectures. The main reservations are that the dynamic SMMF/PMF selection rule is not specified precisely enough to reproduce the headline 'dynamic' advantage, and the empirical claims are reported without variance or a numerical-equivalence protocol. Both are addressable in revision, and the central idea is credible.

major comments (3)
  1. [Section 4.2] The SMMF/PMF selection rule is not specified at a reproducible level. The text states that PMF is used when 'max_v(N(v)) >= Shared memory capacity / Bytes of feature data type' and the SDDMM is a dot product, but the denominator is ambiguous. If 'Bytes of feature data type' means sizeof(float)=4 bytes, then on the A10G the threshold is about 12,288 or 25,600 edges depending on whether the default 48KB or the maximum opt-in shared memory is used; under either value, Ogbg-ppa (max degree 3,241) and Protein (max degree 7,750) would not trigger PMF, contradicting Section 5.2's statement that PMF is used on these super-node graphs. If the denominator instead means d*sizeof(float)=512 bytes for d=128, the threshold drops to about 96-200, which would make Cora and Pubmed (max degrees 168 and 171) trigger PMF under the lower shared-memory assumption, although they are presented as ordinary SMMF full-graph cases. The manuscript must state the exact formula, the numeric shared-memory capacity used, and which datasets satisfy the rule; because the reported average speedups depend on this runtime choice, the central dynamic-selection claim is not anchored as written.
  2. [Section 5.1 and Section 5.2] All speedup numbers are reported as single-point normalized values without trial counts, confidence intervals, or error bars (e.g., Figures 9 and 10, and Table 2). Without run-to-run variance, the phrases 'consistently outperforms' and 'average speedup of 2.16x' are not statistically supported. In addition, the only correctness statement is that DF-GNN kernels are 'configured to produce results consistent with the baselines' (Section 5.1); there is no numerical-equivalence check, such as max absolute difference in attention scores, output logits, or gradients, and no model-accuracy comparison. Since the claimed speedups are only meaningful if the fused kernels compute the same function as the DGL/PyG baselines, the paper should report repetition counts with variance and a concrete equivalence protocol.
  3. [Appendix C, Figure 12] The ablation that compares SMMF and PMF is performed only on MNIST, CIFAR10, Reddit, and Protein for GT and GAT. It does not include Ogbg-ppa, the dataset whose max degree lies closest to the plausible threshold, nor does it sweep the threshold to show where PMF becomes beneficial. As a result, the Section 4.2 selection rule lacks direct empirical support at the decision boundary. The authors should either add a threshold sweep or state explicitly that the rule is a hand-tuned heuristic whose validity is demonstrated only on the tested graphs, and then discuss how sensitive the reported average speedups are to the choice of threshold.
minor comments (4)
  1. [Section 5.2] There are small language errors: 'Pumbed' should be 'Pubmed', and 'we also analysis the bandwidth utilization' should be 'we also analyze the bandwidth utilization'.
  2. [Table 2] The '-' entries for Preprocess on full graphs should be explained: clarify whether preprocessing is not measured, not needed, or already included in the forward/backward times.
  3. [Figure 12] The caption and axis labels should clarify which bars correspond to SMMF and which to PMF, and why only these four datasets are shown; currently the figure mixes GT and GAT results without a clear legend for the two bar styles beyond the header text.
  4. [Section 4.2] The notation 'Bytes of feature data type' should be replaced with an explicit formula, e.g., 'sizeof(feature_dtype)' or 'd * sizeof(feature_dtype)', and 'Shared memory capacity' should name the exact per-block capacity used on each GPU model.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; DF-GNN's claims are measured speedups against external baselines, not derivations from fitted or self-referential inputs.

full rationale

DF-GNN is a systems-optimization paper whose central claims are empirical kernel-time and end-to-end speedups measured against external baselines (DGL sparse, PyG, dgNN, cuGraph) on stated datasets, models, and hardware. The SMMF/PMF selection rule in Section 4.2 is a design heuristic based on graph degree and SDDMM operator type; it is not a parameter fitted to the reported speedups and then renamed as a prediction. No equation defines a claimed result in terms of the measured data. The backward-pass formulations in Appendix B are standard matrix derivatives and are not used to establish the framework's performance. Citations to DGL and other GNN systems are used as baselines or background, not as authority for DF-GNN's correctness; the one author overlap with the DGL baseline does not make the comparison circular because DGL's runtimes are independently measured. Reproducibility concerns about the exact threshold interpretation in Section 4.2 would be correctness or engineering issues, not circularity. Therefore no load-bearing step reduces to its own inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the AT-GNN computation pattern (SDDMM-Softmax-SpMM), standard CUDA memory and thread models, and the heuristic SMMF/PMF selection rule. None of these are fitted to data; they are domain assumptions.

assumptions (4)
  • domain assumption AT-GNN layers reduce to the vectorized SDDMM, Softmax, SpMM pipeline in Eq. (1).
    Introduced in Section 2.1; if a target AT-GNN does not follow this pattern (e.g., global attention or edge-conditioned softmax), DF-GNN's fused kernels do not apply.
  • domain assumption GPU kernel launch overhead and global memory traffic dominate AT-GNN kernel cost.
    Stated in Sections 1 and 4.2; the claimed speedups rely on this being true across evaluated workloads.
  • ad hoc to paper The condition max_v deg(v) >= shared_memory_capacity / bytes_per_feature with dot-SDDMM identifies the bottleneck shift to SDDMM.
    Defined in Section 4.2 as the selection rule between SMMF and PMF; it is a heuristic threshold, not derived from a performance model, and is load-bearing for the dynamic choice.
  • domain assumption The benchmark models (GAT, AGNN, GT) and datasets are representative of the AT-GNN family.
    Choice of Section 5.1; the abstract's AT-GNN family claim is broader than the evaluated set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DF-GNN: Dynamic Fusion Framework for Attention Graph Neural Networks on GPUs." pith.science (2026). https://pith.science/paper/IXSAUHZY

@misc{pith2026241116127,
  author       = {Pith},
  title        = {Pith review of: DF-GNN: Dynamic Fusion Framework for Attention Graph Neural Networks on GPUs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IXSAUHZY}},
  note         = {Machine review of arXiv:2411.16127}
}
abstract

Attention Graph Neural Networks (AT-GNNs), such as GAT and Graph Transformer, have demonstrated superior performance compared to other GNNs. However, existing GNN systems struggle to efficiently train AT-GNNs on GPUs due to their intricate computation patterns. The execution of AT-GNN operations without kernel fusion results in heavy data movement and significant kernel launch overhead, while fixed thread scheduling in existing GNN kernel fusion strategies leads to sub-optimal performance, redundant computation and unbalanced workload. To address these challenges, we propose a dynamic kernel fusion framework, DF-GNN, for the AT-GNN family. DF-GNN introduces a dynamic bi-level thread scheduling strategy, enabling flexible adjustments to thread scheduling while retaining the benefits of shared memory within the fused kernel. DF-GNN tailors specific thread scheduling for operations in AT-GNNs and considers the performance bottleneck shift caused by the presence of super nodes. Additionally, DF-GNN is integrated with the PyTorch framework for high programmability. Evaluations across diverse GNN models and multiple datasets reveal that DF-GNN surpasses existing GNN kernel optimization works like cuGraph and dgNN, with speedups up to $7.0\times$ over the state-of-the-art non-fusion DGL sparse library. Moreover, it achieves an average speedup of $2.16\times$ in end-to-end training compared to the popular GNN computing framework DGL.

Figures

Figures reproduced from arXiv: 2411.16127 by the authors.

Figure 1
Figure 1. Workload partition in node and edge parallel. = warp 0 warp 1 0 1 2 3 Block Processed by Block 0 𝐐 𝐀 𝐊 𝐓 ∗ 𝐕 𝐎 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. DF-GNN framework. SDDMM Softmax SpMM SMEM SMEM SDDMM Softmax SpMM GMEM SMEM Kernel 0 Kernel 0 Kernel 1 SMMF PMF edge parallel node parallel [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Dynamic Bi-level Thread Scheduling in GT convolution. 0 1 2 3 4 5 6 7 Block 0 0 1 2 3 Block 1 0 1 2 3 (a) (b) Processed by Block 0 Processed by Thread group 0 in Block 0 smem of Block 0 smem of Block 0 [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figures from the paper (8 more)
Figure 6
Figure 6. Figure 6: (a) Feature parallel SDDMM. (b) Warp-balanced SDDMM in DF-GNN. warp 0 reduction warp 0 𝑧0 reduction (max) 𝑧0 (sum) warp 1 reduction warp 1 𝑧0 reduction (max) 𝑧0 (sum) 𝐒𝟎 warp 0 reduction warp 0 reduction 𝑧1 warp 1 (max) reduction warp 1 reduction 𝐒𝟎 𝐒𝟏 𝑧2 warp 2 (max) …
Figure 8
Figure 8. Figure 8: Kernels in GT backward pass. (a) Without any fusion. (b) After DF-GNN fusion. 4.3 Backward Pass Kernel Fusion Optimization In DF-GNN, we extend the kernel fusion optimization to the backward pass. This extension is motivated by the observation that the backward pass of…
Figure 9
Figure 9. Figure 9: Normalized kernel time speedup and bandwidth utilization on batch graph datasets. (a) (b) PyG dgNN cuGraph DF-GNN dgNN cuGraph DF-GNN 2.03 x1.46 x 1.36 x 2.56 x1.68 x 1.62 x 2.68 x 2.20 x 2.27 x 0 x 1 x 2 x 3 x 4 x 5 x PP RD PR Normalized Speedup (to DGL sparse) 1.65 x…
Figure 10
Figure 10. Figure 10: Normalized kernel time speedup on (a) Full graphs and (b) Full graphs with super nodes. employed for AT-GNNs, including datasets from Benchmarking GNNs [28] (PATTERN, CLUSTER, MNIST, CIFAR10) and Long-Range Graph Benchmark [29] (PascalVOC-SP, COCO-SP). Running AT-GNNs…
Figure 11
Figure 11. Figure 11: Relative performance improvement brought by proposed optimizations on GT. Proposed optimization methods [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Normalized kernel time speedup of SMMF and PMF in DF-GNN. SMMF and PMF [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Normalized speedup of GT against different feature dimension (x-axis). 4.96 x 4.18 x 4.23 x 4.16 x 3.91 x 4.11 x 0 x 1 x 2 x 3 x 4 x 5 x 6 x 64 128 256 512 1024 2048 Normalized Speedup (to DGL sparse) dgNN cuGraph DF-GNN [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 15
Figure 15. Figure 15: Normalized kernel time speedup on NVIDIA H100 80GB. Evaluation on Different GPUs. The DF-GNN exhibits robust generalization capabilities, making it applicable across various GPU architectures [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 14 canonical work pages

  1. [1]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  2. [2]

    Graph attention networks

    Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks. stat, 1050(20):10–48550, 2017

    2017

  3. [3]

    How attentive are graph attention networks? arXiv preprint arXiv:2105.14491, 2021

    Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2105.14491, 2021

    arXiv 2021

  4. [4]

    A generalization of transformer networks to graphs

    Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699, 2020

    arXiv 2012

  5. [5]

    Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877–28888, 2021

    Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877–28888, 2021

    work page 2021

  6. [6]

    Recipe for a general, powerful, scalable graph transformer

    Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35:14501–14515, 2022

    work page 2022

  7. [7]

    Attention-based graph neural network for semi-supervised learning

    Kiran K Thekumparampil, Chong Wang, Sewoong Oh, and Li-Jia Li. Attention-based graph neural network for semi-supervised learning. arXiv preprint arXiv:1803.03735, 2018

    arXiv 2018

  8. [8]

    Semi-supervised classification with graph convolutional networks

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016

    arXiv 2016

Show all 29 references
  1. [9]

    How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018

    2018 arXiv

  2. [10]

    Link prediction based on graph neural networks

    Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems, 31, 2018

    2018

  3. [11]

    Neural bellman-ford networks: A general graph neural network framework for link prediction

    Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 34:29476–29490, 2021

    2021

  4. [12]

    A fair comparison of graph neural networks for graph classification

    Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. A fair comparison of graph neural networks for graph classification. arXiv preprint arXiv:1912.09893, 2019

    1912 arXiv

  5. [13]

    Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization.arXiv preprint arXiv:1908.01000, 2019

    Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization.arXiv preprint arXiv:1908.01000, 2019

    1908 arXiv

  6. [14]

    Deep graph library: Towards efficient and scalable deep learning on graphs

    Minjie Yu Wang. Deep graph library: Towards efficient and scalable deep learning on graphs. In ICLR workshop on representation learning on graphs and manifolds, 2019

    2019

  7. [15]

    Fast graph representation learning with pytorch geometric

    Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019

    1903 arXiv

  8. [16]

    Neural message passing for quantum chemistry

    Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning , pages 1263–1272. PMLR, 2017

    2017

  9. [17]

    Seastar: vertex-centric programming for graph neural networks

    Yidi Wu, Kaihao Ma, Zhenkun Cai, Tatiana Jin, Boyang Li, Chenguang Zheng, James Cheng, and Fan Yu. Seastar: vertex-centric programming for graph neural networks. In Proceedings of the Sixteenth European Conference on Computer Systems, pages 359–375, 2021

    2021

  10. [18]

    Understanding gnn computational graph: A coordinated computation, io, and memory perspective

    Hengrui Zhang, Zhongming Yu, Guohao Dai, Guyue Huang, Yufei Ding, Yuan Xie, and Yu Wang. Understanding gnn computational graph: A coordinated computation, io, and memory perspective. Proceedings of Machine Learning and Systems, pages 467–484, 2022

    2022

  11. [19]

    Tlpgnn: A lightweight two-level parallelism paradigm for graph neural network computation on gpu

    Qiang Fu, Yuede Ji, and H Howie Huang. Tlpgnn: A lightweight two-level parallelism paradigm for graph neural network computation on gpu. InProceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing, pages 122–134, 2022

    2022

  12. [20]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32,...

    2019

  13. [21]

    Flashattention: Fast and memory-efficient exact attention with io-awareness.Advances in Neural Information Processing Systems, 35:16344–16359, 2022

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness.Advances in Neural Information Processing Systems, 35:16344–16359, 2022

    2022

  14. [22]

    Fusedmm: A unified sddmm- spmm kernel for graph embedding and graph neural networks

    Md Khaledur Rahman, Majedul Haque Sujon, and Ariful Azad. Fusedmm: A unified sddmm- spmm kernel for graph embedding and graph neural networks. In 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 256–266. IEEE, 2021

    2021

  15. [23]

    Graphiler: Optimizing graph neural networks with message passing data flow graph

    Zhiqiang Xie, Minjie Wang, Zihao Ye, Zheng Zhang, and Rui Fan. Graphiler: Optimizing graph neural networks with message passing data flow graph. Proceedings of Machine Learning and Systems, 4:515–528, 2022

    2022

  16. [24]

    Featgraph: A flexible and efficient backend for graph neural network systems

    Yuwei Hu, Zihao Ye, Minjie Wang, Jiali Yu, Da Zheng, Mu Li, Zheng Zhang, Zhiru Zhang, and Yida Wang. Featgraph: A flexible and efficient backend for graph neural network systems. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis...

    2020

  17. [25]

    Sparsetir: Composable abstractions for sparse compilation in deep learning

    Zihao Ye, Ruihang Lai, Junru Shao, Tianqi Chen, and Luis Ceze. Sparsetir: Composable abstractions for sparse compilation in deep learning. In Proceedings of the 28th ACM Inter- national Conference on Architectural Support for Programming Languages and Operating Systems, Volume...

    2023

  18. [26]

    Exploiting online locality and reduction parallelism for sampled dense matrix multiplication on gpus

    Zhongming Yu, Guohao Dai, Guyue Huang, Yu Wang, and Huazhong Yang. Exploiting online locality and reduction parallelism for sampled dense matrix multiplication on gpus. In 2021 IEEE 39th International Conference on Computer Design (ICCD), pages 567–574, 2021

    2021

  19. [27]

    Rapids cugraph, 2024

    rapidsai. Rapids cugraph, 2024. URL https://github.com/rapidsai/cugraph

    2024

  20. [28]

    Benchmarking graph neural networks

    Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. Journal of Machine Learning Research, 24(43):1–48, 2023

    2023

  21. [29]

    Long range graph benchmark

    Vijay Prakash Dwivedi, Ladislav Rampášek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long range graph benchmark. Advances in Neural Information Processing Systems, 35:22326–22340, 2022. A Edge Softmax in A T-GNNs Here is the vectorized formulatio...

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.