Pith. sign in

REVIEW 4 major objections 4 minor 35 references

Morphling claims that a GNN-focused code synthesizer can generate backend-specialized training code that runs about 20x faster per epoch on CPUs, 19x faster on GPUs, and 6x faster in distributed settings than mainstream GNN frameworks, whil

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 19:08 UTC pith:IH36KQAA

load-bearing objection Serious GNN-systems engineering with plausible speedups, but the headlining dynamic-sparsity contribution is undone by a one-time static decision, and the evaluation lacks raw data, variance, and artifact. the 4 major comments →

arxiv 2512.01678 v5 pith:IH36KQAA submitted 2025-12-01 cs.LG cs.DCcs.PL

Morphling: Fast, Fused, and Flexible GNN Training at Scale

Anubhab , Rupesh Nasre This is my paper
classification cs.LG cs.DCcs.PL
keywords graph neural networkscode synthesisdomain-specific languagesparsity-aware executionsparse matrix multiplicationperformance portabilitydistributed trainingmemory efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether GNN training can be made much faster without hand-writing kernels per platform, and answers yes. Morphling takes a compact, high-level GNN specification and compiles it into optimized implementations for multicore CPUs, GPUs, and distributed clusters, using a runtime that chooses between sparse and dense execution paths based on measured feature sparsity. Across eleven datasets, the paper reports average per-epoch speedups of about 20x on CPUs, 19x on GPUs, and 6x in distributed settings, with peak speedups to 66x and peak memory reduced up to 15x. If these numbers hold, domain-specific code synthesis is a practical route to high-performance, portable GNN training, especially on graphs whose input features are highly sparse.

Core claim

The paper's central claim is that a single GNN specification, written in a small domain-specific language, can be compiled into backend-specialized training loops that outperform the generic gather-scatter execution of mainstream GNN libraries. At load time, a sparsity-aware engine measures the fraction s of non-zero entries in the input feature matrix; if s exceeds a threshold near 0.80, it selects custom sparse SpMM kernels (CSR for forward, CSC for backward) instead of dense BLAS. Fused kernels avoid materializing edge-sized intermediate tensors, keeping peak memory near O(|V| F). On eleven real-world graphs, the paper reports average per-epoch speedups of 20x on CPUs, 19x on GPUs, and 6x

What carries the argument

The central decision rule is s > 1 − gamma, where gamma is the measured ratio of sustained throughput of the sparse SpMM kernel to the dense GEMM kernel; with gamma ≈ 0.20 the engine sets a load-time threshold tau ≈ 0.80. On the sparse path, a cache-tiled, software-prefetched SpMM kernel with compile-time tile width keeps weights resident in L1; a block-per-row GPU kernel assigns one thread block per output node so warps traverse neighbors in lockstep and accumulate without atomics. For distributed runs, a hierarchical partitioner balances summed vertex degree across ranks, and the runtime overlaps halo exchange and gradient reductions with compute. These mechanisms convert the sparse/dense

Load-bearing premise

The load-bearing premise is that one offline-measured efficiency ratio gamma ≈ 0.20 (threshold tau ≈ 0.80) transfers across all datasets, layers, and backends, and that feature sparsity measured at load time stays representative; if sparsity drops after the first GNN layer, the single load-time dense/sparse choice does little for later layers and the speedups attributed to the sparsity-aware engine do not follow.

What would settle it

A direct test: retrain the same model on a graph whose input features are highly sparse, but with dense features of the same dimension (e.g., random dense vectors). If the reported speedup over mainstream frameworks mostly disappears, the gain comes from the sparse-feature path rather than the custom kernels. Also instrument per-layer feature sparsity during training: if after the first aggregation the fraction of non-zero entries in node features falls below 0.80, then the load-time decision locked the sparse path for layers where it no longer applies.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • GNN workloads with intrinsically sparse features can be trained several times faster on existing CPUs and GPUs without changing model semantics.
  • The fused-kernel design bounds peak memory to roughly O(|V| F) instead of O(|E| F), so graphs that currently exhaust a node's memory can be trained in full rather than subsampled.
  • A single high-level GNN program can replace separate hand-tuned CPU, GPU, and distributed codebases, reducing maintenance burden across heterogeneous hardware.
  • The dense/sparse switching rule can be reused by any system facing a similar kernel-choice problem, using its own measured efficiency ratio.
  • Degree-balanced partitioning offers a template for distributed graph processing beyond GNNs, where load balance can matter more than edge-cut minimization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the sparsity check runs once at load time, the 'dynamic' engine is static with respect to layers; if feature sparsity collapses after the first aggregation, the sparse path mainly benefits the early layer, and the sparsity-attributed speedup is bounded by that layer's runtime share.
  • The largest reported gains occur on datasets with highly sparse features, while a dense-feature benchmark shows only a modest advantage; the result is best read as a claim about sparse-feature workloads.
  • The fixed gamma ≈ 0.20 threshold could be re-measured per backend and per layer at runtime rather than fixed offline, which would make the dense/sparse switch genuinely dynamic and could preserve gains on later, denser layers.
  • In the distributed-motivation section, an empty cross-reference appears where the paper argues that minimizing edge-cuts is a proxy objective; that specific motivation is not supported by text in the manuscript.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. Morphling is a domain-specific code synthesizer that compiles GNN training specifications into backend-specialized C++ code for OpenMP, CUDA, and MPI. The paper's contributions are a GNN DSL on top of StarPlat, a set of architecture-specific kernels (cache-tiled/prefetched CPU SpMM, block-per-row GPU aggregation, and a hierarchical MPI partitioner with pipelined communication), and a sparsity-aware execution engine that dispatches between dense and sparse feature kernels. The evaluation on a 3-layer GCN reports average per-epoch throughput gains of 20.21× over PyG on CPU, 15.48× on GPU (19× in the abstract), and 6.22× in distributed settings, with up to 15.5× lower peak memory and a peak GPU speedup of 66.28×. The central claim is that this synthesis approach achieves large, portable speedups while reducing memory traffic. The paper is clearly written in parts, but the experimental section does not yet substantiate the headline numbers due to internal inconsistencies, missing reproducibility data, and a static sparsity mechanism that contradicts the 'dynamic' description.

Significance. Should the claims hold, Morphling would be a significant systems contribution: it demonstrates that a DSL with backend-specialized primitives can outperform general-purpose frameworks by one to two orders of magnitude across CPU, GPU, and MPI, and it reports a compelling memory-efficiency advantage that enables full-graph training where baselines OOM. The formal cost model for distributed partitioning (Eqs. (8)–(11)) and the explicit Algorithms 1–4 are useful articulations of the design. However, the evidence is currently conditional: the sparsity-aware mechanism is not actually dynamic, the evaluation lacks raw data/error bars/artifact, specialized GNN baselines are absent, and the headline averages are inconsistent across sections. The potential is real, but the manuscript needs substantial experimental revision before the claims can be accepted.

major comments (4)
  1. [§IV.B, Algorithm 1; §III.D(b); §V.C.2] The 'dynamic sparsity-aware execution engine' is not supported as written. Algorithm 1 (Phase 1, INITIALIZE) computes sparsity s from the input X once at load time and fixes MODE for the entire run, while §III.D(b) states that GNN feature sparsity varies across layers and epochs. For the evaluated 3-layer GCN with hidden dimension 32 (§V.B), hidden activations are outputs of dense GEMM+ReLU and will not generally satisfy τ≈0.80. Thus the sparse path chosen from input statistics is applied to dense later-layer inputs, and the NELL 43.52× CPU speedup attributed to sparsity-aware dispatch (§V.C.2) is not established. The threshold is inconsistent (τ≈0.80 in Algorithm 1 vs. 'experimentally tuned thresholds≈0.85' in §V.C.2), and Eq. (5) restates the measured efficiency ratio γ rather than predicting the crossover. Kernel-fusion/layout gains may survive, but this contribution needs a per-layer
  2. [Abstract; §V-A, §V-D; Table II] Headline numbers are internally inconsistent. The abstract reports 19× average GPU speedup and 'eleven real-world datasets'; §V-D reports a 15.48× GPU average (over PyG) and §V-A says 'ten real-world datasets,' with Table II listing exactly ten datasets. The GPU evaluation also mentions AmazonComputers, which is absent from Table II and dataset statistics. Because the paper's central thesis is quantified by these averages, the manuscript must reconcile the abstract, §V, and Table II, and report per-dataset latencies with repetitions and error bars. As written, the reader cannot determine which numbers are authoritative.
  3. [§V-C–V-E; §II] No comparison with specialized GNN systems. GNNAdvisor, DGCL, and FeatGraph are cited in §II as directly relevant systems ([3], [4], [17]), and GNNAdvisor/DGCL are described as attempting to bridge the same gap, yet the evaluation in §V-C–V-E compares only PyG and DGL. The claim of improved over 'state-of-the-art' GNN execution is therefore not established. Additionally, no code artifact, raw measurements, confidence intervals, or run-to-run variability are provided, making the 20×/19×/6× averages impossible to verify. The authors should include at least one specialized baseline or justify its exclusion, and provide a reproducibility artifact with raw data.
  4. [§V-F, Table III; §V-D] Memory comparison uses unequal baselines for AmazonProducts. Table III reports PyG as exceeding 192 GB on the full graph but then lists 140.24 GB from a 75% subsampled graph; §V-D compares GPU throughput against PyG on a 55% subsampled graph. These are not full-data measurements, so the 15.5× memory reduction and the 'effective 9.23×' GPU speedup over PyG are not clean comparisons. Report full-graph baseline measurements or unambiguously label those numbers as extrapolated from subsampled runs.
minor comments (4)
  1. [§III.D(c)] The sentence 'as noted in ,' contains an empty citation; please supply the reference.
  2. [Listing 1; §V.B] Listing 1 demonstrates GraphSAGE (SAGE, 'Max' aggregation), while all experiments use a 3-layer GCN (§V.B). Clarify whether the DSL path for SAGE is actually evaluated or if Listing 1 is only illustrative.
  3. [§IV.C–E; §V] Several tuning constants are introduced without sensitivity analysis (T=32, prefetch distance D=8, OpenMP chunk size 64, METIS imbalance tolerances ϵ=1.03/1.20). Please report whether the conclusions are robust to these choices.
  4. [Global; Table II] Notation and naming should be unified: 'Pyg' appears in Figure 8 caption/Table III while 'PyG' is used elsewhere; also 'AmazonComputers' in §V-D should either be added to Table II or corrected.

Circularity Check

0 steps flagged

No circular derivation; the sparse/dense crossover is a measured/tuned engineering heuristic and the headline speedups are external benchmark measurements.

full rationale

The paper's only analytical decision model—the sparse/dense crossover in §IV.B.d—derives s > 1 − γ from T = W/η and W_sparse = (1−s)W_dense. The parameter γ = η_sparse/η_dense is explicitly an offline-measured hardware heuristic: the paper says the value γ≈0.20 'is obtained from offline microbenchmarks and serves as a default heuristic rather than a per-run calibration,' and §V.C.2 refers to 'experimentally tuned thresholds≈0.85.' Thus the crossover is an engineering calibration, not a first-principles prediction, and it is not used to generate the reported throughput numbers—those are measured end-to-end against PyG and DGL. The one self-citation (StarPlat [12], whose author list includes Rupesh Nasre) is used as the compiler front-end substrate, but no load-bearing claim reduces to it; the evaluation is external. There are real rigor concerns that are not circularity: Algorithm 1 performs the sparsity decision once at load time on the input features, while §III.D(b) motivates 'dynamic sparsity that varies across datasets, layers, and epochs,' so the attribution of NELL's 43.52× CPU speedup specifically to 'sparsity-aware dispatch' is not fully established. There is also a missing reference in §III.D(c) ('as noted in ,'). These are consistency/completeness defects, not circular reductions. Consequently, no circular step is identified.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claims rest on several hand-set constants (tau, gamma, prefetch depth, tile width, partition tolerances) and on the assumptions that feature sparsity is stable across the training loop and that the StarPlat compiler foundation is correct. These are engineering choices rather than derived results; no new physical or mathematical entities are introduced.

free parameters (6)
  • Sparsity threshold tau = 0.80 (Algorithm 1), 0.85 (Section V-C)
    Hand-set or experimentally tuned crossover for dense vs sparse path selection; not predicted per dataset or backend.
  • Efficiency ratio gamma = ~0.20
    Measured offline throughput ratio of SpMM to GEMM on the testbed; directly determines tau and is assumed constant across all datasets and backends.
  • Prefetch lookahead D = 8
    Default software-prefetch distance in the CPU SpMM kernel; a tuning choice, not derived.
  • Feature tile width T = 32 (FP32)
    Compile-time tile width chosen to match 128-byte AVX-512 cache-line granularity; affects CPU kernel performance.
  • METIS imbalance tolerances epsilon = 1.03, 1.20
    Hand-specified constraint relaxation levels in the hierarchical partitioner.
  • OpenMP dynamic schedule chunk size = 64
    Chunk size for dynamic scheduling of node iterations; a tuning choice for power-law degree distributions.
axioms (5)
  • domain assumption Sparse path is beneficial iff feature sparsity s > 1 - gamma, with gamma = eta_sparse/eta_dense.
    Used in Section IV-B Eqs. (2)-(5). Assumes work scales as (1-s)NFH and that gamma is stable across microarchitectures; ignores format-conversion overhead and that activations become dense after the first layer.
  • ad hoc to paper A single efficiency ratio gamma ~ 0.20 measured offline transfers to all evaluated datasets, layers, and backends.
    Algorithm 1 sets tau ~ 0.80 from one offline microbenchmark; Section V-C says the threshold was experimentally tuned to ~0.85. Load-bearing for the sparsity-aware dispatch contribution.
  • domain assumption The StarPlat compiler frontend and IR are correct and adequate as the static foundation for GNN training code synthesis.
    Section IV-A rebuilds on StarPlat; the paper does not formally verify generated code, so any StarPlat code-generation bug inherits to Morphling.
  • domain assumption Feature sparsity is static across training: selecting the path once at load remains optimal for all epochs and layers.
    Algorithm 1 Phase 1 decides the mode before the training loop, while Section III-B says feature sparsity varies across layers and epochs. This is load-bearing and questionable for multi-layer GNNs.
  • domain assumption GNN per-rank wall-clock cost is proportional to sum of local degrees times feature dimension.
    Eqs. (8)-(9) model T_comp as proportional to sum deg(v)*F, ignoring cache effects, communication overlap, and compute-communication interactions; this motivates the entire distributed partitioner design.

pith-pipeline@v1.3.0-alltime-deepseek · 18224 in / 17319 out tokens · 184924 ms · 2026-08-03T19:08:04.149764+00:00 · methodology

0 comments
read the original abstract

Graph Neural Networks (GNNs) present a fundamental hardware challenge by fusing irregular, memory-bound graph traversals with regular, compute-intensive dense matrix operations. While frameworks such as PyTorch Geometric (PyG) and Deep Graph Library (DGL) prioritize high-level usability, they fail to address these divergent execution characteristics. As a result, they rely on generic kernels that suffer from poor cache locality, excessive memory movement, and substantial intermediate allocations. To address these limitations, we present Morphling, a domain-specific code synthesizer designed to bridge this gap. Morphling compiles high-level GNN specifications into portable, backend-specialized implementations targeting OpenMP, CUDA, and MPI. It achieves this by instantiating a library of optimized, architecture-aware primitives tailored to each execution environment. Morphling also incorporates a runtime sparsity-aware execution engine that dynamically selects dense or sparse execution paths using input feature statistics, reducing unnecessary computation on zero-valued entries. We evaluate Morphling on eleven real-world datasets spanning diverse graph structures, feature dimensionalities, and sparsity regimes. Morphling improves per-epoch training throughput by an average of 20X on CPUs, 19X on GPUs, and 6X in distributed settings over PyG and DGL, with peak speedups reaching 66X. Morphling's memory-efficient layouts further reduce peak memory consumption by up to 15X, enabling large-scale GNN training on commodity hardware. These findings demonstrate that specialized, architecture-aware code synthesis provides an effective and scalable path toward high-performance GNN execution across diverse parallel and distributed platforms.

Figures

Figures reproduced from arXiv: 2512.01678 by Anubhab, Rupesh Nasre.

Figure 1
Figure 1. Figure 1: Code Generation framework of Morphling B. Sparsity-Aware Execution Engine Most GNN systems exploit graph sparsity but treat node features as dense tensors. This assumption is suboptimal for real-world workloads where features are intrinsically sparse (e.g., text embeddings). Morphling implements a dual-path execution engine that inspects feature statistics at load time and selects the optimal execution str… view at source ↗
Figure 2
Figure 2. Figure 2: Morphling speedup over PyG and DGL on CPU [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Per-epoch training time of Morphling, PyG, and DGL [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Morphling speedup over PyG and DGL on GPU [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-epoch training time of Morphling, PyG, and DGL [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: Peak memory comparison between Pyg, DGL, and [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 2 canonical work pages

  1. [1]

    Fast Graph Representation Learning with PyTorch Geometric,

    M. Fey and J. E. Lenssen, “Fast Graph Representation Learning with PyTorch Geometric,” inProc. Workshop on Representation Learning on Graphs and Manifolds at ICLR, 2019. https://arxiv.org/abs/1903.02428

    Pith/arXiv arXiv 2019

  2. [2]

    Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks,

    M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X. Song, J. Zhou, C. Ma, L. Yu, Y . Gai, T. Xiao, T. He, G. Karypis, J. Li, and Z. Zhang, “Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks,” inarXiv preprint arXiv:1909.01315, Sept. 2019. https: //arxiv.org/abs/1909.01315

    Pith/arXiv arXiv 1909

  3. [3]

    GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs,

    Y . Wang, B. Feng, G. Li, S. Li, L. Deng, Y . Xie, and Y . Ding, “GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs,” inProc. 15th USENIX Symposium on Oper- ating Systems Design and Implementation (OSDI), 2021, pp. 515–531. https://www.usenix.org/conference/osdi21/presentation/wang-yuke 13

    2021

  4. [4]

    Un- derstanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective,

    H. Zhang, Z. Yu, G. Dai, G. Huang, Y . Ding, Y . Xie, and Y . Wang, “Un- derstanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective,” inProc. 5th Machine Learning and Systems Conference (MLSys), 2022, pp. 467–484. https://arxiv.org/abs/2110.09524

    Pith/arXiv arXiv 2022

  5. [5]

    AliGraph: A Comprehensive Graph Neural Network Platform,

    R. Zhu, K. Zhao, H. Yang, W. Lin, C. Zhou, B. Ai, Y . Li, and J. Zhou, “AliGraph: A Comprehensive Graph Neural Network Platform,” inProc. The 12th International Conference on Very Large Data Bases (VLDB), vol. 12, no. 12, 2019, pp. 2094–2105. doi:10.14778/3352063.3352127. https://doi.org/10.14778/3352063.3352127

    arXiv 2019

  6. [6]

    Euler: A System for Large-Scale Graph Analytics,

    F. Huang, W. Zhang, K. Zhai, J. Zhu, R. Chen, W. Wu, and X. Wan, “Euler: A System for Large-Scale Graph Analytics,” inProc. 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), 2018, pp. 316–324. doi:10.1145/3219819.3220063. https://doi.org/10.1145/3219819.3220063

    arXiv 2018

  7. [7]

    DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs,

    D. Zheng, C. Ma, M. Wang, J. Zhou, Q. Su, X. Song, Q. Gan, Z. Zhang, and G. Karypis, “DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs,” in10th IEEE/ACM Workshop on Irregular Applications: Architectures and Algorithms (IA 3), 2020, pp. 36–44. https://arxiv.org/abs/2010.05337

    Pith/arXiv arXiv 2020

  8. [8]

    GraphIt: A High-Performance DSL for Graph An- alytics,

    Y . Zhang, A. Brahmakshatriya, X. Chen, L. Dhulipala, S. Kamil, S. Ama- rasinghe, and J. Shun, “GraphIt: A High-Performance DSL for Graph An- alytics,” inProc. 33rd ACM SIGPLAN Conference on Programming Lan- guage Design and Implementation (PLDI), 2018. doi:10.1145/3276491. https://doi.org/10.1145/3276491

    doi:10.1145/3276491 2018

  9. [9]

    Ligra: A Lightweight Graph Pro- cessing Framework for Shared Memory,

    J. Shun and G. E. Blelloch, “Ligra: A Lightweight Graph Pro- cessing Framework for Shared Memory,” inProc. 18th ACM SIG- PLAN Symposium on Principles and Practice of Parallel Program- ming (PPoPP), Feb. 23-27 2013, Shenzhen, China, pp. 135–146. doi:10.1145/2517327.2442530

    arXiv 2013

  10. [10]

    The Galois System: Optimistic Parallelization of Irregular Programs,

    M. Kulkarni, K. Pingali, B. Walter, K. Tung, and J. Hollingsworth, “The Galois System: Optimistic Parallelization of Irregular Programs,” in Proc. 20th IEEE International Parallel & Distributed Processing Sympo- sium (IPDPS), 2008, pp. 1-12. https://ecommons.cornell.edu/bitstreams/ 707e6678-bb5b-4dc6-b590-1b58e409d802/download

    2008

  11. [11]

    Gemini: A Computation- Centric Distributed Graph Processing System,

    X. Zhu, W. Chen, W. Zheng, and X. Ma, “Gemini: A Computation- Centric Distributed Graph Processing System,” inProc. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), Savannah, GA, USA, Nov. 2016, pp. 301–316. https://www.usenix.org/ conference/osdi16/technical-sessions/presentation/zhu

    2016

  12. [12]

    StarPlat: A Versatile DSL for Graph Analytics,

    N. Behera, A. Kumar, E. Rajadurai T., S. Nitish, R. Pandian M., and R. Nasre, “StarPlat: A Versatile DSL for Graph Analytics,”Journal of Parallel and Distributed Computing, vol. 183, 2024, Article 104967. doi:10.1016/j.jpdc.2024.104967. https://arxiv.org/abs/2305.03317

    arXiv 2024

  13. [13]

    TensorFlow: A System for Large- Scale Machine Learning,

    M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. Murray, B. Steiner, P. Tucker, V . Vasudevan, P. Warden, M. Wicke, Y . Yu, and X. Zheng, “TensorFlow: A System for Large- Scale Machine Learning,” inProc. 12th USENIX Symposium on Oper- ating Systems Design ...

    2016

  14. [14]

    PyTorch: An Imperative Style, High- Performance Deep Learning Library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “PyTorch: An Imperative Style, High- Performance Deep Learning Library,” inProc. 33rd International Confer- ence on Ne...

    Pith/arXiv arXiv 2019

  15. [15]

    ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table,

    H. Guo, W. Guo, Y . Gao, R. Tang, X. He, and W. Liu, “ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table,” inProc. 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21), July 11- 15 2021, Virtual Event, Canada, 10 pages. doi:10.1145/3404835.3462976. https://a...

    arXiv 2021

  16. [16]

    Seastar: Vertex-Centric Programming for Graph Neural Networks,

    Y . Wu, K. Ma, Z. Cai, T. Jin, B. Li, C. Zheng, J. Cheng, and F. Yu, “Seastar: Vertex-Centric Programming for Graph Neural Networks,” in Proc. ACM / IEEE European Conference on Computer Systems (EuroSys ’21), Apr. 26-29 2021, 17 pages. doi:10.1145/3447786.3456247. https: //doi.org/10.1145/3447786.3456247

    arXiv 2021

  17. [17]

    FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems,

    Y . Hu, Z. Ye, M. Wang, J. Yu, D. Zheng, M. Li, Z. Zhang, Z. Zhang, and Y . Wang, “FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems,” inProc. IEEE/ACM International Conference for High Performance Computing, Networking, Storage and Analysis (SC),

  18. [18]

    Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines,

    J. Ragan-Kelley, A. Adams, S. Paris, M. Levoy, and F. Durand, “Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines,” inProc. ACM SIG- PLAN Conference on Programming Language Design and Implementa- tion (PLDI), 2013, pp. 519–530. https://dl.acm.org/doi/10.1145/2491956. 2462176

    work page doi:10.1145/2491956 2013

  19. [19]

    TVM: An Automated End-to-End Optimizing Compiler for Deep Learning,

    T. Chen, T. Moreau, Z. Jiang, L. Ceze, C. Guestrin, and A. Kr- ishnamurthy, “TVM: An Automated End-to-End Optimizing Compiler for Deep Learning,” inProc. 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2018, pp. 578–594. https: //www.usenix.org/conference/osdi18/presentation/chen

    2018

  20. [20]

    The Tensor Algebra Compiler,

    F. Kjolstad, S. Kamil, S. Chou, D. Lugato, and S. Amarasinghe, “The Tensor Algebra Compiler,” inProc. ACM Program. Lang., vol. 1, no. OOPSLA, Article 77, Oct. 2017. doi:10.1145/3133901. https://doi.org/ 10.1145/3133901

    doi:10.1145/3133901 2017

  21. [21]

    Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code,

    R. Baghdadi, Y . Zou, M. Shirinzadeh, P. Zhao, M. Ripeanu, T. M. Smith, A. Sanan, R. Mullapudi, J. Ragan-Kelley, and S. Amarasinghe, “Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code,” in Proc. IEEE/ACM International Symposium on Code Generation and Optimization (CGO), 2019, pp. 193–205. https://doi.org/10.1109/CGO. 2019.8661197

    arXiv 2019

  22. [22]

    Tensor Com- prehensions: Framework-Agnostic High-Performance Machine Learning Abstractions,

    N. Vasilache, O. Zinenko, T. Theodoridis, P. Goyal, Z. DeVito, W. S. Moses, S. Verdoolaege, A. Adams, and A. Cohen, “Tensor Com- prehensions: Framework-Agnostic High-Performance Machine Learning Abstractions,” inProc. of the 2nd ACM/IEEE Workshop on Machine Learning and Systems (MLSys) — Pre-print / ArXiv, 2018. https://arxiv. org/abs/1802.04730

    Pith/arXiv arXiv 2018

  23. [23]

    Kokkos: Enabling manycore performance portability through polymorphic memory access patterns,

    H. C. Edwards, C. R. Trott, and D. Sunderland, “Kokkos: Enabling manycore performance portability through polymorphic memory access patterns,”Journal of Parallel and Distributed Computing, vol. 74, no. 12, pp. 3202–3216, Dec. 2014. doi:10.1016/j.jpdc.2014.07.003. https://doi. org/10.1016/j.jpdc.2014.07.003

    doi:10.1016/j.jpdc.2014.07.003 2014

  24. [24]

    RAJA: Portable Performance for Large-Scale Scientific Applications,

    D. A. Beckingsale, J. Burmark, R. Hornung, H. Jones, W. Killian, A. J. Kunen, O. Pearce, P. Robinson, B. S. Ryujin, and T. R. W. Scogland, “RAJA: Portable Performance for Large-Scale Scientific Applications,” inProc. IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC), 2019, pp. 71-81. https://www.osti. gov/servlets...

    arXiv 2019

  25. [25]

    SYCL: Single-source C++ accelerator programming,

    R. Reyes and V . Lomüller, “SYCL: Single-source C++ accelerator programming,” inAdvances in Parallel Computing, Vol. 27: Parallel Computing — On the Road to Exascale, R. T. Morris, Ed., IOS Press, 2016, pp. 673-682. https://doi.org/10.3233/978-1-61499-621-7-673

    work page doi:10.3233/978-1-61499-621-7-673 2016

  26. [26]

    Intel® oneAPI Programming Guide,

    Intel Corporation, “Intel® oneAPI Programming Guide,” Intel Corpora- tion, 2025. oneAPI Documentation

    2025

  27. [27]

    OpenCL: A parallel programming standard for heterogeneous computing systems,

    J. E. Stone, D. Gohara, and G. Shi, “OpenCL: A parallel programming standard for heterogeneous computing systems,”Computing in Science & Engineering, vol. 12, no. 3, pp. 66–73, 2010. https://doi.org/10.1109/ MCSE.2010.69

    2010

  28. [28]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” inProc. Int. Conf. Learn. Represent. (ICLR),

  29. [29]

    Inductive representation learning on large graphs,

    W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017. https://arxiv.org/abs/1706.02216

    Pith/arXiv arXiv 2017

  30. [30]

    How powerful are graph neural networks?,

    K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?,” inProc. Int. Conf. Learn. Represent. (ICLR), 2019. https://arxiv.org/abs/1810.00826

    Pith/arXiv arXiv 2019

  31. [31]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y . Ben- gio, “Graph attention networks,” inProc. Int. Conf. Learn. Represent. (ICLR), 2018. https://arxiv.org/abs/1710.10903

    Pith/arXiv arXiv 2018

  32. [32]

    A fast and high quality multilevel scheme for partitioning irregular graphs,

    G. Karypis and V . Kumar, “A fast and high quality multilevel scheme for partitioning irregular graphs,”SIAM Journal on Scientific Computing, vol. 20, no. 1, pp. 359–392, 1998. doi: 10.1137/S1064827595287997

    doi:10.1137/s1064827595287997 1998

  33. [33]

    Sparse tensor support,

    PyTorch Geometric, “Sparse tensor support,”PyTorch Geometric Doc- umentation. Pytorch_Geometric Documentation

  34. [2017]

    https://arxiv.org/abs/1609.02907

    Pith/arXiv arXiv

  35. [2020]

    https://arxiv.org/abs/2008.11359

    Pith/arXiv arXiv 2008