Efficient GNN Training Through Structure-Aware Randomized Mini-Batching

Christos Kozyrakis; Gal Chechik; Haggai Maron; Vignesh Balaji

REVIEW 3 major objections 6 minor 1 cited by

Efficient GNN Training Through Structure-Aware Randomized Mini-Batching

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that COMM-RAND, a mini-batching scheme that biases randomization toward graph communities, cuts GNN training time by up to 2.76x (1.8x on average) while keeping final validation accuracy within 1.79 percentage points…

desk verdict Solid systems paper with a clean two-knob idea; the headline speedup is credible but the mechanism claim is underdetermined by a missing matched-footprint baseline. read the letter →

arxiv 2504.18082 v1 pith:4PVOPSJW submitted 2025-04-25 cs.LG cs.AI

Vignesh Balaji , Christos Kozyrakis , Gal Chechik , Haggai Maron This is my paper · ORCID

classification cs.LGcs.AI

keywords graphneuralnetworksmini-batchtrainingcommunitystructurereorderingcachelocalityrandomizedbatchingsamplingbiasspeedup

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the randomization used in GNN mini-batch training can be made community-aware without sacrificing accuracy, and that doing so accelerates training substantially. It proposes COMM-RAND, which biases two steps—how root nodes are split into batches and how neighbors are sampled—so that batches mostly stay within graph communities. On four benchmarks, this reduces training time by up to 2.76x (1.8x average) with final accuracy within 1.79 percentage points (0.42 average) of uniform-random mini-batching. The practical promise is that practitioners can trade a small, controlled amount of randomness for large per-epoch speedups.

What carries the argument

The mechanism is community-aware biased randomization, applied to both steps of mini-batch construction. In root partitioning, each community is a block that is shuffled as a whole and internally; a knob mixes a percentage of communities into super-blocks to increase randomness. In neighborhood sampling, intra-community edges are sampled with probability p and inter-community edges with 1-p. What carries the argument is that these two biases shrink the average per-batch subgraph footprint, lowering the volume of input features fetched from memory; the paper shows a strong correlation (r=0.90–0.99) between that footprint and per-epoch time.

What would settle it

Run the same comparison on a random graph with no community structure (where community-biased sampling cannot reduce the per-batch feature footprint). If per-epoch speedups remain large, the proposed mechanism is not the cause; alternatively, reduce the feature dimension to 1, which should collapse the correlation shown in Figure 6 and bring speedups down toward 1.0x.

Watch

Extended reading notes

Core claim

COMM-RAND shows that the trade-off between randomized and structure-aware mini-batching is not binary. Root nodes are partitioned by shuffling community blocks (and optionally mixing a fixed percentage of communities into super-blocks), and neighbors are sampled with probability p for intra-community edges versus 1-p for inter-community edges. This constrained randomization keeps batches locally dense while still changing every epoch. The paper reports that, across four graphs, per-epoch time tracks the average size of the input feature matrix fetched per batch (Pearson r = 0.90–0.99), and the best knob setting yields total training speedups of 1.8x on average and up to 2.76x, with an average final-accuracy drop of 0.42 percentage points and a maximum drop of 1.79.

Load-bearing premise

The speedups rest on the assumption that per-epoch training time is dominated by the amount of input feature data fetched for each batch; if feature fetching is not the bottleneck on a given platform or graph, the speedups may shrink or disappear.

Editorial extensions

If this is right

With a fixed wall-clock budget, COMM-RAND trains more epochs than uniform-random mini-batching and can end up with better test accuracy (shown on reddit: 1.54x more epochs, +0.27 points).
The speedup widens as the problem size relative to on-chip cache grows; smaller L2 partitions and software-managed caches both amplify COMM-RAND's advantage.
The benefit transfers beyond GraphSAGE: on reddit, GCN training is 2.03x faster overall and GAT 1.38x faster, with accuracy within 1 point.
Unlike prior graph-partitioning based mini-batching, COMM-RAND keeps its speedups when only a small fraction of nodes is labeled, because it only builds subgraphs from the training set's neighborhoods.
The two knobs define a continuum: users who care about fewest epochs can choose near-uniform settings, while users with tight compute budgets can push toward community-only batches and accept slightly slower convergence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If feature-fetch cost is the true bottleneck, the same community-block shuffling idea could be exported to other subgraph-based models—graph transformers, heterogeneous GNNs—where 'communities' are replaced by any partition of the node set that improves cache reuse.
The paper leaves open whether the knobs can be set automatically; a natural testable extension is to use the per-epoch feature size as a cheap proxy objective and tune the knobs online during training.
Because the correlation between per-epoch time and feature footprint is so strong, the method's benefit should grow with feature dimensionality and graph locality; on datasets with tiny feature vectors or hardware with very large caches, the advantage is likely to shrink.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Zeynep — quick take on 2504.18082 (COMM-RAND). It's a well-executed systems study of structure-aware randomized mini-batching for GNN training. The core contribution is two controllable knobs: mixing k% of communities when partitioning root nodes, and biasing neighborhood sampling toward intra-community edges with probability p. That construction is new relative to ClusterGCN, which randomizes at partition granularity but not within, and the paper is honest about the difference. The empirical sweep is broad: four datasets, three models, plus ablations on cache sensitivity, hyperparameter budget, and comparison to LABOR and ClusterGCN. The 1.8x average total training speedup with 0.42pp average accuracy loss is plausible and useful for practice.

The soft spots are real but not fatal. First, the headline configuration (MIX-12.5%, p=1.0) is selected by sweeping knobs on the same four benchmarks used to report it. They disclose other settings, so it's not hidden, but the best-case number shouldn't be treated as a default. No error bars on speedups or accuracy gaps either. Second, and more substantively, the mechanism claim in Section 6.1.1 is underdetermined. Figure 6 shows per-epoch time tracks the average feature size per batch. But feature size is a deterministic function of the sampling policy. With p=1.0, inter-community edges are never sampled, so the subgraph footprint shrinks; a uniform-random baseline with matched feature footprint (e.g., lower fanout or smaller effective neighborhood) would likely show a similar per-epoch reduction under the paper's own correlation. Without such a baseline, the speedup is not cleanly attributable to community bias improving cache reuse versus simply doing less feature fetch. This does not sink the contribution—a method that reduces work while preserving accuracy is still valuable—but it means the causal story is weaker than the paper implies.

Also, no code is released, which matters for a systems paper whose main evidence is empirical.

Would I engage with it? Yes. The idea is clean, the evaluation is extensive, and the questions about matched baselines and error bars are exactly what a good referee would push on. It deserves a serious peer review with requests for a matched-footprint baseline, error bars, and code release. I'd bring it to the reading group as an example of a solid systems contribution whose evaluation could be tightened.

Referee Report

3 major / 6 minor

Summary. The paper proposes COMM-RAND, a mini-batch construction method for GNN training that introduces community-structure bias into both root-node partitioning and neighborhood sampling, with two user-controlled knobs: k (number of communities mixed before randomization, as a percentage) and p (probability of sampling intra-community edges). The method is evaluated on four graph benchmarks (reddit, igb-small, ogbn-products, ogbn-papers100M) with 3-layer GraphSAGE, plus GCN and GAT on reddit. The authors report up to 2.76x (1.8x on average) total training speedup with accuracy within 1.79 percentage points (0.42 on average) of uniform random mini-batching, and include studies on hyperparameter tuning budgets, comparisons to LABOR and ClusterGCN, software-cache sensitivity, and L2 cache capacity sensitivity. The central claim is that a controlled mix of randomization and community structure improves per-epoch efficiency, mainly through reduced input feature footprint and improved cache reuse, without materially harming convergence or accuracy.

Significance. If validated, COMM-RAND would be a practical, low-effort optimization for GNN training on graphs with community structure, with immediate applicability in existing frameworks like DGL. The paper's strengths include a clean operationalization of the randomization-structure trade-off, extensive empirical coverage across datasets, models, ablations, a hyperparameter-budget experiment (Section 6.2) that accounts for tuning costs, and direct evidence of improved cache reuse in the software-cache miss rates of Section 6.5.1. The correlation-based mechanism story (Figures 6 and 7) is plausible, and the comparison to ClusterGCN correctly identifies a scalability weakness of partition-based mini-batching. However, the headline speedup is obtained after selecting the best knob configuration on the same four datasets used to report the number, no seed-level error bars are given, and the causal mechanism is confounded with subgraph size because no matched-footprint random baseline is tested. These issues must be addressed before the quantitative claims can be fully trusted.

major comments (3)

[Section 6.1.3, Figure 5] The headline average total speedup of 1.8x is selected as the best among many knob combinations on the same four datasets used to report the headline, and no seed-level error bars are reported for either the speedup or the accuracy gap. Since the chosen configuration (k=12.5%, p=1.0) is a post-hoc maximum over the evaluation grid, the expected speedup of COMM-RAND under a fixed practical default is not established. Please report per-seed distributions, and either validate the selected configuration on held-out datasets or justify a fixed default configuration from the trade-off analysis rather than from the test-set maximum.
[Section 6.1.1, Figure 6] The claim that COMM-RAND's per-epoch speedup comes from community-aware cache reuse is confounded by subgraph size. The average input feature size in Figure 6 is a deterministic output of the same knobs that define each configuration, and with p=1.0 the subgraph shrinks sharply because inter-community edges are given zero sampling probability. Under the paper's own near-linear correlation (r=0.90-0.99), a uniform-random baseline with a matched average feature footprint (for example, lower fanout or smaller batch size) would be expected to achieve the same per-epoch time reduction. No such matched-footprint baseline is run, so the observed speedup may be entirely attributable to reduced feature traffic rather than to structure-aware locality. Please add an ablation that matches the average feature size between uniform random mini-batching and COMM-RAND, or otherwise controls for subgraph size.
[Section 6.1.1 and Section 6.5.1] The causal role of cache reuse is inferred from correlations with feature size, but the main experiments do not directly measure on-chip cache hit rates or memory traffic. The software-cache miss rates in Section 6.5.1 provide direct evidence for only ogbn-papers100M and a single cache configuration. To support the mechanism claim, please report cache miss rates or memory traffic for the main configurations across the other datasets, or otherwise test the cache-reuse mechanism independently of the subgraph-size effect.

minor comments (6)

[Section 6.1.3] The phrase 'provides the higher average total training speedups' is grammatically incorrect; it should be 'provides the highest average total training speedup' or similar.
[Figure 5] The accuracy subplots use truncated y-axes that start away from zero, which visually exaggerates the differences between configurations; please either use a full-scale axis or explicitly state the axis range in the caption to avoid misleading the reader.
[Section 6.3, Table 4] The baseline for the ClusterGCN comparison runs on the original graph ordering while COMM-RAND runs on the community-reordered graph, so the per-epoch speedups in Table 4 conflate reordering benefits with the effect of COMM-RAND itself; please clarify how much of the speedup comes from reordering alone, especially since Section 3 reports that reordering alone gives only a 3% average training-time improvement.
[Section 6.5.2] The MIG-based L2 cache sensitivity study also changes the number of SMs and the memory bandwidth along with the L2 capacity; the authors acknowledge this, but a cleaner experiment such as a cache simulator or a configuration that isolates L2 capacity would make the claim about cache-capacity sensitivity more robust.
[References] References [32] and [33] appear to be the same paper with different formatting; please deduplicate them, and consider citing the original METIS paper rather than a multithreaded partitioning library as the source for METIS.
[Table 3] The column 'No. of Epochs' reports fractional values (641.8 and 987.6); please clarify whether these are averages over the five seeds, and if so, report the variance across seeds.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical evaluation against external baselines; speedups are measured outcomes, not derived from inputs by construction.

full rationale

COMM-RAND does not claim a formal derivation or a first-principles prediction. Its central claims are measured runtime and accuracy numbers obtained by running the proposed sampling scheme and comparing against uniform-random mini-batching, ClusterGCN, and LABOR on four standard benchmarks. The reduction in per-batch feature footprint caused by biased neighborhood sampling is a direct consequence of the sampling definition, but the speedups are reported as measurements, not derived from the footprint algebraically; Figure 6 shows an empirical correlation between feature size and per-epoch time rather than defining one quantity in terms of the other. The selection of COMM-RAND-MIX-12.5% with p=1.0 as the best configuration is a post-hoc empirical sweep over disclosed knob settings (Section 6.1.3), with ranges and alternative configurations reported; this is a statistical selection issue, not a case of a fitted parameter being renamed as a prediction. The self-citations (references [6,7,8]) support the general graph-reordering-locality premise, but that premise is also supported by external citations ([5,17,46,49]) and by the paper's own experiments in Section 3; no load-bearing argument reduces to a self-citation, and no uniqueness theorem or ansatz is imported from the authors' prior work. The paper is therefore self-contained as an empirical study, and no circular step is present.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper's central claim is empirical; it does not derive a bound. The only numbers chosen post hoc are the knob settings k=12.5% and p=1.0, selected from a sweep over the same four benchmarks. No new entities are introduced. The main domain assumptions are that community structure is detectable and that feature-fetch locality dominates per-epoch time.

free parameters (2)

Community mixing knob k = 12.5% of communities (best)
Selected after sweeping 0%, 12.5%, 25%, 50% on the four evaluation datasets; yields the headline average speedup.
Intra-community sampling probability p = 1.0 (best)
Selected after sweeping 0.5, 0.9, 1.0; p=1.0 maximizes per-epoch speedup while keeping accuracy loss within 1.79 points.

assumptions (3)

domain assumption The evaluated graphs have detectable community structure whose reordered layout improves cache locality.
Section 5 assumes community-ordered inputs via RABBIT; Section 6.5.3 gives overhead only for reddit (0.78%).
domain assumption Average per-batch input feature size is the dominant driver of per-epoch training time.
Section 6.1.1 and Figure 6 show high correlations (r=0.90-0.99) but not causal isolation.
domain assumption DGL's NeighborSampler with edge probabilities correctly implements the biased sampling used in COMM-RAND.
Section 5 relies on DGL API for biased neighborhood sampling without independent verification of sampling distributions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Efficient GNN Training Through Structure-Aware Randomized Mini-Batching." pith.science (2026). https://pith.science/paper/4PVOPSJW

@misc{pith2026250418082,
  author       = {Pith},
  title        = {Pith review of: Efficient GNN Training Through Structure-Aware Randomized Mini-Batching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4PVOPSJW}},
  note         = {Machine review of arXiv:2504.18082}
}

read the original abstract

Graph Neural Networks (GNNs) enable learning on realworld graphs and mini-batch training has emerged as the de facto standard for training GNNs because it can scale to very large graphs and improve convergence. Current mini-batch construction policies largely ignore efficiency considerations of GNN training. Specifically, existing mini-batching techniques employ randomization schemes to improve accuracy and convergence. However, these randomization schemes are often agnostic to the structural properties of the graph (for eg. community structure), resulting in highly irregular memory access patterns during GNN training that make suboptimal use of on-chip GPU caches. On the other hand, while deterministic mini-batching based solely on graph structure delivers fast runtime performance, the lack of randomness compromises both the final model accuracy and training convergence speed. In this paper, we present Community-structure-aware Randomized Mini-batching (COMM-RAND), a novel methodology that bridges the gap between the above extremes. COMM-RAND allows practitioners to explore the space between pure randomness and pure graph structural awareness during mini-batch construction, leading to significantly more efficient GNN training with similar accuracy. We evaluated COMM-RAND across four popular graph learning benchmarks. COMM-RAND cuts down GNN training time by up to 2.76x (1.8x on average) while achieving an accuracy that is within 1.79% points (0.42% on average) compared to popular random mini-batching approaches.

Figures

Figures reproduced from arXiv: 2504.18082 by the authors.

**Figure 1.** Community-based Graph Reordering: Assigning community members consecutive IDs improves locality 3 Tension between Randomization vs Structure-Awareness in Mini-Batching In the previous section we discussed the use of randomization in mini-batch training of GNNs. This section explores how graph structure and randomization create a trade-off between per-epoch training speed and overall training convergence (number o… view at source ↗

**Figure 2.** Impact of entirely community-based mini [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 4.** Biased Neighborhood Selection: Biased neighborhood selection for the graph in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Figures from the paper (7 more)

**Figure 5.** Figure 5: First, while the level of graph structure bias affects [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 5.** Figure 5: Impact of COMM-RAND across different GNN training metrics (rows) and input graphs (columns): [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Per-epoch training time vs input node fea [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: No. of epochs to converge vs label diversity: [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: Per-Epoch time vs training set size: ClusterGCN performance is invariant to the training set size 6.4 Generalizing to other GNN models The results up to this point have focused on the GraphSAGE model. However, COMM-RAND generalizes to arbitrary GNN models. To illustr…

**Figure 10.** Figure 10: Per-Epoch Speedups vs L2 cache capacity: [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

**Figure 9.** Figure 9: Per-Epoch Training Speedups with a SW cache: COMM-RAND increases performance by reducing UVA transfers between CPU and GPU (for ogbn-papers100M). 6.5.2 Sensitivity to on-chip cache capacity. The main reason for per-epoch training speedups with COMM-RAND is due to reduc…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Geometric deep learning assists protein engineering. Opportunities and Challenges
q-bio.QM 2025-06 conditional novelty 3.0 of 10

A perspective synthesizing geometric deep learning applications in protein engineering and proposing an explainable, structure-aware design pipeline.

Reference graph

Works this paper leans on

52 extracted references · 39 canonical work pages · cited by 1 Pith paper

[1]

2013. UVA. https://developer.nvidia.com/blog/unified-memory-in- cuda-6/. Accessed: Mar 12, 2025

work page 2013
[2]

DGL DataLoader API

2025. DGL DataLoader API. https://www.dgl.ai/dgl_docs/generated/ dgl.dataloading.DataLoader.html. Accessed: Mar 12, 2025

work page 2025
[3]

NVIDIA Merlin GPU Embedding Cache

2025. NVIDIA Merlin GPU Embedding Cache. https://github.com/ NVIDIA-Merlin/HugeCTR/tree/main/gpu_cache. Accessed: Mar 12, 2025

work page 2025
[4]

RedditDataset-DGL

2025. RedditDataset-DGL. Retrieved Accessed: Mar 8, 2025 from https: //www.dgl.ai/dgl_docs/generated/dgl.data.RedditDataset.html

work page 2025
[5]

Junya Arai, Hiroaki Shiokawa, Takeshi Yamamuro, Makoto Onizuka, and Sotetsu Iwamura. 2016. Rabbit order: Just-in-time Parallel Reorder- ing for Fast Graph Analysis. In International Parallel and Distributed Processing Symposium (IPDPS). 22–31

work page 2016
[6]

Vignesh Balaji, Neal Crago, Aamer Jaleel, and Brandon Lucia. 2021. P-opt: Practical optimal cache replacement for graph analytics. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 668–681

work page 2021
[7]

Vignesh Balaji, Neal C Crago, Aamer Jaleel, and Stephen W Keckler

work page
[8]

Vignesh Balaji and Brandon Lucia. 2018. When is Graph Reorder- ing an Optimization? Studying the Effect of Lightweight Graph Re- ordering Across Applications and Input Graphs. In IEEE Interna- tional Symposium on Workload Characterization (IISWC) . 203–214. doi:10.1109/IISWC.2018.8573478

arXiv 2018

Show all 52 references

[9]

Muhammed Fatih Balin and Ümit Çatalyürek. 2023. Layer-Neighbor Sampling—Defusing Neighborhood Explosion in GNNs. Advances in Neural Information Processing Systems 36 (2023), 25819–25836

2023
[10]

Albert-László Barabási. 2009. Scale-free Networks: A Decade and Beyond. Science 325, 5939 (2009), 412–413

2009
[11]

Yoshua Bengio. 2012. Practical recommendations for gradient-based training of deep architectures. In Neural networks: Tricks of the trade: Second edition. Springer, 437–478

2012
[12]

Léon Bottou. 2009. Curiously fast convergence of some stochastic gra- dient descent algorithms. In Proceedings of the symposium on learning and data science, Paris , Vol. 8. Citeseer, 2624–2633

2009
[13]

Jie Chen, Tengfei Ma, and Cao Xiao. 2018. Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:1801.10247 (2018)

2018 arXiv
[14]

Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho- Jui Hsieh. 2019. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data minin...

2019
[15]

DGL. 2024. DGL NeighborSampler API. https://docs.dgl.ai/en/1.1.x/ generated/dgl.dataloading.NeighborSampler.html. Accessed: Oct 31, 2024

2024
[16]

Jialin Dong, Da Zheng, Lin F Yang, and George Karypis. 2021. Global neighbor sampling for mixed CPU-GPU training on giant graphs. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining . 289–299

2021
[17]

Priyank Faldu, Jeff Diamond, and Boris Grot. 2020. Domain-specialized Cache Management for Graph Analytics. In International Symposium on High Performance Computer Architecture (HPCA) . 234–248

2020
[18]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In The world wide web conference . 417–426

2019
[19]

Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019)

2019 arXiv
[20]

Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, et al. 2020. AWB-GCN: A graph convolutional network accelerator with runtime workload rebalancing. In 2020 53rd Annual IEEE/ACM International Symposium o...

2020
[21]

Tong Geng, Chunshu Wu, Yongan Zhang, Cheng Tan, Chenhao Xie, Haoran You, Martin Herbordt, Yingyan Lin, and Ang Li. 2021. I-GCN: A graph convolutional network accelerator with runtime locality en- hancement through islandization. InMICRO-54: 54th annual IEEE/ACM international s...

2021
[22]

Michelle Girvan and Mark EJ Newman. 2002. Community Structure in Social and Biological Networks. Proceedings of the National Academy of Sciences 99, 12 (2002), 7821–7826

2002
[23]

Gürbüzbalaban, A

M. Gürbüzbalaban, A. Ozdaglar, and P. A. Parrilo. 2019. Why random reshuffling beats stochastic gradient descent. Mathematical Program- ming 186, 1–2 (Oct. 2019), 49–84. doi: 10.1007/s10107-019-01440-w

2019 doi
[24]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive repre- sentation learning on large graphs. Advances in neural information processing systems 30 (2017)

2017
[25]

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems 33 (2020), 22118–22133

2020
[26]

Ranggi Hwang, Minhoo Kang, Jiwon Lee, Dongyun Kam, Youngjoo Lee, and Minsoo Rhu. 2023. Grow: A row-stationary sparse-dense gemm accelerator for memory-efficient graph convolutional neural networks. In 2023 IEEE International Symposium on High-Performance Computer Architecture ...

2023
[27]

Zhihao Jia, Sina Lin, Mingyu Gao, Matei Zaharia, and Alex Aiken. 2020. Improving the accuracy, scalability, and performance of graph neural networks with roc. Proceedings of Machine Learning and Systems 2 (2020), 187–198

2020
[28]

Tim Kaler, Nickolas Stathas, Anne Ouyang, Alexandros-Stavros Il- iopoulos, Tao Schardl, Charles E Leiserson, and Jie Chen. 2022. Ac- celerating training and inference of graph neural networks with fast sampling and pipelining. Proceedings of Machine Learning and Systems 4 (202...

2022
[29]

Konstantinos I Karantasis, Andrew Lenharth, Donald Nguyen, María J Garzarán, and Keshav Pingali. 2014. Parallelization of Reordering Algorithms for Bandwidth and Wavefront Reduction. In International Conference for High Performance Computing, Networking, Storage and Analysis (...

2014
[30]

Arpandeep Khatua, Vikram Sharma Mailthody, Bhagyashree Taleka, Tengfei Ma, Xiang Song, and Wen-mei Hwu. 2023. Igb: Addressing the gaps in labeling, features, heterogeneity, and size of public graph datasets for deep learning research. In Proceedings of the 29th ACM SIGKDD Conf...

2023
[31]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

2016 arXiv
[32]

Dominique Lasalle and George Karypis. 2013. Multi-threaded Graph Partitioning. In International Symposium on Parallel and Distributed Processing (IPDPS). 225–236

2013
[33]

Dominique LaSalle and George Karypis. 2013. Multi-threaded graph partitioning. In 2013 IEEE 27th International Symposium on Parallel and Distributed Processing. IEEE, 225–236

2013
[34]

Zhiqi Lin, Cheng Li, Youshan Miao, Yunxin Liu, and Yinlong Xu. 2020. Pagraph: Scaling gnn training on large graphs via computation-aware caching. In Proceedings of the 11th ACM Symposium on Cloud Comput- ing. 401–415

2020
[35]

Summer Liu, Brad Rees, and Pahal Patangia. 2024. Su- percharging Fraud Detection in Financial Services with Graph Neural Networks. Retrieved Accessed: Mar 8, 2025 from https://developer.nvidia.com/blog/supercharging-fraud- detection-in-financial-services-with-graph-neural-netw...

2024
[36]

Tianfeng Liu, Yangrui Chen, Dan Li, Chuan Wu, Yibo Zhu, Jun He, Yanghua Peng, Hongzheng Chen, Hongzhi Chen, and Chuanxiong Guo. 2023. {BGL}:{GPU-Efficient}{ GNN} training by optimizing graph data{I/O} and preprocessing. In 20th USENIX Symposium on Networked Systems Design and ...

2023
[37]

Xin Liu, Mingyu Yan, Lei Deng, Guoqi Li, Xiaochun Ye, and Dongrui Fan. 2021. Sampling methods for efficient training of graph convolu- tional networks: A survey. IEEE/CAA Journal of Automatica Sinica 9, 2 (2021), 205–234

2021
[38]

Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, and Yafei Dai. 2019. {NeuGraph}: Parallel deep neural net- work computation on large graphs. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). 443–458

2019
[39]

Nikolai Merkel, Pierre Toussing, Ruben Mayer, and Hans-Arno Jacob- sen. 2025. Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Study. Proc. VLDB Endow. 18, 2 (Feb. 2025), 293–307. doi: 10.14778/3705829.3705846

2025
[40]

NVIDIA. 2024. MIG User Guide. https://docs.nvidia.com/datacenter/ tesla/pdf/NVIDIA_MIG_User_Guide.pdf. Accessed: Mar 11, 2025

2024
[41]

NVIDIA. 2024. NVIDIA NGC DGL Docker container. https://catalog. ngc.nvidia.com/orgs/nvidia/containers/dgl. Accessed: Oct 31, 2024

2024
[42]

Itay Safran and Ohad Shamir. 2020. How good is SGD with random shuffling?. In Conference on Learning Theory . PMLR, 3250–3284

2020
[43]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention net- works. arXiv preprint arXiv:1710.10903 (2017)

2017 arXiv
[44]

Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al. 2019. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 (2019)

2019 arXiv
[45]

2021.{GNNAdvisor}: An adaptive and efficient runtime system for{GNN} acceleration on{GPUs}

Yuke Wang, Boyuan Feng, Gushu Li, Shuangchen Li, Lei Deng, Yuan Xie, and Yufei Ding. 2021.{GNNAdvisor}: An adaptive and efficient runtime system for{GNN} acceleration on{GPUs}. In 15th USENIX symposium on operating systems design and implementation (OSDI 21) . 515–531

2021
[46]

Hao Wei, Jeffrey Xu Yu, Can Lu, and Xuemin Lin. 2016. Speedup Graph Processing by Graph Ordering. In International Conference on Management of Data (SIGMOD) . 1813–1828

2016
[47]

Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. 2020. Hygcn: A gcn accel- erator with hybrid architecture. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) . IEEE, 15–29

2020
[48]

Shuangyan Yang, Minjia Zhang, Wenqian Dong, and Dong Li. 2023. Betty: Enabling large-scale gnn training with batch-level graph par- titioning. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volu...

2023
[49]

Zhang, V

Y. Zhang, V. Kiriansky, C. Mendis, S. Amarasinghe, and M. Zaharia
[50]

Difan Zou, Ziniu Hu, Yewen Wang, Song Jiang, Yizhou Sun, and Quan- quan Gu. 2019. Layer-dependent importance sampling for training deep and large graph convolutional networks. Advances in neural information processing systems 32 (2019). 13

2019
[2017]

In International Conference on Big Data (Big Data)

Making Caches Work for Graph Analytics. In International Conference on Big Data (Big Data) . 293–302
[2023]

In 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)

Community-based matrix reordering for sparse linear algebra optimization. In 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) . IEEE, 214–223

2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

2013. UVA. https://developer.nvidia.com/blog/unified-memory-in- cuda-6/. Accessed: Mar 12, 2025

work page 2013

[2] [2]

DGL DataLoader API

2025. DGL DataLoader API. https://www.dgl.ai/dgl_docs/generated/ dgl.dataloading.DataLoader.html. Accessed: Mar 12, 2025

work page 2025

[3] [3]

NVIDIA Merlin GPU Embedding Cache

2025. NVIDIA Merlin GPU Embedding Cache. https://github.com/ NVIDIA-Merlin/HugeCTR/tree/main/gpu_cache. Accessed: Mar 12, 2025

work page 2025

[4] [4]

RedditDataset-DGL

2025. RedditDataset-DGL. Retrieved Accessed: Mar 8, 2025 from https: //www.dgl.ai/dgl_docs/generated/dgl.data.RedditDataset.html

work page 2025

[5] [5]

Junya Arai, Hiroaki Shiokawa, Takeshi Yamamuro, Makoto Onizuka, and Sotetsu Iwamura. 2016. Rabbit order: Just-in-time Parallel Reorder- ing for Fast Graph Analysis. In International Parallel and Distributed Processing Symposium (IPDPS). 22–31

work page 2016

[6] [6]

Vignesh Balaji, Neal Crago, Aamer Jaleel, and Brandon Lucia. 2021. P-opt: Practical optimal cache replacement for graph analytics. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 668–681

work page 2021

[7] [7]

Vignesh Balaji, Neal C Crago, Aamer Jaleel, and Stephen W Keckler

work page

[8] [8]

Vignesh Balaji and Brandon Lucia. 2018. When is Graph Reorder- ing an Optimization? Studying the Effect of Lightweight Graph Re- ordering Across Applications and Input Graphs. In IEEE Interna- tional Symposium on Workload Characterization (IISWC) . 203–214. doi:10.1109/IISWC.2018.8573478

arXiv 2018