Pith. sign in

REVIEW 4 major objections 6 minor 52 references

Graph Learning at Scale: Characterizing and Optimizing Pre-Propagation GNNs

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Optimized pre-propagation GNNs train on average 9.9x faster than sampling-based GNNs on large graphs.

desk verdict A solid systems paper with a real contribution, but the headline 9.9x speedup is only as strong as the MP-GNN baselines, and the paper never tests the strongest combination of sampler and caching. read the letter →

arxiv 2504.13266 v1 pith:L57E4V46 submitted 2025-04-17 cs.LG

Zichao Yue , Chenhui Deng , Zhiru Zhang This is my paper · ORCID
classification cs.LG
keywords pre-propagationGNNsgraphsamplingdataloadingchunkreshufflingGNNtrainingsystemslarge-scalelearningnodeclassificationinputexpansionproblem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that pre-propagation GNNs—models that move neighbor feature aggregation into a one-time preprocessing step so training is a dense model on precomputed hop features—can be made practically fast on graphs with up to 100 million nodes. The authors argue that the reason these models have looked slow is not computation but data loading: the lightweight dense training step is dominated by assembling and transferring batches, and the precomputed features can expand beyond host memory. They characterize the bottleneck, then engineer around it with a custom batch assembler, double-buffer prefetching, chunk-level reshuffling that moves assembly to the GPU, and direct storage access. Their central result is that optimized pre-propagation GNNs train on average 9.9x faster than sampling-based message-passing GNNs on large graph benchmarks, with speedups reaching two orders of magnitude and accuracy matching or exceeding the baselines. If true, this converts a niche theoretical idea into a practical default for large-scale node classification.

What carries the argument

The argument rides on treating the precomputed hop-feature matrices as a dense training set and making their movement the object of optimization. The load-bearing pieces are: a custom batch assembler that gathers scattered node vectors with a single index operation; a double-buffer prefetching scheme on the GPU that overlaps data transfer with compute using separate streams; and chunk reshuffling, which shuffles contiguous chunks of node indices rather than individual nodes so that entire chunks move over the bus and batch assembly happens on the GPU. Chunk reshuffling also enables direct storage access for data too large for host memory. The one-time preprocessing cost, multiplying operator matrices by input features, is amortized over training runs and hyperparameter searches.

What would settle it

Run the identical benchmarks with a vanilla pre-propagation baseline that uses many data-loader workers or a CUDA-aware loader, and with sampling-based baselines whose sampler, fanout, and batch size are tuned per dataset; if the optimized pipeline's throughput lead over both falls well below the reported 9.9x average, the central claim would need to be qualified to specific system configurations.

Watch

Extended reading notes

Core claim

Pre-propagation GNNs match sampling-based message-passing GNNs in accuracy while their training time is dominated by data loading, not compute. After applying efficient host-side batch assembly through a single index operation, double-buffer prefetching that pipelines data movement with GPU computation, and chunk reshuffling that lets batches be assembled on the GPU from contiguous chunks, the authors report a 15x average throughput improvement over vanilla implementations. On three large graphs, optimized pre-propagation GNNs achieve on average 9.9x and up to two orders of magnitude higher training throughput than message-passing GNNs run with state-of-the-art samplers, while reaching higher test accuracy in the reported settings. The gain grows with receptive-field size because pre-propagation training cost grows sublinearly with hops, and the proposed storage access path handles input sizes that exceed host memory.

Load-bearing premise

The headline speedups assume the comparison systems—the vanilla pre-propagation baseline and the sampling-based message-passing systems—are each configured at a representative level of effort; a stronger data-loading baseline or differently tuned samplers could shrink the reported margins.

Editorial extensions

If this is right

  • For node classification on graphs with tens to hundreds of millions of nodes, pre-propagation GNNs become a credible default: they deliver comparable or better accuracy with far higher throughput.
  • As the receptive field grows, the pre-propagation advantage increases because training cost grows sublinearly with hops, making deeper propagation affordable.
  • Inputs too large for host memory can be trained on from storage with modest slowdown, removing a major scalability barrier.
  • Data transfer volume during training is one to two orders of magnitude smaller for pre-propagation GNNs than for sampling-based message-passing GNNs, a decisive advantage on bandwidth-limited systems.
  • A one-time preprocessing cost that is smaller than a single training run becomes negligible across many runs, so the comparison favors pre-propagation GNNs in realistic tuning workflows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The data-loading optimizations are not GNN-specific: any dense training pipeline that gathers batches from large precomputed tensors could reuse chunk reshuffling and double-buffer prefetching with similar gains.
  • If the accuracy trend on the largest benchmark persists with more epochs and multiple seeds, the case for pre-propagation GNNs strengthens beyond systems into modeling; this can be tested by running the models to convergence.
  • Chunk granularity creates a tradeoff between shuffling quality and transfer efficiency, and on non-homophilous graphs or tasks that need strict batch randomness, larger chunks may matter more than the reported datasets suggest.
  • The complexity analysis predicts even larger relative speedups when input feature dimension is high, so high-dimensional-feature graphs are a natural next test of the central claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper presents a systematic empirical comparison of pre-propagation GNNs (PP-GNNs: SGC, SIGN, HOGA) against message-passing GNNs (MP-GNNs) trained with sampling-based systems on graphs ranging from roughly 2M to 111M nodes. The authors identify data loading as the dominant cost in vanilla PP-GNN training and identify the 'input expansion problem' as a scalability bottleneck. They propose a customized data loader, double-buffer GPU-side prefetching, chunk reshuffling, and GPU direct storage access, together with an automated data-placement policy. On medium graphs the optimizations yield about 15x throughput improvement over a PyTorch DataLoader baseline; on three large graphs the paper reports an average 9.9x and up to two orders of magnitude higher throughput than the MP-GNN systems evaluated, with comparable or higher accuracy.

Significance. If the quantitative claims hold, this paper would make a strong practical case for PP-GNNs as a default approach for large-scale node classification, countering the common view that PP-GNNs are only a theoretical alternative with limited systems support. The study is valuable as a first systematic separation of model accuracy and systems efficiency for PP-GNNs, and the ablation cleanly attributes the end-to-end speedup to each proposed optimization. The public artifact, the identification of the input-expansion problem, and the careful treatment of preprocessing amortization are concrete strengths. However, the headline speedup is not tested against the strongest MP-GNN configuration that the paper's own components would permit, throughput variability is not reported, and the 9.9x number is not reproducible from the tables as stated; these issues currently overstate the strength of the central claim.

major comments (4)
  1. [§6.4, Tables 3–4] The headline claim of an average 9.9x and up to two orders of magnitude speedup over 'MP-GNN models with state-of-the-art graph samplers' is not tested against the strongest MP-GNN configuration that the paper's own components would permit. DGL is run with LABOR but without GPU-side feature caching, while GNNLab and SALIENT++ provide GPU feature caching but use hardcoded neighbor samplers that the paper itself states produce larger sampled subgraphs than LABOR. A GNNLab-style system using LABOR sampling is a natural 'SOTA sampler + SOTA system' baseline, and it is absent from Tables 3 and 4. Because LABOR reduces the number of sampled nodes, integrating it into GNNLab could materially raise MP-GNN throughput; for example, the 4-layer ogbn-papers100M ratio of SIGN over GNNLab is about 48x, and a reduction in sampled subgraph size would shrink this substantially. The quantitative claim in the abstract and Section 6.4 therefore needs either this baseline or a more restrictive wording, such as 'compared to DGL with LABOR and to GNNLab/SALIENT++ with their default samplers.'
  2. [Abstract and §6.4 (Tables 3–5)] The derivation of the 9.9x average is not specified, and I could not reproduce it from the tables. Using the per-setting ratios of SIGN and HOGA to DGL in Tables 3–5 gives a geometric mean of about 23x; using the best MP-GNN system in each row gives about 8.9x. The text does not state which ratios are included (which model, which layer/hop count, which GPU count, which baseline system, and whether the geometric mean is over settings or over datasets). The claim should be made reproducible by reporting the exact set of ratios and the aggregation formula, or by presenting the ratios in a dedicated table.
  3. [§6 'Baselines' and Fig. 9] No throughput variance is reported anywhere in the paper. Tables 3–5 give single epoch/sec values, and the 15x ablation in Fig. 9 is a single geometric mean over datasets, models, and hops. The statement that the vanilla PP-GNN baseline uses PyTorch DataLoader with pin_memory and 2 workers 'to achieve optimal performance' is not supported by a worker-count sweep; a stronger data-loading baseline (more workers, DALI, or a CUDA-aware loader) could reduce the 15x figure. For a systems paper whose central claims are throughput ratios, mean ± std over repeated runs should be reported at least for the headline settings, and the baseline configuration should be justified by a sweep.
  4. [Table 5 and §6.4] The claim of 'maintaining superior accuracy' on igb-large rests on test accuracies measured after only 3 epochs and a single run. At 3 epochs the DGL and Ginex baselines are far from convergence, so this comparison does not establish that PP-GNNs are more accurate on this dataset; it only compares very early-training behavior. Either the MP-GNN baselines should be run to a comparable convergence point, or the claim should be restricted to 'higher accuracy at the same early training budget.' The one-run no-error-bar reporting also makes the 64.41% versus 63.07% gap unreliable.
minor comments (6)
  1. [Throughout] There are several typos and inconsistent acronyms: 'Practial' in §3.3, 'ognb-papers100M' and 'IGB-meduim' in Table 2, 'Pre-propgation' in §2.5, 'SDG-RR' in Table 4, and 'UV A' should be 'UVA'.
  2. [Table 1] The table uses red and blue coloring to distinguish feature propagation from feature transformation; this will not survive grayscale printing, so the legend should use textual markers or symbols instead of color alone.
  3. [Figure 6] Figure 6 is very dense; the sub-figure labels, buffer diagrams, and stream dependencies are hard to read at normal size and should be enlarged or simplified.
  4. [§3.5 and Table 7] The text says pre-processing overhead is 'usually much smaller than the time required for a single training run,' but for ogbn-papers100M the preprocessing time is 90% of a single run; this exception should be acknowledged in the main text near the claim.
  5. [Appendix E] The acronym 'SGD-CR' is used without definition; it should be defined as 'stochastic gradient descent with chunk reshuffling' at first use.
  6. [§5] The phrase 'The configure system defaults to SGD-RR' is missing the final 'd' in 'configured' and should read 'The configured system defaults to SGD-RR.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: throughput and accuracy claims are measured against external systems and ablations; HOGA self-citation is not load-bearing.

full rationale

This is an empirical systems paper whose central claims are measured throughput and accuracy comparisons against independently developed systems (DGL, GNNLab, SALIENT++, Ginex), not quantities derived from the models' definitions. The 15x optimization speedup is an ablation measurement in Section 6.3 comparing the authors' own vanilla PyTorch DataLoader baseline to their optimized pipeline; an engineering speedup over one's own baseline is a legitimate measured result, not a circular prediction. The 9.9x headline is a ratio over the measurements in Tables 3-5 comparing optimized PP-GNNs to external MP-GNN systems, so it is not forced by construction. HOGA is taken from the authors' prior work (Deng et al. 2024) and that work is cited as motivation and as one of three PP-GNN models, but the accuracy and throughput numbers for HOGA are produced fresh in this paper against public benchmarks; the self-citation is not load-bearing. No parameter is fitted to a subset of data and then renamed a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The characterization of data loading as the bottleneck and input expansion as a scalability challenge is an empirical observation, not a definitional equivalence. The paper is self-contained with respect to its headline comparisons, so no circularity is present.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No mathematical derivation is claimed; the ledger records the experimental hyperparameters and domain assumptions on which the measured speedups and accuracy comparisons depend.

free parameters (5)
  • chunk_size = 8000
    Chosen by hand equal to batch size after testing 1, 1000, 2000, 4000, 8000 in Section 6.2; affects the throughput-accuracy trade-off of chunk reshuffling.
  • batch_size = 8000
    Used for both PP-GNNs and MP-GNNs; influences epoch time, memory, and convergence; justified in Appendix A as a common choice.
  • GraphSAGE fanout = [15, 10, 5] (extended to [15,10,5,3,3,3])
    Chosen in Appendix A to balance accuracy and efficiency; directly affects MP-GNN throughput and accuracy comparisons.
  • GAT fanout = [10, 10, 10] (extended to [10,10,10,5,5,5])
    Chosen in Appendix A to push GAT towards accuracy; affects relative accuracy-efficiency trade-off.
  • PP-GNN hidden dimensions = HOGA 256 (medium)/1024 (large), SIGN 512, GAT 128 per channel x4
    Tuned settings from official repos and Appendix A; affect both accuracy and training throughput.
assumptions (3)
  • domain assumption The selected PP-GNNs (SGC, SIGN, HOGA) are representative of the PP-GNN family.
    The characterization and optimization results are drawn from these three models; if they are not representative, the conclusions may not generalize (Section 3.2, Section 6).
  • domain assumption The chosen MP-GNN training systems and samplers represent state-of-the-art optimized MP-GNN training.
    Speedup claims are measured relative to DGL, GNNLab, SALIENT++, and Ginex with specific samplers; a stronger system could reduce the reported advantage (Section 6, Appendix A).
  • domain assumption The evaluation hardware (NVIDIA GPUs with GDS, 380 GB host memory, PCIe 4.0 SSDs) is representative of large-graph training environments.
    GDS-based optimizations and the input expansion solution depend on this hardware; results on other systems, e.g., without GDS, may differ (Appendix C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Graph Learning at Scale: Characterizing and Optimizing Pre-Propagation GNNs." pith.science (2026). https://pith.science/paper/L57E4V46

@misc{pith2026250413266,
  author       = {Pith},
  title        = {Pith review of: Graph Learning at Scale: Characterizing and Optimizing Pre-Propagation GNNs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L57E4V46}},
  note         = {Machine review of arXiv:2504.13266}
}
abstract

Graph neural networks (GNNs) are widely used for learning node embeddings in graphs, typically adopting a message-passing scheme. This approach, however, leads to the neighbor explosion problem, with exponentially growing computational and memory demands as layers increase. Graph sampling has become the predominant method for scaling GNNs to large graphs, mitigating but not fully solving the issue. Pre-propagation GNNs (PP-GNNs) represent a new class of models that decouple feature propagation from training through pre-processing, addressing neighbor explosion in theory. Yet, their practical advantages and system-level optimizations remain underexplored. This paper provides a comprehensive characterization of PP-GNNs, comparing them with graph-sampling-based methods in training efficiency, scalability, and accuracy. While PP-GNNs achieve comparable accuracy, we identify data loading as the key bottleneck for training efficiency and input expansion as a major scalability challenge. To address these issues, we propose optimized data loading schemes and tailored training methods that improve PP-GNN training throughput by an average of 15$\times$ over the PP-GNN baselines, with speedup of up to 2 orders of magnitude compared to sampling-based GNNs on large graph benchmarks. Our implementation is publicly available at https://github.com/cornell-zhang/preprop-gnn.

Figures

Figures reproduced from arXiv: 2504.13266 by the authors.

Figure 1
Figure 1. General structure of MP-GNN and PP-GNN models. previous efforts (Huang et al., 2020) have shown that within the message-passing framework, feature aggregation is typi￾cally more time-consuming than transformation due to its sparse nature. By restricting training to dense computations, PP-GNNs are expected to achieve greater efficiency. Impor￾tantly, the input data preprocessing is a one-time cost that can be amortiz… view at source ↗
Figure 2
Figure 2. Test accuracy of GNN models with different hop counts or layer counts — LABOR and SAINT represent GraphSAGE with the LABOR sampler and GraphSAINT node sampler, respectively. Our evaluation indicates that, among sampling methods, the LABOR sampler achieves the highest test accuracy across most settings, while HOGA outperforms other PP-GNN models in the majority of cases [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. compares the epoch times among 3-layer MP￾GNNs (GraphSAGE with the LABOR sampler) and 3-hop PP-GNNs (SIGN, HOGA, and SGC). The epoch time for GraphSAGE is measured with a vanilla DGL implementa￾products pokec wiki 10 0 10 1 10 2 Epoch Time (sec) Method SAGE-Vanilla SAGE-UVA SAGE-Preload HOGA SIGN SGC [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (10 more)
Figure 3
Figure 3. Figure 3: Convergence rate comparison among 4-layer(hop) MP￾GNNs and PP-GNNs— The number in the plot denotes the con￾vergence point where 99% of peak validation accuracy is reached. Convergence Rate Comparison. The convergence rate significantly impacts the end-to-end training t…
Figure 5
Figure 5. Figure 5: Training time breakdown of PP-GNNs on ogbn-products. Further investigation reveals that the primary overhead in the baseline implementation stems from data loading. Fig￾ure 5 shows the epoch time breakdown of three PP-GNN models on the ogbn-products dataset, averaged a…
Figure 6
Figure 6. Figure 6: System-level optimizations adopted in our work — For the case with input data residing in the host memory. illustrated in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Accuracy efficiency trade-off comparison among MP￾GNNs and PP-GNNs on wiki— The MP-GNN legend keys show the backbone model and the graph sampler. ’128’ in the labels represents the additional hidden dimension setting. This section compares the accuracy and training eff…
Figure 9
Figure 9. Figure 9: Ablation study with input data in the host memory — X-ticks show the dataset (O: ogbn-products, P: pokec, W: wiki) and the PP-GNN model (HOGA, SIGN and SGC). In this section, we evaluate the efficacy of our techniques for improving data loading efficiency: efficient ho…
Figure 8
Figure 8. Figure 8: Validation accuracy of HOGA with 4 hops on three datasets — The number in the legend denotes the chunk size. In this section, we investigate the impact of our proposed chunk reshuffling training method on model accuracy and convergence rate using the three medium-sized…
Figure 10
Figure 10. Figure 10: Convergence rate comparison among MP-GNNs and PP-GNNs— The number in the plot denotes the convergence point where 99% of peak validation accuracy is reached. 66.45%, and 66.75%, respectively, with a maximum drop of 0.2% compared to SGD-RR. For SIGN, the test accuracie…
Figure 11
Figure 11. Figure 11: Accuracy efficiency trade-off comparison among MP-GNNs and PP-GNNs— The Y axis represents mean test accuracy obtained from 5 runs, each with 400 epochs. In the legend, for MP-GNNs, the name to the left of the dash denotes the foundation model (GraphSAGE or GAT), while…
Figure 12
Figure 12. Figure 12: Validation accuracy of HOGA and SIGN under different numbers of hops on three datasets — The number in the legend denotes the chunk size. 0 100 200 300 400 Epochs 0.6 0.65 Validation Accuracy 34 21 2 hops 0 100 200 300 400 Epochs 0.6 0.65 22 28 3 hops 0 100 200 300 40…
Figure 13
Figure 13. Figure 13: Convergence rate of HOGA and SIGN on ogbn￾papers100M— The number in the plot denotes the convergence point where 99% of peak validation accuracy is reached. relative increase in overall training time. On average, direct storage loading achieves 36% of the training eff…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 44 canonical work pages

  1. [1]

    u rek, \

    Balin, M. F. and C ataly \"u rek, \"U . Layer-Neighbor Sampling---Defusing Neighborhood Explosion in GNNs . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2024

    work page 2024

  2. [2]

    DSP: Efficient GNN Training with Multiple GPUs

    Cai, Z., Zhou, Q., Yan, X., Zheng, D., Song, X., Zheng, C., Cheng, J., and Karypis, G. DSP: Efficient GNN Training with Multiple GPUs . ACM SIGPLAN Symp. on Principles and Practice of Parallel Programming (PPoPP), 2023

    work page 2023

  3. [3]

    Stochastic Training of Graph Convolutional Networks with Variance Reduction

    Chen, J., Zhu, J., and Song, L. Stochastic Training of Graph Convolutional Networks with Variance Reduction . arXiv preprint arXiv:1710.10568, 2017

    arXiv 2017

  4. [4]

    FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

    Chen, J., Ma, T., and Xiao, C. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling . International Conference on Learning Representations (ICLR), 2018

    work page 2018

  5. [5]

    On Graph Neural Networks Versus Graph-Augmented MLPs

    Chen, L., Chen, Z., and Bruna, J. On Graph Neural Networks Versus Graph-Augmented MLPs . arXiv preprint arXiv:2010.15116, 2020 a

    arXiv 2010

  6. [6]

    Scalable Graph Neural Networks via Bidirectional Propagation

    Chen, M., Wei, Z., Ding, B., Li, Y., Yuan, Y., Du, X., and Wen, J.-R. Scalable Graph Neural Networks via Bidirectional Propagation . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2020 b

    work page 2020

  7. [7]

    Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

    Chiang, W.-L., Liu, X., Si, S., Li, Y., Bengio, S., and Hsieh, C.-J. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks . ACM SIGKDD Conf. on Knowledge Discovery & Data Mining (KDD), 2019

    work page 2019

  8. [8]

    Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits

    Deng, C., Yue, Z., Yu, C., Sarar, G., Carey, R., Jain, R., and Zhang, Z. Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits . Design Automation Conf. (DAC), 2024

    work page 2024

Show all 52 references
  1. [9]

    On the Equivalence of Decoupled Graph Convolution Network and Label Propagation

    Dong, H., Chen, J., Feng, F., He, X., Bi, S., Ding, Z., and Cui, P. On the Equivalence of Decoupled Graph Convolution Network and Label Propagation . Int'l World Wide Web Conf. (WWW), 2021

    2021

  2. [10]

    SIGN: Scalable Inception Graph Neural Networks

    Frasca, F., Rossi, E., Eynard, D., Chamberlain, B., Bronstein, M., and Monti, F. SIGN: Scalable Inception Graph Neural Networks . arXiv preprint arXiv:2004.11198, 2020

    2004 arXiv

  3. [11]

    Diffusion Improves Graph Learning

    Gasteiger, J., Wei enberger, S., and G \"u nnemann, S. Diffusion Improves Graph Learning . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2019

    2019

  4. [12]

    S., Riley, P

    Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural Message Passing for Quantum Chemistry . Int'l Conf. on Machine Learning (ICML), 2017

    2017

  5. [13]

    Inductive Representation Learning on Large Graphs

    Hamilton, W., Ying, Z., and Leskovec, J. Inductive Representation Learning on Large Graphs . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2017

    2017

  6. [14]

    Open Graph Benchmark: Datasets for Machine Learning on Graphs

    Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open Graph Benchmark: Datasets for Machine Learning on Graphs . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2020

    2020

  7. [15]

    GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks

    Huang, G., Dai, G., Wang, Y., and Yang, H. GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks . Int'l Conf. on High Performance Computing Networking, Storage and Analysis (SC), 2020

    2020

  8. [16]

    WiseGraph: Optimizing GNN with Joint Workload Partition of Graph and Operations

    Huang, K., Zhai, J., Zheng, L., Wang, H., Jin, Y., Zhang, Q., Zhang, R., Zheng, Z., Yi, Y., and Shen, X. WiseGraph: Optimizing GNN with Joint Workload Partition of Graph and Operations . European Conf. on Computer Systems (EuroSys), 2024

    2024

  9. [17]

    E., and Chen, J

    Kaler, T., Iliopoulos, A., Murzynowski, P., Schardl, T., Leiserson, C. E., and Chen, J. Communication-Efficient Graph Neural Networks with Probabilistic Neighborhood Expansion Analysis and Caching . Machine Learning and Systems (MLSys), 2023

    2023

  10. [18]

    S., Taleka, B., Ma, T., Song, X., and Hwu, W.-m

    Khatua, A., Mailthody, V. S., Taleka, B., Ma, T., Song, X., and Hwu, W.-m. IGB: Addressing the Gaps in Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Research . ACM SIGKDD Conf. on Knowledge Discovery & Data Mining (KDD), 2023

    2023

  11. [19]

    Kipf, T. N. and Welling, M. Semi-Supervised Classification with Graph Convolutional Networks . International Conference on Learning Representations (ICLR), 2017

    2017

  12. [20]

    SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization

    Liao, N., Mo, D., Luo, S., Li, X., and Yin, P. SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization . Int'l Conf. on Very Large Data Bases (VLDB), 2022

    2022

  13. [21]

    LD2: Scalable Heterophilous Graph Neural Network with Decoupled Embeddings

    Liao, N., Luo, S., Li, X., and Shi, J. LD2: Scalable Heterophilous Graph Neural Network with Decoupled Embeddings . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2024

    2024

  14. [22]

    L., Gupta, V., Bhalerao, O., and Lim, S

    Lim, D., Hohne, F., Li, X., Huang, S. L., Gupta, V., Bhalerao, O., and Lim, S. N. Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2021

    2021

  15. [23]

    PaGraph: Scaling GNN Training on Large Graphs via Computation-Aware Caching

    Lin, Z., Li, C., Miao, Y., Liu, Y., and Xu, Y. PaGraph: Scaling GNN Training on Large Graphs via Computation-Aware Caching . ACM Symp. on Cloud Computing (SoCC), 2020

    2020

  16. [24]

    BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing

    Liu, T., Chen, Y., Li, D., Wu, C., Zhu, Y., He, J., Peng, Y., Chen, H., Chen, H., and Guo, C. BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing . USENIX Symp. on Networked Systems Design and Implementation (NSDI), 2023

    2023

  17. [25]

    Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling

    Meng, Q., Chen, W., Wang, Y., Ma, Z.-M., and Liu, T.-Y. Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling . Neurocomputing , 337: 0 46--57, 2019

    2019

  18. [26]

    Random Reshuffling: Simple Analysis With Vast Improvements

    Mishchenko, K., Khaled, A., and Richt \'a rik, P. Random Reshuffling: Simple Analysis With Vast Improvements . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2020

    2020

  19. [27]

    T., Trahay, F., Domke, J., Drozd, A., Vatai, E., Liao, J., Wahib, M., and Gerofi, B

    Nguyen, T. T., Trahay, F., Domke, J., Drozd, A., Vatai, E., Liao, J., Wahib, M., and Gerofi, B. Why Globally Re-Shuffle? Revisiting Data Shuffling in Large Scale Deep Learning . Int'l Parallel and Distributed Processing Symp. (IPDPS), 2022

    2022

  20. [28]

    and Maehara, T

    Nt, H. and Maehara, T. Revisiting Graph Neural Networks: All We Have Is Low-Pass Filters . arXiv preprint arXiv:1905.09550, 2019

    1905 arXiv

  21. [29]

    Park, Y., Min, S., and Lee, J. W. Ginex: SSD-Enabled Billion-Scale Graph Neural Network Training on a Single Machine via Provably Optimal In-Memory Caching . Int'l Conf. on Very Large Data Bases (VLDB), 2022

    2022

  22. [30]

    TPUGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs

    Phothilimthana, M., Abu-El-Haija, S., Cao, K., Fatemi, B., Burrows, M., Mendis, C., and Perozzi, B. TPUGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2024

    2024

  23. [31]

    Schroeder, T. C. Peer-to-Peer & Unified Virtual Addressing . GPU Technology Conference, NVIDIA , 2011

    2011

  24. [32]

    u tt, K., Kindermans, P.-J., Sauceda Felix, H. E., Chmiela, S., Tkatchenko, A., and M \

    Sch \"u tt, K., Kindermans, P.-J., Sauceda Felix, H. E., Chmiela, S., Tkatchenko, A., and M \"u ller, K.-R. SchNet: A Continuous-Filter Convolutional Neural Network for Modeling Quantum Interactions . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2017

    2017

  25. [33]

    Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Training

    Sun, J., Su, L., Shi, Z., Shen, W., Wang, Z., Wang, L., Zhang, J., Li, Y., Yu, W., Zhou, J., et al. Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Training . USENIX Annual Technical Conference (USENIX ATC), 2023

    2023

  26. [34]

    Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness

    Tan, Z., Yuan, X., He, C., Sit, M.-K., Li, G., Liu, X., Ai, B., Zeng, K., Pietzuch, P., and Mai, L. Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness . arXiv preprint arXiv:2305.10863, 2023

    2023 arXiv

  27. [35]

    and Newburn, C

    Thompson, A. and Newburn, C. J. GPUDirect Storage: A Direct Path Between Storage and GPU Memory . NVIDIA Developer Whitepapers , 8, 2019

    2019

  28. [36]

    Graph Clustering with Graph Neural Networks

    Tsitsulin, A., Palowitch, J., Perozzi, B., and M \"u ller, E. Graph Clustering with Graph Neural Networks . Journal of Machine Learning Research, 24 0 (127): 0 1--21, 2023

    2023

  29. [37]

    Graph Attention Networks

    Veli c kovi \'c , P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. Graph Attention Networks . International Conference on Learning Representations (ICLR), 2018

    2018

  30. [38]

    Wang, M. Y. Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs . ICLR workshop on representation learning on graphs and manifolds, 2019

    2019

  31. [39]

    Simplifying Graph Convolutional Networks

    Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., and Weinberger, K. Simplifying Graph Convolutional Networks . Int'l Conf. on Machine Learning (ICML), 2019

    2019

  32. [40]

    Gamora: Graph Learning Based Symbolic Reasoning for Large-Scale Boolean Networks

    Wu, N., Li, Y., Hao, C., Dai, S., Yu, C., and Xie, Y. Gamora: Graph Learning Based Symbolic Reasoning for Large-Scale Boolean Networks . Design Automation Conf. (DAC), 2023

    2023

  33. [41]

    and Cong, G

    Yang, C.-C. and Cong, G. Accelerating Data Loading in Deep Neural Network Training . Int'l Conf. on High-Performance Computing, Data, and Analytics (HiPC), 2019

    2019

  34. [42]

    GNNLab: A Factored System for Sample-Based GNN Training Over GPUs

    Yang, J., Tang, D., Song, X., Wang, L., Yin, Q., Chen, R., Yu, W., and Zhou, J. GNNLab: A Factored System for Sample-Based GNN Training Over GPUs . European Conf. on Computer Systems (EuroSys), 2022

    2022

  35. [43]

    L., and Leskovec, J

    Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., and Leskovec, J. Graph Convolutional Neural Networks for Web-Scale Recommender Systems . ACM SIGKDD Conf. on Knowledge Discovery & Data Mining (KDD), 2018 a

    2018

  36. [44]

    Hierarchical Graph Representation Learning with Differentiable Pooling

    Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W., and Leskovec, J. Hierarchical Graph Representation Learning with Differentiable Pooling . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2018 b

    2018

  37. [45]

    Scalable Graph Neural Networks for Heterogeneous Graphs

    Yu, L., Shen, J., Li, J., and Lerer, A. Scalable Graph Neural Networks for Heterogeneous Graphs . arXiv preprint arXiv:2011.09679, 2020

    2011 arXiv

  38. [46]

    GraphSAINT: Graph Sampling Based Inductive Learning Method

    Zeng, H., Zhou, H., Srivastava, A., Kannan, R., and Prasanna, V. GraphSAINT: Graph Sampling Based Inductive Learning Method . International Conference on Learning Representations (ICLR), 2020

    2020

  39. [47]

    and Chen, Y

    Zhang, M. and Chen, Y. Link Prediction Based on Graph Neural Networks . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2018

    2018

  40. [48]

    Graph Attention Multi-Layer Perceptron

    Zhang, W., Yin, Z., Sheng, Z., Li, Y., Ouyang, W., Li, X., Tao, Y., Yang, Z., and Cui, B. Graph Attention Multi-Layer Perceptron . ACM SIGKDD Conf. on Knowledge Discovery & Data Mining (KDD), 2022

    2022

  41. [49]

    Attributed Graph Clustering via Adaptive Graph Convolution

    Zhang, X., Liu, H., Li, Q., and Wu, X.-M. Attributed Graph Clustering via Adaptive Graph Convolution . arXiv preprint arXiv:1906.01210, 2019

    1906 arXiv

  42. [50]

    and Koniusz, P

    Zhu, H. and Koniusz, P. Simple Spectral Graph Convolution . International Conference on Learning Representations (ICLR), 2020

    2020

  43. [51]

    Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks

    Zou, D., Hu, Z., Wang, Y., Jiang, S., Sun, Y., and Gu, Q. Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks . Conf. on Neural Information Processing Systems ( NeurIPS ) , 2019

    2019

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.