REVIEW 3 major objections 5 minor 102 references
OMEGA: A Low-Latency GNN Serving System for Large Graphs
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read OMEGA serves GNNs on billion-edge graphs with up to 159× lower latency
desk verdict A solid, well-evaluated GNN serving system whose deployed recomputation heuristic is better than the theory used to justify it; referee with revisions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is SRPE's recomputation policy, which ranks candidate nodes by the ratio $|N_Q(u)|/|N(u)|$ — the number of edges from query nodes to node $u$ divided by $u$'s total degree. This ratio proxies the theoretically optimal variance-minimizing weights $p_u \propto ||\sum_l q^{(l)}_u||$ while avoiding the impossible computation of full query embeddings. CGP then carries the execution: each machine aggregates messages from its local partition, exchanges partial aggregations through all-to-all, and applies a model-specific merge function (identity for sum, max, softmax with logits, or power-mean with pow operations) before the update function.
What would settle it
Run the reported Yelp GAT workload with a recomputation budget tuned to keep accuracy drop under 1% and then re-run it with the trained attention weights re-initialized or with edges rewired so that high-degree nodes have low query-edge ratios; if accuracy drops exceed one point under budget, the top-query-edges-ratio proxy is not carrying the accuracy claim.
Extended reading notes
Core claim
OMEGA's core discovery is that approximation errors in precomputed embeddings are highly skewed: a small fraction of nodes produces most of the error when a new query node connects to them. Recomputing just that top fraction restores accuracy almost fully. The system therefore proposes Selective Recomputation of Precomputed Embeddings (SRPE), with a top-query-edges-ratio policy that recomputes embeddings of nodes whose neighborhood has the highest ratio of query edges, and proves in Theorem 1 that recomputation probabilities proportional to $||\sum_{l=1}^{k-1} q^{(l)}_u||$ minimize estimator variance. To remove the remaining communication bottleneck, OMEGA adds Computation Graph Parallelism (CGP), where each machine builds and executes a local partition of the computation graph with local aggregation, then merges partial results with all-to-all collectives and custom merge functions for sum, max, power-mean, normalized-moment, and softmax-based aggregations. The evaluation reports up to 159× lower latency than full-computation-graph DGL serving and up to 10.8× lower latency than sampling-based DGL serving, with accuracy within 1 point of the full model.
Load-bearing premise
The load-bearing premise is that a recomputation candidate's error can be predicted by the ratio of query edges to its total degree, with aggregation treated as a mean of neighbor messages and GNN layers treated as statistically independent; for attention-based aggregators with correlated layers, this proxy has no proven guarantee and accuracy recovery depends on per-dataset budgets.
Editorial extensions
If this is right
- Serving latency for GNNs on billion-edge graphs can drop from seconds to tens of milliseconds, making real-time inference feasible on datasets like the 10-billion-edge FB10B workload.
- Accuracy stays within one percentage point of the full model when the recomputation budget is set per dataset, whereas neighborhood sampling can lose 2–6 points on attention-based and convolutional models.
- Computation graph parallelism turns communication from a dominant bottleneck into a few megabytes of collective traffic, and it scales with GPUs: OMEGA's latency drops 67% from 2 to 8 GPUs while sampling-based serving barely improves.
- Because PEs shrink computation graphs to direct neighbors, deeper GNN layers cost roughly linearly in latency rather than exponentially, as shown with GCNII up to six layers.
- The system also serves models with attention and generalized arithmetic aggregation by translating their local aggregations into merge functions, so the approach is not limited to sum or mean aggregators.
Reading between the lines
- The variance-minimization proof relies on treating GNN layers as statistically independent and aggregation as a mean; for attention models with learned, input-dependent weights and correlated layers, the top-query-edges-ratio policy has no formal error guarantee beyond the empirical budgets reported.
- One testable extension is caching recomputed embeddings for frequently queried nodes across requests, which the paper does not explore but which could reduce recomputation costs further on skewed query workloads.
- The system assumes query nodes attach only to existing training nodes; handling dynamic edge insertions or node deletions after deployment would require a staleness or invalidation mechanism that OMEGA explicitly leaves for future work.
- The reported accuracy numbers are measured against a fixed training/test split with 25% of test nodes held out; a realistic deployment with drifting query distributions could require re-tuning the recomputation budget, and the paper does not provide an online adaptation rule.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents OMEGA, a distributed GNN serving system that combines two techniques: selective recomputation of precomputed embeddings (SRPE), which reuses layer embeddings of training nodes and recomputes a small fraction of error-prone embeddings, and computation graph parallelism (CGP), which partitions the construction and execution of computation graphs across machines with custom merge functions for different GNN aggregations. The system is implemented on top of DGL and evaluated on six graph datasets and three GNN models. The paper reports large latency reductions, up to 159× versus full-graph serving and up to 10.8× versus sampling-based serving, with accuracy drops kept below 1 percentage point. A theoretical contribution is claimed: Theorem 1 states that the recomputation probabilities minimize the variance of unbiased embedding estimators, and the paper argues that the deployed top-query-edges-ratio policy statistically minimizes approximation errors.
Significance. If the claims are scoped carefully, the paper is a solid systems contribution. The evaluation is extensive: six datasets, three models, multiple batch sizes, scaling studies, latency breakdowns, and a latency-throughput analysis. The system design is plausible and the reported latency benefits are large. The paper also provides an analytical latency model for CGP in Appendix D, which is a useful check on the empirical results. The main weakness is that the theoretical statement is used to support a guarantee that the deployed heuristic does not actually satisfy, and the accuracy-loss claims are partly established by tuning the recomputation budget on the same workload used for evaluation. These issues do not invalidate the empirical latency findings, but they require a careful reframing of the paper's accuracy and optimality claims.
major comments (3)
- [§5.2.1–5.2.2, Appendix A] Theorem 1 derives optimal recomputation probabilities p_u ∝ ||Σ_{l=1}^{k-1} Σ_{v∈N_Q(u)} m_v^(l)/|N(u)| ||, which depend on the actual query-node messages m_v^(l). The deployed top-query-edges-ratio policy replaces this quantity with |N_Q(u)|/|N(u)|, effectively dropping the message terms. As stated in §5.2.2, the exact calculation is infeasible because it requires full query embeddings. Consequently, the abstract's phrase 'statistically minimizes approximation errors' and the §5.2.2 reference to 'optimal probabilities' are not properties of the system as implemented. The paper should either prove a separate approximation guarantee for the degree-ratio proxy under explicit conditions (e.g., homogeneous message norms), or clearly label the proxy as a heuristic and remove the language that ties its accuracy to Theorem 1.
- [§5.2.2, Table 3, §8.2] The 'minimal accuracy loss' claim is weaker than it appears because for the majority of dataset-model combinations in Table 3 the recomputation budget is γ=0. For Reddit, Products, and Papers, all models use γ=0, meaning SRPE does not perform any recomputation and the reported accuracy drops are entirely due to plain PE reuse. For the configurations where γ>0 (e.g., Yelp GCN with γ=20% and Yelp GAT with γ=7%), the budget is explicitly selected so that the accuracy drop is below 1 percentage point on the same workload used to report the final accuracy. This makes the <1% drop a tuning target rather than an independent prediction. The paper should separate the measured accuracy of the policy at a fixed budget from the budget-selection procedure, and should report results at representative budgets without per-dataset tuning.
- [§5.2.2, §8.2, Table 4] For attention-based models such as GAT, the degree-ratio proxy has no demonstrated error guarantee. GAT's softmax attention weights are learned and can be highly nonuniform per edge, so a single high-attention query edge may dominate approximation error regardless of the ratio |N_Q(u)|/|N(u)|. The empirical comparison in Fig. 18 shows that the policy works on the tested GAT workloads, but this does not establish the general claim that it 'statistically minimizes approximation errors.' The paper should either include an analysis or experiments specifically probing attention-weight distributions (e.g., comparing the degree-ratio ranking against rankings based on actual attention-weighted messages), or explicitly restrict the theoretical claim to mean-aggregation models and present the GAT results as empirical evidence only.
minor comments (5)
- [§6.1, Eq. (3)] The symbol U is used both for the GNN update function in Eq. (1) and for the CGP merge function in Eq. (3); these are different operations and should use distinct symbols to avoid confusion.
- [§5.2.1] The notation q_u^(l) is introduced in the text but the estimator \f\u005e(l)_u is written with q_u^(l) and t_u^(l) without explicitly defining t_u^(l) in the main text; it appears only in Appendix A. A short definition in the main text would improve readability.
- [Figure 6 (right)] The legend lists '10' next to the OMEGA line, which appears to be a leftover artifact; the labeled curves are RANDOM, AE, IS, and OMEGA, so the stray '10' should be removed.
- [§7] The paper does not state whether the implementation or evaluation scripts will be released; a reproducibility or artifact availability statement would strengthen the systems contribution.
- [Throughout] The abstract in the submitted text does not include the quantitative speedup numbers (159×, 10.8×) that are emphasized in the paper's evaluation and in the reader's summary; adding these to the abstract would make the contribution more visible, though it is not required.
Circularity Check
No circular derivation: SRPE's theorem is adapted from external prior work, the deployed heuristic is an explicitly approximate policy, and the γ budget is a transparently configured tradeoff parameter.
full rationale
The paper's derivation chain is self-contained rather than circular. Theorem 1 is explicitly adapted from Theorem 3.2 of GraphSAINT [90], an external result, and the proof in Appendix A states its simplifying assumption (independent GNN layers) rather than smuggling in the conclusion. The deployed top-query-edges-ratio policy is transparently a simplification: §5.2.2 says the exact optimal probabilities require full query-node messages, 'which is infeasible in serving', and therefore the policy approximates p_u ∝ |N_Q(u)|/|N(u)| 'without considering the message terms'. The accuracy claims are therefore supported by empirical comparisons against RANDOM, IS, and DGL baselines (§5.2.2, Fig. 6, Fig. 18, Table 4), not by equating the heuristic with the theorem. The only parameter that could look fitted, γ, is explicitly a user-controlled budget: Table 3's caption states that γ is 'the recomputation budget to achieve less than 1% points of accuracy drop', and §5.2.2 says users adjust it based on acceptable accuracy drop and latency. Selecting a threshold and then reporting the resulting accuracy is a configuration choice, not a fitted quantity renamed as a prediction, and the latency speedups (Figs. 10, 11) are measured end-to-end against DGL baselines independently of the accuracy target. The only coauthor self-citation is [34], used as related-work context and not load-bearing for any central claim. No circular step can be exhibited, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (1)
- Recomputation budget gamma =
Yelp GCN 20%, Amazon GCN 3%, Yelp GAT 7%, Amazon GAT 1%, 0% for others (Table 3)
assumptions (4)
- domain assumption Each GNN layer independently learns embeddings, enabling statistical analysis despite nonlinear activations.
- domain assumption Aggregation is a mean of neighbor messages in the SRPE error model.
- domain assumption Approximation error is dominated by direct neighbors of query nodes, so recomputation candidates are restricted to direct neighbors.
- domain assumption Synthetic serving workload, created by removing 25% random test nodes and adding query edges, represents real serving traffic.
Cite this review
Pith. "Pith review of OMEGA: A Low-Latency GNN Serving System for Large Graphs." pith.science (2026). https://pith.science/paper/U4IN3O5J
@misc{pith2026250108547,
author = {Pith},
title = {Pith review of: OMEGA: A Low-Latency GNN Serving System for Large Graphs},
year = {2026},
howpublished = {\url{https://pith.science/paper/U4IN3O5J}},
note = {Machine review of arXiv:2501.08547}
}
read the original abstract
Graph Neural Networks (GNNs) have been widely adopted for their ability to compute expressive node representations in graph datasets. However, serving GNNs on large graphs is challenging due to the high communication, computation, and memory overheads of constructing and executing computation graphs, which represent information flow across large neighborhoods. Existing approximation techniques in training can mitigate the overheads but, in serving, still lead to high latency and/or accuracy loss. To this end, we propose OMEGA, a system that enables low-latency GNN serving for large graphs with minimal accuracy loss through two key ideas. First, OMEGA employs selective recomputation of precomputed embeddings, which allows for reusing precomputed computation subgraphs while selectively recomputing a small fraction to minimize accuracy loss. Second, we develop computation graph parallelism, which reduces communication overhead by parallelizing the creation and execution of computation graphs across machines. Our evaluation with large graph datasets and GNN models shows that OMEGA significantly outperforms state-of-the-art techniques.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
-
[1]
https : / / www.nvidia.com/en- us/data- center/a100/
NVIDIA A100 Tensor Core GPU. https : / / www.nvidia.com/en- us/data- center/a100/ . [Ac- cessed 06-16-2024]
2024
-
[2]
https://www.nvidia.com/ en - us / networking / ethernet / connectx - 4 - lx/
NVIDIA ConnectX-4 Lx. https://www.nvidia.com/ en - us / networking / ethernet / connectx - 4 - lx/. [Accessed 06-16-2024]
2024
-
[3]
https://www.nvidia.com/en- us/networking/ethernet/connectx-5/
NVIDIA ConnectX-5. https://www.nvidia.com/en- us/networking/ethernet/connectx-5/. [Accessed 06-16-2024]
2024
-
[4]
https://www.nvidia.com/ en - us / networking / ethernet / connectx - 6 - dx/
NVIDIA ConnectX-6 Dx. https://www.nvidia.com/ en - us / networking / ethernet / connectx - 6 - dx/. [Accessed 06-16-2024]
2024
-
[5]
https : / / www.nvidia.com/en- us/data- center/h100/
NVIDIA H100 Tensor Core GPU. https : / / www.nvidia.com/en- us/data- center/h100/ . [Ac- cessed 06-16-2024]
2024
-
[6]
https://www.nvidia.com/en- us/data- center/tesla- p100/
NVIDIA Tesla P100. https://www.nvidia.com/en- us/data- center/tesla- p100/ . [Accessed 06-16- 2024]
2024
-
[7]
https : / / www.nvidia.com/en- us/data- center/v100/
NVIDIA V100 Tensor Core GPU. https : / / www.nvidia.com/en- us/data- center/v100/ . [Ac- cessed 06-16-2024]
2024
-
[8]
Build a GNN-based real-time fraud detection solu- tion using Amazon Sagemaker, Amazon Neptune, and the Deep Graph Library. https://aws .amazon.com/ blogs / machine - learning / build - a - gnn - based - real - time - fraud - detection - solution - using - amazon - sagemaker - amazon - neptune - and - the - deep-graph-library/, 2024. [Accessed 10-09-2024]
2024
Show all 102 references
-
[9]
https: //github.com/facebookincubator/gloo, 2024
Gloo: a collective communications library. https: //github.com/facebookincubator/gloo, 2024. [Ac- cessed 10-09-2024]
2024
-
[10]
https://github.com/NVIDIA/nccl,
NVIDIA NCCL. https://github.com/NVIDIA/nccl,
-
[11]
Abdine, M
H. Abdine, M. Chatzianastasis, C. Bouyioukos, and M. Vazirgiannis. Prot2text: Multimodal protein’s func- tion generation with gnns and transformers. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10757–10765, 2024
2024
-
[12]
Achiam, S
J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023
2023 arXiv
-
[13]
Atkinson, A
O. Atkinson, A. Bhardwaj, C. Englert, V . S. Ngairang- bam, and M. Spannowsky. Anomaly detection with convolutional graph neural networks. Journal of High Energy Physics, 2021(8):1–19, 2021
2021
-
[14]
L. Bai, L. Yao, C. Li, X. Wang, and C. Wang. Adap- tive graph convolutional recurrent network for traffic forecasting. Advances in neural information processing systems, 33:17804–17815, 2020
2020
-
[15]
Battaglia, J
P. Battaglia, J. B. C. Hamrick, V . Bapst, A. Sanchez, V . Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, C. Gulcehre, F. Song, A. Bal- lard, J. Gilmer, G. E. Dahl, A. Vaswani, K. Allen, C. Nash, V . J. Langston, C. Dyer, N. Heess, D. Wierstra, P. K...
2018
-
[16]
J. S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to sta- tistical pattern recognition. In Neurocomputing: Algo- rithms, architectures and applications, pages 227–236. Springer, 1990
1990
-
[17]
Brody, U
S. Brody, U. Alon, and E. Yahav. How atten- tive are graph attention networks? arXiv preprint arXiv:2105.14491, 2021
2021 arXiv
-
[18]
Brown, B
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Ka- plan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. ...
1901
-
[19]
J. Chen, T. Ma, and C. Xiao. FastGCN: Fast learn- ing with graph convolutional networks via importance sampling. In International Conference on Learning Representations, 2018
2018
-
[20]
J. Chen, J. Zhu, and L. Song. Stochastic training of graph convolutional networks with variance reduction. In In- ternational Conference on Machine Learning , pages 941–949, 2018
2018
-
[21]
M. Chen, Z. Wei, Z. Huang, B. Ding, and Y . Li. Sim- ple and deep graph convolutional networks. In Inter- national conference on machine learning, pages 1725–
-
[22]
Chiang, X
W.-L. Chiang, X. Liu, S. Si, Y . Li, S. Bengio, and C.-J. Hsieh. Cluster-GCN: An efficient algorithm for train- ing deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, pages ...
2019
-
[23]
Chien, W.-C
E. Chien, W.-C. Chang, C.-J. Hsieh, H.-F. Yu, J. Zhang, O. Milenkovic, and I. S. Dhillon. Node feature ex- traction by self-supervised multi-scale neighborhood prediction. In International Conference on Learning Representations (ICLR), 2022
2022
-
[24]
Corso, L
G. Corso, L. Cavalleri, D. Beaini, P. Liò, and P. Veliˇckovi´c. Principal neighbourhood aggregation for graph nets. Advances in Neural Information Processing Systems, 33:13260–13271, 2020
2020
-
[25]
Damania, S
P. Damania, S. Li, A. Desmaison, A. Azzolini, B. Vaughan, E. Yang, G. Chanan, G. J. Chen, H. Jia, H. Huang, J. Spisak, L. Wehrstedt, L. Hosseini, M. Kr- ishnan, O. Salpekar, P. Belevich, R. Varma, S. Gera, W. Liang, S. Xu, S. Chintala, C. He, A. Ziashahabi, S. Avestimehr, and ...
2023
-
[26]
T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré. FlashAt- tention: Fast and memory-efficient exact attention with io-awareness. In S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 163...
2022
-
[27]
S. Deng, H. Rangwala, and Y . Ning. Learning dynamic context graphs for predicting social events. In Proceed- ings of the 25th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining , KDD ’19, pages 1007–1016, New York, NY , USA, 2019. As- sociation for Com...
2019
-
[28]
Derrow-Pinion, J
A. Derrow-Pinion, J. She, D. Wong, O. Lange, T. Hes- ter, L. Perez, M. Nunkesser, S. Lee, X. Guo, B. Wilt- shire, et al. Eta prediction with graph neural networks in google maps. In Proceedings of the 30th ACM In- ternational Conference on Information & Knowledge Management, p...
2021
-
[29]
Edunov, D
S. Edunov, D. Logothetis, C. Wang, A. Ching, and M. Kabiljo. Generating synthetic social graphs with Darwini. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pages 567– 577, 2018
2018
-
[30]
W. Fan, Y . Ma, Q. Li, Y . He, E. Zhao, J. Tang, and D. Yin. Graph neural networks for social recommenda- tion. In The World Wide Web Conference, WWW ’19, page 417–426, New York, NY , USA, 2019. Association for Computing Machinery
2019
-
[31]
Fey and J
M. Fey and J. E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019
2019
-
[32]
M. Fey, J. E. Lenssen, F. Weichert, and J. Leskovec. GN- NAutoScale: Scalable and expressive graph neural net- works via historical embeddings. In International con- ference on machine learning, pages 3294–3304. PMLR, 2021
2021
-
[33]
A. Fout, J. Byrd, B. Shariat, and A. Ben-Hur. Protein interface prediction using graph convolutional networks. Advances in neural information processing systems, 30, 2017
2017
-
[34]
Gandhi and A
S. Gandhi and A. P. Iyer. P3: Distributed deep graph learning at scale. In 15th USENIX Symposium on Oper- ating Systems Design and Implementation (OSDI 21), pages 551–568, 2021
2021
-
[35]
Geyer and S
F. Geyer and S. Bondorf. DeepTMA: Predicting ef- fective contention models for network calculus using graph neural networks. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pages 1009–1017, 2019
2019
-
[36]
Gilmer, S
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chem- istry. In International conference on machine learning, pages 1263–1272. PMLR, 2017
2017
-
[37]
W. L. Hamilton, R. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Infor- mation Processing Systems, NIPS’17, page 1025–1035, Red Hook, NY , USA, 2017. Curran Associates Inc. 14
2017
-
[38]
W. L. Hamilton, R. Ying, and J. Leskovec. Represen- tation learning on graphs: Methods and applications. arXiv preprint arXiv:1709.05584, 2017
2017 arXiv
-
[39]
Hochreiter and J
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997
1997
-
[40]
Hoffmann, S
J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022
2022 arXiv
-
[41]
H. Hu, F. Liu, Q. Pei, Y . Yuan, Z. Xu, and L. Wang. λGrapher: A resource-efficient serverless system for gnn serving through graph sharing. In The World Wide Web Conference, WWW ’24. Association for Computing Machinery, 2024
2024
-
[42]
W. Hu, M. Fey, H. Ren, M. Nakata, Y . Dong, and J. Leskovec. OGB-LSC: A large-scale challenge for machine learning on graphs. In J. Vanschoren and S. Ye- ung, editors, Proceedings of the Neural Information Pro- cessing Systems Track on Datasets and Benchmarks 1, NeurIPS Datase...
2021
-
[43]
W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec. Open Graph Benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , 33:22118– 22133, 2020
2020
-
[44]
Huang, T
W. Huang, T. Zhang, Y . Rong, and J. Huang. Adap- tive sampling towards fast graph representation learning. Advances in neural information processing systems, 31, 2018
2018
-
[45]
Z. Jia, S. Lin, M. Gao, M. Zaharia, and A. Aiken. Im- proving the accuracy, scalability, and performance of graph neural networks with ROC. In I. Dhillon, D. Pa- pailiopoulos, and V . Sze, editors,Proceedings of Ma- chine Learning and Systems, volume 2, pages 187–198, 2020
2020
-
[46]
Z. Jia, S. Lin, R. Ying, J. You, J. Leskovec, and A. Aiken. Redundancy-free computation for graph neural net- works. In Proceedings of the 26th ACM SIGKDD Inter- national Conference on Knowledge Discovery & Data Mining, KDD ’20, page 997–1005, New York, NY , USA,
-
[47]
Jiang and J
W. Jiang and J. Luo. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications, page 117921, 2022
2022
-
[48]
M. Jin, Y . Liu, Y . Zheng, L. Chi, Y .-F. Li, and S. Pan. ANEMONE: graph anomaly detection with multi-scale contrastive learning. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 3122–3126, 2021
2021
-
[49]
Kaler, N
T. Kaler, N. Stathas, A. Ouyang, A.-S. Iliopoulos, T. Schardl, C. E. Leiserson, and J. Chen. Accelerat- ing training and inference of graph neural networks with fast sampling and pipelining. Proceedings of Machine Learning and Systems, 4:172–189, 2022
2022
-
[50]
Karypis and V
G. Karypis and V . Kumar. A fast and high quality multi- level scheme for partitioning irregular graphs. SIAM J. Sci. Comput., 20(1):359–392, Dec. 1998
1998
-
[51]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[52]
T. N. Kipf and M. Welling. Semi-Supervised Classifica- tion with Graph Convolutional Networks. In Proceed- ings of the 5th International Conference on Learning Representations, ICLR ’17, 2017
2017
-
[53]
G. Li, M. Muller, A. Thabet, and B. Ghanem. Deep- GCNs: Can GCNs go as deep as CNNs? In Proceedings of the IEEE/CVF international conference on computer vision, pages 9267–9276, 2019
2019
-
[54]
G. Li, C. Xiong, A. Thabet, and B. Ghanem. Deep- erGCN: All you need to train deeper gcns. arXiv preprint arXiv:2006.07739, 2020
2006 arXiv
-
[55]
Q. Li, Z. Han, and X.-M. Wu. Deeper insights into graph convolutional networks for semi-supervised learn- ing. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018
2018
-
[56]
D. Lin, S. Sun, J. Ding, X. Ke, H. Gu, X. Huang, C. Song, X. Zhang, L. Yi, J. Wen, et al. Platogl: Effective and scalable deep graph learning system for graph-enhanced real-time recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edg...
2022
-
[57]
Z. Lin, C. Li, Y . Miao, Y . Liu, and Y . Xu. PaGraph: Scal- ing gnn training on large graphs via computation-aware caching. In Proceedings of the 11th ACM Symposium on Cloud Computing, SoCC ’20, pages 401–415, New York, NY , USA, 2020. Association for Computing Machinery
2020
-
[58]
T. Liu, Y . Chen, D. Li, C. Wu, Y . Zhu, J. He, Y . Peng, H. Chen, H. Chen, and C. Guo. BGL: GPU-Efficient GNN training by optimizing graph data I/O and prepro- cessing. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 103–118, Boston, M...
2023
-
[59]
Y .-C. Lo, S. E. Rensi, W. Torng, and R. B. Altman. Ma- chine learning in chemoinformatics and drug discovery. Drug Discovery Today, 23(8):1538 – 1546, 2018
2018
-
[60]
M. Lu, Z. Han, S. X. Rao, Z. Zhang, Y . Zhao, Y . Shan, R. Raghunathan, C. Zhang, and J. Jiang. Bright-graph neural networks in real-time fraud detection. In Pro- ceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 3342– 3351, 2022
2022
-
[61]
L. Ma, Z. Yang, Y . Miao, J. Xue, M. Wu, L. Zhou, and Y . Dai. NeuGraph: Parallel deep neural network compu- tation on large graphs. In 2019 USENIX Annual Tech- nical Conference (USENIX ATC 19) , pages 443–458, Renton, W A, July 2019. USENIX Association
2019
-
[62]
V . Md, S. Misra, G. Ma, R. Mohanty, E. Georganas, A. Heinecke, D. Kalamkar, N. K. Ahmed, and S. Avan- cha. DistGNN: Scalable distributed training for large- scale graph neural networks. In Proceedings of the International Conference for High Performance Com- puting, Networkin...
2021
-
[63]
Ong and P
E. Ong and P. Veli ˇckovi´c. Learnable commutative monoids for graph neural networks. In Learning on Graphs Conference, pages 43–1. PMLR, 2022
2022
-
[64]
Paszke, S
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017
2017
-
[65]
J. Peng, Z. Chen, Y . Shao, Y . Shen, L. Chen, and J. Cao. Sancus: staleness-aware communication-avoiding full- graph decentralized training in large-scale graph neu- ral networks. Proceedings of the VLDB Endowment , 15(9):1937–1950, 2022
1937
-
[66]
J. Shi, V . Chaurasiya, Y . Liu, S. Vij, Y . Wu, S. Kanduri, N. Shah, P. Yu, N. Srivastava, L. Shi, G. Venkataraman, and J. Yu. Embedding based retrieval in friend recom- mendation. In H. Chen, W. E. Duh, H. Huang, M. P. Kato, J. Mothe, and B. Poblete, editors, Proceedings of ...
2023
-
[67]
Y . Shi, A. Zhang, E. Zhang, Z. Liu, and X. Wang. ReLM: Leveraging language models for enhanced chemical re- action prediction. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023
2023
-
[68]
Sinha, Z
A. Sinha, Z. Shen, Y . Song, H. Ma, D. Eide, B.-J. P. Hsu, and K. Wang. An overview of Microsoft Academic Service (MAS) and applications. In Proceedings of the 24th International Conference on World Wide Web, WWW ’15 Companion, page 243–246, New York, NY , USA, 2015. Associati...
2015
-
[69]
J. M. Stokes, K. Yang, K. Swanson, W. Jin, A. Cubillos- Ruiz, N. M. Donghia, C. R. MacNair, S. French, L. A. Carfrae, Z. Bloom-Ackermann, V . M. Tran, A. Chiappino-Pepe, A. H. Badran, I. W. Andrews, E. J. Chory, G. M. Church, E. D. Brown, T. S. Jaakkola, R. Barzilay, and J. J....
2020
-
[70]
Suarez-Varela, P
J. Suarez-Varela, P. Almasan, M. Ferriol-Galmes, K. Rusek, F. Geyer, X. Cheng, X. Shi, S. Xiao, F. Scarselli, A. Cabellos-Aparicio, and P. Barlet-Ros. Graph neural networks for communication networks: Context, use cases and opportunities. IEEE Network, pages 1–8, 2022
2022
-
[71]
J. Sun, L. Su, Z. Shi, W. Shen, Z. Wang, L. Wang, J. Zhang, Y . Li, W. Yu, J. Zhou, and F. Wu. Legion: Automatically pushing the envelope of Multi-GPU sys- tem for Billion-Scale GNN training. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pages 165–179, Boston, MA...
2023
-
[72]
Z. Tan, X. Yuan, C. He, M.-K. Sit, G. Li, X. Liu, B. Ai, K. Zeng, P. Pietzuch, and L. Mai. Quiver: Sup- porting GPUs for low-latency, high-throughput GNN serving with workload awareness. arXiv preprint arXiv:2305.10863, 2023
2023 arXiv
-
[73]
Thorpe, Y
J. Thorpe, Y . Qiao, J. Eyolfson, S. Teng, G. Hu, Z. Jia, J. Wei, K. V ora, R. Netravali, M. Kim, and G. H. Xu. Dorylus: Affordable, scalable, and accurate GNN train- ing with distributed CPU servers and serverless threads. In 15th USENIX Symposium on Operating Systems De- sig...
2021
-
[74]
Y . Tian, H. Song, Z. Wang, H. Wang, Z. Hu, F. Wang, N. V . Chawla, and P. Xu. Graph neural prompting with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19080–19088, 2024
2024
-
[75]
Veliˇckovi´c, G
P. Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y . Bengio. Graph attention networks. In International Conference on Learning Representations, 2018
2018
-
[76]
Virinchi, A
S. Virinchi, A. Saladi, and A. Mondal. Recommend- ing related products using graph neural networks in di- rected graphs. In Joint European Conference on Ma- chine Learning and Knowledge Discovery in Databases, pages 541–557. Springer, 2022. 16
2022
-
[77]
C. Wan, Y . Li, A. Li, N. S. Kim, and Y . Lin. BNS-GCN: Efficient full-graph training of graph convolutional net- works with partition-parallelism and random boundary node sampling. Proceedings of Machine Learning and Systems, 4:673–693, 2022
2022
-
[78]
M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X. Song, J. Zhou, C. Ma, L. Yu, Y . Gai, T. Xiao, T. He, G. Karypis, J. Li, and Z. Zhang. Deep Graph Library: A graph- centric, highly-performant package for graph neural net- works. arXiv preprint arXiv:1909.01315, 2019
1909 arXiv
-
[79]
Y . Wang, B. Feng, G. Li, S. Li, L. Deng, Y . Xie, and Y . Ding. GNNAdvisor: An adaptive and efficient run- time system for GNN acceleration on GPUs. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21), pages 515–531. USENIX Association, July 2021
2021
-
[80]
Y . Wang, B. Feng, Z. Wang, T. Geng, K. Barker, A. Li, and Y . Ding. MGG: Accelerating graph neural net- works with Fine-Grained Intra-Kernel Communication- Computation pipelining on Multi-GPU platforms. In 17th USENIX Symposium on Operating Systems Design and Implementation (...
2023
-
[81]
J. Wei, Y . Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Met- zler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022
2022
-
[82]
Wen and Y
Z. Wen and Y . Fang. Augmenting low-resource text clas- sification with graph-grounded pre-training and prompt- ing. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in In- formation Retrieval, SIGIR ’23, page 506–516, New York, NY , US...
2023
-
[83]
Y . Wu, D. Lian, Y . Xu, L. Wu, and E. Chen. Graph convo- lutional networks with markov random field reasoning for social spammer detection. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1054–1061, 2020
2020
-
[84]
Y . Wu, K. Ma, Z. Cai, T. Jin, B. Li, C. Zheng, J. Cheng, and F. Yu. Seastar: vertex-centric programming for graph neural networks. In Proceedings of the Sixteenth European Conference on Computer Systems, pages 359– 375, 2021
2021
-
[85]
K. Xu, C. Li, Y . Tian, T. Sonobe, K.-i. Kawarabayashi, and S. Jegelka. Representation learning on graphs with jumping knowledge networks. In International confer- ence on machine learning , pages 5453–5462. PMLR, 2018
2018
-
[86]
H. Yang. AliGraph: A comprehensive graph neural network platform. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Dis- covery & Data Mining , KDD ’19, pages 3165–3166, New York, NY , USA, 2019. Association for Computing Machinery
2019
-
[87]
J. Yang, Z. Liu, S. Xiao, C. Li, D. Lian, S. Agrawal, A. Singh, G. Sun, and X. Xie. GraphFormers: GNN- nested transformers for representation learning on tex- tual graph. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, editors,Advances in Neu- ral Info...
-
[88]
J. Yang, D. Tang, X. Song, L. Wang, Q. Yin, R. Chen, W. Yu, and J. Zhou. GNNLab: a factored system for sample-based gnn training over gpus. In Proceedings of the Seventeenth European Conference on Computer Systems, pages 417–434, 2022
2022
-
[89]
R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamil- ton, and J. Leskovec. Graph convolutional neural net- works for web-scale recommender systems. In Proceed- ings of the 24th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining , KDD ’18, pages 974–98...
2018
-
[90]
H. Zeng, H. Zhou, A. Srivastava, R. Kannan, and V . K. Prasanna. GraphSAINT: Graph sampling based induc- tive learning method. In8th International Conference on Learning Representations, ICLR, Addis Ababa, Ethiopia, April 26-30, 2020
2020
-
[91]
L. Zeng, P. Huang, K. Luo, X. Zhang, Z. Zhou, and X. Chen. Fograph: Enabling real-time deep graph infer- ence with fog computing. In Proceedings of the ACM Web Conference 2022, pages 1774–1784, 2022
2022
-
[92]
Zhang, X
D. Zhang, X. Huang, Z. Liu, J. Zhou, Z. Hu, X. Song, Z. Ge, L. Wang, Z. Zhang, and Y . Qi. AGL: A scalable system for industrial-purpose graph machine learning. Proc. VLDB Endow., 13(12):3125–3137, Aug. 2020
2020
-
[93]
Zhang, Y
S. Zhang, Y . Liu, Y . Sun, and N. Shah. Graph-less neural networks: Teaching old mlps new tricks via distillation. In The Tenth International Conference on Learning Rep- resentations, ICLR, Virtual Event, April 25-29, 2022
2022
-
[94]
Zhang, A
X. Zhang, A. Bosselut, M. Yasunaga, H. Ren, P. Liang, C. D. Manning, and J. Leskovec. GreaseLM: Graph REASoning enhanced language models. In Interna- tional Conference on Learning Representations, 2021. 17
2021
-
[95]
Zheng, H
C. Zheng, H. Chen, Y . Cheng, Z. Song, Y . Wu, C. Li, J. Cheng, H. Yang, and S. Zhang. ByteGNN: efficient graph neural network training at large scale. Proceed- ings of the VLDB Endowment, 15(6):1228–1242, 2022
2022
-
[96]
Zheng, C
D. Zheng, C. Ma, M. Wang, J. Zhou, Q. Su, X. Song, Q. Gan, Z. Zhang, and G. Karypis. DistDGL: Dis- tributed graph neural network training for billion-scale graphs. In 2020 IEEE/ACM 10th Workshop on Irregu- lar Applications: Architectures and Algorithms (IA3) , pages 36–44, Los...
2020
-
[97]
Zheng, X
D. Zheng, X. Song, C. Yang, D. LaSalle, and G. Karypis. Distributed hybrid cpu and gpu training for graph neural networks on billion-scale heterogeneous graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 4582– 4591, 2022
2022
-
[98]
G. Zhou, X. Zhu, C. Song, Y . Fan, H. Zhu, X. Ma, Y . Yan, J. Jin, H. Li, and K. Gai. Deep interest network for click-through rate prediction. In Y . Guo and F. Farooq, editors, Proceedings of the 24th ACM SIGKDD Inter- national Conference on Knowledge Discovery & Data Mining,...
-
[99]
J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun. Graph neural networks: A review of methods and applications. AI Open, 1:57–81, 2020. A Proof of Theorem 1 We describe the formal proof of Theorem 1. To prove the theorem, we assume each GNN layer in...
2020
-
[102]
B Evaluation on Recomputation Policies In this section, we evaluate the performance of OMEGA ’s recomputation policy against RANDOM and IS policies (§5.2.2)
≥ ( ∑ u∈R || k−1 ∑ l=1 q(l) u ||)2 (6) Since γ = ∑u∈R √pu2 and the right-hand side are constants, the first term is minimized when the following equality con- dition holds: ∀u ∈ R, || k−1 ∑ l=1 q(l) u || 1√pu ∝ √pu (7) Therefore, we conclude S is minimized if pu ∝ || ∑k−1 l=1 ...
-
[2020]
Association for Computing Machinery
-
[2024]
[Accessed 10-09-2024]
2024
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.