Pith. sign in

REVIEW 4 major objections 5 minor 59 references

SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SNI-GNN: SmartNIC-side prediction cuts GNN communication by 21–45%

desk verdict Genuinely new SmartNIC-based embedding prediction for GNN training, well-evaluated on real hardware, but the theoretical section overclaims and needs a major fix or downgrade. read the letter →

arxiv 2608.06441 v1 pith:RDJCSCTB submitted 2026-08-06 cs.LG cs.DC

Guofan Yu , Sitian Chen , Zhenheng Tang , Xiaowen Chu , Amelie Chi Zhou This is my paper · ORCID
classification cs.LGcs.DC
keywords graphneuralnetworksdistributedtrainingSmartNICoffloadingembeddingpredictionstalenessmitigationfull-graphcommunicationreductionin-networkcomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SNI-GNN claims that the inter-node communication bottleneck in distributed full-graph GNN training can be attacked at the network itself: a SmartNIC, a programmable network card, sits on the data path, caches remote embeddings, and uses a lightweight linear-trend predictor to extrapolate their current values between synchronization rounds. This lets workers skip most remote embedding fetches while keeping the aggregated embeddings nearly fresh, so accuracy stays within about one percentage point of fully synchronous training. The paper reports that adding this in-network predictor to existing full-graph training systems cuts communication by 21–45%, yields 1.3–3.6× end-to-end speedups over a sampling-based baseline and up to 1.29× over an optimized staleness-aware baseline, and scales to 16 GPUs on graphs with tens of millions of edges. If correct, it shows that active in-network staleness compensation is a practical complement to partitioning and compression for communication-efficient full-graph GNN training.

What carries the argument

The load-bearing object is the in-network linear-trend predictor, $\hat{h}_v(t)=\bar{h}_v+\Delta t\,\Phi_v e^{-\lambda\Delta t}$, run on the SmartNIC between synchronization epochs. It estimates a baseline via exponential moving average and a trend vector from recent slopes, then extrapolates the embedding forward and exponentially decays the trend so predicted values do not run away as staleness grows. The companion machinery is the bounded-second-order-dynamics assumption, $\|H(t)-2H(t-1)+H(t-2)\|_\infty\le\epsilon$, which turns the extrapolation error into the quadratic bound $\epsilon_H=\frac{1}{2}(\Delta t)^2\epsilon$; importance-based boundary-node sampling keeps the DPU's compute and memory footprint small, while the asynchronous pipeline hides prediction latency behind GPU computation. Together these pieces convert the old trick of reusing stale historical embeddings into an active, hardware-isolated compensation mechanism.

What would settle it

Run SNI-GNN's exact setup on a graph where, during steady training, the measured maximum second-order difference of embeddings exceeds the assumed epsilon, for instance under a high learning rate without warm-up, and check whether the predictor error follows the claimed $\frac{1}{2}(\Delta t)^2\epsilon$ bound; if the error grows faster than quadratic in the sync interval, or if final accuracy at sync interval 5 drops more than 0.01 below synchronous training, the paper's central accuracy-and-convergence claim would be falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that remote GNN embeddings are smooth enough in time that a single lightweight linear-trend model, executed on the SmartNIC rather than on the host or GPU, can predict them accurately enough to replace most inter-node traffic during training. The predictor extrapolates from a ring-buffer history using a baseline plus a decayed trend, and the paper proves that as long as embeddings have bounded second-order dynamics, the prediction error stays below $\frac{1}{2}(\Delta t)^2\epsilon$, which in turn keeps the gradient error bounded and preserves the $O(1/\sqrt{N})$ non-convex SGD convergence rate. The system pairs the predictor with boundary-node importance sampling, meaning only the most-connected remote neighbors get predicted, plus an asynchronous DPU–GPU pipeline and self-feeding reuse of intermediate trend results. Evaluated by integrating into SANCUS and NeutronTP on BlueField-3 hardware, SNI-GNN claims communication reductions of 21–45%, end-to-end speedups of 1.3–3.6× over BNS-GCN and up to 1.29× over SANCUS, with peak accuracy loss no greater than 0.01 on the tested datasets.

Load-bearing premise

The entire argument leans on the premise that node embeddings change smoothly enough between synchronization rounds that a straight-line extrapolation from recent history stays close to the true value, formally that the second-order difference of embeddings is bounded by a small epsilon; the paper's own robustness experiments show this smoothness fails during early epochs and high learning rates, so the formal guarantee is conditional even though the system degrades gracefully.

Editorial extensions

If this is right

  • Full-graph GNN training can safely skip inter-node embedding broadcasts for several epochs at a time, because in-network prediction recovers most of the freshness that full synchronization would provide.
  • Existing full-graph training systems, both partition-parallel and tensor-parallel, can absorb the SmartNIC predictor as a drop-in enhancement, so communication savings compound with the system's own overlapping and compression strategies.
  • The safe synchronization interval grows with the smoothness of embedding dynamics: when second-order differences are small, the same accuracy can be kept at reduced communication frequency.
  • Because the predictor is hardware-isolated on the DPU, it adds no PCIe traffic or host CPU contention, so the speedup is not bought at the price of congesting the host I/O path.
  • Accuracy loss stays bounded, at most 0.01 on the evaluated datasets, and standard non-convex convergence with inexact gradients still holds, so users do not face a stark accuracy-versus-speed trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same in-network prediction pattern could apply to other communication-heavy distributed training loops, such as attention over large key-value caches or coordinate-descent-style updates, wherever the exchanged state is smooth in time and the network card has idle compute.
  • Editorial inference: the theory's dependence on bounded second-order dynamics suggests a practical adaptive controller, one that measures empirical curvature online and lengthens or shortens the synchronization interval, which the paper does not itself develop.
  • Editorial inference: if embedding smoothness degrades as model depth or learning rate grows, the claimed 0.01 accuracy bound may require re-tuning the sampling rate and sync interval; the paper's robustness study indicates graceful degradation, but the formal guarantee remains conditional.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents SNI-GNN, a distributed full-graph GNN training system that offloads embedding prediction to BlueField-3 SmartNICs (DPUs). A lightweight linear-trend predictor refines cached historical embeddings for remote boundary nodes, reducing inter-node communication frequency while mitigating staleness. The system combines importance-based boundary-node sampling, an asynchronous DPU–GPU pipeline, and intermediate-result reuse. The authors claim theoretical guarantees: a prediction error bound under bounded second-order embedding dynamics, a gradient error bound, and standard non-convex SGD convergence with inexact gradients. On four datasets and up to 16 GPUs, they report 21–45% communication reduction, 1.3–3.6× end-to-end speedups over BNS-GCN, up to 1.29× over a SANCUS baseline, and accuracy within 0.01 of synchronous training.

Significance. The empirical contribution is substantial and well executed: a real BlueField-3 prototype, integration with two state-of-the-art full-graph systems (SANCUS and NeutronTP), evaluation across four datasets and multiple GNN architectures, and ablation and microbenchmark studies. If the results hold, SNI-GNN is a practical complement to existing communication-reduction techniques, and the idea of using SmartNICs for active staleness mitigation is novel and timely. The theoretical section, however, is the weakest part of the paper. The convergence and error bounds as stated are not rigorously derived from the actual predictor, and Theorem IV.3 does not have the form of a standard non-convex convergence guarantee. These issues are load-bearing for the paper's claimed 'theoretical rigor' contribution, though they do not undermine the empirical findings by themselves.

major comments (4)
  1. [Section IV-D, Theorem IV.1] The proof sketch asserts that the EMA baseline and the weighted trend satisfy h_bar ≈ H(τ) and Φ ≈ H'(τ), but these are exactly the approximations that need to be justified. Algorithm 1 computes h_bar as an exponential moving average and Φ as a decay-weighted average of past slopes, while Theorem IV.1 assumes a linear predictor that exactly captures the first-order Taylor term. The second-order difference bound alone does not imply that the specific EMA/weighted-trend quantities approximate the true value and derivative. The theorem's bound ||Ĥ(t) − H(t)||∞ ≤ (1/2)(Δt)²ε is therefore not established for the predictor actually used in the system.
  2. [Section IV-D, Lemma IV.2 and Theorem IV.3] The gradient error bound K applies only to the subset of nodes selected for prediction. With the sampling ratios used in Section V (0.3, 0.1, 0.05 for Reddit, IGB-small, ogbn-products), the remaining 70–95% of remote embeddings are raw stale caches. A bound on the second-order difference of embeddings does not control the staleness error h(t) − h(t−s) for those unsampled nodes: a constant first difference has zero second difference but an O(s) staleness error. Consequently, K in Lemma IV.2 does not bound the true gradient error of the evaluated system, and Theorem IV.3's convergence conclusion does not follow for the system as actually configured.
  3. [Section IV-D, Theorem IV.3] The displayed bound min_t E[||∇L(W_t)||²_F] ≤ [2(L0 − L*) + ρK]/√N is not a standard non-convex SGD convergence rate. The numerator contains the constant term 2(L0 − L*) and there is no dependence on the number of iterations T (or on the learning rate in the usual 1/(ηT) form). As stated, the bound does not imply convergence to a stationary point as training progresses. The proof sketch's reference to 'standard SGD analysis with inexact gradients' is not sufficient; a correct derivation with the appropriate T dependence and a proper treatment of the bias term is needed.
  4. [Section V-D4 and Theorem IV.1] The paper's own robustness experiments show that the smoothness premise of Theorem IV.1 is violated in early epochs and at high learning rates. Since the formal error and convergence guarantees rely on bounded second-order dynamics, the theoretical claims do not hold unconditionally for the evaluated training regimes. The authors should either extend the analysis to cover non-smooth dynamics with a weaker but still useful bound, or explicitly delimit the scope of the theoretical contributions and explain how the empirical fail-safe behavior (periodic synchronization) relates to the formal analysis.
minor comments (5)
  1. [Table II] The IGB-medium edge count is given as '120,077, 694' with an extra space; this should be corrected to a single number.
  2. [Figure 2] The x-axis labels appear garbled (e.g., '8 N_1G4 N_2G2 N_4G8'); please fix the label formatting so the node configurations are legible.
  3. [Section V-D1, Table III] The baseline row labeled '0' is clear in context, but it would be more readable to label it 'SR=0 (no prediction)' to match the other rows.
  4. [Section V-B] The text states 'SNI-GNN maintains accuracy loss within 0.01 across all datasets,' but Table III shows accuracy improvements over the SANCUS baseline. Please clarify that the 0.01 loss is relative to the ideal synchronous (E=1) training, not to the stale baseline.
  5. [Section V-G, Table VI] The PCIe bandwidth comparison appears to be based on a theoretical Dell XE9680 topology rather than the actual testbed. Please clarify in the text whether these numbers are analytical or measured, and note the implication for the CPU-vs-DPU comparison on the testbed hardware.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SNI-GNN's predictor is evaluated against external baselines and its theoretical bounds are assumption-based rather than reductions to fitted outputs.

full rationale

The derivation chain is not circular. The core prediction mechanism (Algorithm 1) extrapolates from cached historical embeddings, and its empirical evaluation is against external full-graph systems (SANCUS, NeutronTP, BNS-GCN), not against its own fitted outputs. Theorem IV.1 is a conditional mathematical statement: if the second-order embedding difference is bounded by ε, then the linear-trend predictor error is bounded by (1/2)(Δt)^2 ε. Even though the proof sketch relies on approximations such as ¯h≈H(τ) and Φ≈H′(τ), this is an assumption-based bound, not a definitional equivalence or a fitted parameter renamed as a prediction. Theorem IV.3 invokes standard inexact-gradient SGD analysis from an external reference [24] and does not derive its conclusion from the system's own fitted values. The boundary-node sampling scheme does not create circularity: it selects which nodes receive prediction, while unsampled nodes use stale caches; this is a coverage limitation of the formal bound, not a circular step. Likewise, Algorithm 2's self-feeding of predicted values into the history is a system optimization that may weaken the bounded-dynamics premise, but it is not an equation-level reduction of the claimed result to its inputs. The paper's own robustness section (Section V-D4) concedes that the smoothness assumption can be violated in early epochs or with large learning rates; this is a limitation of the guarantee, not circularity. The citations that may share authors (e.g., NeutronTP [36] and OS2G [29]) are used only as integration targets or related work and are not load-bearing for the paper's central claim or its convergence proof. The notable mathematical issues — the displayed O(1/√N) bound in Theorem IV.3 lacks the usual 1/(ηT) term, and the gradient-error bound does not account for unsampled stale nodes — are correctness and proof-completeness concerns, not evidence that the derivation is equivalent to its own inputs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the smooth-evolution observation (Figure 5), the bounded second-order dynamics assumption (Theorem IV.1), and several hand-set hyperparameters (alpha, lambda, per-dataset sampling ratio, sync interval, warm-up). The predictor itself is not an invented entity but a calibrated transformation of history. The system's performance numbers also depend on the unstated hardware behavior of the BlueField-3 DPU in the measurement setup.

free parameters (6)
  • sampling_ratio (per dataset) = Reddit 0.3, IGB-small 0.1, ogbn-products 0.05
    Set to the maximum ratio that keeps DPU prediction latency hidden behind GPU computation in the measured idle window (Section V-D1, Table IV). This is a per-dataset tuning choice that directly controls the accuracy/efficiency trade-off.
  • EMA weight alpha = 0.3
    Hand-set in Algorithm 1 (h_bar = 0.3*H[tau] + 0.7*mean(H)); no sensitivity analysis reported.
  • trend decay lambda = 0.2
    Hand-set in Algorithm 1 (e^{-0.2*Delta t}); no sensitivity analysis reported.
  • ring buffer capacity K / window W = unspecified
    The history length and trend accumulation window are parameters of Equation 5 and Algorithm 2, but their values are not reported.
  • sync interval E = 5 for main results; 1/3/5 in study
    The communication interval is an operating point; main results use E=5 for SNI-GNN and compare with SANCUS and NeutronTP at their default or matched intervals. The choice affects both speedup and accuracy.
  • warm-up epochs = 20
    Warm-up of 20 epochs is standard for staleness-tolerant training, but it is a chosen constant (Section V-B).
assumptions (4)
  • domain assumption Bounded second-order embedding dynamics: ||H(t)-2H(t-1)+H(t-2)||_inf <= epsilon
    Used in Theorem IV.1 to bound predictor error by 0.5*(Delta t)^2*epsilon. The paper's own robustness study (Section V-D4) shows this is violated during warm-up and at high learning rates.
  • standard math rho-smooth loss and activations, weight matrices bounded by C, adjacency blocks bounded by B
    Standard assumptions in non-convex SGD analysis, invoked in Lemma IV.2 and Theorem IV.3.
  • domain assumption Embeddings evolve smoothly and gradually over time
    Empirical observation from Figure 5 motivates the whole predictor; if it fails, prediction accuracy degrades. Section III presents it as the design rationale.
  • standard math Standard non-convex SGD convergence with inexact gradients (following SANCUS [24])
    Theorem IV.3 relies on the standard O(1/sqrt(T)) analysis for SGD with bounded gradient error, which is a known result in optimization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction." pith.science (2026). https://pith.science/paper/RDJCSCTB

@misc{pith2026260806441,
  author       = {Pith},
  title        = {Pith review of: SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RDJCSCTB}},
  note         = {Machine review of arXiv:2608.06441}
}
abstract

Full-graph GNN training delivers high accuracy but scales poorly on multi-server clusters due to heavy, irregular inter-node embedding exchanges. We present SNI-GNN, a SmartNIC-assisted full-graph training system that reduces communication while preserving accuracy by predicting remote embeddings in-network. SNI-GNN deploys a lightweight linear-trend predictor on SmartNICs to refine cached historical embeddings, coupled with an importance-based boundary-node sampling policy and an asynchronous DPU--GPU data pipeline with intermediate-result reuse. We provide error and convergence bounds showing that predictor bias remains controlled under bounded second-order dynamics and yields standard non-convex convergence with inexact gradients. Implemented on NVIDIA BlueField-3, SNI-GNN integrates with state-of-the-art full-graph systems, cuts communication by 21--45\%, achieves 1.3--3.6$\times$ end-to-end speedups over BNS-GCN and up to 1.29$\times$ over baseline SANCUS, with accuracy loss $\leq 0.01$, and scales efficiently to 16 GPUs on graphs with up to tens of millions of edges. These results indicate SmartNIC-based in-network prediction is a practical complement to partitioning and compression techniques for communication-efficient full-graph GNN training at scale.

Figures

Figures reproduced from arXiv: 2608.06441 by the authors.

Figure 2
Figure 2. Full-Graph GNN training time breakdown embedding h (l−1,t) u for a remote neighbor u, a worker uses a cached version h˜ (l−1,t) u = h (l−1,t−1) u . While this approach reduces network pressure, it introduces embedding staleness due to the discrepancy between the cached historical value and the current, updated embedding. As training progresses, stale embeddings can distort the aggregation step, potentially degrading… view at source ↗
Figure 5
Figure 5. Predictable embedding evolution across training epochs. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 4
Figure 4. System architecture with BlueField-3 SmartNIC. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (11 more)
Figure 6
Figure 6. Figure 6: System architecture for SmartNIC-assisted full-graph GNN training. Each node has a GPU and a SmartNIC. SmartNICs [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Comparison of prediction accuracy curves across different forecasting methods TABLE I: Computation time and peak accuracy of differ￾ent predictors Method Time (s) Acc. (%) Ours 48.2 92.88 HOLT 46.83 92.14 ARIMA 425.2 93.18 LSTM 57.37 92.39 Backend DOCA Pytorch OpenMP C…
Figure 8
Figure 8. Figure 8: Predictor design overview. The blue modules represent [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: The asynchronous communication pipeline in SNI￾GNN (illustrated with a 2-worker, 2-layer GNN configuration). historical embeddings. This selective prediction reduces the DPU’s workload while focusing its computational power on the nodes most likely to affect model con…
Figure 11
Figure 11. Figure 11: Runtime breakdown (synchronization, computation, [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: The accuracy results using 8 nodes for 3-layer GCN [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]
Figure 13
Figure 13. Figure 13: Breakdown of time overhead (synchronization, computation, communication) for SCS, SCS-SNI, NTP, and NTP-SNI [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]
Figure 14
Figure 14. Figure 14: Deeper-model study (4/6 layers): convergence under multi-node training. TABLE III: Accuracy results across datasets and Sampling Rates (SR). SR Reddit IGB-small ogbn-products IGB-medium 0 91.19% 69.84% 69.54% 60.81% 0.01 91.35% (+0.16) 69.88% (+0.04) 69.91% (+0.37) 60…
Figure 17
Figure 17. Figure 17: Training loss curves of SANCUS with SNI prediction on GAT/GraphSAGE models under 8 GPUs. E1/2/3/4/6 denotes the communication intervals of 1/2/3/4/6. TABLE V: Cumulative impact of our three primary optimiza￾tions on end-to-end time (s). Method IGB-small Reddit ogbn-pr…
Figure 16
Figure 16. Figure 16: Robustness under non-smooth dynamics. optimization settings using significantly larger learning rates. As shown in [PITH_FULL_IMAGE:figures/full_fig_p011_16.png]
Figure 18
Figure 18. Figure 18: Prediction error over epochs with/without self-feeding on Reddit Dataset. TABLE VI: PCIe bandwidth contention: CPU vs. DPU com￾munication patterns (theoretical, Dell XE9680 PCIe Gen5 with 2:1 switch-uplink oversubscription). Pattern #GPU BW/GPU (GB/s) CPU → GPU (1:1) …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 47 canonical work pages

  1. [1]

    Early: Efficient and reliable graph neural network for dynamic graphs,

    H. Li and L. Chen, “Early: Efficient and reliable graph neural network for dynamic graphs,”Proceedings of the ACM on Management of Data, vol. 1, no. 2, pp. 1–28, 2023

    work page 2023

  2. [2]

    Towards deeper graph neural networks,

    M. Liu, H. Gao, and S. Ji, “Towards deeper graph neural networks,” inProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 338–348, 2020

    work page 2020

  3. [3]

    Link prediction based on graph neural networks,

    M. Zhang and Y . Chen, “Link prediction based on graph neural networks,” Advances in neural information processing systems, vol. 31, 2018

    2018

  4. [4]

    A multi-scale approach for graph link prediction,

    L. Cai and S. Ji, “A multi-scale approach for graph link prediction,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 3308–3315, 2020

    work page 2020

  5. [5]

    How powerful are graph neural networks?,

    K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?,”arXiv preprint arXiv:1810.00826, 2018

    arXiv 2018

  6. [6]

    Topology-aware graph pooling networks,

    H. Gao, Y . Liu, and S. Ji, “Topology-aware graph pooling networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 12, pp. 4512–4518, 2021

    work page 2021

  7. [7]

    Content matters: A gnn-based model combined with text semantics for social network cascade prediction,

    Y . Liu, K. Zeng, H. Wang, X. Song, and B. Zhou, “Content matters: A gnn-based model combined with text semantics for social network cascade prediction,” inPacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 728–740, Springer, 2021

    work page 2021

  8. [8]

    Deepinf: Social influence prediction with deep learning,

    J. Qiu, J. Tang, H. Ma, Y . Dong, K. Wang, and J. Tang, “Deepinf: Social influence prediction with deep learning,” inProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2110–2119, 2018

    work page 2018

Show all 59 references
  1. [9]

    A survey of graph neural network based recommendation in social networks,

    X. Li, L. Sun, M. Ling, and Y . Peng, “A survey of graph neural network based recommendation in social networks,”Neurocomputing, vol. 549, p. 126441, 2023

    2023

  2. [10]

    Neural graph collaborative filtering,

    X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua, “Neural graph collaborative filtering,” inProceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pp. 165–174, 2019

    2019

  3. [11]

    Enhancing graph neural network-based fraud detectors against camouflaged fraudsters,

    Y . Dou, Z. Liu, L. Sun, Y . Deng, H. Peng, and P. S. Yu, “Enhancing graph neural network-based fraud detectors against camouflaged fraudsters,” in Proceedings of the 29th ACM international conference on information & knowledge management, pp. 315–324, 2020

    2020

  4. [12]

    Fp-gnn: a versatile deep learning architecture for enhanced molecular property prediction,

    H. Cai, H. Zhang, D. Zhao, J. Wu, and L. Wang, “Fp-gnn: a versatile deep learning architecture for enhanced molecular property prediction,” Briefings in bioinformatics, vol. 23, no. 6, p. bbac408, 2022

    2022

  5. [13]

    A comprehensive survey on distributed training of graph neural networks,

    H. Lin, M. Yan, X. Ye, D. Fan, S. Pan, W. Chen, and Y . Xie, “A comprehensive survey on distributed training of graph neural networks,” Proceedings of the IEEE, vol. 111, no. 12, pp. 1572–1606, 2023

    2023

  6. [14]

    Graph neural network training systems: A performance comparison of full-graph and mini-batch,

    S. Bajaj, H. Guan, and M. Serafini, “Graph neural network training systems: A performance comparison of full-graph and mini-batch,”arXiv preprint arXiv:2406.00552, 2024

    2024 arXiv

  7. [15]

    Cogkr: Cognitive graph for multi-hop knowledge reasoning,

    Z. Du, C. Zhou, J. Yao, T. Tu, L. Cheng, H. Yang, J. Zhou, and J. Tang, “Cogkr: Cognitive graph for multi-hop knowledge reasoning,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 2, pp. 1283–1295, 2021

    2021

  8. [16]

    Fragnet: A graph neural network for molecular property prediction with four layers of interpretability,

    G. Panapitiya, P. Gao, C. M. Maupin, and E. G. Saldanha, “Fragnet: A graph neural network for molecular property prediction with four layers of interpretability,”arXiv preprint arXiv:2410.12156, 2024

    2024 arXiv

  9. [17]

    Distdgl: Distributed graph neural network training for billion-scale graphs,

    D. Zheng, C. Ma, M. Wang, J. Zhou, Q. Su, X. Song, Q. Gan, Z. Zhang, and G. Karypis, “Distdgl: Distributed graph neural network training for billion-scale graphs,” in2020 IEEE/ACM 10th Workshop on Irregular Applications: Architectures and Algorithms (IA3), pp. 36–44, IEEE, 2020

    2020

  10. [18]

    Ginex: Ssd-enabled billion-scale graph neural network training on a single machine via provably optimal in- memory caching,

    Y . Park, S. Min, and J. W. Lee, “Ginex: Ssd-enabled billion-scale graph neural network training on a single machine via provably optimal in- memory caching,”Proceedings of the VLDB Endowment, vol. 15, no. 11, pp. 2626–2639, 2022

    2022

  11. [19]

    Sequential aggregation and rematerialization: Distributed full-batch training of graph neural networks on large graphs,

    H. Mostafa, “Sequential aggregation and rematerialization: Distributed full-batch training of graph neural networks on large graphs,”Proceedings of Machine Learning and Systems, vol. 4, pp. 265–275, 2022

    2022

  12. [20]

    Flexgraph: a flexible and efficient distributed framework for gnn training,

    L. Wang, Q. Yin, C. Tian, J. Yang, R. Chen, W. Yu, Z. Yao, and J. Zhou, “Flexgraph: a flexible and efficient distributed framework for gnn training,” inProceedings of the Sixteenth European Conference on Computer Systems, pp. 67–82, 2021

    2021

  13. [21]

    Scalable and efficient full-graph gnn training for large graphs,

    X. Wan, K. Xu, X. Liao, Y . Jin, K. Chen, and X. Jin, “Scalable and efficient full-graph gnn training for large graphs,”Proceedings of the ACM on Management of Data, vol. 1, no. 2, pp. 1–23, 2023

    2023

  14. [22]

    Distributed hybrid cpu and gpu training for graph neural networks on billion- scale heterogeneous graphs,

    D. Zheng, X. Song, C. Yang, D. LaSalle, and G. Karypis, “Distributed hybrid cpu and gpu training for graph neural networks on billion- scale heterogeneous graphs,” inProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4582–4591, 2022

    2022

  15. [23]

    Pipegcn: Efficient full-graph training of graph convolutional networks with pipelined feature communication,

    C. Wan, Y . Li, C. R. Wolfe, A. Kyrillidis, N. S. Kim, and Y . Lin, “Pipegcn: Efficient full-graph training of graph convolutional networks with pipelined feature communication,”arXiv preprint arXiv:2203.10428, 2022

    2022 arXiv

  16. [24]

    Sancus: staleness-aware communication-avoiding full-graph decentralized train- ing in large-scale graph neural networks,

    J. Peng, Z. Chen, Y . Shao, Y . Shen, L. Chen, and J. Cao, “Sancus: staleness-aware communication-avoiding full-graph decentralized train- ing in large-scale graph neural networks,”Proceedings of the VLDB Endowment, vol. 15, no. 9, pp. 1937–1950, 2022

    1937

  17. [25]

    Adaptive message quantization and parallelization for distributed full-graph gnn training,

    B. Wan, J. Zhao, and C. Wu, “Adaptive message quantization and parallelization for distributed full-graph gnn training,”Proceedings of Machine Learning and Systems, vol. 5, 2023

    2023

  18. [26]

    Sylvie: 3d-adaptive and universal system for large-scale graph neural network training,

    M. Zhang, Q. Hu, C. Wan, H. Wang, P. Sun, Y . Wen, and T. Zhang, “Sylvie: 3d-adaptive and universal system for large-scale graph neural network training,” in2024 IEEE 40th International Conference on Data Engineering (ICDE), pp. 3823–3836, IEEE, 2024

    2024

  19. [27]

    Granndis: Fast distributed graph neural network training framework for multi-server clusters,

    J. Song, H. Jang, H. Lim, J. Jung, Y . Kim, and J. Lee, “Granndis: Fast distributed graph neural network training framework for multi-server clusters,” inProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques, pp. 91–107, 2024

    2024

  20. [28]

    Guo,Software and hardware codesign of SmartNIC-based heteroge- neous HPC clusters with machine learning case studies

    A. Guo,Software and hardware codesign of SmartNIC-based heteroge- neous HPC clusters with machine learning case studies. PhD thesis, Boston University, 2024

    2024

  21. [29]

    Os2g: A high-performance dpu offloading architecture for gpu-based deep learning with object storage,

    Z. Jin, Y . Chen, M. Liang, Y . Wang, G. Fang, A. Zhou, K. Zhang, J. Xu, W. Lin, Y . Lin,et al., “Os2g: A high-performance dpu offloading architecture for gpu-based deep learning with object storage,” in Proceedings of the 30th ACM International Conference on Architectural Sup...

    2025

  22. [30]

    Bns-gcn: Efficient full-graph training of graph convolutional networks with partition-parallelism and random boundary node sampling,

    C. Wan, Y . Li, A. Li, N. Kim, and Y . Lin, “Bns-gcn: Efficient full-graph training of graph convolutional networks with partition-parallelism and random boundary node sampling,”ArXiv, vol. abs/2203.10983, 2022

    2022 arXiv

  23. [31]

    Neural message passing for quantum chemistry,

    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” inInternational conference on machine learning, pp. 1263–1272, PMLR, 2017

    2017

  24. [32]

    Dgcl: An efficient communication library for distributed gnn training,

    Z. Cai, X. Yan, Y . Wu, K. Ma, J. Cheng, and F. Yu, “Dgcl: An efficient communication library for distributed gnn training,” inProceedings of the Sixteenth European Conference on Computer Systems, pp. 130–144, 2021

    2021

  25. [33]

    P3: Distributed deep graph learning at scale,

    S. Gandhi and A. P. Iyer, “P3: Distributed deep graph learning at scale,” in15th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI}21), pp. 551–568, 2021

    2021

  26. [34]

    Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings,

    M. Fey, J. E. Lenssen, F. Weichert, and J. Leskovec, “Gnnautoscale: Scalable and expressive graph neural networks via historical embeddings,” inInternational conference on machine learning, pp. 3294–3304, PMLR, 2021

    2021

  27. [35]

    Gnnlab: a factored system for sample-based gnn training over gpus,

    J. Yang, D. Tang, X. Song, L. Wang, Q. Yin, R. Chen, W. Yu, and J. Zhou, “Gnnlab: a factored system for sample-based gnn training over gpus,” inProceedings of the Seventeenth European Conference on Computer Systems, pp. 417–434, 2022

    2022

  28. [36]

    Neutrontp: Load-balanced distributed full-graph gnn training with tensor parallelism,

    X. Ai, H. Yuan, Z. Ling, Q. Wang, Y . Zhang, Z. Fu, C. Chen, Y . Gu, and G. Yu, “Neutrontp: Load-balanced distributed full-graph gnn training with tensor parallelism,”Proc. VLDB Endow., vol. 18, p. 173–186, Oct. 2024

    2024

  29. [37]

    A fast and high quality multilevel scheme for partitioning irregular graphs,

    G. Karypis and V . Kumar, “A fast and high quality multilevel scheme for partitioning irregular graphs,”SIAM Journal on scientific Computing, vol. 20, no. 1, pp. 359–392, 1998

    1998

  30. [38]

    Streaming graph partitioning for large distributed graphs,

    I. Stanton and G. Kliot, “Streaming graph partitioning for large distributed graphs,” inProceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1222–1230, 2012

    2012

  31. [39]

    Scalable Edge Partitioning,

    S. Schlag, C. Schulz, D. Seemaier, and D. Strash, “Scalable Edge Partitioning,” inProceedings of the 21th Workshop on Algorithm Engineering and Experimentation (ALENEX), pp. 211–225, SIAM, 2019

    2019

  32. [40]

    Open Graph Benchmark: Datasets for Machine Learning on Graphs,

    W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, “Open Graph Benchmark: Datasets for Machine Learning on Graphs,”arXiv e-prints, p. arXiv:2005.00687, May 2020

    2005 arXiv

  33. [41]

    Inductive representation learning on large graphs,

    W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,”Advances in neural information processing systems, vol. 30, 2017

    2017

  34. [42]

    Graphfm: Im- proving large-scale gnn training via feature momentum,

    H. Yu, L. Wang, B. Wang, M. Liu, T. Yang, and S. Ji, “Graphfm: Im- proving large-scale gnn training via feature momentum,” inInternational conference on machine learning, pp. 25684–25701, PMLR, 2022

    2022

  35. [43]

    Freshgnn: reducing memory access via stable historical embeddings for graph neural network training,

    K. Huang, H. Jiang, M. Wang, G. Xiao, D. Wipf, X. Song, Q. Gan, Z. Huang, J. Zhai, and Z. Zhang, “Freshgnn: reducing memory access via stable historical embeddings for graph neural network training,”arXiv preprint arXiv:2301.07482, 2023

    2023 arXiv

  36. [44]

    Gpus, cpus, and... nics: Rethinking the network’s role in serving complex ai pipelines,

    M. Wong, U. Butler, E. Farkash, P. Tammana, A. Sivaraman, and R. Netravali, “Gpus, cpus, and... nics: Rethinking the network’s role in serving complex ai pipelines,”arXiv preprint arXiv:2502.15712, 2025

    2025 arXiv

  37. [45]

    Large-message nonblocking mpi_iallgather and mpi ibcast offload via bluefield-2 dpu,

    N. Sarkauskas, M. Bayatpour, T. Tran, B. Ramesh, H. Subramoni, and D. K. Panda, “Large-message nonblocking mpi_iallgather and mpi ibcast offload via bluefield-2 dpu,” in2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC), pp. 388– ...

    2021

  38. [46]

    Network-offloaded bandwidth-optimal broadcast and allgather for distributed ai,

    M. Khalilov, S. Di Girolamo, M. Chrapek, R. Nudelman, G. Bloch, and T. Hoefler, “Network-offloaded bandwidth-optimal broadcast and allgather for distributed ai,” inSC24: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–17, IEEE, 2024

    2024

  39. [47]

    Squeezenic: Low-latency in-nic compression for distributed deep learn- ing,

    A. Rebai, M. A. Ojewale, A. Ullah, M. Canini, and S. A. Fahmy, “Squeezenic: Low-latency in-nic compression for distributed deep learn- ing,” inProceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing, pp. 61–68, 2024

    2024

  40. [48]

    Omniccl: Zero-cost sparse allreduce with direct cache access and smartnics,

    T. Gu, J. Fei, and M. Canini, “Omniccl: Zero-cost sparse allreduce with direct cache access and smartnics,” inProceedings of the 2024 SIGCOMM Workshop on Networks for AI Computing, pp. 75–83, 2024

    2024

  41. [49]

    Luwu: An end-to-end in-network out-of-core optimizer for 100b-scale model- in-network data-parallel training on distributed gpus,

    M. Sun, Z. Yang, C. Liao, Y . Li, F. Wu, and Z. Wang, “Luwu: An end-to-end in-network out-of-core optimizer for 100b-scale model- in-network data-parallel training on distributed gpus,”arXiv preprint arXiv:2409.00918, 2024

    2024 arXiv

  42. [50]

    Analysis: Forecasting and control,

    G. Box and G. Jenkins, “Analysis: Forecasting and control,”San francisco, 1976

    1976

  43. [51]

    Forecasting seasonals and trends by exponentially weighted moving averages,

    C. C. Holt, “Forecasting seasonals and trends by exponentially weighted moving averages,”International journal of forecasting, vol. 20, no. 1, pp. 5–10, 2004

    2004

  44. [52]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  45. [53]

    Igb: Addressing the gaps in labeling, features, heterogeneity, and size of public graph datasets for deep learning research,

    A. Khatua, V . S. Mailthody, B. Taleka, T. Ma, X. Song, and W. mei W. Hwu, “Igb: Addressing the gaps in labeling, features, heterogeneity, and size of public graph datasets for deep learning research,”Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Dat...

    2023

  46. [54]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,”arXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  47. [55]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y . Bengio, “Graph attention networks,”arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  48. [56]

    Staleness-alleviated distributed gnn training via online dynamic-embedding prediction,

    G. Bai, Z. Yu, Z. Chai, Y . Cheng, and L. Zhao, “Staleness-alleviated distributed gnn training via online dynamic-embedding prediction,” in Proceedings of the 2025 SIAM International Conference on Data Mining (SDM), pp. 578–587, SIAM, 2025

    2025

  49. [57]

    Conspirator:{SmartNIC-Aided} control plane for distributed {ML} workloads,

    Y . Xiao, D. Z. Tootaghaj, A. Dhakal, L. Cao, P. Sharma, and A. Kuz- manovic, “Conspirator:{SmartNIC-Aided} control plane for distributed {ML} workloads,” in2024 USENIX Annual Technical Conference (USENIX ATC 24), pp. 767–784, 2024

    2024

  50. [58]

    Optimusnic: Offloading optimizer state to smartnics for efficient large-scale ai training,

    A. Rebai and M. Canini, “Optimusnic: Offloading optimizer state to smartnics for efficient large-scale ai training,” inProceedings of the 5th Workshop on Machine Learning and Systems, pp. 176–182, 2025

    2025

  51. [59]

    Accelerating cpu-based distributed dnn training on modern hpc clusters using bluefield-2 dpus,

    A. Jain, N. Alnaasan, A. Shafi, H. Subramoni, and D. K. Panda, “Accelerating cpu-based distributed dnn training on modern hpc clusters using bluefield-2 dpus,” in2021 IEEE Symposium on High-Performance Interconnects (HOTI), pp. 17–24, IEEE, 2021

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.