Pith. sign in

REVIEW 3 major objections 5 minor 93 references

MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Accelerators

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that full-graph GCN training can be made scalable without the growing communication and duplication of partition parallelism, by splitting feature tensors rather than graph partitions, and by pairing sparse and dense…

desk verdict Useful MoP idea with honest limitations, but the headline constant-communication claim is narrower than advertised once you account for the replicated adjacency matrix. read the letter →

arxiv 2501.01951 v3 pith:RGPCREXZ submitted 2025-01-03 cs.LG cs.AI

Cheng Wan , Runkai Tao , Zheng Du , Yang Katie Zhao , Yingyan Celine Lin This is my paper · ORCID
classification cs.LGcs.AI
keywords graphconvolutionalnetworksdistributedGCNtrainingmixtureofparallelismfeature-levelnode-levelacceleratorsS-SpMMbandwidthnodereordering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the two obstacles to scalable full-graph GCN training — giant feature tensors and alternating sparse/dense computation — can both be addressed by changing how the work is divided. Instead of partitioning the graph and copying remote neighbors onto each worker, MixGCN partitions the feature matrix along its feature dimension for neighbor aggregation and along the node dimension for node update, keeping the propagation matrix replicated on every aggregation accelerator. The authors argue that this 'mixture of parallelism' keeps communication volume and feature-memory per layer at $O(N)$ regardless of accelerator count, gives a perfectly balanced workload in $O(1)$ time, and lets sparse aggregation and dense update run on different accelerators. A reader should care because, if the claims hold, GCN training on full graphs would scale by adding accelerators without the communication blow-up that current partition-parallel systems suffer.

What carries the argument

The load-bearing object is the propagation matrix $\hat{A}$ replicated on every aggregation accelerator, together with the two-way split of the feature tensor. MoP splits $H$ along the feature dimension so each aggregation worker computes $\hat{A}H$ for a column slice, then all-to-all communicates so each update worker holds complete features for its node slice; the same flow runs in reverse for the backward pass. Because the split is uniform, work is balanced by construction, and because no remote-neighbor copies exist, communication stays flat in the number of workers. The second mechanism is S-SpMM, a fused sampled sparse matrix-matrix multiplication that applies the output sparsity mask (dropout in forward, ReLU in backward) inside the aggregation kernel, supported by a hybrid sparse index module that selects only needed neighbor features. A third mechanism is node reordering via a bandwidth-reducing BFS-based ordering, which lets a fine-grain pipeline start the next layer before all dependent updates finish.

What would settle it

Measure per-worker communication volume and peak memory while scaling MixGCN from 1 to, say, 128 accelerators on a fixed graph, using a network where all-to-all bandwidth does not grow with worker count (e.g., a single-switch cluster). If total communication per layer grows with the number of workers, or if the replicated adjacency matrix forces out-of-memory before the feature tensors do, Proposition 3.4's constant-volume and feature-memory claims are falsified.

Watch

Extended reading notes

Core claim

The central discovery is a parallelism scheme, MoP, in which no node's features are ever duplicated merely because an edge crosses a worker boundary. Feature-level parallelism splits each layer's feature matrix horizontally across aggregation accelerators that each hold the entire propagation matrix; after aggregation, all-to-all communication reassembles features, and node-level parallelism splits the update across dense accelerators. The paper proves that workload balancing becomes $O(1)$ time (Proposition 3.3), and that communication volume plus feature memory is $O(N)$ for the training process (Proposition 3.4), in contrast to partition parallelism's $O(|\mathcal{R}|)$ remote-neighbor cost (Proposition 3.2). On top of this, MoA assigns aggregation to sparse accelerators and update to dense accelerators, fuses dropout/ReLU masking with SpMM into a single S-SpMM operation, and uses graph-bandwidth-reducing node reordering to make a fine-grain pipeline idle-free. The empirical claim is end-to-end throughput up to $10.4\times$ over existing baselines on a 4-node cluster, with further gains attributed to the dedicated sparse accelerator.

Load-bearing premise

The load-bearing premise is that the graph's propagation matrix fits in a single accelerator, because every aggregation accelerator keeps a full copy; for graphs whose adjacency matrix alone overflows one device, the constant-communication and $O(N)$-memory claims no longer hold as stated.

Editorial extensions

If this is right

  • If the $O(N)$ communication claim (Proposition 3.4) holds, full-graph GCN training can scale horizontally by adding accelerators without per-element communication growing, unlike partition parallelism.
  • Feature tensors, not graph structure, become the scaling dimension: node features and intermediate activations distribute across workers, while the adjacency matrix stays a one-time replicated copy.
  • Sparse and dense accelerators become usable together in training, because MoP cleanly separates neighbor aggregation from node update into two parallel phases.
  • The pipeline condition $(n-b)/(2n)\ge 1/s$ gives a concrete criterion for choosing how many pipeline stages a given graph's bandwidth allows, so systems can be provisioned without trial and error.
  • Bandwidth-reducing node reordering offers the same asymptotic preprocessing cost as graph partitioning but with much lower measured overhead, making the ordering cost negligible relative to training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the 'constant communication' claim is with respect to accelerator count at fixed graph size; wall-clock time still depends on all-to-all network bandwidth, so the scheme's advantage may shrink on machines where bisection bandwidth does not scale with node count.
  • Inference: MoP is restricted to element-wise aggregation (GCN-style sum/mean), so the approach does not directly transfer to attention-based GNNs; the authors note this, but it means the scalability win is tied to the GCN/GraphSAGE/GIN family.
  • Inference: the paper's own limitation suggests a testable hybrid: combine MoP with partition parallelism for graphs whose adjacency matrix exceeds one accelerator, which would restore scalability at the price of reintroducing some remote-neighbor communication.
  • Inference: one could test the pipeline claim directly by generating synthetic graphs with controlled bandwidth; the speedup from node reordering should track $(n-b)/(2n)$, a quantitative prediction not reported for real datasets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes MixGCN, a distributed full-graph GCN training system that combines mixture of parallelism (MoP) and mixture of accelerators (MoA). MoP splits node features along the feature dimension for neighbor aggregation and splits nodes across update workers, avoiding the remote-neighbor duplication of partition parallelism and using a regular all-to-all communication pattern. MoA assigns sparse aggregation and dense update operations to different accelerators, introduces a fused S-SpMM operation with a dedicated sparse accelerator design, and uses reverse Cuthill-McKee node reordering to improve fine-grain pipeline utilization. Experiments on five large-scale graphs on a 4-node cluster report substantial throughput improvements over DistDGL, CAGNET, PipeGCN, and Sancus, together with profiling results intended to support constant communication volume, bounded memory usage, and balanced workload.

Significance. If the MoP scalability claim were established with its conditions fully stated, the paper would be a useful contribution: it constructs a parallelism scheme that avoids remote-neighbor duplication by design, has a regular all-to-all communication pattern, and combines a real CPU/GPU implementation with accelerator simulation. The empirical results on four nodes are plausible, and the ablation studies support the value of the MoA components. However, the central theoretical claim, Proposition 3.4, is stated without proof, and the paper's own Section 5.2.1 concedes that the scheme depends on replicating the full adjacency matrix on every aggregation accelerator. The abstract and introduction present the constant-communication and scalability properties without this scope restriction, so the headline claim is narrower than the paper claims.

major comments (3)
  1. [Section 3.1.2, Proposition 3.4] Proposition 3.4 is the load-bearing theoretical claim: 'Both the communication volume and feature memory requirement are O(N) for GCN training with MoP.' No proof or derivation is provided anywhere in the manuscript. Since the abstract states that theoretical analysis verifies the constant-communication property, the paper should either supply a rigorous proof (including the communication model and the exact quantities being counted) or explicitly downgrade this statement to a conditional/empirical observation and revise the abstract accordingly.
  2. [Section 5.2.1, Proposition 3.4] The O(N) communication and feature-memory claim is conditional on replicating the propagation matrix A on every aggregation accelerator. Section 5.2.1 admits this and states that for 'extremely giant graphs' where A does not fit in one accelerator, MoP cannot start; the suggested fallback of combining partition parallelism with MoP would reintroduce remote-neighbor communication. Because per-accelerator memory is at least Theta(E) for the replicated A plus Theta(NF/m) for features, and total memory across all workers is m*Theta(E), the claim that adding accelerators yields scalable GCN training holds only in the regime where Theta(E) fits in a single accelerator. This scope restriction must be stated in the abstract, the introduction, and Proposition 3.4 itself.
  3. [Section 4.3, Figures 9 and 10] The empirical support for Proposition 3.4 is incomplete as presented. The memory normalization in Figure 9 is ambiguous ('against DistDGL on a single node'), and the text's 'linear memory scaling' does not distinguish total memory from per-accelerator memory; with A replicated, total memory grows as m*Theta(E). In addition, the experiments cover only 1, 2, and 4 nodes and only graphs whose adjacency matrices fit within roughly 24 GB, so they cannot demonstrate the asymptotically constant behavior as the accelerator count grows. Please clarify the metric being plotted and restrict the empirical claim to the tested regime.
minor comments (5)
  1. [Section 3.1.2] The paragraph contains the typo 'partiton parallelism' instead of 'partition parallelism'.
  2. [Section 4.1] The implementation description contains the typo 'equiped' instead of 'equipped'.
  3. [Algorithm 2, line 8] Line 8 uses the notation bA_i although the design replicates the full propagation matrix on every aggregation accelerator; please use bA (or explicitly define bA_i := bA) to avoid suggesting a distributed partition of the matrix.
  4. [Section 3.1.3] The all-to-all scalability argument cites butterfly networks and 2048-device Transformer training as evidence of feasibility, but those systems do not perform the same per-layer all-to-all exchange with the same message sizes as MoP; the analogy should be stated more carefully.
  5. [Section 5.2.1] The statement that storing A for ogbn-papers100M requires only 24 GB should clarify whether this is the size of one full copy or the total across all aggregation accelerators, since the relevant quantity for the memory bottleneck is per-accelerator memory when A is replicated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: MixGCN's scaling claims follow from the MoP construction; the self-cited partition-parallelism bound is not load-bearing for MixGCN's own result, and the adjacency-matrix replication caveat is a scope limitation, not a circular step.

full rationale

The paper's claimed derivation chain does not reduce any result back into its own assumptions. Proposition 3.1 is proved by a genuine reduction from identical-machines scheduling. Proposition 3.2 is cited from the authors' prior BNS-GCN work [65], but it characterizes partition parallelism (a baseline), and MixGCN's central claim—constant communication volume and O(N) feature memory under MoP—does not rely on [65] for its truth; it is a direct reading of Algorithm 2, where features are split along the feature dimension and exchanged by all-to-all, so per-iteration communication is O(Nd) independently of the accelerator count. Proposition 3.3 is the trivial observation that uniform feature/node splits balance work. Proposition 3.4 is stated without proof and is overbroad: Section 5.2.1 concedes that every aggregation accelerator replicates the propagation matrix A, so total per-device memory is O(E + Nd/m), not O(N); this is a missing-proof/scope-of-claim issue rather than circularity. The S-SpMM accelerator and pipeline scheduler are evaluated against external baselines with stated 28nm/CMOS parameters and RCM preprocessing; no fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported via self-citation. The only self-citation is Proposition 3.2, which is not load-bearing for MixGCN's own derivation, so it does not raise the circularity score. Such caveats belong under correctness risk, not circularity.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No new physical entities are postulated. The dedicated sparse accelerator (SPU) is a designed artifact with simulation-based evidence, not an invented entity invoked to explain results. The main burdens are the full replication of the adjacency matrix on every accelerator and the assumed 50% output sparsity, both acknowledged in the paper.

free parameters (2)
  • Assumed output sparsity ratio for S-SpMM = 0.5
    Section 3.2.1 assumes dropout/ReLU drops about 50% of aggregation outputs, a value not measured per dataset; it directly drives the accelerator's simulated speedup.
  • HBM bandwidth for accelerator simulation = 1024 GB/s
    Section 4.1 sets 1024 GB/s following TPU/GPU designs. This chosen parameter affects all compared accelerators and the relative speedups.
assumptions (5)
  • domain assumption The propagation matrix A can be fully replicated on every accelerator
    MoP requires each aggregation accelerator to hold the full graph adjacency. Section 5.2.1 admits this limits extremely large graphs.
  • domain assumption Input graphs are undirected and symmetric
    Used in Section 3.2.2 to reuse node ordering for backward propagation; not all real graphs are symmetric.
  • domain assumption Adjacency matrix is binary, so no multipliers are needed
    Section 4.1 design; weighted graphs would require multipliers and change the area/power estimates.
  • domain assumption Output sparsity from dropout/ReLU is about 50%
    Assumed in Section 3.2.1; the S-SpMM speedup hinges on this value.
  • domain assumption All-to-all communication can be implemented with butterfly networks with O(n log n) wires
    Section 3.1.3 relies on literature [46,80] for cost-effective all-to-all; not validated on the testbed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Accelerators." pith.science (2026). https://pith.science/paper/RGPCREXZ

@misc{pith2026250101951,
  author       = {Pith},
  title        = {Pith review of: MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Accelerators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGPCREXZ}},
  note         = {Machine review of arXiv:2501.01951}
}
read the original abstract

Graph convolutional networks (GCNs) have demonstrated superiority in graph-based learning tasks. However, training GCNs on full graphs is particularly challenging, due to the following two challenges: (1) the associated feature tensors can easily explode the memory and block the communication bandwidth of modern accelerators, and (2) the computation workflow in training GCNs alternates between sparse and dense matrix operations, complicating the efficient utilization of computational resources. Existing solutions for scalable distributed full-graph GCN training mostly adopt partition parallelism, which is unsatisfactory as they only partially address the first challenge while incurring scaled-out communication volume. To this end, we propose MixGCN aiming to simultaneously address both the aforementioned challenges towards GCN training. To tackle the first challenge, MixGCN integrates mixture of parallelism. Both theoretical and empirical analysis verify its constant communication volumes and enhanced balanced workload; For handling the second challenge, we consider mixture of accelerators (i.e., sparse and dense accelerators) with a dedicated accelerator for GCN training and a fine-grain pipeline. Extensive experiments show that MixGCN achieves boosted training efficiency and scalability.

Figures

Figures reproduced from arXiv: 2501.01951 by the authors.

Figure 1
Figure 1. An illustrative comparison between partition parallelism and the proposed MixGCN, where MixGCN avoids the scaled-out communication volume needed for duplicated remote neighbor features (highlighted in red in (b)) as required by partition parallelism. composed of dependent dense operations [1, 8, 40]. Conse￾quently, a straightforward deployment of GCN training into a distributed DNN system would suffer from low hardw… view at source ↗
Figure 2
Figure 2. Illustrating the workflow of our proposed mixture of parallelism (MoP) where we adopt 3 pairs of aggregation and update accelerators for visual clarity. Algorithm 1: Partition parallelism for GCN training. Input: adjacency matrix 𝐴b𝑖 , node feature 𝑋𝑖 , label 𝑌𝑖 , initial model weight 𝑊0 Output: trained model weight 𝑊𝑇 after 𝑇 iterations 1 𝐻 (0) 𝑖𝑛 ← 𝑋𝑖 ; 2 for 𝑡 ← 0 : 𝑇 − 1 do 3 for 𝑙 ← 0 : 𝐿 − 1 do 4 Receiving rem… view at source ↗
Figure 3
Figure 3. An illustration of S-SpMM in the accelerator for neighbor aggregation. memory usage, regardless of the number of accelerators. Therefore, MoP offers better scalability. • All-to-All Communication. Both partition parallelism and MoP employ all-to-all communication. Nevertheless, in addition to the required constant communication vol￾ume as mentioned above, MoP enjoys a more regular communication pattern, securing bal… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: An illustration of the proposed mixture of accelerators (MoA), which integrates a dedicated accelerator for computing S-SpMM (Sampled Sparse Matrix-Matrix Multiplication). Specifically, as shown in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: An example that illustrates the comparison of the temporal execution flow among different pipeline designs between the sparse and dense accelerators. We assume that the training graph is identical to the graph in Figure 1a. 3.2.2 A Pipeline Scheduler with Node Reorderi…
Figure 6
Figure 6. Figure 6: b), the first processing batch only require the first 4 nodes (i.e., the first two processing batches), enabling the pipeline to eliminate idle periods, as shown in Figure 5d. Method. Building upon this motivation, the core strategy to reduce idleness in MoA is to elim…
Figure 7
Figure 7. Figure 7: Speedup comparison between MixGCN and baselines against DistDGL on 4 nodes [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The time breakdown of MixGCN-CPU without fine-grain pipeline. under this setting is dubbed as MixGCN-CPU. (2) We also evaluate the performance of MoA with the proposed sparse accelerator through simulation. The dedicated accelerator is implemented with a commercial 28n…
Figure 9
Figure 9. Figure 9: Normalized memory for GCN training between MixGCN and baseline methods against DistDGL on a single node. 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes Configuration 0 1 2 3 4 5 Normalized CV OOM OOM O…
Figure 10
Figure 10. Figure 10: Normalized communication volume for GCN between MixGCN and baselines against DistDGL on a single node. 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes 1 Node 2 Nodes 4 Nodes Configuration 0 1 2 3 Max FLOPs / Mean FLOPs OOM O…
Figure 11
Figure 11. Figure 11: Workload balance comparison between MixGCN and baselines. significant performance gains, achieving up to 4.2×, 10.4×, 1.9×, and 5.5× higher throughput compared to DistDGL, CAGNET, PipeGCN, and Sancus, respectively. Additionally, MixGCN-SPU further enhances the efficie…
Figure 12
Figure 12. Figure 12: Speedup over CPU of the proposed sparse acceler￾ator and the baseline accelerators: HyGCN and AWB-GCN [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

93 extracted references · 55 canonical work pages

  1. [1]

    Tensorflow: Large-scale machine learning on heteroge- neous distributed systems

    Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. Tensorflow: Large-scale machine learning on heteroge- neous distributed systems. arXiv preprint arXiv:1603.04467, 2016

    arXiv 2016

  2. [2]

    Hardware accel- eration of graph neural networks

    Adam Auten, Matthew Tomei, and Rakesh Kumar. Hardware accel- eration of graph neural networks. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1–6. IEEE, 2020

    2020

  3. [3]

    Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction

    Guangji Bai, Ziyang Yu, Zheng Chai, Yue Cheng, and Liang Zhao. Staleness-alleviated distributed gnn training via online dynamic- embedding prediction. arXiv preprint arXiv:2308.13466, 2023

    work page Pith review arXiv 2023

  4. [4]

    Pathways: Asynchronous distributed dataflow for ml

    Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Daniel Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, et al. Pathways: Asynchronous distributed dataflow for ml. Proceedings of Machine Learning and Systems , 4:430–449, 2022

    2022

  5. [5]

    Distributed Graph Neural Network Training with Periodic Stale Representation Synchronization

    Zheng Chai, Guangji Bai, Liang Zhao, and Yue Cheng. Distributed graph neural network training with periodic stale representation syn- chronization. arXiv preprint arXiv:2206.00057, 2022

    work page Pith review arXiv 2022

  6. [6]

    Dygnn: Algorithm and architecture support of dynamic pruning for graph neural net- works

    Cen Chen, Kenli Li, Xiaofeng Zou, and Yangfan Li. Dygnn: Algorithm and architecture support of dynamic pruning for graph neural net- works. In 2021 58th ACM/IEEE Design Automation Conference (DAC) , pages 1201–1206. IEEE, 2021

    2021

  7. [7]

    Graph representation learning: a survey

    Fenxiao Chen, Yun-Cheng Wang, Bin Wang, and C-C Jay Kuo. Graph representation learning: a survey. APSIPA Transactions on Signal and Information Processing, 9, 2020

    2020

  8. [8]

    Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems

    Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274, 2015

    arXiv 2015

Show all 93 references
  1. [9]

    Rubik: A hierarchical architecture for efficient graph neural network training

    Xiaobing Chen, Yuke Wang, Xinfeng Xie, Xing Hu, Abanti Basak, Ling Liang, Mingyu Yan, Lei Deng, Yufei Ding, Zidong Du, et al. Rubik: A hierarchical architecture for efficient graph neural network training. IEEE Transactions on Computer-Aided Design of Integrated Circuits and S...

    2021

  2. [10]

    The bandwidth problem for graphs and matrices—a survey

    Phyllis Z Chinn, Jarmila Chvátalová, Alexander K Dewdney, and Nor- man E Gibbs. The bandwidth problem for graphs and matrices—a survey. Journal of Graph Theory , 6(3):223–254, 1982

    1982

  3. [11]

    Reducing the bandwidth of sparse symmetric matrices

    Elizabeth Cuthill and James McKee. Reducing the bandwidth of sparse symmetric matrices. In Proceedings of the 1969 24th national conference, pages 157–172, 1969

    1969

  4. [12]

    Hardware acceleration of sparse and irregular tensor computations of ml models: A survey and insights

    Shail Dave, Riyadh Baghdadi, Tony Nowatzki, Sasikanth Avancha, Aviral Shrivastava, and Baoxin Li. Hardware acceleration of sparse and irregular tensor computations of ml models: A survey and insights. Proceedings of the IEEE , 109(10):1706–1752, 2021

    2021

  5. [13]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961, 2021

    2021 arXiv

  6. [14]

    Gn- nautoscale: Scalable and expressive graph neural networks via histori- cal embeddings

    Matthias Fey, Jan E Lenssen, Frank Weichert, and Jure Leskovec. Gn- nautoscale: Scalable and expressive graph neural networks via histori- cal embeddings. arXiv preprint arXiv:2106.05609, 2021

    2021 arXiv

  7. [15]

    Tlpgnn: A lightweight two- level parallelism paradigm for graph neural network computation on gpu

    Qiang Fu, Yuede Ji, and H Howie Huang. Tlpgnn: A lightweight two- level parallelism paradigm for graph neural network computation on gpu. In Proceedings of the 31st International Symposium on High- Performance Parallel and Distributed Computing , pages 122–134, 2022

    2022

  8. [16]

    P3: Distributed deep graph learning at scale

    Swapnil Gandhi and Anand Padmanabha Iyer. P3: Distributed deep graph learning at scale. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21) , pages 551–568, 2021

    2021

  9. [17]

    Understanding the design space of sparse/dense multiphase dataflows for mapping graph neural networks on spatial accelerators

    Raveesh Garg, Eric Qin, Francisco Muñoz-Martínez, Robert Guirado, Akshay Jain, Sergi Abadal, José L Abellán, Manuel E Acacio, Eduard Alarcón, Sivasankaran Rajamanickam, et al. Understanding the design space of sparse/dense multiphase dataflows for mapping graph neural networks...

    2021 arXiv

  10. [18]

    Awb-gcn: A graph convolutional network accelerator with runtime workload rebalancing

    Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, et al. Awb-gcn: A graph convolutional network accelerator with runtime workload rebalancing. In 2020 53rd Annual IEEE/ACM International Symposium on Micr...

    2020

  11. [19]

    I-gcn: A graph convolutional network accelerator with runtime locality enhancement through islandization

    Tong Geng, Chunshu Wu, Yongan Zhang, Cheng Tan, Chenhao Xie, Haoran You, Martin Herbordt, Yingyan Lin, and Ang Li. I-gcn: A graph convolutional network accelerator with runtime locality enhancement through islandization. In MICRO-54: 54th Annual IEEE/ACM Interna- tional Sympos...

    2021

  12. [20]

    Data-efficient graph grammar learning for molecu- lar generation

    Minghao Guo, Veronika Thost, Beichen Li, Payel Das, Jie Chen, and Wojciech Matusik. Data-efficient graph grammar learning for molecu- lar generation. In International Conference on Learning Representations, 2022

    2022

  13. [21]

    Inductive represen- tation learning on large graphs

    Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive represen- tation learning on large graphs. In Advances in neural information processing systems, pages 1024–1034, 2017

    2017

  14. [22]

    Pipedream: Fast and efficient pipeline parallel dnn training

    Aaron Harlap, Deepak Narayanan, Amar Phanishayee, Vivek Seshadri, Nikhil Devanur, Greg Ganger, and Phil Gibbons. Pipedream: Fast and efficient pipeline parallel dnn training. arXiv preprint arXiv:1806.03377, 2018

    2018 arXiv

  15. [23]

    Open graph benchmark: Datasets for machine learning on graphs

    Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. arXiv preprint arXiv:2005.00687, 2020

    2005 arXiv

  16. [24]

    Recurrent graph convolutional network-based multi- task transient stability assessment framework in power system

    Jiyu Huang, Lin Guan, Yinsheng Su, Haicheng Yao, Mengxuan Guo, and Zhi Zhong. Recurrent graph convolutional network-based multi- task transient stability assessment framework in power system. IEEE Access, 8:93283–93296, 2020

    2020

  17. [25]

    Wisegraph: Optimizing gnn with joint workload partition of graph and operations

    Kezhao Huang, Jidong Zhai, Liyan Zheng, Haojie Wang, Yuyang Jin, Qihao Zhang, Runqing Zhang, Zhen Zheng, Youngmin Yi, and Xipeng Shen. Wisegraph: Optimizing gnn with joint workload partition of graph and operations. In Proceedings of the Nineteenth European Conference on Compu...

    2024

  18. [26]

    Gpipe: Efficient training of giant neural networks using pipeline parallelism

    Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al. Gpipe: Efficient training of giant neural networks using pipeline parallelism. In Advances in neural information processing systems, pages ...

    2019

  19. [27]

    Graphpipe: Improving performance and scala- bility of dnn training with graph pipeline parallelism

    Byungsoo Jeon, Mengdi Wu, Shiyi Cao, Sunghyun Kim, Sunghyun Park, Neeraj Aggarwal, Colin Unger, Daiyaan Arfeen, Peiyuan Liao, Xupeng Miao, et al. Graphpipe: Improving performance and scala- bility of dnn training with graph pipeline parallelism. arXiv preprint arXiv:2406.17145, 2024

    2024 arXiv

  20. [28]

    A survey on knowledge graphs: Representation, acquisition, and applications

    Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 2021

    2021

  21. [29]

    Improving the accuracy, scalability, and performance of graph neural networks with roc

    Zhihao Jia, Sina Lin, Mingyu Gao, Matei Zaharia, and Alex Aiken. Improving the accuracy, scalability, and performance of graph neural networks with roc. Proceedings of Machine Learning and Systems (MLSys), pages 187–198, 2020

    2020

  22. [30]

    A survey of frequent subgraph mining algorithms

    Chuntao Jiang, Frans Coenen, and Michele Zito. A survey of frequent subgraph mining algorithms. The Knowledge Engineering Review , 28(1):75–105, 2013

    2013

  23. [31]

    A unified architecture for accelerating distributed{DNN} 12 training in heterogeneous{GPU/CPU} clusters

    Yimin Jiang, Yibo Zhu, Chang Lan, Bairen Yi, Yong Cui, and Chuanx- iong Guo. A unified architecture for accelerating distributed{DNN} 12 training in heterogeneous{GPU/CPU} clusters. In 14th USENIX Sym- posium on Operating Systems Design and Implementation (OSDI 20) , pages 463...

    2020

  24. [32]

    In-datacenter performance analysis of a tensor pro- cessing unit

    Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al. In-datacenter performance analysis of a tensor pro- cessing unit. In Proceedings of the 44th annual international symposium ...

    2017

  25. [33]

    A fast and high quality multilevel scheme for partitioning irregular graphs

    George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing, 20(1):359–392, 1998

    1998

  26. [34]

    Grip: A graph neu- ral network accelerator architecture

    Kevin Kiningham, Christopher Re, and Philip Levis. Grip: A graph neu- ral network accelerator architecture. arXiv preprint arXiv:2007.13828, 2020

    2007 arXiv

  27. [35]

    Semi-supervised classification with graph convolutional networks

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  28. [36]

    What is twitter, a social network or a news media? InProceedings of the 19th international conference on World wide web , pages 591–600, 2010

    Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon. What is twitter, a social network or a news media? InProceedings of the 19th international conference on World wide web , pages 591–600, 2010

    2010

  29. [37]

    Maeri: En- abling flexible dataflow mapping over dnn accelerators via reconfig- urable interconnects

    Hyoukjun Kwon, Ananda Samajdar, and Tushar Krishna. Maeri: En- abling flexible dataflow mapping over dnn accelerators via reconfig- urable interconnects. ACM SIGPLAN Notices, 53(2):461–475, 2018

    2018

  30. [38]

    Gshard: Scaling giant models with conditional com- putation and automatic sharding

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional com- putation and automatic sharding. arXiv preprint arXiv:2006.16668 , 2020

    2006 arXiv

  31. [39]

    Gcnax: A flexible and energy-efficient accelerator for graph convolutional neural networks

    Jiajun Li, Ahmed Louri, Avinash Karanth, and Razvan Bunescu. Gcnax: A flexible and energy-efficient accelerator for graph convolutional neural networks. In 2021 IEEE International Symposium on High- Performance Computer Architecture (HPCA), pages 775–788. IEEE, 2021

    2021

  32. [40]

    Pytorch distributed: Experiences on accelerating data parallel training

    Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, et al. Pytorch distributed: Experiences on accelerating data parallel training. arXiv preprint arXiv:2006.15704, 2020

    2006 arXiv

  33. [41]

    Terapipe: Token-level pipeline parallelism for training large-scale language models

    Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, and Ion Stoica. Terapipe: Token-level pipeline parallelism for training large-scale language models. In International Conference on Machine Learning, pages 6543–6552. PMLR, 2021

    2021

  34. [42]

    Engn: A high-throughput and energy-efficient accelerator for large graph neural networks

    Shengwen Liang, Ying Wang, Cheng Liu, Lei He, LI Huawei, Dawen Xu, and Xiaowei Li. Engn: A high-throughput and energy-efficient accelerator for large graph neural networks. IEEE Transactions on Computers, 70(9):1511–1525, 2020

    2020

  35. [43]

    Nvidia tesla: A unified graphics and computing architecture

    Erik Lindholm, John Nickolls, Stuart Oberman, and John Montrym. Nvidia tesla: A unified graphics and computing architecture. IEEE micro, 28(2):39–55, 2008

    2008

  36. [44]

    Flexflow: A flexible dataflow accelerator architecture for convolutional neural networks

    Wenyan Lu, Guihai Yan, Jiajun Li, Shijun Gong, Yinhe Han, and Xi- aowei Li. Flexflow: A flexible dataflow accelerator architecture for convolutional neural networks. In 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) , pages 553–564. IEEE, 2017

    2017

  37. [45]

    NeuGraph: Parallel deep neural network compu- tation on large graphs

    Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, and Yafei Dai. NeuGraph: Parallel deep neural network compu- tation on large graphs. In 2019 USENIX Annual Technical Conference (USENIX ATC 19), pages 443–458, 2019

    2019

  38. [46]

    All-to-all personalized communication on multi- stage interconnection networks

    Annalisa Massini. All-to-all personalized communication on multi- stage interconnection networks. Discrete applied mathematics, 128(2- 3):435–446, 2003

    2003

  39. [47]

    Distgnn: Scalable distributed training for large-scale graph neural networks

    Vasimuddin Md, Sanchit Misra, Guixiang Ma, Ramanarayan Mohanty, Evangelos Georganas, Alexander Heinecke, Dhiraj Kalamkar, Nes- reen K Ahmed, and Sasikanth Avancha. Distgnn: Scalable distributed training for large-scale graph neural networks. In Proceedings of the International...

    2021

  40. [48]

    Device placement optimization with reinforce- ment learning

    Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, and Jeff Dean. Device placement optimization with reinforce- ment learning. In International Conference on Machine Learning, pages 2430–2439. PMLR, 2017

    2017

  41. [49]

    Pipedream: generalized pipeline parallelism for dnn train- ing

    Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. Pipedream: generalized pipeline parallelism for dnn train- ing. In Proceedings of the 27th ACM Symposium on Operating Systems Principles,...

    2019

  42. [50]

    Sancus: staleness-aware communication-avoiding full- graph decentralized training in large-scale graph neural networks

    Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. Sancus: staleness-aware communication-avoiding full- graph decentralized training in large-scale graph neural networks. Proceedings of the VLDB Endowment , 15(9):1937–1950, 2022

    1937

  43. [51]

    Fusedmm: A unified sddmm-spmm kernel for graph embedding and graph neural networks

    Md Khaledur Rahman, Majedul Haque Sujon, and Ariful Azad. Fusedmm: A unified sddmm-spmm kernel for graph embedding and graph neural networks. In 2021 IEEE International Parallel and Dis- tributed Processing Symposium (IPDPS) , pages 256–266. IEEE, 2021

    2021

  44. [52]

    Deepspeed-moe: Advancing mixture-of-experts infer- ence and training to power next-generation ai scale

    Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. Deepspeed-moe: Advancing mixture-of-experts infer- ence and training to power next-generation ai scale. arXiv preprint arXiv:2201.05596, 2022

    2022 arXiv

  45. [53]

    Zero: Memory optimizations toward training trillion parameter mod- els

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter mod- els. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–16. IEEE, 2020

    2020

  46. [54]

    Learn locally, correct globally: A distributed algorithm for training graph neural networks

    Morteza Ramezani, Weilin Cong, Mehrdad Mahdavi, Mahmut T Kan- demir, and Anand Sivasubramaniam. Learn locally, correct globally: A distributed algorithm for training graph neural networks. arXiv preprint arXiv:2111.08202, 2021

    2021 arXiv

  47. [55]

    Deepspeed: System optimizations enable training deep learning mod- els with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning mod- els with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Min- ing, pag...

    2020

  48. [56]

    Algorithms for scheduling independent tasks

    Sartaj K Sahni. Algorithms for scheduling independent tasks. Journal of the ACM (JACM) , 23(1):116–127, 1976

    1976

  49. [57]

    Horovod: fast and easy dis- tributed deep learning in tensorflow

    Alexander Sergeev and Mike Del Balso. Horovod: fast and easy dis- tributed deep learning in tensorflow. arXiv preprint arXiv:1802.05799, 2018

    2018 arXiv

  50. [58]

    Mesh-tensorflow: Deep learning for supercomputers

    Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, et al. Mesh-tensorflow: Deep learning for supercomputers. Advances in neural information processing systems, 31, 2018

    2018

  51. [59]

    Megatron-lm: Training multi- billion parameter language models using model parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi- billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019

    1909 arXiv

  52. [60]

    Synopsys design compiler

    Synopsys. Synopsys design compiler. https://www.synopsys.com/ implementation-and-signoff/rtl-synthesis-test/dc-ultra.html . Ac- cessed: 2022-02-17

    2022

  53. [61]

    Dorylus: affordable, scalable, and accurate gnn training with distributed cpu servers and serverless threads

    John Thorpe, Yifan Qiao, Jonathan Eyolfson, Shen Teng, Guanzhou Hu, Zhihao Jia, Jinliang Wei, Keval Vora, Ravi Netravali, Miryung Kim, et al. Dorylus: affordable, scalable, and accurate gnn training with distributed cpu servers and serverless threads. In 15th USENIX Symposium ...

    2021

  54. [62]

    Reducing com- munication in graph neural network training

    Alok Tripathy, Katherine Yelick, and Aydin Buluc. Reducing com- munication in graph neural network training. arXiv preprint arXiv:2005.03300, 2020

    2005 arXiv

  55. [63]

    Graph attention networks

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. 13 arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  56. [64]

    Adaptive message quan- tization and parallelization for distributed full-graph gnn training

    Borui Wan, Juntao Zhao, and Chuan Wu. Adaptive message quan- tization and parallelization for distributed full-graph gnn training. Proceedings of Machine Learning and Systems , 5, 2023

    2023

  57. [65]

    BNS- GCN: Efficient full-graph training of graph convolutional networks with partition-parallelism and random boundary node sampling

    Cheng Wan, Youjie Li, Ang Li, Nam Sung Kim, and Yingyan Lin. BNS- GCN: Efficient full-graph training of graph convolutional networks with partition-parallelism and random boundary node sampling. Fifth Conference on Machine Learning and Systems , 2022

    2022

  58. [66]

    Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin

    Cheng Wan, Youjie Li, Cameron R. Wolfe, Anastasios Kyrillidis, Nam Sung Kim, and Yingyan Lin. PipeGCN: Efficient full-graph training of graph convolutional networks with pipelined feature com- munication. In International Conference on Learning Representations , 2022

    2022

  59. [67]

    Towards cognitive ai systems: a survey and prospective on neuro-symbolic ai

    Zishen Wan, Che-Kai Liu, Hanchen Yang, Chaojian Li, Haoran You, Yonggan Fu, Cheng Wan, Tushar Krishna, Yingyan Lin, and Arijit Ray- chowdhury. Towards cognitive ai systems: a survey and prospective on neuro-symbolic ai. arXiv preprint arXiv:2401.01040, 2024

    2024 arXiv

  60. [68]

    Flexgraph: a flexible and efficient distributed framework for gnn training

    Lei Wang, Qiang Yin, Chao Tian, Jianbang Yang, Rong Chen, Wenyuan Yu, Zihang Yao, and Jingren Zhou. Flexgraph: a flexible and efficient distributed framework for gnn training. In Proceedings of the Sixteenth European Conference on Computer Systems , pages 67–82, 2021

    2021

  61. [69]

    Supporting very large models using automatic dataflow graph partitioning

    Minjie Wang, Chien-chin Huang, and Jinyang Li. Supporting very large models using automatic dataflow graph partitioning. In Proceed- ings of the Fourteenth EuroSys Conference 2019 , pages 1–17, 2019

    2019

  62. [70]

    Deep graph library: A graph-centric, highly-performant package for graph neural networks

    Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv prepr...

    1909 arXiv

  63. [71]

    Neutronstar: distributed gnn training with hybrid dependency management

    Qiange Wang, Yanfeng Zhang, Hao Wang, Chaoyi Chen, Xiaodong Zhang, and Ge Yu. Neutronstar: distributed gnn training with hybrid dependency management. In Proceedings of the 2022 International Conference on Management of Data , pages 1301–1315, 2022

    2022

  64. [72]

    Gnnadvisor: An adaptive and efficient runtime system for gnn acceleration on gpus

    Yuke Wang, Boyuan Feng, Gushu Li, Shuangchen Li, Lei Deng, Yuan Xie, and Yufei Ding. Gnnadvisor: An adaptive and efficient runtime system for gnn acceleration on gpus. arXiv preprint arXiv:2006.06608, 2020

    2006 arXiv

  65. [73]

    how graph neural networks go beyond weisfeiler-lehman?

    Asiri Wijesinghe and Qing Wang. A new perspective on" how graph neural networks go beyond weisfeiler-lehman?". In International Conference on Learning Representations , 2021

    2021

  66. [74]

    A comprehensive survey on graph neural networks

    Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems , 32(1):4–24, 2020

    2020

  67. [75]

    Graph learning: A survey

    Feng Xia, Ke Sun, Shuo Yu, Abdul Aziz, Liangtian Wan, Shirui Pan, and Huan Liu. Graph learning: A survey. IEEE Transactions on Artificial Intelligence, 2(2):109–127, 2021

    2021

  68. [76]

    How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018

    2018 arXiv

  69. [77]

    Gspmd: general and scalable parallelization for ml computation graphs

    Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Blake Hechtman, Yan- ping Huang, Rahul Joshi, Maxim Krikun, Dmitry Lepikhin, Andy Ly, Marcello Maggioni, et al. Gspmd: general and scalable parallelization for ml computation graphs. arXiv preprint arXiv:2105.04663, 2021

    2021 arXiv

  70. [78]

    Hygcn: A gcn accelerator with hybrid architecture

    Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. Hygcn: A gcn accelerator with hybrid architecture. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 15–29. IEEE, 2020

    2020

  71. [79]

    Defining and evaluating network com- munities based on ground-truth

    Jaewon Yang and Jure Leskovec. Defining and evaluating network com- munities based on ground-truth. In Proceedings of the ACM SIGKDD workshop on mining data semantics , pages 1–8, 2012

    2012

  72. [80]

    Optimal all-to-all personalized exchange in self-routable multistage networks

    Yuanyuan Yang and Jianchao Wang. Optimal all-to-all personalized exchange in self-routable multistage networks. IEEE Transactions on Parallel and Distributed Systems , 11(3):261–274, 2000

    2000

  73. [81]

    Graph convolutional neural networks for web-scale recommender systems

    Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Min- ing, pages 974...

    2018

  74. [82]

    Gcod: Graph convolutional network acceleration via dedicated al- gorithm and accelerator co-design

    Haoran You, Tong Geng, Yongan Zhang, Ang Li, and Yingyan Lin. Gcod: Graph convolutional network acceleration via dedicated al- gorithm and accelerator co-design. arXiv preprint arXiv:2112.11594, 2021

    2021 arXiv

  75. [83]

    Graphact: Accelerating gcn training on cpu-fpga heterogeneous platforms

    Hanqing Zeng and Viktor Prasanna. Graphact: Accelerating gcn training on cpu-fpga heterogeneous platforms. In Proceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pages 255–265, 2020

    2020

  76. [84]

    Hardware accel- eration of large scale gcn inference

    Bingyi Zhang, Hanqing Zeng, and Viktor Prasanna. Hardware accel- eration of large scale gcn inference. In 2020 IEEE 31st International Conference on Application-specific Systems, Architectures and Processors (ASAP), pages 61–68. IEEE, 2020

    2020

  77. [85]

    Autosync: Learning to synchronize for data-parallel distributed deep learning

    Hao Zhang, Yuan Li, Zhijie Deng, Xiaodan Liang, Lawrence Carin, and Eric Xing. Autosync: Learning to synchronize for data-parallel distributed deep learning. Advances in Neural Information Processing Systems, 33:906–917, 2020

    2020

  78. [86]

    Understanding gnn computational graph: A coordinated computation, io, and memory perspective

    Hengrui Zhang, Zhongming Yu, Guohao Dai, Guyue Huang, Yufei Ding, Yuan Xie, and Yu Wang. Understanding gnn computational graph: A coordinated computation, io, and memory perspective. Pro- ceedings of Machine Learning and Systems , 4:467–484, 2022

    2022

  79. [87]

    Sylvie: 3d-adaptive and universal system for large-scale graph neural network training

    Meng Zhang, Qinghao Hu, Cheng Wan, Haozhao Wang, Peng Sun, Yonggang Wen, and Tianwei Zhang. Sylvie: 3d-adaptive and universal system for large-scale graph neural network training. In2024 IEEE 40th International Conference on Data Engineering (ICDE) , pages 3823–3836. IEEE, 2024

    2024

  80. [88]

    A survey on graph neural network acceleration: Algorithms, systems, and customized hardware

    Shichang Zhang, Atefeh Sohrabizadeh, Cheng Wan, Zijie Huang, Ziniu Hu, Yewen Wang, Jason Cong, Yizhou Sun, et al. A survey on graph neural network acceleration: Algorithms, systems, and customized hardware. arXiv preprint arXiv:2306.14052, 2023

    2023

  81. [89]

    G-cos: Gnn-accelerator co-search towards both better accuracy and efficiency

    Yongan Zhang, Haoran You, Yonggan Fu, Tong Geng, Ang Li, and Yingyan Lin. G-cos: Gnn-accelerator co-search towards both better accuracy and efficiency. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD) , pages 1–9. IEEE, 2021

    2021

  82. [90]

    Pytorch fsdp: experiences on scaling fully sharded data parallel

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, et al. Pytorch fsdp: experiences on scaling fully sharded data parallel. arXiv preprint arXiv:2304.11277, 2023

    2023 arXiv

  83. [91]

    Distdgl: dis- tributed graph neural network training for billion-scale graphs

    Da Zheng, Chao Ma, Minjie Wang, Jinjing Zhou, Qidong Su, Xiang Song, Quan Gan, Zheng Zhang, and George Karypis. Distdgl: dis- tributed graph neural network training for billion-scale graphs. In 2020 IEEE/ACM 10th Workshop on Irregular Applications: Architectures and Algorithms...

    2020

  84. [92]

    Alpa: Automating inter-and intra- operator parallelism for distributed deep learning

    Lianmin Zheng, Zhuohan Li, Hao Zhang, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Joseph E Gonzalez, et al. Alpa: Automating inter-and intra- operator parallelism for distributed deep learning. arXiv preprint arXiv:2201.12023, 2022

    2022 arXiv

  85. [93]

    Aligraph: A comprehensive graph neural network platform

    Rong Zhu, Kun Zhao, Hongxia Yang, Wei Lin, Chang Zhou, Baole Ai, Yong Li, and Jingren Zhou. Aligraph: A comprehensive graph neural network platform. arXiv preprint arXiv:1902.08730, 2019. 14 A Proof of Proposition 3.1 We reduce a well-known NP-Hard problem identical-machines s...

    1902 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.