REVIEW 4 cited by
Stochastic Training of Graph Convolutional Networks with Variance Reduction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Graph convolutional networks (GCNs) are powerful deep neural networks for graph-structured data. However, GCN computes the representation of a node recursively from its neighbors, making the receptive field size grow exponentially with the number of layers. Previous attempts on reducing the receptive field size by subsampling neighbors do not have a convergence guarantee, and their receptive field size per node is still in the order of hundreds. In this paper, we develop control variate based algorithms which allow sampling an arbitrarily small neighbor size. Furthermore, we prove new theoretical guarantee for our algorithms to converge to a local optimum of GCN. Empirical results show that our algorithms enjoy a similar convergence with the exact algorithm using only two neighbors per node. The runtime of our algorithms on a large Reddit dataset is only one seventh of previous neighbor sampling algorithms.
Forward citations
Cited by 4 Pith papers
-
Enhancing LLMs for Graph Tasks via Graph-aware LoRA Generation
GaRA generates task-specific LoRA weight updates conditioned on graph structures to enable better whole-graph encoding in LLMs for zero-shot graph learning.
-
RapidGNN: Energy and Communication-Efficient Distributed Training on Large-Scale Graph Neural Networks
Deterministic sampling plus hot-node caching and prefetching speeds up distributed GNN training by about 2.5x to 3x on tested graphs while cutting remote fetches and energy.
-
On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks
Excess risk of SGD and ridge regression on GNNs is characterized through graph spectra, showing graph shape decides which algorithm generalizes better and deeper networks amplify the difference.
-
Neighbor-Sampling Based Momentum Stochastic Methods for Training Graph Neural Networks
The paper creates Adam-style optimizers that combine neighbor sampling and control variates for graph neural networks, with optimal convergence rates and better node-classification performance than control-variate SGD.
Discussion (0). Sign in to comment.