Sampling methods for efficient training of graph convolutional networks: A survey

Xin Liu, Mingyu Yan, Lei Deng, Guoqi Li, Xiaochun Ye, Dongrui Fan

Main figure for Sampling methods for efficient training of graph convolutional networks: A survey

Abstract

Graph Convolutional Networks (GCNs) have received significant attention from various research fields due to the excellent performance in learning graph representations. Although GCN performs well compared with other methods, it still faces challenges. Training a GCN model for large-scale graphs in a conventional way requires high computation and storage costs. Therefore, motivated by an urgent need in terms of efficiency and scalability in training GCN, sampling methods have been proposed and achieved a significant effect. In this paper, we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of GCN. To highlight the characteristics and differences of sampling methods, we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all categories. Finally, we discuss some challenges and future research directions of the sampling methods.

Benchmark Results

Tested on 2 datasets 6 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelF1 ↑Paper
1
EnGCN
0.9714 ±0.0003
2
SAGN+SLE
0.9710
3
GAMLP(JK)
0.9704 ±0.0001
17
Cluster-GCN
actual #17 vs paper #52 35 places worse
0.9660
best reported
actual #17 vs paper #52 35 places worse
paper 0.9540 ±0.0010
paper 0.9540 ±0.0010 Δ at pub 0.0120
15
GraphSAINT-EDGE
0.9660 ±0.0010
This paper
16
GraphSAINT-RW
0.9660 ±0.0010
This paper
14
GraphSAGE
0.9660
23
AS-GCN
actual #23 vs paper #118 95 places worse
0.9640 ±0.0010
best reported
actual #23 vs paper #118 95 places worse
paper 0.8900 ±0.0130
41
Parallelized Graph Sampling
0.9600 ±0.0020
This paper
91
FastGCN
actual #91 vs paper #120 29 places worse
0.9370
best reported
actual #91 vs paper #120 29 places worse
paper 0.8250 ±0.0060
104
LADIES
0.9320 ±0.0010
This paper
#ModelF1 ↑Paper
1
GAMLP(JK)
0.9982 ±0.0001
2
GraphCON-GCN
0.9960
3
GCNII
0.9953 ±0.0001
9
Cluster-GCN
actual #9 vs paper #90 81 places worse
0.9940
best reported
actual #9 vs paper #90 81 places worse
paper 0.8750 ±0.0040
best today: Graph-Coupled Oscillator Networks (ICML 2022)
paper 0.8750 ±0.0040 Δ at pub 0.1186 Δ today 0.1190
21
GCN
0.9850
30
GraphSAINT-EDGE
0.9810 ±0.0070
This paper
31
GraphSAINT-RW
0.9810 ±0.0040
This paper
130
Parallelized Graph Sampling
0.6960 ±0.0040
This paper
135
AS-GCN
actual #135 vs paper #158 23 places worse
0.6870 ±0.0120
best reported
actual #135 vs paper #158 23 places worse
paper 0.5990 ±0.0040
paper 0.5990 ±0.0040 Δ at pub 0.0880 Δ today 0.0880
149
FastGCN
actual #149 vs paper #167 18 places worse
0.6370 ±0.0060
best reported
actual #149 vs paper #167 18 places worse
paper 0.5020 ±0.0030
paper 0.5020 ±0.0030 Δ at pub 0.1350
162
LADIES
0.5740 ±0.0030
This paper
173
MLP
0.4620
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)