Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks

Huixuan Chi, Yuying Wang, Qinfen Hao, Hong Xia

Main figure for Residual Network and Embedding Usage: New Tricks of Node Classification with Graph Convolutional Networks

Abstract

Graph Convolutional Networks (GCNs) and subsequent variants have been proposed to solve tasks on graphs, especially node classification tasks. In the literature, however, most tricks or techniques are either briefly mentioned as implementation details or only visible in source code. In this paper, we first summarize some existing effective tricks used in GCNs mini-batch training. Based on this, two novel tricks named GCN_res Framework and Embedding Usage are proposed by leveraging residual network and pre-trained embedding to improve baseline's test accuracy in different datasets. Experiments on Open Graph Benchmark (OGB) show that, by combining these techniques, the test accuracy of various GCNs increases by 1.21\ We open source our implementation at.

Benchmark Results

Tested on 3 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

OGB

#ModelAccuracy ↑Paper
1
Jacobi
0.8968
2
GPR
0.8953
3
LEMP4HG
0.8939
4
SAGE
0.8933
108
GraphSAGE w/NS + BN + C&S + node2vec
0.8154 ±0.0050
This paper
138
GraphSAGE w/NS + BN + C&S
0.8041 ±0.0022
This paper
168
GraphSAGE w/NS + FLAG
0.7936 ±0.0057
This paper
200
GraphSAGE w/NS
0.7870 ±0.0036
This paper
209
Full-batch GraphSAGE
0.7850 ±0.0014
This paper
351
MLP
0.6605 ±0.0020
#ModelROC-AUC ↑Paper
1
LD
0.8942 ±0.0007
2
X_WO/LD
0.8915 ±0.0012
3
X_PNE
0.8894 ±0.0014
16
GraphSAGE
0.8430
18
GEN + FLAG + node2vec
0.8251 ±0.0043
This paper
20
GEN
actual #20 vs paper #20
0.8130 ±0.0065
actual #20 vs paper #20
paper 0.8130 ±0.0065
21
GEN + FLAG
0.8129 ±0.0067
This paper
#ModelAccuracy ↑Paper
1
LMSPS+LP+MS
0.5784 ±0.0022
2
SeHGNN+emb+ms
0.5719 ±0.0012
3
SeHGNN+ms
0.5671 ±0.0014
45
GraphSAINT + metapath2vec
0.4969 ±0.0022
This paper
53
GraphSAINT (R-GCN aggr)
0.4751 ±0.0022
This paper
57
GraphSAGE
0.4646 ±0.0060
76
MLP
0.3243
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported value improved over the best known when published