WWW 2021

SelfGNN: Self-supervised Graph Neural Networks without explicit negative sampling

Zekarias T. Kefato, Sarunas Girdzijauskas

Main figure for SelfGNN: Self-supervised Graph Neural Networks without explicit negative sampling

Abstract

Real world data is mostly unlabeled or only few instances are labeled. Manually labeling data is a very expensive and daunting task. This calls for unsupervised learning techniques that are powerful enough to achieve comparable results as semi-supervised/supervised techniques. Contrastive self-supervised learning has emerged as a powerful direction, in some cases outperforming supervised techniques. In this study, we propose, SelfGNN, a novel contrastive self-supervised graph neural network (GNN) without relying on explicit contrastive terms. We leverage Batch Normalization, which introduces implicit contrastive terms, without sacrificing performance. Furthermore, as data augmentation is key in contrastive learning, we introduce four feature augmentation (FA) techniques for graphs. Though graph topological augmentation (TA) is commonly used, our empirical findings show that FA perform as good as TA. Moreover, FA incurs no computational overhead, unlike TA, which often has $O(N^3)$ time complexity, $N$ -- number of nodes. Our empirical evaluation on seven publicly available real-world data shows that, SelfGNN is powerful and leads to a performance comparable with SOTA supervised GNNs and always better than SOTA semi-supervised and unsupervised GNNs. The source code is available at https://github.com/zekarias-tilahun/SelfGNN.

Benchmark Results

Tested on 4 datasets 5 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphSAGE+CNA
0.9418 ±0.0033
2
GCN+CNA
0.9366 ±0.0048
3
TransformerConv
0.9350 ±0.0058
21
GCN
0.9004
177
DGI
actual #177 vs paper #606 429 places worse
0.8630 ±0.0020
best reported
actual #177 vs paper #606 429 places worse
paper 0.8100 ±0.0024
paper 0.8100 ±0.0024 Δ at pub 0.0130 Δ today 0.0530
610
SelfBFA
0.8100 ±0.0018
This paper
694
SelfHK
0.7901 ±0.0031
This paper
696
SelfPPR
0.7898 ±0.0025
This paper
722
SelfKatz
0.7771 ±0.0034
This paper
724
MLP
0.7768
770
MvGrlPPR
0.7530 ±0.0072
This paper
780
MvGrlHK
0.7450 ±0.0071
This paper
781
MvGrlKatz
0.7450 ±0.0060
This paper
#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
44
GCN
0.8033 ±0.0088
66
DGI
actual #66 vs paper #739 673 places worse
0.7890 ±0.0020
best reported
actual #66 vs paper #739 673 places worse
paper 0.6630 ±0.0028
paper 0.6630 ±0.0028 Δ at pub 0.0550 Δ today 0.1260
350
MLP
0.7289 ±0.0010
685
SelfPPR
0.6808 ±0.0048
This paper
686
SelfHK
0.6801 ±0.0043
This paper
717
SelfBFA
0.6719 ±0.0037
This paper
751
SelfKatz
0.6614 ±0.0032
This paper
820
MvGrlKatz
0.6150
This paper
843
MvGrlPPR
0.5820
This paper
845
MvGrlHK
0.5750 ±0.0007
This paper
#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
18
GCN
actual #18 vs paper #585 567 places worse
0.9296 ±0.0015
best reported
actual #18 vs paper #585 567 places worse
paper 0.7900 ±0.0001
20
GAT
actual #20 vs paper #711 691 places worse
0.9233 ±0.0018
best reported
actual #20 vs paper #711 691 places worse
paper 0.7750 ±0.0001
116
SAGE
0.8881 ±0.0015
125
GraphSAGE
actual #125 vs paper #764 639 places worse
0.8864 ±0.0030
best reported
actual #125 vs paper #764 639 places worse
paper 0.7620 ±0.0001
paper 0.7620 ±0.0001 Δ at pub 0.1090 Δ today 0.1244
210
MLP
0.8690 ±0.0074
311
SelfGNN PPR (GAT)
0.8270 ±0.0001
This paper
314
SelfGNN hk (GAT)
0.8260 ±0.0001
This paper
322
SelfGNN Standard (GCN encoder)
0.8230 ±0.0001
This paper
358
SelfGNN Split (GCN encoder)
0.8130 ±0.0001
This paper
374
SelfGNN Katz (GCN encoder)
0.8100 ±0.0001
This paper
737
SelfGNN LDP (GCN encoder)
0.7710 ±0.0001
This paper
770
SelfGNN Paste (GAT)
0.7610 ±0.0001
This paper
#ModelAccuracy ↑Paper
1
GrokFormer
0.9831 ±0.0018
2
PolyFormer
0.9808 ±0.0027
3
SGFormer
0.9796 ±0.0081
6
GAT
actual #6 vs paper #49 43 places worse
0.9620 ±0.0010
actual #6 vs paper #49 43 places worse
paper 0.9620 ±0.0010
paper 0.9620 ±0.0010 Δ at pub 0.0073 Δ today 0.0162
9
GCN
actual #9 vs paper #42 33 places worse
0.9650 ±0.0010
actual #9 vs paper #42 33 places worse
paper 0.9650 ±0.0010
paper 0.9650 ±0.0010 Δ at pub 0.0085 Δ today 0.0124
17
SAGE
0.9739 ±0.0018
58
MLP
0.9597 ±0.0007
121
SelfGNN Split
0.9550 ±0.0030
This paper
122
SelfGNN Paste
0.9550 ±0.0020
This paper
160
SelfGNN LDP
0.9500 ±0.0020
This paper
169
SelfGNN Standard
0.9480
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result