PyG-SSL: A Graph Self-Supervised Learning Toolkit

Lecheng Zheng, Baoyu Jing, Zihao Li, Zhichen Zeng, Tianxin Wei, Mengting Ai, Xinrui He, Lihui Liu, Dongqi Fu, Jiaxuan You, Hanghang Tong, Jingrui He

Main figure for PyG-SSL: A Graph Self-Supervised Learning Toolkit

Abstract

Graph Self-Supervised Learning (SSL) has emerged as a pivotal area of research in recent years. By engaging in pretext tasks to learn the intricate topological structures and properties of graphs using unlabeled data, these graph SSL models achieve enhanced performance, improved generalization, and heightened robustness. Despite the remarkable achievements of these graph SSL methods, their current implementation poses significant challenges for beginners and practitioners due to the complex nature of graph structures, inconsistent evaluation metrics, and concerns regarding reproducibility hinder further progress in this field. Recognizing the growing interest within the research community, there is an urgent need for a comprehensive, beginner-friendly, and accessible toolkit consisting of the most representative graph SSL algorithms. To address these challenges, we present a Graph SSL toolkit named PyG-SSL, which is built upon PyTorch and is compatible with various deep learning and scientific computing backends. Within the toolkit, we offer a unified framework encompassing dataset loading, hyper-parameter configuration, model training, and comprehensive performance evaluation for diverse downstream tasks. Moreover, we provide beginner-friendly tutorials and the best hyper-parameters of each graph SSL algorithm on different graph datasets, facilitating the reproduction of results. The GitHub repository of the library is.

Benchmark Results

Tested on 4 datasets 6 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
0.9669 ±0.0014
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
130
AFGRL
actual #130 vs paper #190 60 places worse
0.9361 ±0.0020
best reported
actual #130 vs paper #190 60 places worse
paper 0.9320 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.9320 ±0.0030 Δ at pub 0.0041
172
BGRL
actual #172 vs paper #190 18 places worse
0.9337 ±0.0021
best reported
actual #172 vs paper #190 18 places worse
paper 0.9320 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.9320 ±0.0030 Δ at pub 0.0017
191
GCA
actual #191 vs paper #279 88 places worse
0.9320 ±0.0060
best reported
actual #191 vs paper #279 88 places worse
paper 0.9250 ±0.0010
194
SUGRL
0.9320 ±0.0020
This paper
206
MVGRL
actual #206 vs paper #500 294 places worse
0.9310 ±0.0040
best reported
actual #206 vs paper #500 294 places worse
paper 0.8650 ±0.0120
paper 0.8650 ±0.0120 Δ at pub 0.0660
213
DGI
actual #213 vs paper #308 95 places worse
0.9309 ±0.0008
best reported
actual #213 vs paper #308 95 places worse
paper 0.9230 ±0.0070

Difference within reported uncertainty; not counted as underperformance.

paper 0.9230 ±0.0070 Δ at pub 0.0079
272
ReGCL
0.9260 ±0.0030
This paper
320
GraphCL
actual #320 vs paper #525 205 places worse
0.8170 ±0.0090
actual #320 vs paper #525 205 places worse
paper 0.8170 ±0.0090
390
MLP
0.9128 ±0.0063
#ModelAccuracy ↑Paper
1
GCN
0.9302 ±0.0011
2
RSE + RGCN (70b)
0.8852 ±0.0019
3
LEMP+T
0.8825
9
SAGE
0.8771
100
BGRL
0.8000 ±0.0010
This paper
138
SUGRL
actual #138 vs paper #266 128 places worse
0.7897 ±0.0022
best reported
actual #138 vs paper #266 128 places worse
paper 0.7000 ±0.0110
paper 0.7000 ±0.0110 Δ at pub 0.0897 Δ today 0.0897
143
AFGRL
actual #143 vs paper #143
0.7890 ±0.0060
actual #143 vs paper #143
paper 0.7890 ±0.0060
paper 0.7890 ±0.0060 Δ at pub 0.0128
148
ReGCL
0.7870 ±0.0070
This paper
151
DGI
actual #151 vs paper #229 78 places worse
0.7865 ±0.0690
best reported
actual #151 vs paper #229 78 places worse
paper 0.7530 ±0.0110
paper 0.7530 ±0.0110 Δ at pub 0.0295 Δ today 0.0335
162
GCA
actual #162 vs paper #166 4 places worse
0.7840 ±0.0013
best reported
actual #162 vs paper #166 4 places worse
paper 0.7830

Difference within reported uncertainty; not counted as underperformance.

paper 0.7830 Δ at pub 0.0010 Δ today 0.0010
163
GraphCL
actual #163 vs paper #195 32 places worse
0.7837 ±0.0077
best reported
actual #163 vs paper #195 32 places worse
paper 0.7730 ±0.0080

Difference within reported uncertainty; not counted as underperformance.

paper 0.7730 ±0.0080 Δ today 0.0107
190
MVGRL
actual #190 vs paper #248 58 places worse
0.7757 ±0.0046
best reported
actual #190 vs paper #248 58 places worse
paper 0.7270 ±0.0090
paper 0.7270 ±0.0090 Δ at pub 0.0487 Δ today 0.0487
237
MLP
0.7469 ±0.0085
#ModelAccuracy ↑Paper
1
DoG
0.9690 ±0.0030
2
GWN-fa
0.9667 ±0.0026
3
GWN-sym
0.9666 ±0.0026
6
GraphSAGE
0.9593 ±0.0025
33
ReGCL
0.9370 ±0.0030
This paper
43
DGI
actual #43 vs paper #43
0.9350 ±0.0030
actual #43 vs paper #43
paper 0.9350 ±0.0030
51
GCA
actual #51 vs paper #63 12 places worse
0.9332 ±0.0012
best reported
actual #51 vs paper #63 12 places worse
paper 0.9310
53
BGRL
actual #53 vs paper #54 1 place worse
0.9331 ±0.0013
best reported
actual #53 vs paper #54 1 place worse
paper 0.9330 ±0.0010

Difference within reported uncertainty; not counted as underperformance.

paper 0.9330 ±0.0010 Δ at pub 0.0001 Δ today 0.0001
56
AFGRL
actual #56 vs paper #56
0.9330 ±0.0020
actual #56 vs paper #56
paper 0.9330 ±0.0020

Improvement within reported uncertainty; not highlighted.

paper 0.9330 ±0.0020 Δ at pub 0.0003
83
SUGRL
actual #83 vs paper #83
0.9290 ±0.0010
actual #83 vs paper #83
paper 0.9290 ±0.0010
88
MLP
0.9280 ±0.0020
116
MVGRL
actual #116 vs paper #183 67 places worse
0.9211 ±0.0012
best reported
actual #116 vs paper #183 67 places worse
paper 0.8550 ±0.0040
paper 0.8550 ±0.0040 Δ at pub 0.0661 Δ today 0.0661
190
GraphCL
0.8400 ±0.0060
This paper

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
257
InfoGraph
actual #257 vs paper #272 15 places worse
0.8930 ±0.0514
best reported
actual #257 vs paper #272 15 places worse
paper 0.8900 ±0.0110

Difference within reported uncertainty; not counted as underperformance.

paper 0.8900 ±0.0110 Δ at pub 0.0030
258
GraphMAE
actual #258 vs paper #412 154 places worse
0.8930 ±0.0110
best reported
actual #258 vs paper #412 154 places worse
paper 0.8620 ±0.0410

Difference within reported uncertainty; not counted as underperformance.

paper 0.8620 ±0.0410 Δ at pub 0.0310
354
GraphCL
actual #354 vs paper #396 42 places worse
0.8766 ±0.0103
best reported
actual #354 vs paper #396 42 places worse
paper 0.8680 ±0.0130

Difference within reported uncertainty; not counted as underperformance.

paper 0.8680 ±0.0130 Δ at pub 0.0086
403
MLP
0.8660 ±0.0495
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere