ICLR 2020

Wasserstein Embedding for Graph Learning

Soheil Kolouri, Navid Naderializadeh, G. Rohde, Heiko Hoffmann

Main figure for Wasserstein Embedding for Graph Learning

Abstract

We present Wasserstein Embedding for Graph Learning (WEGL), a novel and fast framework for embedding entire graphs in a vector space, in which various machine learning models are applicable for graph-level prediction tasks. We leverage new insights on defining similarity between graphs as a function of the similarity between their node embedding distributions. Specifically, we use the Wasserstein distance to measure the dissimilarity between node embeddings of different graphs. Unlike prior work, we avoid pairwise calculation of distances between graphs and reduce the computational complexity from quadratic to linear in the number of graphs. WEGL calculates Monge maps from a reference distribution to each node embedding and, based on these maps, creates a fixed-sized vector representation of the graph. We evaluate our new graph embedding approach on various benchmark graph-property prediction tasks, showing state-of-the-art classification performance while having superior computational efficiency. The code is available at.

Proposed method: WL

Benchmark Results

Tested on 6 datasets 4 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
159
WL
actual #159 vs paper #790 631 places worse
0.9040 ±0.0570
best reported
actual #159 vs paper #790 631 places worse
paper 0.5700 ±0.0200
paper 0.5700 ±0.0200 Δ at pub 0.2920 Δ today 0.3340
263
WEGL + GBDT (Concat)
0.8930 ±0.0660
This paper
321
WEGL + RF + Concat
0.8830 ±0.0510
This paper
403
MLP
0.8660 ±0.0495
491
WEGL + SVM-RBF
0.8400 ±0.0760
This paper
567
WEGL + MLP (Concat)
0.8200 ±0.0620
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
131
GraphSAGE
0.8200 ±0.0100
303
WEGL + GBDT (Concat)
0.7840 ±0.0160
This paper
311
WEGL + GBDT + Avg
0.7820 ±0.0290
This paper
379
WEGL + GBDT + Final
0.7640 ±0.0250
This paper
507
WEGL + SVM - RBF (Concat)
0.6780 ±0.0210
This paper
524
WEGL + SVM - RBF + Avg
0.6510 ±0.0210
This paper
528
WEGL + SVM - RBF + Final
0.6490 ±0.0230
This paper
547
WEGL + MLP (Concat)
0.6130 ±0.0460
This paper
554
WEGL + MLP (Avg)
0.6000 ±0.0380
This paper
556
WEGL + MLP (Final)
0.5990 ±0.0370
This paper
562
WEGL + SVM - Linear (Concat)
0.5890 ±0.0720
This paper
564
WEGL + SVM - Linear + Final
0.5740 ±0.0500
This paper
568
WEGL + SVM - Linear (Avg)
0.5620 ±0.0360
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
63
GraphNorm
0.7740 ±0.0490
This paper
108
WEGL + Random Forest
0.7650 ±0.0420
This paper
126
CapsGNN
0.7630 ±0.0360
This paper
129
WEGL + GBDT
0.7630 ±0.0390
This paper
132
DGCNN
actual #132 vs paper #370 238 places worse
0.7626
best reported
actual #132 vs paper #370 238 places worse
paper 0.7290 ±0.0350
best today: Orthogonal Graph Neural Networks (AAAI 2021)

Difference within reported uncertainty; not counted as underperformance.

paper 0.7290 ±0.0350 Δ at pub 0.0030 Δ today 0.0336
139
RetGK
actual #139 vs paper #224 85 places worse
0.7620 ±0.0050
best reported
actual #139 vs paper #224 85 places worse
paper 0.7520 ±0.0030
paper 0.7520 ±0.0030 Δ today 0.0100
151
WEGL + SVM-RBF
0.7600 ±0.0440
This paper
149
MLP
0.7600 ±0.0320
178
DGK
actual #178 vs paper #420 242 places worse
0.7568 ±0.0054
best reported
actual #178 vs paper #420 242 places worse
paper 0.7170 ±0.0050
best today: Hierarchical Graph Capsule Network (AAAI 2020)
paper 0.7170 ±0.0050 Δ at pub 0.0398 Δ today 0.0398
190
WL
actual #190 vs paper #370 180 places worse
0.7560 ±0.0040
best reported
actual #190 vs paper #370 180 places worse
paper 0.7290 ±0.0060
paper 0.7290 ±0.0060 Δ at pub 0.0270
300
WWL
0.7430 ±0.0060
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
23
GCN
0.8260 ±0.0220
82
RetGK
actual #82 vs paper #270 188 places worse
0.8100 ±0.0030
best reported
actual #82 vs paper #270 188 places worse
paper 0.7360 ±0.0030
paper 0.7360 ±0.0030 Δ at pub 0.0740 Δ today 0.0740
98
WEGL + GBDT
0.8060 ±0.0200
This paper
114
GraphNorm
0.8020 ±0.0100
This paper
133
WEGL + Random Forest
0.7980 ±0.0150
This paper
136
GraphSAGE
actual #136 vs paper #267 131 places worse
0.7390 ±0.0170
actual #136 vs paper #267 131 places worse
paper 0.7390 ±0.0170
prev best: Memory-Based Graph Networks (ICLR 2020)
paper 0.7390 ±0.0170 Δ at pub 0.0565 Δ today 0.0580
140
CapsGNN
actual #140 vs paper #140
0.7962 ±0.0091
best reported
actual #140 vs paper #140
paper 0.7960 ±0.0090

Difference within reported uncertainty; not counted as underperformance.

paper 0.7960 ±0.0090 Δ today 0.0002
151
MLP
0.7920 ±0.0240
153
WL
actual #153 vs paper #253 100 places worse
0.7910 ±0.0010
best reported
actual #153 vs paper #253 100 places worse
paper 0.7480 ±0.0020
paper 0.7480 ±0.0020 Δ at pub 0.0430
167
WEGL + SVM-RBF
0.7860 ±0.0100
This paper
191
DGCNN
actual #191 vs paper #290 99 places worse
0.7759 ±0.0224
best reported
actual #191 vs paper #290 99 places worse
paper 0.7120 ±0.0190
best at pub: Graph U-Nets (2019)
paper 0.7120 ±0.0190 Δ at pub 0.0256 Δ today 0.0639
265
AWE
actual #265 vs paper #267 2 places worse
0.7393 ±0.0194
best reported
actual #265 vs paper #267 2 places worse
paper 0.7390 ±0.0190

Difference within reported uncertainty; not counted as underperformance.

paper 0.7390 ±0.0190 Δ today 0.0003
274
DGK
actual #274 vs paper #274
0.7310 ±0.0030
actual #274 vs paper #274
paper 0.7310 ±0.0030

Improvement within reported uncertainty; not highlighted.

paper 0.7310 ±0.0030 Δ at pub 0.0001
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
7
WWL
actual #7 vs paper #106 99 places worse
0.7325 ±0.0087
best reported
actual #7 vs paper #106 99 places worse
paper 0.5910 ±0.0080
paper 0.5910 ±0.0080 Δ at pub 0.1415
17
GatedGCN
0.7000 ±0.0494
18
WL
actual #18 vs paper #145 127 places worse
0.6950 ±0.0320
best reported
actual #18 vs paper #145 127 places worse
paper 0.5320 ±0.0110
paper 0.5320 ±0.0110 Δ at pub 0.0585 Δ today 0.1630
26
GraphSAGE
0.6817 ±0.0545
91
WEGL + Random Forest
0.6050 ±0.0590
This paper
96
WEGL + GBDT
0.6000 ±0.0630
This paper
107
RetGK
0.5910 ±0.0110
This paper
118
WEGL + SVM-RBF
0.5730 ±0.0420
This paper
119
DGCNN
actual #119 vs paper #191 72 places worse
0.5712
best reported
actual #119 vs paper #191 72 places worse
paper 0.3890 ±0.0570
paper 0.3890 ±0.0570 Δ at pub 0.0210 Δ today 0.1822
124
MLP
0.5583 ±0.0352
127
CapsGNN
actual #127 vs paper #136 9 places worse
0.5531 ±0.0423
best reported
actual #127 vs paper #136 9 places worse
paper 0.5470 ±0.0570

Difference within reported uncertainty; not counted as underperformance.

paper 0.5470 ±0.0570 Δ today 0.0061
145
DGK
actual #145 vs paper #246 101 places worse
0.5343 ±0.0091
best reported
actual #145 vs paper #246 101 places worse
paper 0.2710 ±0.0080
paper 0.2710 ±0.0080 Δ at pub 0.2633
169
AWE
actual #169 vs paper #205 36 places worse
0.3580 ±0.0590
actual #169 vs paper #205 36 places worse
paper 0.3580 ±0.0590
prev best: Anonymous Walk Embeddings (ICML 2018)

Improvement within reported uncertainty; not highlighted.

paper 0.3580 ±0.0590 Δ at pub 0.0003 Δ today 0.0978

OGB

#ModelROC-AUC ↑Paper
1
Multi-RF Fusion with Multi-GNN Blending
0.8476 ±0.0002
2
HyperFusion
0.8475 ±0.0003
3
PAS+FPs
0.8420 ±0.0015
9
GatedGCN
0.8090 ±0.0160
58
WEGL + Virtual Node + AutoML
0.7910 ±0.0030
This paper
70
HIMP
0.7880 ±0.0080
This paper
71
GCN+GraphNorm
0.7880 ±0.0100
This paper
78
DeeperGCN
0.7860 ±0.0120
This paper
109
GIN
0.7780 ±0.0182
146
GIN+virtual node
0.7710 ±0.0150
This paper
176
WL + Virtual Node + Random Forest
0.7650 ±0.0180
This paper
184
GCN
actual #184 vs paper #202 18 places worse
0.7630 ±0.0030
best reported
actual #184 vs paper #202 18 places worse
paper 0.7600 ±0.0120

Difference within reported uncertainty; not counted as underperformance.

paper 0.7600 ±0.0120 Δ today 0.0030
224
WL + Random Forest
0.7550 ±0.0150
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere