NeurIPS 2019

Wasserstein Weisfeiler-Lehman Graph Kernels

Matteo Togninalli, M. Ghisu, Felipe Llinares-López, Bastian Alexander Rieck, Karsten M. Borgwardt

Main figure for Wasserstein Weisfeiler-Lehman Graph Kernels

Abstract

Most graph kernels are an instance of the class of $ $-Convolution kernels, which measure the similarity of objects by comparing their substructures. Despite their empirical success, most graph kernels use a naive aggregation of the final set of substructures, usually a sum or average, thereby potentially discarding valuable information about the distribution of individual components. Furthermore, only a limited instance of these approaches can be extended to continuously attributed graphs. We propose a novel method that relies on the Wasserstein distance between the node feature vector distributions of two graphs, which allows finding subtler differences in data sets by considering graphs as high-dimensional objects rather than simple means. We further propose a Weisfeiler--Lehman-inspired embedding scheme for graphs with continuous node attributes and weighted edges, enhance it with the computed Wasserstein distance, and thereby improve the state-of-the-art prediction performance on several graph classification tasks.

Benchmark Results

Tested on 5 datasets 11 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
159
WL
actual #159 vs paper #429 270 places worse
0.9040 ±0.0570
best reported
actual #159 vs paper #429 270 places worse
paper 0.8578 ±0.0083

Difference within reported uncertainty; not counted as underperformance.

paper 0.8578 ±0.0083 Δ at pub 0.0022 Δ today 0.0462
372
Weisfeiler--Lehman-inspired embedding scheme
0.8727 ±0.0150
This paper
388
WL-OA
actual #388 vs paper #388
0.8715 ±0.0182
actual #388 vs paper #388
paper 0.8715 ±0.0182
paper 0.8715 ±0.0182 Δ at pub 0.0265
403
MLP
0.8660 ±0.0495
439
V
actual #439 vs paper #441 2 places worse
0.8540 ±0.0070
best reported
actual #439 vs paper #441 2 places worse
paper 0.8539 ±0.0073

Difference within reported uncertainty; not counted as underperformance.

paper 0.8539 ±0.0073 Δ at pub 0.0001
445
E
actual #445 vs paper #480 35 places worse
0.8520 ±0.0060
best reported
actual #445 vs paper #480 35 places worse
paper 0.8417 ±0.0144
paper 0.8417 ±0.0144 Δ at pub 0.0103
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
8
WL-OA
actual #8 vs paper #8
0.8610 ±0.0020
best reported
actual #8 vs paper #8
paper 0.8608 ±0.0027

Difference within reported uncertainty; not counted as underperformance.

paper 0.8608 ±0.0027 Δ at pub 0.0002 Δ today 0.0002
12
WL
actual #12 vs paper #12
0.8590 ±0.0010
best reported
actual #12 vs paper #12
paper 0.8583 ±0.0009
best at pub: Propagation Kernels (2014)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8583 ±0.0009 Δ at pub 0.0007
14
Weisfeiler--Lehman-inspired embedding scheme
0.8575 ±0.0025
This paper
131
GraphSAGE
0.8200 ±0.0100
517
E
actual #517 vs paper #534 17 places worse
0.6620 ±0.0010
best reported
actual #517 vs paper #534 17 places worse
paper 0.6357 ±0.0012
paper 0.6357 ±0.0012 Δ at pub 0.0263
529
V
actual #529 vs paper #533 4 places worse
0.6460 ±0.0010
best reported
actual #529 vs paper #533 4 places worse
paper 0.6422 ±0.0011
paper 0.6422 ±0.0011 Δ at pub 0.0038
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
117
WL-OA
actual #117 vs paper #124 7 places worse
0.7640 ±0.0040
best reported
actual #117 vs paper #124 7 places worse
paper 0.7637 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.7637 ±0.0030 Δ at pub 0.0003
149
MLP
0.7600 ±0.0320
190
WL
actual #190 vs paper #253 63 places worse
0.7560 ±0.0040
best reported
actual #190 vs paper #253 63 places worse
paper 0.7499 ±0.0028
paper 0.7499 ±0.0028 Δ at pub 0.0061
342
E
actual #342 vs paper #403 61 places worse
0.7350 ±0.0020
best reported
actual #342 vs paper #403 61 places worse
paper 0.7218 ±0.0042
paper 0.7218 ±0.0042 Δ at pub 0.0132
381
V
actual #381 vs paper #404 23 places worse
0.7212 ±0.0019
actual #381 vs paper #404 23 places worse
paper 0.7212 ±0.0019
paper 0.7212 ±0.0019 Δ at pub 0.0022 Δ today 0.0048
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
7
WWL
0.7325 ±0.0087
This paper
17
GatedGCN
0.7000 ±0.0494
18
WL
actual #18 vs paper #145 127 places worse
0.6950 ±0.0320
best reported
actual #18 vs paper #145 127 places worse
paper 0.5333 ±0.0093
paper 0.5333 ±0.0093 Δ at pub 0.0572 Δ today 0.1617
21
RBF-WL
0.6843 ±0.0147
This paper
26
GraphSAGE
0.6817 ±0.0545
38
HGK-SP
0.6636 ±0.0037
This paper
46
gh
0.6565 ±0.0080
This paper
66
HGK-WL
0.6304 ±0.0065
This paper
98
WL-OA
actual #98 vs paper #108 10 places worse
0.5990 ±0.0110
best reported
actual #98 vs paper #108 10 places worse
paper 0.5897 ±0.0082
paper 0.5897 ±0.0082 Δ at pub 0.0093
124
MLP
0.5583 ±0.0352
167
VH-C
0.4715 ±0.0079
This paper
244
E
actual #244 vs paper #281 37 places worse
0.2740 ±0.0080
best reported
actual #244 vs paper #281 37 places worse
paper 0.2187 ±0.0064
paper 0.2187 ±0.0064 Δ at pub 0.0553
261
V
actual #261 vs paper #268 7 places worse
0.2340 ±0.0110
best reported
actual #261 vs paper #268 7 places worse
paper 0.2272 ±0.0056
paper 0.2272 ±0.0056 Δ at pub 0.0068
#ModelAccuracy ↑Paper
1
GC-GNN (GCN)
0.9210 ±0.0810
2
SUGAR
0.8403 ±0.0133
3
Ortho-g-U-Nets
0.8387
15
SAGE
0.8088 ±0.0233
25
WL
actual #25 vs paper #67 42 places worse
0.7978 ±0.0036
best reported
actual #25 vs paper #67 42 places worse
paper 0.7829 ±0.0030
paper 0.7829 ±0.0030 Δ at pub 0.0149 Δ today 0.0149
28
WWL
0.7969 ±0.0050
This paper
40
WL-OA
actual #40 vs paper #41 1 place worse
0.7920 ±0.0040
best reported
actual #40 vs paper #41 1 place worse
paper 0.7915 ±0.0033

Difference within reported uncertainty; not counted as underperformance.

paper 0.7915 ±0.0033 Δ at pub 0.0005
67
E
actual #67 vs paper #129 62 places worse
0.7830 ±0.0050
best reported
actual #67 vs paper #129 62 places worse
paper 0.7549 ±0.0021
paper 0.7549 ±0.0021 Δ at pub 0.0281
68
V
actual #68 vs paper #68
0.7824 ±0.0028
actual #68 vs paper #68
paper 0.7824 ±0.0028

Improvement within reported uncertainty; not highlighted.

paper 0.7824 ±0.0028 Δ at pub 0.0004
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result