NeurIPS 2016

On Valid Optimal Assignment Kernels and Applications to Graph Classification

Nils M. Kriege, P. Giscard, Richard C. Wilson

Main figure for On Valid Optimal Assignment Kernels and Applications to Graph Classification

Abstract

The success of kernel methods has initiated the design of novel positive semidefinite functions, in particular for structured data. A leading design paradigm for this is the convolution kernel, which decomposes structured objects into their parts and sums over all pairs of parts. Assignment kernels, in contrast, are obtained from an optimal bijection between parts, which can provide a more valid notion of similarity. In general however, optimal assignments yield indefinite functions, which complicates their use in kernel methods. We characterize a class of base kernels used to compare parts that guarantees positive semidefinite optimal assignment kernels. These base kernels give rise to hierarchies from which the optimal assignment kernels are computed in linear time by histogram intersection. We apply these results by developing the Weisfeiler-Lehman optimal assignment kernel for graphs. It provides high classification accuracy on widely-used benchmark data sets improving over the original Weisfeiler-Lehman kernel.

Proposed method: WL-OA

Benchmark Results

Tested on 6 datasets 1 baseline value reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
0.9330 ±0.0290
159
WL
actual #159 vs paper #421 262 places worse
0.8600 ±0.0170
actual #159 vs paper #421 262 places worse
paper 0.8600 ±0.0170
prev best: The Multiscale Laplacian Graph Kernel (NeurIPS 2016)

Improvement within reported uncertainty; not highlighted.

paper 0.8600 ±0.0170 Δ at pub 0.0150
388
WL-OA
actual #388 vs paper #466 78 places worse
0.8715 ±0.0182
best reported
actual #388 vs paper #466 78 places worse
paper 0.8450 ±0.0170
paper 0.8450 ±0.0170 Δ today 0.0265
403
MLP
0.8660 ±0.0495
426
SP
actual #426 vs paper #519 93 places worse
0.8580 ±0.0020
best reported
actual #426 vs paper #519 93 places worse
paper 0.8300 ±0.0140
best at pub: Propagation Kernels (2014)
paper 0.8300 ±0.0140 Δ at pub 0.0280
439
V
0.8540 ±0.0070
This paper
445
E
0.8520 ±0.0060
This paper
444
GL
0.8520 ±0.0090
This paper
546
V-OA
0.8250 ±0.0110
This paper
590
E-OA
0.8100 ±0.0110
This paper
#ModelAccuracy ↑Paper
1
WL-MLP
0.9954
2
GIN
0.9934
3
GCN
0.9727
8
WL-OA
0.8610 ±0.0020
This paper
12
WL
actual #12 vs paper #13 1 place worse
0.8590 ±0.0010
best reported
actual #12 vs paper #13 1 place worse
paper 0.8580 ±0.0020
best at pub: Propagation Kernels (2014)

Difference within reported uncertainty; not counted as underperformance.

paper 0.8580 ±0.0020 Δ at pub 0.0010
131
GraphSAGE
0.8200 ±0.0100
421
SP
actual #421 vs paper #421
0.7450 ±0.0030
actual #421 vs paper #421
paper 0.7450 ±0.0030
prev best: Propagation Kernels (2014)

Improvement within reported uncertainty; not highlighted.

paper 0.7450 ±0.0030 Δ at pub 0.0010
478
GL
0.7050 ±0.0020
This paper
500
E-OA
0.6890 ±0.0030
This paper
517
E
0.6620 ±0.0010
This paper
520
V-OA
0.6560 ±0.0030
This paper
529
V
0.6460 ±0.0010
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
0.7880 ±0.0410
117
WL-OA
0.7640 ±0.0040
This paper
149
MLP
0.7600 ±0.0320
164
SP
0.7580 ±0.0050
This paper
190
WL
0.7560 ±0.0040
This paper
290
E-OA
0.7450 ±0.0060
This paper
325
V-OA
0.7380 ±0.0050
This paper
342
E
0.7350 ±0.0020
This paper
377
GL
0.7270 ±0.0060
This paper
381
V
actual #381 vs paper #412 31 places worse
0.7260
best reported
actual #381 vs paper #412 31 places worse
paper 0.7190 ±0.0040
paper 0.7190 ±0.0040 Δ today 0.0070
#ModelAccuracy ↑Paper
1
WL-MLP
0.9792
2
MSH-GNN
0.9640 ±0.0070
3
CoCN
0.8722 ±0.0013
23
GCN
0.8260 ±0.0220
94
WL-OA
0.8070 ±0.0010
This paper
151
MLP
0.7920 ±0.0240
153
WL
0.7910 ±0.0010
This paper
309
E-OA
0.6820 ±0.0030
This paper
321
GL
0.6470 ±0.0010
This paper
330
V-OA
0.5930 ±0.0010
This paper
332
SP
0.5880 ±0.0020
This paper
340
V
0.5620
This paper
346
E
0.5200
This paper
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
17
GatedGCN
0.7000 ±0.0494
18
WL
actual #18 vs paper #141 123 places worse
0.6950 ±0.0320
best reported
actual #18 vs paper #141 123 places worse
paper 0.5370 ±0.0140
best at pub: The Multiscale Laplacian Graph Kernel (NeurIPS 2016)
paper 0.5370 ±0.0140 Δ at pub 0.0005 Δ today 0.1580
26
GraphSAGE
0.6817 ±0.0545
98
WL-OA
0.5990 ±0.0110
This paper
124
MLP
0.5583 ±0.0352
179
SP
actual #179 vs paper #179
0.4260 ±0.0160
actual #179 vs paper #179
paper 0.4260 ±0.0160
prev best: The Multiscale Laplacian Graph Kernel (NeurIPS 2016)

Improvement within reported uncertainty; not highlighted.

paper 0.4260 ±0.0160 Δ at pub 0.0029
203
E-OA
0.3740 ±0.0180
This paper
211
V-OA
0.3510 ±0.0110
This paper
230
GL
0.3060 ±0.0120
This paper
244
E
0.2740 ±0.0080
This paper
261
V
0.2340 ±0.0110
This paper
#ModelAccuracy ↑Paper
1
GC-GNN (GCN)
0.9210 ±0.0810
2
SUGAR
0.8403 ±0.0133
3
Ortho-g-U-Nets
0.8387
15
SAGE
0.8088 ±0.0233
25
WL
actual #25 vs paper #43 18 places worse
0.7900 ±0.0040
actual #25 vs paper #43 18 places worse
paper 0.7900 ±0.0040
paper 0.7900 ±0.0040 Δ at pub 0.0105 Δ today 0.0078
26
GL
0.7970 ±0.0070
This paper
40
WL-OA
0.7920 ±0.0040
This paper
44
E-OA
0.7900 ±0.0040
This paper
45
SP
0.7900 ±0.0060
This paper
51
V-OA
0.7880 ±0.0030
This paper
67
E
0.7830 ±0.0050
This paper
68
V
actual #68 vs paper #70 2 places worse
0.7824 ±0.0028
best reported
actual #68 vs paper #70 2 places worse
paper 0.7820 ±0.0040

Difference within reported uncertainty; not counted as underperformance.

paper 0.7820 ±0.0040 Δ today 0.0004
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere