AAAI 2019

Graph Neural Ordinary Differential Equations

Michael Poli, Stefano Massaroli, Junyoung Park, A. Yamashita, H. Asama, Jinkyoo Park

Main figure for Graph Neural Ordinary Differential Equations

Abstract

We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input--output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with various static and autoregressive GNN models. Results prove general effectiveness of GDEs: in static settings they offer computational advantages by incorporating numerical methods in their forward pass; in dynamic settings, on the other hand, they are shown to improve performance by exploiting the geometry of the underlying dynamics.

Benchmark Results

Tested on 3 datasets 1 baseline value reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
GraphSAGE+CNA
0.9418 ±0.0033
2
GCN+CNA
0.9366 ±0.0048
3
TransformerConv
0.9350 ±0.0058
21
GCN*
actual #21 vs paper #439 418 places worse
0.9004
best reported
actual #21 vs paper #439 418 places worse
paper 0.8280 ±0.0030
paper 0.8280 ±0.0030 Δ at pub 0.0437 Δ today 0.0724
39
SAGE
0.8911 ±0.0012
321
GCDE--rk4
0.8380 ±0.0050
This paper
431
GCDE--rk2
0.8300 ±0.0060
This paper
557
GCDE--dpr5
0.8180 ±0.0120
This paper
724
MLP
0.7768
#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
44
GCN*
actual #44 vs paper #538 494 places worse
0.7120 ±0.0040
actual #44 vs paper #538 494 places worse
paper 0.7120 ±0.0040

Improvement within reported uncertainty; not highlighted.

paper 0.7120 ±0.0040 Δ at pub 0.0010 Δ today 0.0913
61
GraphSAGE
0.7923 ±0.0053
350
MLP
0.7289 ±0.0010
399
GCDE--rk4
0.7250 ±0.0050
This paper
435
GCDE--rk2
0.7230 ±0.0050
This paper
676
GCDE--dpr5
0.6830 ±0.0120
This paper
#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
18
GCN*
actual #18 vs paper #504 486 places worse
0.7950 ±0.0040
actual #18 vs paper #504 486 places worse
paper 0.7950 ±0.0040
116
SAGE
0.8881 ±0.0015
210
MLP
0.8690 ±0.0074
460
GCDE--rk2 (2)
0.7990 ±0.0030
This paper
506
GCDE--rk4 (4)
0.7950 ±0.0040
This paper
634
GCDE--dpr5 (158)
0.7850 ±0.0070
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere