Graph Neural Ordinary Differential Equations
Abstract
We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input--output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with various static and autoregressive GNN models. Results prove general effectiveness of GDEs: in static settings they offer computational advantages by incorporating numerical methods in their forward pass; in dynamic settings, on the other hand, they are shown to improve performance by exploiting the geometry of the underlying dynamics.
Benchmark Results
Tested on 3 datasets 1 baseline value reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Classic
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GraphSAGE+CNA | 0.9418
±0.0033 | |
| 2 | GCN+CNA | 0.9366
±0.0048 | |
| 3 | TransformerConv | 0.9350
±0.0058 | |
| 21 | GCN* | 0.9004 best reported | best at pub: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (2017) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) |
| 39 | SAGE | 0.8911
±0.0012 | |
| 321 | GCDE--rk4 | 0.8380
±0.0050 | This paper |
| 431 | GCDE--rk2 | 0.8300
±0.0060 | This paper |
| 557 | GCDE--dpr5 | 0.8180
±0.0120 | This paper |
| 724 | MLP | 0.7768 |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | CNA | 0.9575
±0.0058 | |
| 2 | IS-GIB | 0.9390
±0.0187 | |
| 3 | EERM | 0.9112
±0.0145 | |
| 44 | GCN* | 0.7120
±0.0040 | prev best: Fast Graph Representation Learning with PyTorch Geometric (ICLR 2019) Improvement within reported uncertainty; not highlighted. |
| 61 | GraphSAGE | 0.7923
±0.0053 | |
| 350 | MLP | 0.7289
±0.0010 | |
| 399 | GCDE--rk4 | 0.7250
±0.0050 | This paper |
| 435 | GCDE--rk2 | 0.7230
±0.0050 | This paper |
| 676 | GCDE--dpr5 | 0.6830
±0.0120 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | LGTL | 0.9518
±0.0011 | |
| 2 | MixHop (LO) | 0.9513
±0.0038 | |
| 3 | GraphSAGE (LO) | 0.9511
±0.0036 | |
| 18 | GCN* | 0.7950
±0.0040 | |
| 116 | SAGE | 0.8881
±0.0015 | |
| 210 | MLP | 0.8690
±0.0074 | |
| 460 | GCDE--rk2 (2) | 0.7990
±0.0030 | This paper |
| 506 | GCDE--rk4 (4) | 0.7950
±0.0040 | This paper |
| 634 | GCDE--dpr5 (158) | 0.7850
±0.0070 | This paper |
