Graph Neural Networks With Convolutional ARMA Filters

F. Bianchi, Daniele Grattarola, L. Livi, C. Alippi

Main figure for Graph Neural Networks With Convolutional ARMA Filters

Abstract

Popular graph neural networks implement convolution operations on graphs based on polynomial spectral filters. In this paper, we propose a novel graph convolutional layer inspired by the auto-regressive moving average (ARMA) filter that, compared to polynomial ones, provides a more flexible frequency response, is more robust to noise, and better captures the global graph structure. We propose a graph neural network implementation of the ARMA filter with a recursive and distributed formulation, obtaining a convolutional layer that is efficient to train, localized in the node space, and can be transferred to new graphs at test time. We perform a spectral analysis to study the filtering effect of the proposed ARMA layer and report experiments on four downstream tasks: semi-supervised node classification, graph signal classification, graph classification, and graph regression. Results show that the proposed ARMA layer brings significant improvements over graph neural networks based on polynomial filters.

Proposed method: ARMA

Benchmark Results

Tested on 7 datasets 2 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
CNA
0.9575 ±0.0058
2
IS-GIB
0.9390 ±0.0187
3
EERM
0.9112 ±0.0145
35
GAT
actual #35 vs paper #561 526 places worse
0.8080 ±0.0026
best reported
actual #35 vs paper #561 526 places worse
paper 0.7090 ±0.0060
paper 0.7090 ±0.0060 Δ at pub 0.0160 Δ today 0.0990
44
GCN
actual #44 vs paper #616 572 places worse
0.8033 ±0.0088
best reported
actual #44 vs paper #616 572 places worse
paper 0.7010 ±0.0070
paper 0.7010 ±0.0070 Δ at pub 0.0020 Δ today 0.1023
61
GraphSAGE
actual #61 vs paper #756 695 places worse
0.7923 ±0.0053
best reported
actual #61 vs paper #756 695 places worse
paper 0.6590 ±0.0090
paper 0.6590 ±0.0090 Δ today 0.1333
259
SAGE
0.7361 ±0.0190
350
MLP
0.7289 ±0.0010
390
ARMA
0.7250 ±0.0040
This paper
389
GIN
actual #389 vs paper #805 416 places worse
0.7250 ±0.0080
best reported
actual #389 vs paper #805 416 places worse
paper 0.6310 ±0.0200
paper 0.6310 ±0.0200 Δ today 0.0940
617
Chebyshev
0.7010 ±0.0080
This paper
719
CayleyNet
0.6710 ±0.0240
This paper
#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
18
GCN
actual #18 vs paper #585 567 places worse
0.9296 ±0.0015
best reported
actual #18 vs paper #585 567 places worse
paper 0.7900 ±0.0050
20
GAT
actual #20 vs paper #632 612 places worse
0.9233 ±0.0018
best reported
actual #20 vs paper #632 612 places worse
paper 0.7850 ±0.0030
paper 0.7850 ±0.0030 Δ at pub 0.0050 Δ today 0.1383
116
SAGE
0.8881 ±0.0015
125
GraphSAGE
actual #125 vs paper #632 507 places worse
0.8864 ±0.0030
best reported
actual #125 vs paper #632 507 places worse
paper 0.7850 ±0.0060
paper 0.7850 ±0.0060 Δ today 0.1014
157
GIN
actual #157 vs paper #732 575 places worse
0.8810
best reported
actual #157 vs paper #732 575 places worse
paper 0.7710 ±0.0070
paper 0.7710 ±0.0070 Δ today 0.1100
210
MLP
0.8690 ±0.0074
217
ARMA
actual #217 vs paper #602 385 places worse
0.8678 ±0.0021
best reported
actual #217 vs paper #602 385 places worse
paper 0.7890 ±0.0030
780
CayleyNet
0.7560 ±0.0360
This paper
807
Chebyshev
0.7440 ±0.0110
This paper
#ModelF1 ↑Paper
1
GAMLP(JK)
0.9982 ±0.0001
2
GraphCON-GCN
0.9960
3
GCNII
0.9953 ±0.0001
21
GCN
actual #21 vs paper #97 76 places worse
0.9850
best reported
actual #21 vs paper #97 76 places worse
paper 0.8080 ±0.0010
best today: Graph-Coupled Oscillator Networks (ICML 2022)
paper 0.8080 ±0.0010 Δ today 0.1770
57
GAT
actual #57 vs paper #97 40 places worse
0.9732 ±0.0032
best reported
actual #57 vs paper #97 40 places worse
paper 0.8130 ±0.0010
89
ARMA
0.9050 ±0.0030
This paper
92
GIN
actual #92 vs paper #102 10 places worse
0.8724 ±0.0002
best reported
actual #92 vs paper #102 10 places worse
paper 0.7810 ±0.0260
paper 0.7810 ±0.0260 Δ today 0.0914
93
Chebyshev
0.8640 ±0.0010
This paper
96
CayleyNet
0.8490 ±0.0120
This paper
101
GraphSAGE
actual #101 vs paper #125 24 places worse
0.7000
actual #101 vs paper #125 24 places worse
paper 0.7000
132
SAGE
0.6928
173
MLP
0.4620

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
39
GIN
actual #39 vs paper #408 369 places worse
0.9330 ±0.0290
best reported
actual #39 vs paper #408 369 places worse
paper 0.8630 ±0.0910

Difference within reported uncertainty; not counted as underperformance.

paper 0.8630 ±0.0910 Δ today 0.0700
63
GCN
actual #63 vs paper #429 366 places worse
0.9220 ±0.0440
best reported
actual #63 vs paper #429 366 places worse
paper 0.8570 ±0.0660

Difference within reported uncertainty; not counted as underperformance.

paper 0.8570 ±0.0660 Δ today 0.0650
89
ARMA
0.9150 ±0.0420
This paper
158
GraphSAGE
actual #158 vs paper #429 271 places worse
0.9040 ±0.0780
best reported
actual #158 vs paper #429 271 places worse
paper 0.8570 ±0.0470

Difference within reported uncertainty; not counted as underperformance.

paper 0.8570 ±0.0470 Δ today 0.0470
183
GAT
actual #183 vs paper #371 188 places worse
0.9010 ±0.0580
best reported
actual #183 vs paper #371 188 places worse
paper 0.8730 ±0.0530

Difference within reported uncertainty; not counted as underperformance.

paper 0.8730 ±0.0530 Δ today 0.0280
344
CayleyNet
0.8780 ±0.1000
This paper
403
MLP
0.8660 ±0.0495
540
Chebyshev
0.8260 ±0.0520
This paper
718
SAGE
0.7490 ±0.0870
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
actual #32 vs paper #429 397 places worse
0.7880 ±0.0410
best reported
actual #32 vs paper #429 397 places worse
paper 0.7140 ±0.0450

Difference within reported uncertainty; not counted as underperformance.

paper 0.7140 ±0.0450 Δ today 0.0740
49
GAT
actual #49 vs paper #398 349 places worse
0.7800 ±0.0300
best reported
actual #49 vs paper #398 349 places worse
paper 0.7230 ±0.0310
67
SAGE
0.7726 ±0.0228
148
GCN
actual #148 vs paper #441 293 places worse
0.7600 ±0.0320
best reported
actual #148 vs paper #441 293 places worse
paper 0.7100 ±0.0270

Difference within reported uncertainty; not counted as underperformance.

paper 0.7100 ±0.0270 Δ today 0.0500
149
MLP
0.7600 ±0.0320
155
GraphSAGE
actual #155 vs paper #455 300 places worse
0.7590 ±0.0320
best reported
actual #155 vs paper #455 300 places worse
paper 0.7020 ±0.0390

Difference within reported uncertainty; not counted as underperformance.

paper 0.7020 ±0.0390 Δ today 0.0570
326
ARMA
0.7370 ±0.0340
This paper
405
Chebyshev
0.7210 ±0.0350
This paper
482
CayleyNet
0.6560 ±0.0570
This paper
#ModelAccuracy ↑Paper
1
GMN
0.7866
2
MemGNN
0.7550
3
TFGW SP
0.7510 ±0.0500
17
GatedGCN
0.7000 ±0.0494
20
GAT
actual #20 vs paper #152 132 places worse
0.6850 ±0.0524
best reported
actual #20 vs paper #152 132 places worse
paper 0.5170 ±0.0430
best today: GNNBenchmark (JMLR 2023)
paper 0.5170 ±0.0430 Δ today 0.1680
26
GraphSAGE
actual #26 vs paper #92 66 places worse
0.6817 ±0.0545
best reported
actual #26 vs paper #92 66 places worse
paper 0.6030 ±0.0710
best today: GNNBenchmark (JMLR 2023)

Difference within reported uncertainty; not counted as underperformance.

paper 0.6030 ±0.0710 Δ today 0.0787
32
GIN
actual #32 vs paper #167 135 places worse
0.6767 ±0.0583
best reported
actual #32 vs paper #167 135 places worse
paper 0.4570 ±0.0770
best today: GNNBenchmark (JMLR 2023)
paper 0.4570 ±0.0770 Δ today 0.2197
36
GCN
actual #36 vs paper #145 109 places worse
0.6650 ±0.0691
best reported
actual #36 vs paper #145 109 places worse
paper 0.5300 ±0.0530
best today: Hierarchical Graph Capsule Network (AAAI 2020)
paper 0.5300 ±0.0530 Δ today 0.1350
90
ARMA
0.6060 ±0.0720
This paper
117
Chebyshev
0.5790 ±0.0260
This paper
124
MLP
0.5583 ±0.0352
126
SAGE
0.5550 ±0.0511
177
CayleyNet
0.4310 ±0.1070
This paper
#ModelAccuracy ↑Paper
1
GC-GNN (GCN)
0.9210 ±0.0810
2
SUGAR
0.8403 ±0.0133
3
Ortho-g-U-Nets
0.8387
15
SAGE
0.8088 ±0.0233
43
GCN
actual #43 vs paper #140 97 places worse
0.7912 ±0.0307
best reported
actual #43 vs paper #140 97 places worse
paper 0.7470 ±0.0380
best today: Hierarchical Graph Capsule Network (AAAI 2020)

Difference within reported uncertainty; not counted as underperformance.

paper 0.7470 ±0.0380 Δ today 0.0442
74
GIN
actual #74 vs paper #166 92 places worse
0.7805 ±0.0189
best reported
actual #74 vs paper #166 92 places worse
paper 0.7120 ±0.0540

Difference within reported uncertainty; not counted as underperformance.

paper 0.7120 ±0.0540 Δ today 0.0685
86
ARMA
0.7760 ±0.0270
This paper
99
GAT
actual #99 vs paper #167 68 places worse
0.7730 ±0.0368
best reported
actual #99 vs paper #167 68 places worse
paper 0.7090 ±0.0400

Difference within reported uncertainty; not counted as underperformance.

paper 0.7090 ±0.0400 Δ today 0.0640
109
GraphSAGE
actual #109 vs paper #153 44 places worse
0.7660
best reported
actual #109 vs paper #153 44 places worse
paper 0.7360 ±0.0410

Difference within reported uncertainty; not counted as underperformance.

paper 0.7360 ±0.0410 Δ today 0.0300
153
Chebyshev
0.7370 ±0.0370
This paper
170
CayleyNet
0.7030 ±0.1160
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere