Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach

Qingfeng Chen, Shiyuan Li, Yixin Liu, Shirui Pan, Geoffrey I. Webb, Shichao Zhang

Main figure for Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach

Abstract

Graph neural networks (GNNs) excel in graph representation learning by integrating graph structure and node features. Existing GNNs, unfortunately, fail to account for the uncertainty of class probabilities that vary with the depth of the model, leading to unreliable and risky predictions in real-world scenarios. To bridge the gap, in this paper, we propose a novel vidence using raph eural etwork (EFGNN for short) to achieve trustworthy prediction, enhance node classification accuracy, and make explicit the risk of wrong predictions. In particular, we integrate the evidence theory with multi-hop propagation-based GNN architecture to quantify the prediction uncertainty of each node with the consideration of multiple receptive fields. Moreover, a parameter-free cumulative belief fusion (CBF) mechanism is developed to leverage the changes in prediction uncertainty and fuse the evidence to improve the trustworthiness of the final prediction. To effectively optimize the EFGNN model, we carefully design a joint learning objective composed of evidence cross-entropy, dissonance coefficient, and false confident penalty. The experimental results on various datasets and theoretical analyses demonstrate the effectiveness of the proposed model in terms of accuracy and trustworthiness, as well as its robustness to potential attacks. The source code of EFGNN is available at https://github.com/Shiy-Li/EFGNN.

Proposed method: EFGNN

Benchmark Results

Tested on 4 datasets 18 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelAccuracy ↑Paper
1
LGTL
0.9518 ±0.0011
2
MixHop (LO)
0.9513 ±0.0038
3
GraphSAGE (LO)
0.9511 ±0.0036
17
APPNP
actual #17 vs paper #439 422 places worse
0.9355 ±0.0060
best reported
actual #17 vs paper #439 422 places worse
paper 0.8010 ±0.0020
paper 0.8010 ±0.0020 Δ at pub 0.1345
18
GCN
actual #18 vs paper #541 523 places worse
0.9296 ±0.0015
best reported
actual #18 vs paper #541 523 places worse
paper 0.7930 ±0.0070
20
GAT
actual #20 vs paper #585 565 places worse
0.9233 ±0.0018
best reported
actual #20 vs paper #585 565 places worse
paper 0.7900 ±0.0030
36
ResGCN
actual #36 vs paper #652 616 places worse
0.9110
best reported
actual #36 vs paper #652 616 places worse
paper 0.7830 ±0.0060
paper 0.7830 ±0.0060 Δ at pub 0.1280
116
SAGE
0.8881 ±0.0015
135
JK-Net
actual #135 vs paper #614 479 places worse
0.8841 ±0.0001
best reported
actual #135 vs paper #614 479 places worse
paper 0.7880 ±0.0070
paper 0.7880 ±0.0070 Δ at pub 0.0961
210
MLP
0.8690 ±0.0074
320
EFGNN
0.8240 ±0.0020
This paper
390
GAMLP
0.8080 ±0.0050
This paper
444
AP-GCN
actual #444 vs paper #482 38 places worse
0.8010 ±0.0070
best reported
actual #444 vs paper #482 38 places worse
paper 0.7970 ±0.0030

Difference within reported uncertainty; not counted as underperformance.

paper 0.7970 ±0.0030 Δ at pub 0.0040
466
S²GC
0.7990 ±0.0030
This paper
465
AGNN
actual #465 vs paper #732 267 places worse
0.7990 ±0.0700
best reported
actual #465 vs paper #732 267 places worse
paper 0.7710 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.7710 ±0.0050 Δ at pub 0.0280
478
Flip-APPNP
0.7980 ±0.0060
This paper
484
SIGN
actual #484 vs paper #504 20 places worse
0.7970 ±0.0043
best reported
actual #484 vs paper #504 20 places worse
paper 0.7950 ±0.0050

Difference within reported uncertainty; not counted as underperformance.

paper 0.7950 ±0.0050 Δ today 0.0020
548
CaGCN
0.7930 ±0.0150
This paper
579
S-BGCN-T
0.7910 ±0.0020
This paper
580
GKDE
0.7910 ±0.0020
This paper
594
Drop-GCN
0.7900 ±0.0030
This paper
#ModelAccuracy ↑Paper
1
GraphTARIF
0.9703 ±0.0019
2
GAT
actual #2 vs paper #408 406 places worse
0.9669 ±0.0014
best reported
actual #2 vs paper #408 406 places worse
paper 0.9080 ±0.0100
3
GAT+T
0.9660 ±0.0007
7
SAGE
0.9643 ±0.0027
8
GCN
actual #8 vs paper #395 387 places worse
0.9632 ±0.0008
best reported
actual #8 vs paper #395 387 places worse
paper 0.9120 ±0.0060
21
APPNP
actual #21 vs paper #378 357 places worse
0.9573 ±0.0012
best reported
actual #21 vs paper #378 357 places worse
paper 0.9140 ±0.0030
paper 0.9140 ±0.0030 Δ at pub 0.0433
50
SGC
actual #50 vs paper #366 316 places worse
0.9511 ±0.0009
best reported
actual #50 vs paper #366 316 places worse
paper 0.9160 ±0.0070
paper 0.9160 ±0.0070 Δ at pub 0.0351
117
EFGNN
0.9380 ±0.0010
This paper
271
GAMLP
0.9260 ±0.0080
This paper
313
CaGCN
0.9230 ±0.0060
This paper
332
AP-GCN
0.9210 ±0.0030
This paper
343
Drop-GCN
0.9200 ±0.0010
This paper
349
JK-Net
0.9190 ±0.0070
This paper
350
Flip-APPNP
0.9190 ±0.0030
This paper
363
SIGN
0.9170 ±0.0070
This paper
388
ResGCN
0.9130 ±0.0090
This paper
390
MLP
0.9128 ±0.0063
478
AGNN
0.8890 ±0.0150
This paper
487
GKDE
0.8830 ±0.0160
This paper
525
S-BGCN-T
0.8180 ±0.0130
This paper
#ModelAccuracy ↑Paper
1
GraphTARIF
0.9461 ±0.0017
2
GCN
actual #2 vs paper #390 388 places worse
0.9412 ±0.0008
best reported
actual #2 vs paper #390 388 places worse
paper 0.8240 ±0.0040
3
GAT
actual #3 vs paper #428 425 places worse
0.9398 ±0.0022
best reported
actual #3 vs paper #428 425 places worse
paper 0.8010 ±0.0060
8
SAGE
0.9351 ±0.0006
58
APPNP
actual #58 vs paper #409 351 places worse
0.9093 ±0.0018
best reported
actual #58 vs paper #409 351 places worse
paper 0.8170 ±0.0030
paper 0.8170 ±0.0030 Δ at pub 0.0923
82
SGC
actual #82 vs paper #398 316 places worse
0.9022 ±0.0021
best reported
actual #82 vs paper #398 316 places worse
paper 0.8220 ±0.0090
paper 0.8220 ±0.0090 Δ at pub 0.0802
308
AP-GCN
actual #308 vs paper #357 49 places worse
0.8518 ±0.0023
best reported
actual #308 vs paper #357 49 places worse
paper 0.8370 ±0.0060
paper 0.8370 ±0.0060 Δ at pub 0.0148
320
MLP
0.8463
326
EFGNN
0.8450 ±0.0040
This paper
343
GAMLP
actual #343 vs paper #343
0.8420 ±0.0050
actual #343 vs paper #343
paper 0.8420 ±0.0050
paper 0.8420 ±0.0050 Δ at pub 0.0340
378
SIGN
0.8310 ±0.0080
This paper
380
S2GC
0.8310 ±0.0070
This paper
404
JK-Net
0.8200 ±0.0060
This paper
406
CaGCN
0.8200 ±0.0160
This paper
417
Drop-GCN
0.8120 ±0.0050
This paper
418
ResGCN
0.8110 ±0.0070
This paper
423
Flip-APPNP
0.8080 ±0.0010
This paper
440
S-BGCN-T
0.7830 ±0.0150
This paper
442
GKDE
0.7810 ±0.0200
This paper
449
AGNN
0.7570 ±0.0240
This paper

Heterophilic Graphs

#ModelAccuracy ↑Paper
1
DiGGR
0.4535 ±0.0353
2
TRIGON
0.4402 ±0.0028
3
DoG
0.4320 ±0.0120
33
SIGN
actual #33 vs paper #385 352 places worse
0.4122 ±0.0096
best reported
actual #33 vs paper #385 352 places worse
paper 0.2850 ±0.0010
paper 0.2850 ±0.0010 Δ at pub 0.1272
41
APPNP
actual #41 vs paper #453 412 places worse
0.4046 ±0.0064
best reported
actual #41 vs paper #453 412 places worse
paper 0.2310 ±0.0050
paper 0.2310 ±0.0050 Δ at pub 0.1736
42
MLP
0.4026 ±0.0108
68
GraphSAGE
0.3899 ±0.0085
135
GCN
actual #135 vs paper #449 314 places worse
0.3690
best reported
actual #135 vs paper #449 314 places worse
paper 0.2350 ±0.0050
paper 0.2350 ±0.0050 Δ at pub 0.1340
182
GAT
actual #182 vs paper #421 239 places worse
0.3605 ±0.0035
best reported
actual #182 vs paper #421 239 places worse
paper 0.2570 ±0.0080
paper 0.2570 ±0.0080 Δ at pub 0.1035
298
AGNN
actual #298 vs paper #442 144 places worse
0.3220
best reported
actual #298 vs paper #442 144 places worse
paper 0.2400 ±0.0300
353
EFGNN
0.3000 ±0.0030
This paper
355
Flip-APPNP
0.2990 ±0.0110
This paper
422
CaGCN
0.2570 ±0.0100
This paper
436
JK-Net
0.2490 ±0.0040
This paper
445
ResGCN
0.2380 ±0.0280
This paper
450
S²GC
0.2350 ±0.0020
This paper
468
GAMLP
0.2220 ±0.0140
This paper
475
S-BGCN-T
0.2160 ±0.0060
This paper
484
AP-GCN
0.2070 ±0.0290
This paper
487
Drop-GCN
0.2050 ±0.0060
This paper
489
GKDE
0.1970 ±0.0110
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Best today — OK at publication; today's leaderboard best is elsewhere