Two Birds with One Stone: Enhancing Uncertainty Quantification and Interpretability with Graph Functional Neural Process

Lingkai Kong, Haotian Sun, Yuchen Zhuang, Haorui Wang, Wenhao Mu, Chao Zhang

Main figure for Two Birds with One Stone: Enhancing Uncertainty Quantification and Interpretability with Graph Functional Neural Process

Abstract

Graph neural networks (GNNs) are powerful tools on graph data. However, their predictions are mis-calibrated and lack interpretability, limiting their adoption in critical applications. To address this issue, we propose a new uncertainty-aware and interpretable graph classification model that combines graph functional neural process and graph generative model. The core of our method is to assume a set of latent rationales which can be mapped to a probabilistic embedding space; the predictive distribution of the classifier is conditioned on such rationale embeddings by learning a stochastic correlation matrix. The graph generator serves to decode the graph structure of the rationales from the embedding space for model interpretability. For efficient model training, we adopt an alternating optimization procedure which mimics the well known Expectation-Maximization (EM) algorithm. The proposed method is general and can be applied to any existing GNN architecture. Extensive experiments on five graph classification datasets demonstrate that our framework outperforms state-of-the-art methods in both uncertainty quantification and GNN interpretability. We also conduct case studies to show that the decoded rationale structure can provide meaningful explanations.

Proposed method: BNNs

Benchmark Results

Tested on 3 datasets 5 baseline values reported too low Non-standard metric on 1 dataset

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
8
GSAT
actual #8 vs paper #650 642 places worse
0.9637 ±0.0215
best reported
actual #8 vs paper #650 642 places worse
paper 0.7889 ±0.0058
39
GIN
0.9330 ±0.0290
116
DropEdge
actual #116 vs paper #686 570 places worse
0.9100 ±0.0570
best reported
actual #116 vs paper #686 570 places worse
paper 0.7650 ±0.0163
paper 0.7650 ±0.0163 Δ at pub 0.1450
331
GNNExplainer
actual #331 vs paper #777 446 places worse
0.8800 ±0.0300
best reported
actual #331 vs paper #777 446 places worse
paper 0.6546 ±0.0164
393
PGExplainer
actual #393 vs paper #719 326 places worse
0.8700 ±0.0300
best reported
actual #393 vs paper #719 326 places worse
paper 0.7479 ±0.0257
403
MLP
0.8660 ±0.0495
478
Vanilla
actual #478 vs paper #638 160 places worse
0.8430 ±0.0320
best reported
actual #478 vs paper #638 160 places worse
paper 0.7938 ±0.0087
paper 0.7938 ±0.0087 Δ at pub 0.0492
628
GraphFNP
0.7980 ±0.0071
This paper
645
DeepEnsemble
0.7903 ±0.0065
This paper
656
MC-Dropout
0.7834 ±0.0081
This paper
658
Graph-GP
0.7816 ±0.0103
This paper
667
SGLD
0.7789 ±0.0059
This paper
684
BBP
0.7682 ±0.0059
This paper
711
SubgraphX
0.7519 ±0.0073
This paper
766
GstarX
0.6832 ±0.0117
This paper
774
XGNN
0.6586 ±0.0196
This paper

MoleculeNet

BACE

Global leaderboard →
Non-default metric. This paper reports AU-ROC; the dataset default is ROC-AUC. Values below use AU-ROC only.

Sorted by AU-ROC. Highlighted rows are from other papers.

#ModelAU-ROC ↑Paper
1
BNNs
0.8036 ±0.0089
This paper
2
DropEdge
0.8010 ±0.0576
This paper
3
DeepEnsemble
0.7920
This paper
4
MC-Dropout
0.7840 ±0.0065
This paper
5
GSAT
0.7791 ±0.0078
This paper
6
Vanilla
0.7752 ±0.0066
This paper
7
Graph-GP
0.7696 ±0.0097
This paper
8
SGLD
0.7674 ±0.0187
This paper
9
BBP
0.7534 ±0.0137
This paper
#ModelROC-AUC ↑Paper
1
MAGPrompt+
0.9432 ±0.0238
2
CamS-LLaMA
0.9420
3
SE(3)-I MPPH
0.9400 ±0.0210
34
GIN
0.8875 ±0.0049
105
DeepEnsemble
0.7198
This paper
114
GSAT
actual #114 vs paper #114
0.7155 ±0.0156
actual #114 vs paper #114
paper 0.7155 ±0.0156

Improvement within reported uncertainty; not highlighted.

paper 0.7155 ±0.0156 Δ at pub 0.0242
139
BNNs
0.7025 ±0.0070
This paper
149
MC-Dropout
0.7003 ±0.0198
This paper
150
DropEdge
0.7002 ±0.0069
This paper
156
Vanilla
0.6982 ±0.0217
This paper
165
BBP
0.6954 ±0.0226
This paper
182
Graph-GP
0.6898 ±0.0176
This paper
205
SGLD
0.6787 ±0.0234
This paper
283
GNNExplainer
0.6393 ±0.0266
This paper
284
SubgraphX
0.6370 ±0.0049
This paper
305
PGExplainer
0.5975 ±0.0246
This paper
308
GstarX
0.5855 ±0.0240
This paper
311
XGNN
0.5623 ±0.0425
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)