NeurIPS 2023

Interpretable Prototype-based Graph Information Bottleneck

Sangwoo Seo, Sungwon Kim, Chanyoung Park

Main figure for Interpretable Prototype-based Graph Information Bottleneck

Abstract

The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB), that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability. The source code of PGIB is available at.

Benchmark Results

Tested on 2 datasets 3 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

TU Dortmund

#ModelAccuracy ↑Paper
1
MSH-GNN
0.9910 ±0.0030
2
ID-GNN+BEC
0.9830 ±0.0360
3
SupCosine
0.9830 ±0.0250
8
GSAT
actual #8 vs paper #592 584 places worse
0.9637 ±0.0215
best reported
actual #8 vs paper #592 584 places worse
paper 0.8088 ±0.0894
39
GIN
actual #39 vs paper #598 559 places worse
0.9330 ±0.0290
best reported
actual #39 vs paper #598 559 places worse
paper 0.8050 ±0.0789
paper 0.8050 ±0.0789 Δ at pub 0.1280
63
GCN
actual #63 vs paper #720 657 places worse
0.9220 ±0.0440
best reported
actual #63 vs paper #720 657 places worse
paper 0.7450 ±0.0789
paper 0.7450 ±0.0789 Δ at pub 0.1460 Δ today 0.1770
81
ProtGNN
actual #81 vs paper #598 517 places worse
0.9170 ±0.0290
best reported
actual #81 vs paper #598 517 places worse
paper 0.8050 ±0.0907

Difference within reported uncertainty; not counted as underperformance.

paper 0.8050 ±0.0907 Δ at pub 0.1120
158
GraphSAGE
0.9040 ±0.0780
183
GAT
actual #183 vs paper #735 552 places worse
0.9010 ±0.0580
best reported
actual #183 vs paper #735 552 places worse
paper 0.7350 ±0.0743
paper 0.7350 ±0.0743 Δ at pub 0.1660 Δ today 0.1660
299
GIB
actual #299 vs paper #645 346 places worse
0.8870 ±0.0530
best reported
actual #299 vs paper #645 346 places worse
paper 0.7900 ±0.0624

Difference within reported uncertainty; not counted as underperformance.

paper 0.7900 ±0.0624 Δ at pub 0.0970
403
MLP
0.8660 ±0.0495
592
VGIB
0.8100 ±0.0663
This paper
#ModelAccuracy ↑Paper
1
MSH-GNN
0.9410 ±0.0330
2
GRDL
0.8260 ±0.0120
3
GMN
0.8225
32
GIN
actual #32 vs paper #454 422 places worse
0.7880 ±0.0410
best reported
actual #32 vs paper #454 422 places worse
paper 0.7030 ±0.0484
49
GAT
actual #49 vs paper #430 381 places worse
0.7800 ±0.0300
best reported
actual #49 vs paper #430 381 places worse
paper 0.7135 ±0.0485

Difference within reported uncertainty; not counted as underperformance.

paper 0.7135 ±0.0485 Δ at pub 0.0665
67
SAGE
0.7726 ±0.0228
148
GCN
actual #148 vs paper #372 224 places worse
0.7600 ±0.0320
best reported
actual #148 vs paper #372 224 places worse
paper 0.7283 ±0.0423

Difference within reported uncertainty; not counted as underperformance.

paper 0.7283 ±0.0423 Δ at pub 0.0317 Δ today 0.0317
149
MLP
0.7600 ±0.0320
223
GIB
0.7525 ±0.0592
This paper
324
ProtGNN
0.7383 ±0.0422
This paper
334
VGIB
0.7366 ±0.0332
This paper
460
GSAT
0.6964 ±0.0471
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere