The expressive power of pooling in Graph Neural Networks
Abstract
In Graph Neural Networks (GNNs), hierarchical pooling operators generate local summaries of the data by coarsening the graph structure and the vertex features. While considerable attention has been devoted to analyzing the expressive power of message-passing (MP) layers in GNNs, a study on how graph pooling affects the expressiveness of a GNN is still lacking. Additionally, despite the recent advances in the design of pooling operators, there is not a principled criterion to compare them. In this work, we derive sufficient conditions for a pooling operator to fully preserve the expressive power of the MP layers before it. These conditions serve as a universal and theoretically grounded criterion for choosing among existing pooling operators or designing new ones. Based on our theoretical findings, we analyze several existing pooling operators and identify those that fail to satisfy the expressiveness conditions. Finally, we introduce an experimental setup to verify empirically the expressive power of a GNN equipped with pooling layers, in terms of its capability to perform a graph isomorphism test.
Benchmark Results
Tested on 5 datasets 2 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
TU Dortmund
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | MSH-GNN | 0.9910
±0.0030 | |
| 2 | ID-GNN+BEC | 0.9830
±0.0360 | |
| 3 | SupCosine | 0.9830
±0.0250 | |
| 39 | GIN | 0.9330
±0.0290 | |
| 281 | Rand-dense | 0.8890
±0.0430 | This paper |
| 403 | MLP | 0.8660
±0.0495 | |
| 516 | Cmp-Graclus | 0.8320
±0.0910 | This paper |
| 762 | Rand-sparse | 0.6890
±0.1730 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9954 | |
| 2 | GIN | 0.9934 | |
| 3 | GCN | 0.9727 | |
| 5 | DiffPool | 0.7780
±0.0390 | prev best: Contrastive Self-supervised Learning for Graph Classification (AAAI 2020) best today: Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders (IJCAI 2024) Improvement within reported uncertainty; not highlighted. |
| 131 | GraphSAGE | 0.8200
±0.0100 | |
| 170 | Graclus | 0.8120
±0.0340 | prev best: Rethinking pooling in graph neural networks (NeurIPS 2020) Improvement within reported uncertainty; not highlighted. |
| 221 | MinCut | 0.8010
±0.0260 | prev best: Graph Neural Networks in TensorFlow and Keras with Spektral (2020) |
| 231 | ecpool | 0.7980
±0.0330 | This paper |
| 259 | SAGPool | 0.7910
±0.0300 | prev best: Contrastive Self-supervised Learning for Graph Classification (AAAI 2020) Improvement within reported uncertainty; not highlighted. |
| 294 | DMoN | 0.7850
±0.0140 | prev best: Total Variation Graph Neural Networks (ICML 2022) |
| 310 | Rand-dense | 0.7820
±0.0200 | This paper |
| 327 | Cmp-Graclus | 0.7780
±0.0180 | This paper |
| 332 | k-MIS | 0.7760
±0.0300 | This paper |
| 449 | ASAPool | 0.7310
±0.0250 | This paper |
| 457 | Top-k | 0.7260
±0.0310 | prev best: Graph Neural Networks in TensorFlow and Keras with Spektral (2020) Improvement within reported uncertainty; not highlighted. |
| 465 | PanPool | 0.7168
±0.0445 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) Difference within reported uncertainty; not counted as underperformance. |
| 495 | Rand-sparse | 0.6910
±0.0330 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9792 | |
| 2 | MSH-GNN | 0.9640
±0.0070 | |
| 3 | CoCN | 0.8722
±0.0013 | |
| 23 | GCN | 0.8260
±0.0220 | |
| 33 | DiffPool | 0.8213
±0.0043 best reported | best at pub: CogDL: A Comprehensive Library for Graph Deep Learning (WWW 2021) best today: Scalable Graph Compressed Convolutions (2024) |
| 39 | SAGPool | 0.8176
±0.0157 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 84 | DMoN | 0.8090
±0.0070 | prev best: Total Variation Graph Neural Networks (ICML 2022) |
| 85 | ecpool | 0.8090
±0.0140 | This paper |
| 107 | Graclus | 0.8040
±0.0150 | prev best: Fast Graph Representation Learning with PyTorch Geometric (ICLR 2019) Improvement within reported uncertainty; not highlighted. |
| 111 | PanPool | 0.8022
±0.0202 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) Difference within reported uncertainty; not counted as underperformance. |
| 149 | MinCut | 0.7920
±0.0150 | This paper |
| 151 | MLP | 0.7920
±0.0240 | |
| 171 | ASAPool | 0.7840
±0.0160 | This paper |
| 180 | Top-k | 0.7790
±0.0210 | prev best: Total Variation Graph Neural Networks (ICML 2022) Improvement within reported uncertainty; not highlighted. |
| 270 | k-MIS | 0.7370
±0.0140 | This paper |
| 307 | Rand-dense | 0.6930
±0.0160 | This paper |
| 308 | Rand-sparse | 0.6880
±0.0160 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GMN | 0.7866 | |
| 2 | MemGNN | 0.7550 | |
| 3 | TFGW SP | 0.7510
±0.0500 | |
| 17 | GatedGCN | 0.7000
±0.0494 | |
| 26 | GraphSAGE | 0.6817
±0.0545 | |
| 64 | k-MIS | 0.6350
±0.0710 | This paper |
| 70 | DiffPool | 0.6253 best reported | best at pub: MathNet: Haar-Like Wavelet Multiresolution-Analysis for Graph Representation and Learning (2020) Difference within reported uncertainty; not counted as underperformance. |
| 72 | MinCut | 0.6230
±0.0380 | This paper |
| 82 | DMoN | 0.6100
±0.0500 | This paper |
| 83 | Graclus | 0.6100
±0.0660 | |
| 101 | ecpool | 0.5960
±0.0370 | This paper |
| 124 | MLP | 0.5583
±0.0352 | |
| 171 | Top-k | 0.4550
±0.1050 | prev best: Hierarchical Representation Learning in Graph Neural Networks with Node Decimation Pooling (2019) Improvement within reported uncertainty; not highlighted. |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GraphSNN | 0.7790
±0.0360 | |
| 2 | SIN | 0.7560
±0.0320 | |
| 3 | CIN | 0.7560
±0.0370 | |
| 5 | GIN | 0.7510
±0.0510 | |
| 43 | DiffPool | 0.5131
±0.0072 best reported | best at pub: Structural Entropy Guided Graph Hierarchical Pooling (ICML 2022) best today: Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention (NeurIPS 2024) |
| 73 | SAGPool | 0.4933
±0.0490 best reported | best at pub: Pooling Architecture Search for Graph Classification (2021) Difference within reported uncertainty; not counted as underperformance. |
| 76 | ecpool | 0.4920
±0.0290 | This paper |
| 88 | PanPool | 0.4830
±0.0370 | This paper |
| 94 | DMoN | 0.4700
±0.0550 | This paper |
| 96 | k-MIS | 0.4680
±0.0460 | This paper |
| 97 | MinCut | 0.4660
±0.0400 | This paper |
| 101 | Graclus | 0.4620
±0.0440 | This paper |
| 102 | ASAPool | 0.4610
±0.0420 | This paper |
| 103 | Rand-sparse | 0.4580
±0.0370 | This paper |
| 105 | Rand-dense | 0.4520
±0.0560 | This paper |
| 108 | Top-k | 0.4480
±0.0290 | This paper |
