Rethinking pooling in graph neural networks
Abstract
Graph pooling is a central component of a myriad of graph neural network (GNN) architectures. As an inheritance from traditional CNNs, most approaches formulate graph pooling as a cluster assignment problem, extending the idea of local patches in regular grids to graphs. Despite the wide adherence to this design choice, no work has rigorously evaluated its influence on the success of GNNs. In this paper, we build upon representative GNNs and introduce variants that challenge the need for locality-preserving representations, either using randomization or clustering on the complement graph. Strikingly, our experiments demonstrate that using these variants does not result in any decrease in performance. To understand this phenomenon, we study the interplay between convolutional layers and the subsequent pooling ones. We show that the convolutions play a leading role in the learned representations. In contrast to the common belief, local pooling is not responsible for the success of GNNs on relevant and widely-used benchmarks.
Benchmark Results
Tested on 2 datasets 1 baseline value reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
TU Dortmund
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9954 | |
| 2 | GIN | 0.9934 best reported | best at pub: Provably Powerful Graph Networks (NeurIPS 2019) best today: 1-WL Expressiveness Is (Almost) All You Need (2022) |
| 3 | GCN | 0.9727 | |
| 131 | GraphSAGE | 0.8200
±0.0100 best reported | best at pub: Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels (NeurIPS 2019) best today: Weisfeiler and Leman Go Infinite: Spectral and Combinatorial Pre-Colorings (TMLR 2022) |
| 170 | Graclus | 0.8120
±0.0340 best reported | best today: The expressive power of pooling in Graph Neural Networks (NeurIPS 2023) Difference within reported uncertainty; not counted as underperformance. |
| 216 | Complement | 0.8020
±0.0140 | This paper |
| 342 | DiffPool v2 | 0.7740
±0.0200 | This paper |
| 343 | N-DiffPool v2 | 0.7740
±0.0140 | This paper |
| 368 | MINCUTPOOL | 0.7680
±0.0160 best reported | best today: Higher-order Clustering and Pooling for Graph Neural Networks (2022) Difference within reported uncertainty; not counted as underperformance. |
| 372 | N-MinCutPool | 0.7660
±0.0160 | This paper |
GNNBenchmark
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | Schrödinger | 0.9913
±0.0004 | |
| 2 | CNN | 0.9907
±0.0007 | |
| 3 | MPNN | 0.9895
±0.0006 | |
| 11 | GatedGCN | 0.9855
±0.0002 | |
| 32 | GIN | 0.9833
±0.0011 best reported | |
| 95 | GraphSAGE | 0.9731
±0.0010 best reported | best today: Differential Encoding for Improved Representation Learning Over Graphs (2024) Difference within reported uncertainty; not counted as underperformance. |
| 97 | SAGE | 0.9731
±0.0010 | |
| 113 | Graclus | 0.9621
±0.0018 best reported | |
| 126 | MLP | 0.9534
±0.0014 | |
| 132 | Complement | 0.9450
±0.0033 | This paper |
| 140 | DiffPool v2 | 0.9070
±0.0030 | This paper |
| 143 | MINCUTPOOL | 0.8930
±0.0100 | This paper |
| 144 | N-DiffPool v2 | 0.8900
±0.0050 | This paper |
| 150 | N-MinCutPool | 0.8200
±0.0160 | This paper |
