On Exploring Node-feature and Graph-structure Diversities for Node Drop Graph Pooling
Abstract
A pooling operation is essential for effective graph-level representation learning, where the node drop pooling has become one mainstream graph pooling technology. However, current node drop pooling methods usually keep the top-k nodes according to their significance scores, which ignore the graph diversity in terms of the node features and the graph structures, thus resulting in suboptimal graph-level representations. To address the aforementioned issue, we propose a novel plug-and-play score scheme and refer to it as MID, which consists of a Multidimensional score space with two operations, i.e., flIpscore and Dropscore. Specifically, the multidimensional score space depicts the significance of nodes through multiple criteria; the flipscore encourages the maintenance of dissimilar node features; and the dropscore forces the model to notice diverse graph structures instead of being stuck in significant local structures. To evaluate the effectiveness of our proposed MID, we perform extensive experiments by applying it to a wide variety of recent node drop pooling methods, including TopKPool, SAGPool, GSAPool, and ASAP. Specifically, the proposed MID can efficiently and consistently achieve about 2.8\
Benchmark Results
Tested on 3 datasets 14 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
TU Dortmund
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | MSH-GNN | 0.9910
±0.0030 | |
| 2 | ID-GNN+BEC | 0.9830
±0.0360 | |
| 3 | SupCosine | 0.9830
±0.0250 | |
| 39 | GIN | 0.9330
±0.0290 | |
| 101 | TopKPool | 0.9130
±0.0070 best reported | best at pub: Mind the Label Shift of Augmentation-based Graph OOD Generalization (2023) |
| 141 | MINCUTPOOL | 0.9060
±0.0460 best reported | best at pub: Total Variation Graph Neural Networks (ICML 2022) best today: Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024) |
| 190 | SortPool | 0.9000
±0.0224 best reported | best at pub: CogDL: A Comprehensive Library for Graph Deep Learning (WWW 2021) |
| 202 | GMT | 0.8990
±0.0420 best reported | best at pub: Structural Entropy Guided Graph Hierarchical Pooling (ICML 2022) best today: Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024) |
| 239 | DiffPool | 0.8940
±0.0460 best reported | best at pub: Total Variation Graph Neural Networks (ICML 2022) best today: Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024) |
| 367 | ASAP | 0.8740
±0.0570 best reported | best at pub: Multi-grained Semantics-aware Graph Neural Networks (2020) best today: Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024) |
| 403 | MLP | 0.8660
±0.0495 | |
| 560 | ASAP + MID | 0.8233
±0.0340 | This paper |
| 589 | EdgePool | 0.8101
±0.0082 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 593 | Set2Set | 0.8084
±0.0067 best reported | best at pub: Graph Convolutional Networks with EigenPooling (KDD 2019) |
| 606 | TopKPool + MID | 0.8038
±0.0226 | This paper |
| 623 | SAGPool | 0.7986
±0.0236 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 671 | HaarPool | 0.7760
±0.0894 best reported | best at pub: Hierarchical Graph Capsule Network (AAAI 2020) Difference within reported uncertainty; not counted as underperformance. |
| 744 | SAGPool + MID | 0.7272
±0.0196 | This paper |
| 754 | GSAPool + MID | 0.7172
±0.0104 | This paper |
| 761 | MemPool | 0.6917
±0.0204 | This paper |
| 763 | GSAPool | 0.6883
±0.0154 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9954 | |
| 2 | GIN | 0.9934 | |
| 3 | GCN | 0.9727 | |
| 5 | DiffPool | 0.9210
±0.0260 best reported | best at pub: The expressive power of pooling in Graph Neural Networks (NeurIPS 2023) best today: Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders (IJCAI 2024) |
| 131 | GraphSAGE | 0.8200
±0.0100 | |
| 226 | GMT | 0.7990
±0.0040 best reported | best at pub: Structural Entropy Guided Graph Hierarchical Pooling (ICML 2022) best today: Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024) |
| 259 | SAGPool | 0.7910
±0.0300 best reported | best at pub: The expressive power of pooling in Graph Neural Networks (NeurIPS 2023) |
| 267 | EdgePool | 0.7893
±0.0222 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) best today: Graph Parsing Networks (ICLR 2024) Difference within reported uncertainty; not counted as underperformance. |
| 295 | TopKPool + MID | 0.7850
±0.0050 | This paper |
| 309 | ASAP | 0.7821
±0.0175 best reported | best at pub: Multi-grained Semantics-aware Graph Neural Networks (2020) |
| 355 | TopKPool | 0.7706
±0.0090 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 368 | MINCUTPOOL | 0.7680
±0.0160 best reported | best at pub: Higher-order Clustering and Pooling for Graph Neural Networks (2022) |
| 403 | GSAPool + MID | 0.7572
±0.0092 | This paper |
| 408 | ASAP + MID | 0.7530
±0.0043 | This paper |
| 415 | SortPool | 0.7496
±0.0134 best reported | best at pub: CogDL: A Comprehensive Library for Graph Deep Learning (WWW 2021) |
| 438 | SAGPool + MID | 0.7408
±0.0040 | This paper |
| 463 | Set2Set | 0.7170
±0.0073 | prev best: Graph Convolutional Networks with EigenPooling (KDD 2019) Improvement within reported uncertainty; not highlighted. |
| 468 | GSAPool | 0.7133
±0.0092 | This paper |
| 508 | MemPool | 0.6765
±0.0067 | This paper |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GC-GNN (GCN) | 0.9210
±0.0810 | |
| 2 | SUGAR | 0.8403
±0.0133 | |
| 3 | Ortho-g-U-Nets | 0.8387 | |
| 15 | SAGE | 0.8088
±0.0233 | |
| 18 | DiffPool | 0.8064 best reported | best at pub: Graph U-Nets (2019) best today: Orthogonal Graph Neural Networks (AAAI 2021) |
| 32 | ASAP | 0.7950
±0.0180 best reported | best at pub: Multi-grained Semantics-aware Graph Neural Networks (2020) |
| 41 | EdgePool | 0.7920
±0.0261 best reported | best at pub: Contrastive Self-supervised Learning for Graph Classification (AAAI 2020) Difference within reported uncertainty; not counted as underperformance. |
| 55 | GMT | 0.7872
±0.0059 best reported | best at pub: Structural Entropy Guided Graph Hierarchical Pooling (ICML 2022) best today: Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention (NeurIPS 2024) Difference within reported uncertainty; not counted as underperformance. |
| 56 | SAGPool | 0.7870
±0.0229 best reported | best at pub: Contrastive Self-supervised Learning for Graph Classification (AAAI 2020) Difference within reported uncertainty; not counted as underperformance. |
| 69 | MINCUTPOOL | 0.7822
±0.0054 best reported | best at pub: Structural Entropy Guided Graph Hierarchical Pooling (ICML 2022) best today: Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention (NeurIPS 2024) |
| 88 | ASAP + MID | 0.7757
±0.0079 | This paper |
| 90 | SortPool | 0.7750
±0.0050 | prev best: Structural Entropy Guided Graph Hierarchical Pooling (ICML 2022) |
| 106 | SAGPool + MID | 0.7697
±0.0072 | This paper |
| 108 | GSAPool + MID | 0.7670
±0.0056 | This paper |
| 110 | TopKPool | 0.7640
±0.0232 best reported | best at pub: Embedding Graphs on Grassmann Manifold (2022) |
| 121 | GSAPool | 0.7591
±0.0077 | This paper |
| 142 | TopKPool + MID | 0.7465
±0.0168 | This paper |
| 146 | Set2Set | 0.7450 best reported | best at pub: Graph Convolutional Networks with EigenPooling (KDD 2019) |
| 156 | MemPool | 0.7296
±0.0084 | This paper |
