Search to aggregate neighborhood for graph neural network
Abstract
Recent years have witnessed the popularity and success of graph neural networks (GNN) in various scenarios. To obtain data-specific GNN architectures, researchers turn to neural architecture search (NAS), which have made impressive success in discovering effective architectures in convolutional neural networks. However, it is non-trivial to apply NAS approaches to GNN due to challenges in search space design and expensive searching cost of existing NAS methods. In this work, to obtain the data-specific GNN architectures and address the computational challenges facing by NAS approaches, we propose a framework, which tries to Search to Aggregate NEighborhood (SANE), to automatically design data-specific GNN architectures. By designing a novel and expressive search space, we propose a differentiable search algorithm, which is more efficient than previous reinforcement learning based methods. Experimental results on four tasks and seven real-world datasets demonstrate the superiority of SANE compared to existing GNN models and NAS approaches in terms of effectiveness and efficiency.. Correspondence is to Q.Yao.
Benchmark Results
Tested on 1 dataset 4 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Classic
| # | Model | F1 ↑ | Paper |
|---|---|---|---|
| 1 | GAMLP(JK) | 0.9982
±0.0001 | |
| 2 | GraphCON-GCN | 0.9960 | |
| 3 | GCNII | 0.9953
±0.0001 | |
| 20 | SANE | 0.9856
±0.0120 | This paper |
| 21 | GCN | 0.9850 best reported | best at pub: Graph Neural Networks With Convolutional ARMA Filters (2019) best today: Graph-Coupled Oscillator Networks (ICML 2022) |
| 22 | GeniePath | 0.9850 best reported | best at pub: Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks (KDD 2019) best today: Graph-Coupled Oscillator Networks (ICML 2022) |
| 55 | GAT-JK | 0.9749 | This paper |
| 57 | GAT | 0.9732
±0.0032 best reported | |
| 67 | GeniePath-JK | 0.9694 | This paper |
| 77 | Bayesian | 0.9583
±0.0082 | This paper |
| 81 | Random | 0.9517
±0.0032 | prev best: Inductive Representation Learning on Large Graphs (NeurIPS 2017) |
| 83 | GIN-JK | 0.9467 | This paper |
| 92 | GIN | 0.8724
±0.0002 | prev best: Graph Neural Networks With Convolutional ARMA Filters (2019) |
| 98 | GCN-JK | 0.8078 | This paper |
| 100 | GraphSAGE-JK | 0.8019 | This paper |
| 101 | GraphSAGE | 0.7860 best reported | best at pub: Graph Neural Networks With Convolutional ARMA Filters (2019) best today: Optimization-Induced Graph Implicit Nonlinear Diffusion (ICML 2022) |
| 103 | LGCN | 0.7720
±0.0020 | This paper |
| 132 | SAGE | 0.6928 | |
| 173 | MLP | 0.4620 |
