Reinforcement learning on graphs: A survey
Abstract
Graph mining tasks arise from many different application domains, ranging from social networks, transportation to E-commerce, etc., which have been receiving great attention from the theoretical and algorithmic design communities in recent years, and there has been some pioneering work employing the research-rich Reinforcement Learning (RL) techniques to address graph data mining tasks. However, these graph mining methods and RL models are dispersed in different research areas, which makes it hard to compare them. In this survey, we provide a comprehensive overview of RL and graph mining methods and generalize these methods to Graph Reinforcement Learning (GRL) as a unified formulation. We further discuss the applications of GRL methods across various domains and summarize the method descriptions, open-source codes, and benchmark datasets of GRL methods. Furthermore, we propose important directions and challenges to be solved in the future. As far as we know, this is the latest work on a comprehensive survey of GRL, this work provides a global view and a learning resource for scholars. In addition, we create an online open-source for both interested scholars who want to enter this rapidly developing domain and experts who would like to compare GRL methods.
Benchmark Results
Tested on 3 datasets 9 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
Classic
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | GraphSAGE+CNA | 0.9418
±0.0033 | |
| 2 | GCN+CNA | 0.9366
±0.0048 | |
| 3 | TransformerConv | 0.9350
±0.0058 | |
| 6 | Policy-GNN | 0.9190
±0.0140 | This paper |
| 21 | GCN | 0.9004 best reported | best at pub: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (2017) best today: Understanding Heterophily for Graph Neural Networks (ICML 2024) |
| 39 | SAGE | 0.8911
±0.0012 | |
| 44 | GAT | 0.8897
±0.0014 best reported | best at pub: Learned Low Precision Graph Neural Networks (2020) |
| 177 | DGI | 0.8630
±0.0020 best reported | best at pub: Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization (2022) best today: Oversmoothing: A Nightmare for Graph Contrastive Learning? (2023) |
| 346 | AGNN | 0.8360
±0.0030 | This paper |
| 391 | LGCN | 0.8330
±0.0050 | This paper |
| 724 | MLP | 0.7768 |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | CNA | 0.9575
±0.0058 | |
| 2 | IS-GIB | 0.9390
±0.0187 | |
| 3 | EERM | 0.9112
±0.0145 | |
| 9 | Policy-GNN | 0.8970
±0.0210 | This paper |
| 35 | GAT | 0.8080
±0.0026 best reported | best at pub: Gophormer: Ego-Graph Transformer for Node Classification (2021) best today: Feature Expansion for Graph Neural Networks (ICML 2023) |
| 44 | GCN | 0.8033
±0.0088 best reported | best at pub: Gophormer: Ego-Graph Transformer for Node Classification (2021) |
| 61 | GraphSAGE | 0.7923
±0.0053 | |
| 66 | DGI | 0.7890
±0.0020 best reported | best at pub: Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization (2022) best today: Oversmoothing: A Nightmare for Graph Contrastive Learning? (2023) |
| 242 | AGNN | 0.7380
±0.0070 | prev best: Attention-based Graph Neural Network for Semi-supervised Learning (2018) Improvement within reported uncertainty; not highlighted. |
| 318 | GraphNAS | 0.7310
±0.0090 | This paper |
| 327 | LGCN | 0.7300
±0.0060 | This paper |
| 350 | MLP | 0.7289
±0.0010 |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | LGTL | 0.9518
±0.0011 | |
| 2 | MixHop (LO) | 0.9513
±0.0038 | |
| 3 | GraphSAGE (LO) | 0.9511
±0.0036 | |
| 18 | GCN | 0.9296
±0.0015 best reported | best at pub: Tackling Over-Smoothing for General Graph Convolutional Networks (2020) |
| 20 | GAT | 0.9233
±0.0018 best reported | best at pub: Tackling Over-Smoothing for General Graph Convolutional Networks (2020) |
| 21 | Policy-GNN | 0.9210
±0.0220 | This paper |
| 116 | SAGE | 0.8881
±0.0015 | |
| 204 | DGI | 0.8708
±0.0017 best reported | |
| 210 | MLP | 0.8690
±0.0074 | |
| 465 | AGNN | 0.7990
±0.0700 best reported | best at pub: Attention-based Graph Neural Network for Semi-supervised Learning (2018) Difference within reported uncertainty; not counted as underperformance. |
| 493 | GraphNAS | 0.7960
±0.0040 | This paper |
| 505 | LGCN | 0.7950
±0.0020 | This paper |
