Ensemble Learning for Graph Neural Networks

Zhen Hao Wong, Ling Yue, Quanming Yao

Main figure for Ensemble Learning for Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) have shown success in various fields for learning from graph-structured data. This paper investigates the application of ensemble learning techniques to improve the performance and robustness of Graph Neural Networks (GNNs). By training multiple GNN models with diverse initializations or architectures, we create an ensemble model named ELGNN that captures various aspects of the data and uses the Tree-Structured Parzen Estimator algorithm to determine the ensemble weights. Combining the predictions of these models enhances overall accuracy, reduces bias and variance, and mitigates the impact of noisy data. Our findings demonstrate the efficacy of ensemble learning in enhancing GNN capabilities for analyzing complex graph-structured data. The code is public at.

Proposed method: ELGNN

Benchmark Results

Tested on 1 dataset 1 baseline value reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

OGB

#ModelHits@20 ↑Paper
1
ELGNN
0.9777 ±0.0037
This paper
2
GIDN
0.9542
This paper
3
AGDN
0.9538 ±0.0094
This paper
4
PSG
0.9284 ±0.0047
This paper
5
PLNLP
0.9088 ±0.0313
This paper
9
GraphSAGE + Edge Attr
0.8781 ±0.0474
This paper
20
SAGE
0.6719 ±0.0118
24
GCN
actual #24 vs paper #53 29 places worse
0.6476 ±0.0145
best reported
actual #24 vs paper #53 29 places worse
paper 0.3707 ±0.0507
paper 0.3707 ±0.0507 Δ at pub 0.1283 Δ today 0.2769
29
GCN+JKNet
0.6056 ±0.0869
This paper
38
GraphSAGE
0.5390 ±0.0474
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)