Neighbourhood Transformer: Switchable Attention for Monophily-Aware Graph Learning

Yi Luo, Xu Sun, Guangchun Luo, Aiguo Chen

Main figure for Neighbourhood Transformer: Switchable Attention for Monophily-Aware Graph Learning

Abstract

Graph neural networks (GNNs) have been widely adopted in engineering applications such as social network analysis, chemical research and computer vision. However, their efficacy is severely compromised by the inherent homophily assumption, which fails to hold for heterophilic graphs where dissimilar nodes are frequently connected. To address this fundamental limitation in graph learning, we first draw inspiration from the recently discovered monophily property of real-world graphs, and propose Neighbourhood Transformers (NT), a novel paradigm that applies self-attention within every local neighbourhood instead of aggregating messages to the central node as in conventional message-passing GNNs. This design makes NT inherently monophily-aware and theoretically guarantees its expressiveness is no weaker than traditional message-passing frameworks. For practical engineering deployment, we further develop a neighbourhood partitioning strategy equipped with switchable attentions, which reduces the space consumption of NT by over 95\ Extensive experiments on 10 real-world datasets (5 heterophilic and 5 homophilic graphs) show that NT outperforms all current state-of-the-art methods on node classification tasks, demonstrating its superior performance and cross-domain adaptability. The full implementation code of this work is publicly available at to facilitate reproducibility and industrial adoption.

Proposed method: NT*

Benchmark Results

Tested on 3 datasets 2 baseline values reported too low Non-standard metric on 3 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

Heterophily Benchmark

Minesweeper

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is ROC-AUC. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
GraphTARIF
0.9903 ±0.0019
2
BuNN
0.9899 ±0.0016
3
M3Dphormer
0.9827 ±0.0020
4
NT*
0.9792 ±0.0028
This paper
8
Polynormer
actual #8 vs paper #8
0.9749 ±0.0048
best reported
actual #8 vs paper #8
paper 0.9746 ±0.0036

Difference within reported uncertainty; not counted as underperformance.

paper 0.9746 ±0.0036 Δ today 0.0003
9
GCN
actual #9 vs paper #24 15 places worse
0.9744 ±0.0017
best reported
actual #9 vs paper #24 15 places worse
paper 0.8975 ±0.0052
11
SAGE
0.9709 ±0.0100
15
Dir-NT
0.9392 ±0.0059
This paper
17
GraphGPS
actual #17 vs paper #23 6 places worse
0.9229 ±0.0061
best reported
actual #17 vs paper #23 6 places worse
paper 0.9063 ±0.0067
paper 0.9063 ±0.0067 Δ at pub 0.0166
21
DIFFormer
0.9089 ±0.0058
This paper
23
Exphormer
actual #23 vs paper #23
0.9074 ±0.0053
actual #23 vs paper #23
paper 0.9074 ±0.0053

Improvement within reported uncertainty; not highlighted.

paper 0.9074 ±0.0053 Δ at pub 0.0032
27
NodeFormer
actual #27 vs paper #29 2 places worse
0.8771 ±0.0069
best reported
actual #27 vs paper #29 2 places worse
paper 0.8671 ±0.0088

Difference within reported uncertainty; not counted as underperformance.

paper 0.8671 ±0.0088 Δ today 0.0100
29
Dir-GNN
0.8705 ±0.0069
This paper
34
NAGphormer
0.8419 ±0.0066
This paper
37
GOAT
0.8109 ±0.0102
This paper
43
MLP
0.8000

Tolokers

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is ROC-AUC. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
NT*
0.8591 ±0.0083
This paper
2
Polynormer
0.8591 ±0.0074
This paper
3
Dir-NT
0.8502 ±0.0077
This paper
4
Exphormer
0.8377 ±0.0078
This paper
6
GraphGPS
0.8371 ±0.0048
This paper
7
GCN
0.8364 ±0.0067
This paper
8
DIFFormer
0.8357 ±0.0068
This paper
10
GOAT
0.8311 ±0.0104
This paper
12
Dir-GNN
0.8119 ±0.0105
This paper
25
NAGphormer
0.7832 ±0.0095
This paper
28
MLP
0.7816 ±0.0002
30
NodeFormer
0.7810 ±0.0103
This paper

Questions

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is ROC-AUC. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
MLP
0.9733 ±0.0006
2
AGS-GNN
0.9727
3
MeanGNN
0.9716 ±0.0006
4
GCN
actual #4 vs paper #34 30 places worse
0.9715 ±0.0004
best reported
actual #4 vs paper #34 30 places worse
paper 0.7609 ±0.0127
paper 0.7609 ±0.0127 Δ at pub 0.2106 Δ today 0.2106
23
GraphGPS
actual #23 vs paper #44 21 places worse
0.9349 ±0.0040
best reported
actual #23 vs paper #44 21 places worse
paper 0.7173 ±0.0147
26
NT*
0.7936 ±0.0069
This paper
27
Polynormer
0.7892 ±0.0089
This paper
30
Dir-NT
0.7799 ±0.0100
This paper
33
SAGE
0.7644
34
Dir-GNN
0.7613 ±0.0124
This paper
35
GOAT
0.7576 ±0.0166
This paper
38
NodeFormer
0.7427 ±0.0146
This paper
39
Exphormer
0.7394 ±0.0106
This paper
43
DIFFormer
0.7215 ±0.0131
This paper
49
NAGphormer
0.6817 ±0.0153
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere