Contrastive Learning for Non-Local Graphs with Multi-Resolution Structural Views

Asif Khan, A. Storkey

Main figure for Contrastive Learning for Non-Local Graphs with Multi-Resolution Structural Views

Abstract

Learning node-level representations of heterophilic graphs is crucial for various applications, including fraudster detection and protein function prediction. In such graphs, nodes share structural similarity identified by the equivalence of their connectivity which is implicitly encoded in the form of higher-order hierarchical information in the graphs. The contrastive methods are popular choices for learning the representation of nodes in a graph. However, existing contrastive methods struggle to capture higher-order graph structures. To address this limitation, we propose a novel multiview contrastive learning approach that integrates diffusion filters on graphs. By incorporating multiple graph views as augmentations, our method captures the structural equivalence in heterophilic graphs, enabling the discovery of hidden relationships and similarities not apparent in traditional node representations. Our approach outperforms baselines on synthetic and real structural datasets, surpassing the best baseline by $16.06\

Benchmark Results

Tested on 4 datasets 7 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Heterophilic Graphs

#ModelAccuracy ↑Paper
1
P^2GNN
0.9672 ±0.0147
2
DoG
0.9660 ±0.0210
3
GPF-plus
0.9583 ±0.0419
34
MLP
0.9230 ±0.0070
53
GraphSAGE
0.9000 ±0.0365
304
DGI
actual #304 vs paper #576 272 places worse
0.7581 ±0.0503
best reported
actual #304 vs paper #576 272 places worse
paper 0.5297 ±0.0564
351
MVGRL
actual #351 vs paper #555 204 places worse
0.7301 ±0.0552
best reported
actual #351 vs paper #555 204 places worse
paper 0.5675 ±0.0597
358
Ours (dedicated)
0.7289 ±0.0660
This paper
396
NWR-GAE
0.6962 ±0.0666
This paper
523
VGAE
actual #523 vs paper #608 85 places worse
0.5920
best reported
actual #523 vs paper #608 85 places worse
paper 0.3918 ±0.0896
526
GAE
actual #526 vs paper #530 4 places worse
0.5909 ±0.1626
best reported
actual #526 vs paper #530 4 places worse
paper 0.5878 ±0.0341

Difference within reported uncertainty; not counted as underperformance.

paper 0.5878 ±0.0341 Δ at pub 0.0031
585
struc2vec
actual #585 vs paper #594 9 places worse
0.4931 ±0.0322
best reported
actual #585 vs paper #594 9 places worse
paper 0.4729 ±0.0721

Difference within reported uncertainty; not counted as underperformance.

paper 0.4729 ±0.0721 Δ at pub 0.0202
589
GraphCL
actual #589 vs paper #597 8 places worse
0.4867 ±0.0437
best reported
actual #589 vs paper #597 8 places worse
paper 0.4648 ±0.0585

Difference within reported uncertainty; not counted as underperformance.

paper 0.4648 ±0.0585 Δ at pub 0.0219
590
Node2Vec
0.4864 ±0.0292
This paper
595
DeepWalk
actual #595 vs paper #606 11 places worse
0.4700 ±0.2928
best reported
actual #595 vs paper #606 11 places worse
paper 0.4189 ±0.0781

Difference within reported uncertainty; not counted as underperformance.

paper 0.4189 ±0.0781 Δ at pub 0.0511
603
ARGVA
0.4324 ±0.0538
This paper
605
RolX
0.4256 ±0.0713
This paper
613
GraphWave
0.3745 ±0.0709
This paper
#ModelAccuracy ↑Paper
1
P^2GNN
0.9541 ±0.0272
2
ACM-GCN
0.9480 ±0.0380
3
RF-GCN
0.9429
24
MLP
0.9136 ±0.0070
138
SAGE
0.8269
236
mechanism (shared)
0.7470 ±0.0756
This paper
340
MVGRL
actual #340 vs paper #451 111 places worse
0.6430
best reported
actual #340 vs paper #451 111 places worse
paper 0.5351 ±0.0326
350
DGI
actual #350 vs paper #520 170 places worse
0.6335
best reported
actual #350 vs paper #520 170 places worse
paper 0.4648 ±0.0797
354
GAE
actual #354 vs paper #524 170 places worse
0.6309 ±0.0987
best reported
actual #354 vs paper #524 170 places worse
paper 0.4540 ±0.0999
paper 0.4540 ±0.0999 Δ at pub 0.1769
386
VGAE
actual #386 vs paper #494 108 places worse
0.5919
best reported
actual #386 vs paper #494 108 places worse
paper 0.4932 ±0.0919

Difference within reported uncertainty; not counted as underperformance.

paper 0.4932 ±0.0919 Δ at pub 0.0987
394
NWR-GAE
0.5864 ±0.0561
This paper
500
DeepWalk
actual #500 vs paper #549 49 places worse
0.4867 ±0.2784
best reported
actual #500 vs paper #549 49 places worse
paper 0.4121 ±0.0340

Difference within reported uncertainty; not counted as underperformance.

paper 0.4121 ±0.0340 Δ at pub 0.0746
513
GraphCL
actual #513 vs paper #527 14 places worse
0.4722 ±0.0450
best reported
actual #513 vs paper #527 14 places worse
paper 0.4486 ±0.0373

Difference within reported uncertainty; not counted as underperformance.

paper 0.4486 ±0.0373 Δ at pub 0.0236
524
GraphWave
0.4596 ±0.0220
This paper
542
Node2Vec
actual #542 vs paper #553 11 places worse
0.4294
best reported
actual #542 vs paper #553 11 places worse
paper 0.4054 ±0.0162

Difference within reported uncertainty; not counted as underperformance.

paper 0.4054 ±0.0162 Δ at pub 0.0240
551
ARGVA
0.4108 ±0.0485
This paper
571
struc2vec
actual #571 vs paper #576 5 places worse
0.3022 ±0.0587
best reported
actual #571 vs paper #576 5 places worse
paper 0.2372 ±0.1369

Difference within reported uncertainty; not counted as underperformance.

paper 0.2372 ±0.1369 Δ at pub 0.0650
575
RolX
0.2567 ±0.1178
This paper
#ModelAccuracy ↑Paper
1
DiGGR
0.4535 ±0.0353
2
TRIGON
0.4402 ±0.0028
3
DoG
0.4320 ±0.0120
42
MLP
0.4026 ±0.0108
68
GraphSAGE
0.3899 ±0.0085
184
mechanism (dedicated)
0.3601 ±0.0068
This paper
247
MVGRL
actual #247 vs paper #325 78 places worse
0.3464 ±0.0020
best reported
actual #247 vs paper #325 78 places worse
paper 0.3107 ±0.0029
paper 0.3107 ±0.0029 Δ at pub 0.0102 Δ today 0.0357
309
VGAE
actual #309 vs paper #417 108 places worse
0.3157 ±0.0075
best reported
actual #309 vs paper #417 108 places worse
paper 0.2641 ±0.0107
paper 0.2641 ±0.0107 Δ at pub 0.0202 Δ today 0.0516
310
GraphCL
actual #310 vs paper #382 72 places worse
0.3151 ±0.0154
best reported
actual #310 vs paper #382 72 places worse
paper 0.2864 ±0.0128
paper 0.2864 ±0.0128 Δ today 0.0287
335
DGI
actual #335 vs paper #487 152 places worse
0.3067 ±0.0063
best reported
actual #335 vs paper #487 152 places worse
paper 0.2045 ±0.0132
348
NWR-GAE
0.3017 ±0.0017
This paper
374
ARGVA
0.2897 ±0.0117
This paper
383
GAE
actual #383 vs paper #383
0.2863 ±0.0105
actual #383 vs paper #383
paper 0.2863 ±0.0105

Improvement within reported uncertainty; not highlighted.

paper 0.2863 ±0.0105 Δ at pub 0.0060
390
Node2Vec
actual #390 vs paper #441 51 places worse
0.2828
best reported
actual #390 vs paper #441 51 places worse
paper 0.2414 ±0.0102
409
GraphWave
0.2729 ±0.0309
This paper
424
DeepWalk
actual #424 vs paper #443 19 places worse
0.2550 ±0.0028
best reported
actual #424 vs paper #443 19 places worse
paper 0.2384 ±0.0214

Difference within reported uncertainty; not counted as underperformance.

paper 0.2384 ±0.0214 Δ at pub 0.0166
428
RolX
0.2542 ±0.0055
This paper
434
struc2vec
0.2513 ±0.0079
This paper

Hypergraph Benchmarks

#ModelAccuracy ↑Paper
1
HealHGNN
0.7718 ±0.0235
2
Deep-HGNN
0.7526 ±0.0176
3
mechanism (shared)
0.7470 ±0.0756
This paper
12
MLP
0.6793 ±0.0233
20
NWR-GAE
0.5864 ±0.0561
This paper
21
MVGRL
0.5351 ±0.0326
This paper
22
VGAE
0.4932 ±0.0919
This paper
23
DGI
0.4648 ±0.0797
This paper
24
GraphWave
0.4596 ±0.0220
This paper
25
GAE
0.4540 ±0.0999
This paper
26
GraphCL
0.4486 ±0.0373
This paper
27
DeepWalk
0.4121 ±0.0340
This paper
28
ARGVA
0.4108 ±0.0485
This paper
29
Node2Vec
0.4054 ±0.0162
This paper
30
RolX
0.2567 ±0.1178
This paper
31
struc2vec
0.2372 ±0.1369
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best today — OK at publication; today's leaderboard best is elsewhere