Heterogeneous Graph Contrastive Multi-view Learning

Zehong Wang, Qi Li, Donghua Yu [2], Xiaolong Han, Xiao-Zhi Gao, Shigen Shen [2]

Main figure for Heterogeneous Graph Contrastive Multi-view Learning

Abstract

Inspired by the success of Contrastive Learning (CL) in computer vision and natural language processing, Graph Contrastive Learning (GCL) has been developed to learn discriminative node representations on graph datasets. However, the development of GCL on Heterogeneous Information Networks (HINs) is still in the infant stage. For example, it is unclear how to augment the HINs without substantially altering the underlying semantics, and how to design the contrastive objective to fully capture the rich semantics. Moreover, early investigations demonstrate that CL suffers from sampling bias, whereas conventional debiasing techniques are empirically shown to be inadequate for GCL. How to mitigate the sampling bias for heterogeneous GCL is another important problem. To address the aforementioned challenges, we propose a novel Heterogeneous Graph Contrastive Multi-view Learning (HGCML) model. In particular, we use metapaths as the augmentation to generate multiple subgraphs as multi-views, and propose a contrastive objective to maximize the mutual information between any pairs of metapath-induced views. To alleviate the sampling bias, we further propose a positive sampling strategy to explicitly select positives for each node via jointly considering semantic and structural information preserved on each metapath view. Extensive experiments demonstrate HGCML consistently outperforms state-of-the-art baselines on five real-world benchmark datasets. To enhance the reproducibility of our work, we make all the code publicly available at https://github.com/Zehong-Wang/HGCML.: Graph contrastive learning, heterogeneous information network, self-supervised learning, graph neural network, multi-view learning.

Benchmark Results

Tested on 4 datasets 10 baseline values reported too low Non-standard metric on 1 dataset

Tables show presented results in global context (best 3 papers + GCN + MLP).

Heterogeneous Graph Benchmarks

#ModelMicro-F1 ↑Paper
1
LMSPS
0.9566 ±0.0020
2
PSHGCN
0.9561 ±0.0012
3
RpHGNN
0.9555 ±0.0029
21
GCN
actual #21 vs paper #66 45 places worse
0.9460 ±0.0031
best reported
actual #21 vs paper #66 45 places worse
paper 0.9204 ±0.0103
23
GAT
actual #23 vs paper #58 35 places worse
0.9448 ±0.0022
best reported
actual #23 vs paper #58 35 places worse
paper 0.9251 ±0.0128
best at pub: Heterogeneous Graph Benchmarks (KDD 2021)

Difference within reported uncertainty; not counted as underperformance.

paper 0.9251 ±0.0128 Δ at pub 0.0088 Δ today 0.0197
37
HAN
actual #37 vs paper #45 8 places worse
0.9327 ±0.0058
actual #37 vs paper #45 8 places worse
paper 0.9327 ±0.0058

Improvement within reported uncertainty; not highlighted.

paper 0.9327 ±0.0058 Δ at pub 0.0036
40
HGCML-P
0.9344 ±0.0008
This paper
44
DMGI
actual #44 vs paper #70 26 places worse
0.9160 ±0.0066
actual #44 vs paper #70 26 places worse
paper 0.9160 ±0.0066
45
HGCML
0.9329 ±0.0012
This paper
46
GraphSAGE
0.9306 ±0.0047
47
Mp2vec
actual #47 vs paper #86 39 places worse
0.9304
best reported
actual #47 vs paper #86 39 places worse
paper 0.8767 ±0.0012
paper 0.8767 ±0.0012 Δ at pub 0.0537
57
STENCIL
0.9256 ±0.0022
This paper
62
HeCo
0.9224 ±0.0048
This paper
72
HIN2Vec
0.9140 ±0.0008
This paper
74
GRACE
actual #74 vs paper #74
0.9128 ±0.0007
actual #74 vs paper #74
paper 0.9128 ±0.0007
paper 0.9128 ±0.0007 Δ at pub 0.2015
75
HERec
0.9075 ±0.0039
This paper
79
DGI
actual #79 vs paper #79
0.9016 ±0.0060
actual #79 vs paper #79
paper 0.9016 ±0.0060
paper 0.9016 ±0.0060 Δ at pub 0.1638
81
DeepWalk
actual #81 vs paper #85 4 places worse
0.8809 ±0.0007
actual #81 vs paper #85 4 places worse
paper 0.8809 ±0.0007
prev best: Implicit Graph Neural Networks (NeurIPS 2020)
paper 0.8809 ±0.0007 Δ at pub 0.3439 Δ today 0.0203
176
MLP
0.6447 ±0.0136
#ModelMicro-F1 ↑Paper
1
LMSPS
0.9469 ±0.0036
2
PSHGCN
0.9427 ±0.0023
3
SlotGAT
0.9406 ±0.0022
19
GAT
actual #19 vs paper #70 51 places worse
0.9273
best reported
actual #19 vs paper #70 51 places worse
paper 0.8884 ±0.0061
paper 0.8884 ±0.0061 Δ at pub 0.0389
29
GCN
actual #29 vs paper #63 34 places worse
0.9229
best reported
actual #29 vs paper #63 34 places worse
paper 0.8987 ±0.0079
paper 0.8987 ±0.0079 Δ at pub 0.0242
43
HAN
actual #43 vs paper #65 22 places worse
0.9147 ±0.0022
best reported
actual #43 vs paper #65 22 places worse
paper 0.8950 ±0.0121
best at pub: Heterogeneous Graph Benchmarks (KDD 2021)
paper 0.8950 ±0.0121 Δ at pub 0.0129 Δ today 0.0197
45
HGCML-P
0.9134 ±0.0017
This paper
48
HGCML
0.9102 ±0.0013
This paper
49
GIN
0.9092
64
DMGI
actual #64 vs paper #104 40 places worse
0.8980 ±0.0010
best reported
actual #64 vs paper #104 40 places worse
paper 0.7676 ±0.0123
best at pub: Implicit Graph Neural Networks (NeurIPS 2020)
paper 0.7676 ±0.0123 Δ at pub 0.1304 Δ today 0.1304
68
HeCo
0.8897 ±0.0112
This paper
74
DGI
actual #74 vs paper #74
0.8844 ±0.0030
actual #74 vs paper #74
paper 0.8844 ±0.0030
prev best: Implicit Graph Neural Networks (NeurIPS 2020)

Improvement within reported uncertainty; not highlighted.

paper 0.8844 ±0.0030 Δ at pub 0.0034
76
STENCIL
0.8823 ±0.0091
This paper
77
Mp2vec
actual #77 vs paper #103 26 places worse
0.8810 ±0.0030
best reported
actual #77 vs paper #103 26 places worse
paper 0.7982 ±0.0023
paper 0.7982 ±0.0023 Δ today 0.0828
84
GRACE
actual #84 vs paper #84
0.8764 ±0.0031
actual #84 vs paper #84
paper 0.8764 ±0.0031
paper 0.8764 ±0.0031 Δ at pub 0.0171
95
HIN2Vec
0.8523 ±0.0009
This paper
100
DeepWalk
actual #100 vs paper #100
0.8178 ±0.0004
actual #100 vs paper #100
paper 0.8178 ±0.0004
paper 0.8178 ±0.0004 Δ at pub 0.0689
107
HERec
actual #107 vs paper #110 3 places worse
0.7445 ±0.0045
best reported
actual #107 vs paper #110 3 places worse
paper 0.6715 ±0.0085
paper 0.6715 ±0.0085 Δ today 0.0730
109
MLP
0.7102 ±0.0221
#ModelMicro-F1 ↑Paper
1
ELLA
0.7930 ±0.0010
2
MHGCN
0.7820 ±0.0138
3
PSHGCN
0.7446 ±0.0032
14
GCN
actual #14 vs paper #60 46 places worse
0.6813 ±0.0083
best reported
actual #14 vs paper #60 46 places worse
paper 0.5842 ±0.0142
best at pub: Heterogeneous Graph Benchmarks (KDD 2021)
paper 0.5842 ±0.0142 Δ at pub 0.0640 Δ today 0.0971
15
GAT
actual #15 vs paper #62 47 places worse
0.6808 ±0.0049
best reported
actual #15 vs paper #62 47 places worse
paper 0.5797 ±0.0164
best at pub: Heterogeneous Graph Benchmarks (KDD 2021)
paper 0.5797 ±0.0164 Δ at pub 0.0689 Δ today 0.1011
18
HAN
actual #18 vs paper #74 56 places worse
0.6769 ±0.0064
best reported
actual #18 vs paper #74 56 places worse
paper 0.5478 ±0.0101
best at pub: Heterogeneous Graph Benchmarks (KDD 2021)
paper 0.5478 ±0.0101 Δ at pub 0.0985 Δ today 0.1291
33
DMGI
actual #33 vs paper #82 49 places worse
0.6480
best reported
actual #33 vs paper #82 49 places worse
paper 0.5116 ±0.0063
best at pub: Implicit Graph Neural Networks (NeurIPS 2020)
paper 0.5116 ±0.0063 Δ at pub 0.1364 Δ today 0.1364
37
GRACE
actual #37 vs paper #74 37 places worse
0.6407 ±0.0080
best reported
actual #37 vs paper #74 37 places worse
paper 0.5480 ±0.0082
paper 0.5480 ±0.0082 Δ at pub 0.0927
44
GraphSAGE
0.6276
50
HGCML-P
0.6102 ±0.0049
This paper
52
HGCML
0.6075 ±0.0071
This paper
54
DGI
actual #54 vs paper #82 28 places worse
0.6060
best reported
actual #54 vs paper #82 28 places worse
paper 0.5200 ±0.0094
best at pub: Implicit Graph Neural Networks (NeurIPS 2020)
paper 0.5200 ±0.0094 Δ at pub 0.0860
63
STENCIL
0.5783 ±0.0062
This paper
68
DeepWalk
actual #68 vs paper #68
0.5636 ±0.0033
actual #68 vs paper #68
paper 0.5636 ±0.0033
prev best: Implicit Graph Neural Networks (NeurIPS 2020)
paper 0.5636 ±0.0033 Δ at pub 0.0136
74
MLP
0.5514
82
HeCo
0.5212 ±0.0072
This paper
85
Mp2vec
0.5078 ±0.0018
This paper
86
HIN2Vec
0.5073 ±0.0023
This paper
89
HERec
0.4912 ±0.0022
This paper

Freebase

Global leaderboard →
Non-default metric. This paper reports Micro-F1; the dataset default is Macro-F1. Values below use Micro-F1 only.

Sorted by Micro-F1. Highlighted rows are from other papers.

#ModelMicro-F1 ↑Paper
1
HGCML-P
0.7153 ±0.0014
This paper
2
HGCML
0.7141 ±0.0004
This paper
3
HAN
0.7098 ±0.0107
This paper
4
GAT
0.7042 ±0.0055
This paper
5
Mp2vec
0.6966 ±0.0011
This paper
6
DeepWalk
0.6963 ±0.0005
This paper
7
GRACE
0.6925 ±0.0014
This paper
8
GCN
0.6913 ±0.0251
This paper
9
HeCo
0.6902 ±0.0007
This paper
10
DGI
0.6842 ±0.0031
This paper
11
STENCIL
0.6826 ±0.0021
This paper
12
HERec
0.6804 ±0.0019
This paper
13
DMGI
0.6769 ±0.0021
This paper
14
HIN2Vec
0.6742 ±0.0014
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Improved at pub — later paper surpassed this result Best today — OK at publication; today's leaderboard best is elsewhere