A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal

K. Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fu Sun

Main figure for A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal

Abstract

Knowledge graph reasoning (KGR), aiming to deduce new facts from existing facts based on mined logic rules underlying knowledge graphs (KGs), has become a fast-growing research direction. It has been proven to significantly benefit the usage of KGs in many AI applications, such as question answering, recommendation systems, and etc. According to the graph types, existing KGR models can be roughly divided into three categories, i.e., static models, temporal models, and multi-modal models. Early works in this domain mainly focus on static KGR, and recent works try to leverage the temporal and multi-modal information, which are more practical and closer to real-world. However, no survey papers and open-source repositories comprehensively summarize and discuss models in this important direction. To fill the gap, we conduct a first survey for knowledge graph reasoning tracing from static to temporal and then to multi-modal KGs. Concretely, the models are reviewed based on bi-level taxonomy, i.e., top-level (graph types) and base-level (techniques and scenarios). Besides, the performances, as well as datasets, are summarized and presented. Moreover, we point out the challenges and potential opportunities to enlighten the readers. The corresponding open-source repository is shared on GitHub https://github.com/LIANGKE23/Awesome-Knowledge-Graph-Reasoning.

Benchmark Results

Tested on 2 datasets 10 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

Knowledge Graphs

#ModelMRR ↑Paper
1
FedSage
0.8426 ±0.0026
2
FED-PUB
0.8348 ±0.0025
3
FedAvg
0.8252 ±0.0023
7
GCN
0.8171 ±0.0030
11
NeuralLP
actual #11 vs paper #77 66 places worse
0.7440
best reported
actual #11 vs paper #77 66 places worse
paper 0.4590
paper 0.4590 Δ today 0.2850
13
RED-GNN
actual #13 vs paper #35 22 places worse
0.7010
best reported
actual #13 vs paper #35 22 places worse
paper 0.5330
paper 0.5330 Δ at pub 0.1680
36
HousE
0.5110
This paper
40
HAKE
0.4970
This paper
47
ComplEX-DURA
0.4890
This paper
48
QuatE
actual #48 vs paper #53 5 places worse
0.4880
best reported
actual #48 vs paper #53 5 places worse
paper 0.4810
paper 0.4810 Δ at pub 0.0070
50
RGHAT
0.4830
This paper
52
DualE
0.4820
This paper
53
DPMPN
0.4820
This paper
54
MuRP
0.4810
This paper
55
ConEx
0.4810
This paper
60
CompGCN
actual #60 vs paper #69 9 places worse
0.4790
best reported
actual #60 vs paper #69 9 places worse
paper 0.4690
paper 0.4690 Δ at pub 0.0100 Δ today 0.0100
62
RotatE
actual #62 vs paper #62
0.4770
best reported
actual #62 vs paper #62
paper 0.4760
paper 0.4760 Δ at pub 0.0010 Δ today 0.0010
63
ConvR
0.4750
This paper
68
TuckER
0.4700
This paper
69
SACN
0.4700
This paper
71
HypER
0.4650
This paper
72
LowFER
0.4650
This paper
74
KBGAT
0.4640
This paper
75
InteractE
0.4630
This paper
77
CURL
0.4600
This paper
78
RESCAL
0.4550
This paper
82
MINERVA
0.4480
This paper
83
pLogicNet
0.4410
This paper
84
ComplEx
0.4400
This paper
85
M-walk
0.4370
This paper
88
DisMult
actual #88 vs paper #93 5 places worse
0.4300
best reported
actual #88 vs paper #93 5 places worse
paper 0.4200
paper 0.4200 Δ at pub 0.0100
89
ConvE
0.4300
This paper
92
RGCN
0.4270
This paper
98
ConvKB
actual #98 vs paper #98
0.2650
actual #98 vs paper #98
paper 0.2650
paper 0.2650 Δ at pub 0.0160
100
TransE
0.2310
This paper

FB15k-237

Global leaderboard →

Default metric is MRR. The paper reports 31 values on MRR and 13 values on Hits@10 .

#ModelMRR ↑ defaultHits@10 ↑Paper
1
sum, Msg_r^3, f(t)=0
0.8500
—
2
sum, Msg_r^3, f(t)=t
0.8440
—
3
PNA, Msg_r^1, f(t)=0
0.8010
—
8
IMF
—
0.5930
Hits@10 0.5930
This paper
8
MOSE
—
0.5650
Hits@10 0.5650
This paper
8
HRGAT
—
0.5420
Hits@10 0.5420
This paper
8
MKBE
—
0.5320
Hits@10 0.5320
This paper
8
RGHAT
0.5220
—
Hits@10 —
This paper
9
MKGformer
—
0.4990
Hits@10 0.4990
This paper
9
MTRL
—
0.4940
Hits@10 0.4940
This paper
9
KBLRN
—
0.4930
Hits@10 0.4930
This paper
9
VBKGC
—
0.4780
Hits@10 0.4780
This paper
9
RSME
—
0.4670
Hits@10 0.4670
This paper
9
TransAE
—
0.4630
Hits@10 0.4630
This paper
9
IKRL
—
0.4580
Hits@10 0.4580
This paper
9
ViLBERT
—
0.4570
Hits@10 0.4570
This paper
9
VisualBERT
—
0.4390
Hits@10 0.4390
This paper
16
RED-GNN
0.3740
—
Hits@10 —
This paper
19
ComplEX-DURA
0.3700
—
Hits@10 —
This paper
20
DPMPN
0.3690
—
Hits@10 —
This paper
23
QuatE
0.3660
best reported
—
Hits@10 —
paper 0.3110
paper 0.3110 Δ at pub 0.0550
24
ConEx
0.3660
—
Hits@10 —
This paper
26
HousE
0.3610
—
Hits@10 —
This paper
31
LowFER
0.3590
—
Hits@10 —
This paper
32
TuckER
0.3580
—
Hits@10 —
This paper
35
CompGCN
0.3550
best reported
—
Hits@10 —
paper 0.3520
paper 0.3520 Δ at pub 0.0030 Δ today 0.0030
36
InteractE
0.3540
—
Hits@10 —
This paper
38
RESCAL
0.3530
—
Hits@10 —
This paper
44
ConvR
0.3500
—
Hits@10 —
This paper
45
SACN
0.3500
—
Hits@10 —
This paper
46
KBGAT
0.3500
—
Hits@10 —
This paper
47
ComplEx
0.3460
—
Hits@10 —
paper 0.3460
prev best: Knowledge Graphs (AAAI 2017)
paper 0.3460 Δ at pub 0.0990
55
HAKE
0.3410
best reported
—
Hits@10 —
paper 0.3350
paper 0.3350 Δ at pub 0.0060
56
HypER
0.3410
—
Hits@10 —
This paper
58
RotatE
0.3380
—
Hits@10 —
This paper
61
MuRP
0.3350
—
Hits@10 —
This paper
65
pLogicNet
0.3320
—
Hits@10 —
This paper
67
TransE
0.3300
best reported
—
Hits@10 —
paper 0.2890
paper 0.2890 Δ at pub 0.0050 Δ today 0.0410
68
DualE
0.3300
—
Hits@10 —
This paper
71
ConvE
0.3250
—
Hits@10 —
This paper
79
CURL
0.3060
—
Hits@10 —
This paper
80
MINERVA
0.2930
best reported
—
Hits@10 —
paper 0.2710
paper 0.2710 Δ at pub 0.0220
83
ConvKB
0.2890
—
Hits@10 —
paper 0.2890
paper 0.2890 Δ at pub 0.0460
87
RGCN
0.2730
best reported
—
Hits@10 —
paper 0.2480
92
DisMult
0.2430
—
Hits@10 —
This paper
93
M-walk
0.2340
—
Hits@10 —
This paper
94
NeuralLP
0.2270
—
Hits@10 —
This paper
SOTA performance (not significantly different from the best) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published Best today — OK at publication; today's leaderboard best is elsewhere