PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
Abstract
Distance based knowledge graph embedding methods show promising results on link prediction task, on which two topics have been widely studied: one is the ability to handle complex relations, such as N-to-1, 1-to-N and N-to-N, the other is to encode various relation patterns, such as symmetry/antisymmetry. However, the existing methods fail to solve these two problems at the same time, which leads to unsatisfactory results. To mitigate this problem, we propose PairRE, a model with paired vectors for each relation representation. The paired vectors enable an adaptive adjustment of the margin in loss function to fit for complex relations. Besides, PairRE is capable of encoding three important relation patterns, symmetry/antisymmetry, inverse and composition. Given simple constraints on relation representations, PairRE can encode subrelation further. Experiments on link prediction benchmarks demonstrate the proposed key capabilities of PairRE. Moreover, We set a new state-of-the-art on two knowledge graph datasets of the challenging Open Graph Benchmark.
Benchmark Results
Tested on 3 datasets 1 baseline value reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
OGB
| # | Model | MRR ↑ | Paper |
|---|---|---|---|
| 1 | InterHT + DigPiece | 0.7293
±0.0018 | |
| 2 | StarGraph + TripleRE' | 0.7290 | |
| 3 | TripleRE + StarGraph | 0.7201
±0.0011 | |
| 20 | PairRE | 0.5805 best reported | |
| 29 | TransE | 0.4536
±0.0030 | This paper |
| 30 | RotatE | 0.4332
±0.0025 | This paper |
| 36 | ComplEx | 0.4028
±0.0030 | This paper |
| 39 | DistMult | 0.3729
±0.0045 | This paper |
| # | Model | MRR ↑ | Paper |
|---|---|---|---|
| 1 | PairRE+LRE | 0.8360 | |
| 2 | NBFnet | 0.8317 | |
| 3 | AutoSF | 0.8309
±0.0008 | |
| 6 | PairRE | 0.8164
±0.0005 | This paper |
| 7 | ComplEx | 0.8095
±0.0007 | This paper |
| 8 | DistMult | 0.8043
±0.0003 | This paper |
| 9 | RotatE | 0.7989
±0.0004 | This paper |
| 11 | TransE | 0.7452
±0.0004 | This paper |
Knowledge Graphs
| # | Model | MRR ↑ | Paper |
|---|---|---|---|
| 1 | sum, Msg_r^3, f(t)=0 | 0.8500 | |
| 2 | sum, Msg_r^3, f(t)=t | 0.8440 | |
| 3 | PNA, Msg_r^1, f(t)=0 | 0.8010 | |
| 27 | OTE | 0.3610 best reported | best at pub: Multi-hop Attention Graph Neural Networks (IJCAI 2020) |
| 29 | GC-OTE | 0.3610 | This paper |
| 33 | DistMult | 0.3580 best reported | best at pub: Knowledge Graphs (AAAI 2017) best today: KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning (ICML 2024) |
| 42 | PairRE | 0.3510
±0.0007 | This paper |
| 47 | ComplEx | 0.3460 best reported | best at pub: Knowledge Graphs (AAAI 2017) |
| 58 | RotatE | 0.3380 | This paper |
| 67 | TransE | 0.3300 best reported | best at pub: Composition-based Multi-Relational Graph Convolutional Networks (ICLR 2019) best today: KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning (ICML 2024) |
| 71 | ConvE | 0.3250 | This paper |
