Knowledge Graphs
FB15k-237
Structure
1Graphs
14,541Nodes
310,116Edges
237Classes
Coverage
142Results
1Variants
19Papers
- GraphNetz: Statistical Benchmarking of Graph Neural Networks with Paired Tests and Rank Aggregation GraphNetz: Statistical Benchmarking of Graph Neural Networks with Paired Tests and Rank Aggregation
- NeurIPS 2024: A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning NeurIPS 2024
- ICML 2024: KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning ICML 2024
- WWW 2023: Knowledge Graph Completion with Counterfactual Augmentation Knowledge Graph Completion with Counterfactual Augmentation WWW 2023
- NeurIPS 2023: A Theory of Link Prediction via Relational Weisfeiler-Leman on Knowledge Graphs A Theory of Link Prediction via Relational Weisfeiler-Leman on Knowledge Graphs NeurIPS 2023
- A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal
- Knowledge Graph Contrastive Learning based on Relation-Symmetrical Structure Knowledge Graph Contrastive Learning based on Relation-Symmetrical Structure
- TripleRE: Knowledge Graph Embeddings via Tripled Relation Vectors TripleRE: Knowledge Graph Embeddings via Tripled Relation Vectors
- NeurIPS 2022: A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs NeurIPS 2022
- ICLR 2022: StarGraph: Knowledge Representation Learning based on Incomplete Two-hop Subgraph StarGraph: Knowledge Representation Learning based on Incomplete Two-hop Subgraph ICLR 2022
- NeurIPS 2021: AutoGEL: An Automated Graph Neural Network with Explicit Link Information AutoGEL: An Automated Graph Neural Network with Explicit Link Information NeurIPS 2021
- WWW 2021: Knowledge Graph Reasoning with Relational Digraph Knowledge Graph Reasoning with Relational Digraph WWW 2021
- NeurIPS 2021: Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction NeurIPS 2021
- PairRE: Knowledge Graph Embeddings via Paired Relation Vectors PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
- IJCAI 2020: Multi-hop Attention Graph Neural Networks Multi-hop Attention Graph Neural Networks IJCAI 2020
- ICML 2019: Inductive Relation Prediction by Subgraph Reasoning Inductive Relation Prediction by Subgraph Reasoning ICML 2019
- ICLR 2019: Composition-based Multi-Relational Graph Convolutional Networks Composition-based Multi-Relational Graph Convolutional Networks ICLR 2019
- ICLR 2018: RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space ICLR 2018
- AAAI 2017: Knowledge Graphs Knowledge Graphs AAAI 2017
237-relation knowledge-graph completion Knowledge graph link prediction on Freebase (14,541 entities, 237 relations). Inputs are directed subject-relation-object triples with learned embeddings. Evaluated by MRR.
Dataset details
Task
Knowledge graph link prediction on Freebase (14,541 entities, 237 relations). The primary catalog metric is MRR.
Data
Knowledge graph link prediction on Freebase (14,541 entities, 237 relations). The cataloged artifact reports 1 graphs, 237 target classes or tasks.
Features
The raw input is a set of directed subject-relation-object triples. Entity and relation embeddings are learned inputs; reciprocal edges, inverse relations, and filtered candidate sets are evaluation or preprocessing choices.
Splits and evaluation
The catalog records these protocols or variants: Standard split. Evaluation uses MRR (higher is better). Exact masks or folds must come from the cited release.
Quirks and pitfalls
Filtered ranking removes other known true triples but the graph is incomplete, so sampled corruptions can be false negatives. WN18 and FB15k contain inverse-relation leakage; use WN18RR or FB15k-237 when evaluating relational generalization.
Sources
Toutanova & Chen 2015 split. Also reports Hits@1, Hits@3, Hits@10.
SOTA Progression
new SOTA
matches SOTA at that time
other proposed result
other baseline result
Leaderboard
| # | Model | MRR ↑ | AUC | AUC-PR | H@10 | Hit@1 | Hit@10 | Hit@3 | Hits@10 | MR | Paper |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | sum, Msg_r^3, f(t)=0 | 0.8500 | — | — | — | — | — | — | — | — | |
| 2 | sum, Msg_r^3, f(t)=t | 0.8440 | — | — | — | — | — | — | — | — | |
| 3 | PNA, Msg_r^1, f(t)=0 | 0.8010 | — | — | — | — | — | — | — | — | |
| 4 | sum, Msg_r^1, f(t)=t | 0.7940 | — | — | — | — | — | — | — | — | |
| 5 | sum, Msg_r^1, f(t)=0 | 0.7840 | — | — | — | — | — | — | — | — | |
| 6 | sum, Msg_r^2, f(t)=0 | 0.7340 | — | — | — | — | — | — | — | — | |
| 7 | sum, Msg_r^2, f(t)=t | 0.7280 | — | — | — | — | — | — | — | — | |
| 8 | RGHAT | 0.5220 | — | — | — | — | — | — | — | — | |
| 9 | KnowFormer | 0.4300 | — | — | — | — | — | — | — | — | |
| 10 | KGCF | 0.4210 | — | — | — | — | — | — | — | — | |
| 11 | AdaProp | 0.4170 | — | — | — | — | — | — | — | — | |
| 12 | NBFnet | 0.4150 | — | — | — | — | — | — | — | — | |
| 13 | A*Net | 0.4110 | — | — | — | — | — | — | — | — | |
| 14 | Multi-Hop | 0.3930 | — | — | — | — | — | — | — | — | |
| 15 | ComplEx+RP | 0.3880 | — | — | — | — | — | — | — | — | |
| 16 | RED-GNN | 0.3740 | — | — | — | — | — | — | — | — | |
| 17 | HittER | 0.3730 | — | — | — | — | — | — | — | — | |
| 18 | ComplEX-DURA-SymCL | 0.3710 | — | — | — | — | — | — | — | — | |
| 19 | ComplEX-DURA | 0.3700 | — | — | — | — | — | — | — | — | |
| 20 | MAGNA + DistMult | 0.3690 | — | — | — | — | — | — | — | — | |
| 21 | DPMPN | 0.3690 | — | — | — | — | — | — | — | — | |
| 22 | ULTRA | 0.3680 | — | — | — | — | — | — | — | — | |
| 23 | ConEx | 0.3660 | — | — | — | — | — | — | — | — | |
| 24 | QuatE | 0.3660 | — | — | — | — | — | — | — | — | |
| 25 | RuleN | 0.3630 | — | — | — | — | — | — | — | — | |
| 26 | HousE | 0.3610 | — | — | — | — | — | — | — | — | |
| 27 | OTE | 0.3610 | — | — | — | — | — | — | — | — | |
| 28 | CoKE | 0.3610 | — | — | — | — | — | — | — | — | |
| 29 | GC-OTE | 0.3610 | — | — | — | — | — | — | — | — | |
| 30 | AutoSF | 0.3600 | — | — | — | — | — | — | — | — | |
| 31 | LowFER | 0.3590 | — | — | — | — | — | — | — | — | |
| 32 | TuckER | 0.3580 | — | — | — | — | — | — | — | — | |
| 33 | DistMult | 0.3580 | — | — | — | — | — | — | — | — | |
| 34 | AutoGEL | 0.3570 | — | — | — | — | — | — | — | — | |
| 35 | CompGCN | 0.3550 | — | — | — | — | — | — | — | — | |
| 36 | InteractE | 0.3540 | — | — | — | — | — | — | — | — | |
| 37 | COMPGCN-SymCL | 0.3540 | — | — | — | — | — | — | — | — | |
| 38 | RESCAL | 0.3530 | — | — | — | — | — | — | — | — | |
| 39 | LTE + ConvE | 0.3520 | — | — | — | — | — | — | — | — | |
| 40 | CompGCN | 0.3520 | — | — | — | — | — | — | — | — | |
| 41 | TripleRE' | 0.3514 | — | — | — | — | — | — | — | — | |
| 42 | TripleRE | 0.3510 Hit@1
0.2510 Hit@10
0.5520 Hit@3
0.3920 MR
142.0000 | — | — | — | 0.2510 | 0.5520 | 0.3920 | — | 142.0000 | |
| 43 | CompGCN + ConvE | 0.3510 | — | — | — | — | — | — | — | — | |
| 44 | OTE | 0.3510 | — | — | — | — | — | — | — | — | |
| 45 | PairRE | 0.3510
±
0.0007 | — | — | — | — | — | — | — | — | |
| 46 | KBGAT | 0.3500 | — | — | — | — | — | — | — | — | |
| 47 | SACN | 0.3500 | — | — | — | — | — | — | — | — | |
| 48 | ConvR | 0.3500 | — | — | — | — | — | — | — | — | |
| 49 | QuatE | 0.3500 | — | — | — | — | — | — | — | — | |
| 50 | ComplEx | 0.3460 | — | — | — | — | — | — | — | — |
SOTA performance
set a new SOTA at the time
matched the SOTA at the time