Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph
Abstract
Heterogeneous graph neural networks have become popular in various domains. However, their generalizability and interpretability are limited due to the discrepancy between their inherent inference flows and human reasoning logic or underlying causal relationships for the learning problem. This study introduces a novel solution, HG-SCM (Heterogeneous Graph as Structural Causal Model). It can mimic the human perception and decision process through two key steps: constructing intelligible variables based on semantics derived from the graph schema and automatically learning task-level causal relationships among these variables by incorporating advanced causal discovery techniques. We compared HG-SCM to seven state-of-the-art baseline models on three real-world datasets, under three distinct and ubiquitous out-of-distribution settings. HG-SCM achieved the highest average performance rank with minimal standard deviation, substantiating its effectiveness and superiority in terms of both predictive power and generalizability. Additionally, the visualization and analysis of the auto-learned causal diagrams for the three tasks aligned well with domain knowledge and human cognition, demonstrating prominent interpretability. HG-SCM's human-like nature and its enhanced generalizability and interpretability make it a promising solution for special scenarios where transparency and trustworthiness are paramount.
Benchmark Results
Tested on 3 datasets 1 baseline value reported too low Non-standard metric on 3 datasets
Tables show presented results in global context (best 3 papers + GCN + MLP).
Heterogeneous Graph Benchmarks
DBLP
Global leaderboard →Default metric is Micro-F1. The paper reports 0 values on Micro-F1 , 11 values on Macro F1 and 11 values on Accuracy .
Sorted by Macro F1. Highlighted rows are from other papers.
| # | Model | Micro-F1 ↑ default | Macro F1 ↑ | Accuracy ↑ | Paper |
|---|---|---|---|---|---|
| 2 | SeHGNN | — | 0.9449
±
0.0020 | 0.9489
±
0.0018 | Micro-F1
— Accuracy
0.9489
±
0.0018 |
| 1 | HG-SCM | — | 0.9451
±
0.0015 | 0.9490
±
0.0015 | Micro-F1
— Accuracy
0.9490
±
0.0015 |
| 3 | simpleHGN | — | 0.9401
±
0.0024 | 0.9446
±
0.0022 | Micro-F1
— Accuracy
0.9446
±
0.0022 |
| 4 | GTN | — | 0.9352
±
0.0055 | 0.9397
±
0.0054 | Micro-F1
— Accuracy
0.9397
±
0.0054 |
| 5 | RSHN | — | 0.9334
±
0.0058 | 0.9381
±
0.0055 | Micro-F1
— Accuracy
0.9381
±
0.0055 |
| 6 | MAGNN | — | 0.9328
±
0.0051 | 0.9376
±
0.0045 | Micro-F1
— Accuracy
0.9376
±
0.0045 |
| 7 | HGT | — | 0.9301
±
0.0023 | 0.9349
±
0.0025 | Micro-F1
— Accuracy
0.9349
±
0.0025 |
| 8 | HetGNN | — | 0.9176
±
0.0043 | 0.9233
±
0.0041 | Micro-F1
— Accuracy
0.9233
±
0.0041 |
| 9 | HAN | — | 0.9167
±
0.0049 | 0.9205
±
0.0062 | Micro-F1
— Accuracy
0.9205
±
0.0062 |
| 10 | RGCN | — | 0.9152
±
0.0050 | 0.9207
±
0.0050 | Micro-F1
— Accuracy
0.9207
±
0.0050 |
| 11 | HetSANN | — | 0.7855
±
0.0242 | 0.8056
±
0.0150 | Micro-F1
— Accuracy
0.8056
±
0.0150 |
ACM
Global leaderboard →Sorted by Accuracy. Highlighted rows are from other papers.
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | SignGT | 0.9545
±0.0049 | |
| 2 | NodeFormer | 0.9493
±0.0035 | |
| 3 | FAGCN | 0.9487
±0.0059 | |
| 16 | HG-SCM | 0.9356
±0.0032 | This paper |
| 17 | SeHGNN | 0.9351
±0.0045 | This paper |
| 21 | simpleHGN | 0.9335
±0.0045 | This paper |
| 34 | GCN | 0.9233
±0.0038 | |
| 35 | HAN | 0.9220
±0.0020 best reported | best at pub: Scalable Graph Neural Networks for Heterogeneous Graphs (2020) |
| 36 | HGT | 0.9190
±0.0030 best reported | best at pub: Scalable Graph Neural Networks for Heterogeneous Graphs (2020) Difference within reported uncertainty; not counted as underperformance. |
| 38 | RGCN | 0.9175
±0.0035 best reported | best today: Revisiting Graph Autoencoders as Implicit Contrastive Learners (KDD 2024) Difference within reported uncertainty; not counted as underperformance. |
| 41 | GTN | 0.9120
±0.0071 | This paper |
| 45 | MAGNN | 0.9077
±0.0065 | This paper |
| 47 | RSHN | 0.9032
±0.0154 | This paper |
| 48 | HetSANN | 0.8991
±0.0037 | This paper |
| 62 | HetGNN | 0.8505
±0.0025 | This paper |
IMDB
Global leaderboard →Default metric is Micro-F1. The paper reports 0 values on Micro-F1 , 10 values on Macro F1 and 11 values on Accuracy .
Sorted by Macro F1. Highlighted rows are from other papers.
| # | Model | Micro-F1 ↑ default | Macro F1 ↑ | Accuracy ↑ | Paper |
|---|---|---|---|---|---|
| 1 | SeHGNN | — | —
| 0.6598
±
0.0012 | Micro-F1
— Accuracy
0.6598
±
0.0012 |
| 1 | HG-SCM | — | 0.6534
±
0.0033 | 0.6690
±
0.0061 | Micro-F1
— Accuracy
0.6690
±
0.0061 |
| 2 | simpleHGN | — | 0.6353
±
0.0136 | 0.6736
±
0.0057 | Micro-F1
— Accuracy
0.6736
±
0.0057 |
| 3 | HGT | — | 0.6300
±
0.0119 | 0.6720
±
0.0057 | Micro-F1
— Accuracy
0.6720
±
0.0057 |
| 4 | GTN | — | 0.6047
±
0.0098 | 0.6514
±
0.0045 | Micro-F1
— Accuracy
0.6514
±
0.0045 |
| 5 | RSHN | — | 0.5985
±
0.0321 | 0.6422
±
0.0103 | Micro-F1
— Accuracy
0.6422
±
0.0103 |
| 6 | RGCN | — | 0.5885
±
0.0026 | 0.6205
±
0.0015 | Micro-F1
— Accuracy
0.6205
±
0.0015 |
| 7 | HAN | — | 0.5774
±
0.0096 | 0.6463
±
0.0058 | Micro-F1
— Accuracy
0.6463
±
0.0058 |
| 8 | MAGNN | — | 0.5649
±
0.0320 | 0.6467
±
0.0167 | Micro-F1
— Accuracy
0.6467
±
0.0167 |
| 9 | HetSANN | — | 0.4947
±
0.0121 | 0.5768
±
0.0044 | Micro-F1
— Accuracy
0.5768
±
0.0044 |
| 10 | HetGNN | — | 0.4825
±
0.0067 | 0.5116
±
0.0065 | Micro-F1
— Accuracy
0.5116
±
0.0065 |
