Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Tianqianjin Lin, Kaisong Song, Zhuoren Jiang, Yangyang Kang, Weikang Yuan, Xurui Li, Changlong Sun, Cui Huang, Xiaozhong Liu

Main figure for Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Abstract

Heterogeneous graph neural networks have become popular in various domains. However, their generalizability and interpretability are limited due to the discrepancy between their inherent inference flows and human reasoning logic or underlying causal relationships for the learning problem. This study introduces a novel solution, HG-SCM (Heterogeneous Graph as Structural Causal Model). It can mimic the human perception and decision process through two key steps: constructing intelligible variables based on semantics derived from the graph schema and automatically learning task-level causal relationships among these variables by incorporating advanced causal discovery techniques. We compared HG-SCM to seven state-of-the-art baseline models on three real-world datasets, under three distinct and ubiquitous out-of-distribution settings. HG-SCM achieved the highest average performance rank with minimal standard deviation, substantiating its effectiveness and superiority in terms of both predictive power and generalizability. Additionally, the visualization and analysis of the auto-learned causal diagrams for the three tasks aligned well with domain knowledge and human cognition, demonstrating prominent interpretability. HG-SCM's human-like nature and its enhanced generalizability and interpretability make it a promising solution for special scenarios where transparency and trustworthiness are paramount.

Proposed method: HG-SCM

Benchmark Results

Tested on 3 datasets 1 baseline value reported too low Non-standard metric on 3 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

Heterogeneous Graph Benchmarks

DBLP

Global leaderboard →

Default metric is Micro-F1. The paper reports 0 values on Micro-F1 , 11 values on Macro F1 and 11 values on Accuracy .

Sorted by Macro F1. Highlighted rows are from other papers.

#ModelMicro-F1 ↑ defaultMacro F1 ↑Accuracy ↑Paper
2
SeHGNN
—
0.9449 ± 0.0020
0.9489 ± 0.0018
Micro-F1 —
Accuracy 0.9489 ± 0.0018
This paper
1
HG-SCM
—
0.9451 ± 0.0015
0.9490 ± 0.0015
Micro-F1 —
Accuracy 0.9490 ± 0.0015
This paper
3
simpleHGN
—
0.9401 ± 0.0024
0.9446 ± 0.0022
Micro-F1 —
Accuracy 0.9446 ± 0.0022
This paper
4
GTN
—
0.9352 ± 0.0055
0.9397 ± 0.0054
Micro-F1 —
Accuracy 0.9397 ± 0.0054
This paper
5
RSHN
—
0.9334 ± 0.0058
0.9381 ± 0.0055
Micro-F1 —
Accuracy 0.9381 ± 0.0055
This paper
6
MAGNN
—
0.9328 ± 0.0051
0.9376 ± 0.0045
Micro-F1 —
Accuracy 0.9376 ± 0.0045
This paper
7
HGT
—
0.9301 ± 0.0023
0.9349 ± 0.0025
Micro-F1 —
Accuracy 0.9349 ± 0.0025
This paper
8
HetGNN
—
0.9176 ± 0.0043
0.9233 ± 0.0041
Micro-F1 —
Accuracy 0.9233 ± 0.0041
This paper
9
HAN
—
0.9167 ± 0.0049
0.9205 ± 0.0062
Micro-F1 —
Accuracy 0.9205 ± 0.0062
This paper
10
RGCN
—
0.9152 ± 0.0050
0.9207 ± 0.0050
Micro-F1 —
Accuracy 0.9207 ± 0.0050
This paper
11
HetSANN
—
0.7855 ± 0.0242
0.8056 ± 0.0150
Micro-F1 —
Accuracy 0.8056 ± 0.0150
This paper

ACM

Global leaderboard →
Non-default metric. This paper reports Accuracy; the dataset default is Micro-F1. Values below use Accuracy only.

Sorted by Accuracy. Highlighted rows are from other papers.

#ModelAccuracy ↑Paper
1
SignGT
0.9545 ±0.0049
2
NodeFormer
0.9493 ±0.0035
3
FAGCN
0.9487 ±0.0059
16
HG-SCM
0.9356 ±0.0032
This paper
17
SeHGNN
0.9351 ±0.0045
This paper
21
simpleHGN
0.9335 ±0.0045
This paper
34
GCN
0.9233 ±0.0038
35
HAN
actual #35 vs paper #44 9 places worse
0.9220 ±0.0020
best reported
actual #35 vs paper #44 9 places worse
paper 0.9079 ±0.0043
paper 0.9079 ±0.0043 Δ at pub 0.0141
36
HGT
actual #36 vs paper #43 7 places worse
0.9190 ±0.0030
best reported
actual #36 vs paper #43 7 places worse
paper 0.9100 ±0.0076

Difference within reported uncertainty; not counted as underperformance.

paper 0.9100 ±0.0076 Δ at pub 0.0090
38
RGCN
actual #38 vs paper #39 1 place worse
0.9175 ±0.0035
best reported
actual #38 vs paper #39 1 place worse
paper 0.9141 ±0.0075

Difference within reported uncertainty; not counted as underperformance.

paper 0.9141 ±0.0075 Δ today 0.0034
41
GTN
0.9120 ±0.0071
This paper
45
MAGNN
0.9077 ±0.0065
This paper
47
RSHN
0.9032 ±0.0154
This paper
48
HetSANN
0.8991 ±0.0037
This paper
62
HetGNN
0.8505 ±0.0025
This paper

IMDB

Global leaderboard →

Default metric is Micro-F1. The paper reports 0 values on Micro-F1 , 10 values on Macro F1 and 11 values on Accuracy .

Sorted by Macro F1. Highlighted rows are from other papers.

#ModelMicro-F1 ↑ defaultMacro F1 ↑Accuracy ↑Paper
1
SeHGNN
—
—
0.6598 ± 0.0012
Micro-F1 —
Accuracy 0.6598 ± 0.0012
This paper
1
HG-SCM
—
0.6534 ± 0.0033
0.6690 ± 0.0061
Micro-F1 —
Accuracy 0.6690 ± 0.0061
This paper
2
simpleHGN
—
0.6353 ± 0.0136
0.6736 ± 0.0057
Micro-F1 —
Accuracy 0.6736 ± 0.0057
This paper
3
HGT
—
0.6300 ± 0.0119
0.6720 ± 0.0057
Micro-F1 —
Accuracy 0.6720 ± 0.0057
This paper
4
GTN
—
0.6047 ± 0.0098
0.6514 ± 0.0045
Micro-F1 —
Accuracy 0.6514 ± 0.0045
This paper
5
RSHN
—
0.5985 ± 0.0321
0.6422 ± 0.0103
Micro-F1 —
Accuracy 0.6422 ± 0.0103
This paper
6
RGCN
—
0.5885 ± 0.0026
0.6205 ± 0.0015
Micro-F1 —
Accuracy 0.6205 ± 0.0015
This paper
7
HAN
—
0.5774 ± 0.0096
0.6463 ± 0.0058
Micro-F1 —
Accuracy 0.6463 ± 0.0058
This paper
8
MAGNN
—
0.5649 ± 0.0320
0.6467 ± 0.0167
Micro-F1 —
Accuracy 0.6467 ± 0.0167
This paper
9
HetSANN
—
0.4947 ± 0.0121
0.5768 ± 0.0044
Micro-F1 —
Accuracy 0.5768 ± 0.0044
This paper
10
HetGNN
—
0.4825 ± 0.0067
0.5116 ± 0.0065
Micro-F1 —
Accuracy 0.5116 ± 0.0065
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)