Enhancing Distribution and Label Consistency for Graph Out-of-Distribution Generalization

Song Wang, Xiaodong Yang, Rashidul Islam, Huiyuan Chen, Minghua Xu, Jundong Li, Yiwei Cai

Main figure for Enhancing Distribution and Label Consistency for Graph Out-of-Distribution Generalization

Abstract

To deal with distribution shifts in graph data, various graph out-of-distribution (OOD) generalization techniques have been recently proposed. These methods often employ a two-step strategy that first creates augmented environments and subsequently identifies invariant subgraphs to improve generalizability. Nevertheless, this approach could be suboptimal from the perspective of consistency. First, the process of augmenting environments by altering the graphs while preserving labels may lead to graphs that are not realistic or meaningfully related to the origin distribution, thus lacking distribution consistency. Second, the extracted subgraphs are obtained from directly modifying graphs, and may not necessarily maintain a consistent predictive relationship with their labels, thereby impacting label consistency. In response to these challenges, we introduce an innovative approach that aims to enhance these two types of consistency for graph OOD generalization. We propose a modifier to obtain both augmented and invariant graphs in a unified manner. With the augmented graphs, we enrich the training data without compromising the integrity of label-graph relationships. The label consistency enhancement in our framework further preserves the supervision information in the invariant graph. We conduct extensive experiments on real-world datasets to demonstrate the superiority of our framework over other state-of-the-art baselines.

Proposed method: DLG

Benchmark Results

Tested on 1 dataset 2 baseline values reported too low

Tables show presented results in global context (best 3 papers + GCN + MLP).

OGB

#ModelROC-AUC ↑Paper
1
Multi-RF Fusion with Multi-GNN Blending
0.8476 ±0.0002
2
HyperFusion
0.8475 ±0.0003
3
PAS+FPs
0.8420 ±0.0015
9
GatedGCN
0.8090 ±0.0160
18
GSN
actual #18 vs paper #247 229 places worse
0.8039 ±0.0090
best reported
actual #18 vs paper #247 229 places worse
paper 0.7453 ±0.0139
paper 0.7453 ±0.0139 Δ at pub 0.0586 Δ today 0.0586
24
GSAT
0.8018 ±0.0182
This paper
34
DLG
0.7980 ±0.0178
This paper
36
CIGA
actual #36 vs paper #36
0.7975
best reported
actual #36 vs paper #36
paper 0.7970 ±0.0192

Difference within reported uncertainty; not counted as underperformance.

paper 0.7970 ±0.0192 Δ today 0.0005
59
GIL
0.7908 ±0.0247
This paper
109
GIN
0.7780 ±0.0182
124
Attention
actual #124 vs paper #209 85 places worse
0.7744 ±0.0127
best reported
actual #124 vs paper #209 85 places worse
paper 0.7584 ±0.0367

Difference within reported uncertainty; not counted as underperformance.

paper 0.7584 ±0.0367 Δ at pub 0.0160
152
DIR
0.7705 ±0.0398
This paper
155
IRM
actual #155 vs paper #248 93 places worse
0.7702 ±0.0107
best reported
actual #155 vs paper #248 93 places worse
paper 0.7446 ±0.0105
paper 0.7446 ±0.0105 Δ at pub 0.0256
187
ERM
0.7620 ±0.0254
This paper
218
V-REx
0.7562 ±0.0331
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)