State Space Models over Directed Graphs

Junzhi She, Xunkai Li, Ronghua Li, Guoren Wang

Main figure for State Space Models over Directed Graphs

Abstract

Directed graphs are ubiquitous across numerous domains, where the directionality of edges encodes critical causal dependencies. However, existing GNNs and graph Transformers tailored for directed graphs face two major challenges: (1) effectively capturing long-range causal dependencies derived from directed edges; (2) balancing accuracy and training efficiency when processing large-scale graph datasets. In recent years, state space models (SSMs) have achieved substantial progress in causal sequence tasks, and their variants designed for graphs have demonstrated state-of-the-art accuracy while maintaining high efficiency across various graph learning benchmarks. However, existing graph state space models are exclusively designed for undirected graphs, which limits their performance in directed graph learning. To this end, we propose an innovative approach DirEgo2Token which sequentializes directed graphs via k-hop ego graphs. This marks the first systematic extension of state space models to the field of directed graph learning. Building upon this, we develop DirGraphSSM, a novel directed graph neural network architecture that implements state space models on directed graphs via the message-passing mechanism. Experimental results demonstrate that DirGraphSSM achieves state-of-the-art performance on three representative directed graph learning tasks while attaining competitive performance on two additional tasks with 1.5× to 2× training speed improvements compared to existing state-of-the-art models.

Proposed method: DirGraphSSM

Benchmark Results

Tested on 2 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

OGB

#ModelF1 ↑Paper
1
GMN
0.2060 ±0.0020
This paper
2
DirGraphSSM
0.2050 ±0.0023
This paper
3
Graph-Mamba
0.2030 ±0.0023
This paper
4
DAG+SAT
0.2020 ±0.0020
This paper
5
Exphormer
0.1990 ±0.0020
This paper
8
SAT
0.1940 ±0.0030
This paper
18
GraphGPS
0.1890 ±0.0020
This paper
19
DAG+Transformer
0.1880 ±0.0020
This paper
22
PACE
0.1780 ±0.0020
This paper
24
DAGNN
0.1750 ±0.0050
This paper
26
MagNet
0.1690 ±0.0020
This paper
28
DiGCN
0.1670 ±0.0020
This paper
29
DirGNN
0.1660 ±0.0020
This paper
33
GAT
0.1570 ±0.0020
This paper
35
GCN
actual #35 vs paper #35
0.1510 ±0.0020
actual #35 vs paper #35
paper 0.1510 ±0.0020

Improvement within reported uncertainty; not highlighted.

paper 0.1510 ±0.0020 Δ at pub 0.0003
37
GIN
actual #37 vs paper #38 1 place worse
0.1495 ±0.0023
best reported
actual #37 vs paper #38 1 place worse
paper 0.1490 ±0.0020

Difference within reported uncertainty; not counted as underperformance.

paper 0.1490 ±0.0020 Δ at pub 0.0005 Δ today 0.0005

Other Graph Benchmarks

#ModelAccuracy ↑Paper
1
GMN
0.9415 ±0.0020
This paper
2
DirGraphSSM
0.9409 ±0.0023
This paper
3
DAG+SAT-plus
0.9407 ±0.0020
This paper
4
Graph-Mamba
0.9406 ±0.0018
This paper
5
DAG+Transformer-plus
0.9404 ±0.0019
This paper
6
Exphormer
0.9402 ±0.0020
This paper
12
GraphGPS
0.9264 ±0.0078
This paper
14
SAT
0.9256 ±0.0031
This paper
19
MagNet
0.9095 ±0.0025
This paper
20
DiGCN
0.9081 ±0.0017
This paper
21
DirGNN
0.9073 ±0.0015
This paper
22
GIN
0.8898 ±0.0055
This paper
25
GAT
0.8509 ±0.0025
This paper
26
GCN
0.8100 ±0.0001
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper)