Simple yet Effective Node Property Prediction on Edge Streams under Distribution Shifts

Jongha Lee, Taehyung Kwon, Heechan Moon, Kijung Shin

Main figure for Simple yet Effective Node Property Prediction on Edge Streams under Distribution Shifts

Abstract

The problem of predicting node properties (e.g., node classes) in graphs has received significant attention due to its broad range of applications. Graphs from real-world datasets often evolve over time, with newly emerging edges and dynamically changing node properties, posing a significant challenge for this problem. In response, temporal graph neural networks (TGNNs) have been developed to predict dynamic node properties from a stream of emerging edges. However, our analysis reveals that most TGNN-based methods are (a) far less effective without proper node features and, due to their complex model architectures, (b) vulnerable to distribution shifts. In this paper, we propose SPLASH, a simple yet powerful method for predicting node properties on edge streams under distribution shifts. Our key contributions are as follows: (1) we propose feature augmentation methods and an automatic feature selection method for edge streams, which improve the effectiveness of TGNNs, (2) we propose a lightweight MLP-based TGNN architecture that is highly efficient and robust under distribution shifts, and (3) we conduct extensive experiments to evaluate the accuracy, efficiency, generalization, and qualitative performance of the proposed method and its competitors on dynamic node classification, dynamic anomaly detection, and node affinity prediction tasks across seven real-world datasets.

Proposed method: SPLASH

Benchmark Results

Tested on 1 dataset 2 baseline values reported too low Non-standard metric on 1 dataset

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

Reddit

Global leaderboard →
Non-default metric. This paper reports AUC; the dataset default is F1. Values below use AUC only.

Sorted by AUC. Highlighted rows are from other papers.

#ModelAUC ↑Paper
1
SPLASH
0.7358 ±0.0028
This paper
2
Backbone (TGAT)
0.6822 ±0.0052
3
PINT
0.6731 ±0.0020
5
TGAT
actual #5 vs paper #11 6 places worse
0.6556 ±0.0070
best reported
actual #5 vs paper #11 6 places worse
paper 0.6100 ±0.0088
paper 0.6100 ±0.0088 Δ at pub 0.0456
7
DyGFormer+RF
0.6406 ±0.0034
This paper
8
DyGFormer
0.6371 ±0.0067
This paper
9
GraphMixer
0.6298 ±0.0147
This paper
11
JODIE
actual #11 vs paper #15 4 places worse
0.6183 ±0.0270
best reported
actual #11 vs paper #15 4 places worse
paper 0.5522 ±0.0103
paper 0.5522 ±0.0103 Δ at pub 0.0661
13
TGN
actual #13 vs paper #13
0.5926 ±0.0041
actual #13 vs paper #13
paper 0.5926 ±0.0041
paper 0.5926 ±0.0041 Δ at pub 0.0374
14
DySAT
0.5690 ±0.0234
This paper
15
TGAT+RF
0.5613 ±0.0407
This paper
16
TGN+RF
0.5385 ±0.0212
This paper
18
DySAT+RF
0.5347 ±0.0151
This paper
20
GraphMixer+RF
0.5139 ±0.0251
This paper
21
JODIE+RF
0.4872 ±0.0056
This paper
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green) Best at pub — reported value improved over the best known when published