A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

Yang Xiang, Li Fan, Chenke Yin, Chengtao Ji

Main figure for A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

Abstract

Foundation models in language and vision benefit from a unified discrete token interface that converts raw inputs into sequences for scalable pre-training and inference. For graphs, an effective tokenizer should yield reusable discrete codes that capture both node semantics and relational structure across scales, yet prior quantization-based graph tokenizers typically combine residual vector quantization (RVQ) levels with fixed rules and often focus on a single structural view, limiting cross-task transfer. We present a hierarchical quantized tokenization framework with task-conditioned routing and dual-view token streams. It produces multi-scale codes and two synchronized sequences: a local stream that preserves node-level information and a diffusion-style multi-hop stream that summarizes connectivity. A lightweight router learns task-dependent mixtures over RVQ depths to select an appropriate granularity, while a gated cross-attention module aligns and fuses the two streams into a single token sequence without altering the downstream backbone encoder. Experiments on node classification and link prediction show consistent gains over strong quantized baselines at matched compute, with ablations verifying contributions from hierarchical quantization, adaptive routing, and fusion.

Benchmark Results

Tested on 3 datasets

Tables show presented results in global context (best 3 papers + GCN + MLP).

Classic

#ModelMAE ↓Paper
1
TIGT
0.0140 ±0.0010
2
FloydNet
0.0160
3
GSSC
0.0190 ±0.0010
11
GRIT
actual #11 vs paper #58 47 places worse
0.0230 ±0.0010
best reported
actual #11 vs paper #58 47 places worse
paper 0.0590 ±0.0020
paper 0.0590 ±0.0020 Δ at pub 0.0360 Δ today 0.0360
55
TAU-GRIT
0.0510 ±0.0010
This paper
69
GIN
0.0880 ±0.0020

GNNBenchmark

#ModelAccuracy ↑Paper
1
Schrödinger
0.9913 ±0.0004
2
CNN
0.9907 ±0.0007
3
MPNN
0.9895 ±0.0006
11
GatedGCN
0.9855 ±0.0002
32
GIN
0.9833 ±0.0011
40
TAU-GRIT
0.9821 ±0.0010
This paper
47
EGT
0.9817 ±0.0009
This paper
56
GRIT
0.9811 ±0.0011
This paper
62
GraphGPS
0.9805 ±0.0013
This paper
126
MLP
0.9534 ±0.0014
#ModelAccuracy ↑Paper
1
RandAlign
0.7942 ±0.0007
2
differential encoding
0.7907 ±0.0027
3
PPGT
0.7856 ±0.0070
4
TAU-GRIT
0.7778 ±0.0049
This paper
7
GRIT
0.7647 ±0.0088
This paper
50
GraphGPS
0.7230 ±0.0036
This paper
62
GatedGCN
0.7133 ±0.0039
82
EGT
0.6870 ±0.0041
This paper
92
GraphSAGE
0.6690 ±0.0020
126
MLP
0.5634 ±0.0018
SOTA performance (not significantly different from the best) Proposed method (this paper) Baseline / comparison (this paper) Global context (other papers) Best at pub — reported result worse than the best known when published (best reported in green)