Graph Contrastive Learning with Personalized Augmentation
Abstract
Graph contrastive learning (GCL) has emerged as an effective tool for learning unsupervised representations of graphs. The key idea is to maximize the agreement between two augmented views of each graph via data augmentation. Existing GCL models mainly focus on applying identical augmentation strategies for all graphs within a given scenario. However, real-world graphs are often not monomorphic but abstractions of diverse natures. Even within the same scenario (e.g., macromolecules and online communities), different graphs might need diverse augmentations to perform effective GCL. Thus, blindly augmenting all graphs without considering their individual characteristics may undermine the performance of GCL arts. To deal with this, we propose the first principled framework, termed as Graph contrastive learning with Personalized Augmentation (GPA), to advance conventional GCL by allowing each graph to choose its own suitable augmentation operations. In essence, GPA infers tailored augmentation strategies for each graph based on its topology and node attributes via a learnable augmentation selector, which is a plug-and-play module and can be effectively trained with downstream GCL models end-to-end. Extensive experiments across 11 benchmark graphs from different types and domains demonstrate the superiority of GPA against state-of-the-art competitors. Moreover, by visualizing the learned augmentation distributions across different types of datasets, we show that GPA can effectively identify the most suitable augmentations for each graph based on its characteristics.
Benchmark Results
Tested on 5 datasets 9 baseline values reported too low
Tables show presented results in global context (best 3 papers + GCN + MLP).
TU Dortmund
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | MSH-GNN | 0.9910
±0.0030 | |
| 2 | ID-GNN+BEC | 0.9830
±0.0360 | |
| 3 | SupCosine | 0.9830
±0.0250 | |
| 39 | GIN | 0.9330
±0.0290 | |
| 178 | AD-GCL | 0.9028
±0.0401 best reported | best at pub: Graph Self-Supervised Learning: A Survey (2021) best today: Unifying Graph Contrastive Learning via Graph Message Augmentation (2024) Difference within reported uncertainty; not counted as underperformance. |
| 221 | GPA | 0.8968
±0.0080 | This paper |
| 222 | JOAOv2 | 0.8967
±0.0524 best reported | best at pub: Graph Contrastive Learning Automated (ICML 2021) best today: Unifying Graph Contrastive Learning via Graph Message Augmentation (2024) Difference within reported uncertainty; not counted as underperformance. |
| 257 | InfoGraph | 0.8930
±0.0514 best reported | best at pub: Graph Contrastive Learning Automated (ICML 2021) best today: Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization (KDD 2023) Difference within reported uncertainty; not counted as underperformance. |
| 351 | JOAO | 0.8770
±0.0080 best reported | best at pub: Graph Self-Supervised Learning: A Survey (2021) best today: Learning Graph Augmentations to Learn Graph Representations (2022) Difference within reported uncertainty; not counted as underperformance. |
| 354 | GraphCL | 0.8766
±0.0103 best reported | best at pub: Graph Masked Autoencoders with Transformers (2022) Difference within reported uncertainty; not counted as underperformance. |
| 403 | MLP | 0.8660
±0.0495 |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9954 | |
| 2 | GIN | 0.9934 | |
| 3 | GCN | 0.9727 | |
| 104 | GraphCL | 0.8250
±0.0013 best reported | best at pub: Label-invariant Augmentation for Semi-Supervised Graph Classification (NeurIPS 2022) |
| 131 | GraphSAGE | 0.8200
±0.0100 | |
| 146 | JOAOv2 | 0.8170
±0.0026 best reported | best at pub: Label-invariant Augmentation for Semi-Supervised Graph Classification (NeurIPS 2022) |
| 201 | GPA | 0.8042
±0.0041 | This paper |
| 236 | GAE | 0.7978
±0.0017 best reported | best at pub: Label-invariant Augmentation for Semi-Supervised Graph Classification (NeurIPS 2022) |
| 301 | JOAO | 0.7840
±0.0050 best reported | best at pub: Omni-Granular Ego-Semantic Propagation for Self-Supervised Graph Representation Learning (ICML 2022) Difference within reported uncertainty; not counted as underperformance. |
| 338 | InfoGraph | 0.7750
±0.0074 best reported | best at pub: Graph Masked Autoencoders with Transformers (2022) Difference within reported uncertainty; not counted as underperformance. |
| 409 | AD-GCL | 0.7518
±0.0031 | prev best: Let Invariant Rationale Discovery Inspire Graph Contrastive Learning (ICML 2022) |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | WL-MLP | 0.9792 | |
| 2 | MSH-GNN | 0.9640
±0.0070 | |
| 3 | CoCN | 0.8722
±0.0013 | |
| 23 | GCN | 0.8260
±0.0220 | |
| 32 | JOAOv2 | 0.8216
±0.0017 best reported | best at pub: Label-invariant Augmentation for Semi-Supervised Graph Classification (NeurIPS 2022) |
| 80 | GraphCL | 0.8108
±0.0017 best reported | best at pub: Label-invariant Augmentation for Semi-Supervised Graph Classification (NeurIPS 2022) |
| 91 | GAE | 0.8078
±0.0012 best reported | best at pub: Label-invariant Augmentation for Semi-Supervised Graph Classification (NeurIPS 2022) |
| 146 | InfoGraph | 0.7940 best reported | best at pub: CogDL: A Comprehensive Library for Graph Deep Learning (WWW 2021) |
| 151 | MLP | 0.7920
±0.0240 | |
| 226 | GPA | 0.7617
±0.0010 | This paper |
| 233 | AD-GCL | 0.7582
±0.0026 | prev best: Graph Self-Supervised Learning: A Survey (2021) |
| 248 | JOAO | 0.7535
±0.0093 best reported | best at pub: Adversarial Graph Augmentation to Improve Graph Contrastive Learning (NeurIPS 2021) best today: Spectral Augmentation for Self-Supervised Learning on Graphs (ICLR 2022) Difference within reported uncertainty; not counted as underperformance. |
| # | Model | Accuracy ↑ | Paper |
|---|---|---|---|
| 1 | OEPG | 0.9630
±0.0090 | |
| 2 | MSH-GNN | 0.9580
±0.0090 | |
| 3 | GNN-AK+ | 0.9480
±0.0080 | |
| 13 | GIN | 0.9240
±0.0250 | |
| 23 | GraphCL | 0.9169
±0.0070 best reported | best at pub: Graph Contrastive Learning Automated (ICML 2021) best today: Graph Contrastive Learning with Cohesive Subgraph Awareness (WWW 2024) |
| 24 | JOAO | 0.8814
±0.0025 | prev best: Graph Self-Supervised Learning: A Survey (2021) best today: Graph Contrastive Learning with Cohesive Subgraph Awareness (WWW 2024) |
| 37 | AD-GCL | 0.9010
±0.0015 | prev best: Let Invariant Rationale Discovery Inspire Graph Contrastive Learning (ICML 2022) Improvement within reported uncertainty; not highlighted. |
| 39 | GPA | 0.8999
±0.0032 | This paper |
| 46 | InfoGraph | 0.8866
±0.0095 | prev best: Graph Self-Supervised Learning: A Survey (2021) |
| 49 | JOAOv2 | 0.8879
±0.0065 | This paper |
| 55 | GAE | 0.8769
±0.0040 | prev best: Omni-Granular Ego-Semantic Propagation for Self-Supervised Graph Representation Learning (ICML 2022) Improvement within reported uncertainty; not highlighted. |
OGB
| # | Model | ROC-AUC ↑ | Paper |
|---|---|---|---|
| 1 | Multi-RF Fusion with Multi-GNN Blending | 0.8476
±0.0002 | |
| 2 | HyperFusion | 0.8475
±0.0003 | |
| 3 | PAS+FPs | 0.8420
±0.0015 | |
| 9 | GatedGCN | 0.8090
±0.0160 | |
| 109 | GIN | 0.7780
±0.0182 | |
| 250 | JOAOv2 | 0.7440
±0.0140 best reported | best at pub: Automated Data Augmentations for Graph Classification (ICLR 2022) |
| 287 | GraphCL | 0.7060
±0.0160 best reported | best at pub: Omni-Granular Ego-Semantic Propagation for Self-Supervised Graph Representation Learning (ICML 2022) |
| 306 | GPA | 0.6076
±0.0101 | This paper |
