SGL-PT: A Strong Graph Learner with Graph Prompt Tuning

Recently, much exertion has been paid to design graph self-supervised methods to obtain generalized pre-trained models, and adapt pre-trained models onto downstream tasks through standard fine-tuning. But the gap between pretext and downstream tasks can limit pre-trained models' potential, leading to negative transfer. Meanwhile, prompt tuning has seen emerging success in natural language processing (NLP) by aligning pre-training and fine-tuning with consistent training objectives. In this paper, we identify the challenges for graph prompt tuning: The first is the lack of a strong and universal pre-training task across sundry pre-training methods in graph domain. Such a task should be easily emulated by downstream tasks, akin to the Masked Language Modeling (MLM) in the NLP domain. The second challenge lies in the difficulty of designing a consistent training objective for both pre-training and downstream tasks due to the inherent abstraction of graph data. To overcome above obstacles, we propose a novel framework named SGL-PT which follows the learning strategy ``Pre-train, Prompt, and Predict''. Specifically, we raise a strong and universal pre-training task coined as SGL that acquires the complementary merits of generative and contrastive self-supervised graph learning. And motivated by prompt design in NLP, we reformulate the downstream task as maksed node prediction by designing a novel verbalizer-free prompting function, resulting in unifying the objectives of pre-text and downstream tasks. Empirical results show that our pre-training method surpasses other baselines under unsupervised setting, and our prompt tuning method can significantly facilitate models on biological datasets over standard fine-tuning and other graph prompt methods.