Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced Training

Besides of the existing neighbor sampling techniques applied on common Graph Neural Networks (GNNs), scalable methods allowing normal minibatch training can more easily scale to large scaled graphs. They decouple graph convolutions and other learnable transformations into preprocessing and a scalable classifier. A complex and graph structure-aware classifier is important to achieve competitive performances. By replacing redundant concatenation operation in Scalable Inception Graph Neural Networks (SIGN) with a more graph structure-aware attention mechanism, we propose Scalable and Adaptive Graph Neural Networks (SAGN). SAGN can adaptively gather neighborhood information among different hops. To further improve scalable GNNs by introducing the existing techniques applied on common GNNs for semi-supervised learning tasks, we propose Self-Label-Enhanced (SLE) training approach combining the self-training approach and label propagation in depth. We add the base model with a scalable label model. Then we iteratively train models and enhance the training set in several stages. To generate input of the label model, we apply label propagation based on one-hot encoded label vectors without inner random masking. We find out that empirically the label leakage has been effectively alleviated with enough propagation depth. The hard pseudo labels in the enhanced training set participate in label propagation with true labels, which propagates model knowledge and label information into the whole graph. Experiments on both inductive and transductive datasets demonstrate that, compared with other sampling-based and sampling-free methods, SAGN achieves better or comparable results and SLE can further improve performance.