Multi-Augmentation Contrastive Learning as Multi-Objective Optimization for Graph Neural Networks

Recently self-supervised learning is gaining popularity for Graph Neural Networks (GNN) by leveraging unlabeled data. Augmentation plays a key role in self-supervision. While there is a common set of image augmentation methods that preserve image labels in general, graph augmentation methods do not guarantee consistent graph semantics and are usually domain dependent. Existing self-supervised GNN models often handpick a small set of augmentation techniques that limit the performance of the model. In this paper, we propose a common set of graph augmentation methods to a wide range of GNN tasks, and rely on the Pareto optimality to select and balance among these possibly conflicting augmented versions, called Pareto Graph Contrastive Learning (PGCL) framework. We show that while random selection of the same set of augmentation leads to slow convergence or even divergence, PGCL converges much faster with lower error rate. Extensive experiments on multiple datasets of different domains and scales demonstrate superior or comparable performance of PGCL.