Alpha divergence minimization in multi-class Gaussian process classification

This paper analyzes the minimization of alpha-divergences in the context of multi-class Gaussian process classification. For this task, several methods are explored, including memory and computationally efficient variants of the Power Expectation Propagation algorithm, which allow for efficient training using stochastic gradients and mini-batches. When these methods are used for training, very large datasets (several millions of instances) can be considered. The proposed methods are also very general as they can interpolate between other popular approaches for approximate inference based on Expectation Propagation (EP) (alpha -> 1) and Variational Bayes (VB) (alpha -> 0) simply by varying the alpha parameter. An exhaustive empirical evaluation analyzes the generalization properties of each of the proposed methods for different values of the alpha parameter. The results obtained show that one can do better than EP and VB by considering intermediate values of alpha. (C) 2019 Elsevier B.V. All rights reserved.