Optimal Regularization Parameter Estimation for Regularized Discriminant Analysis

Regularized linear discriminant analysis (RLDA) is a popular LDA-based method for dimension reduction. Despite its good performance, how to choose the parameter of the regularizer efficiently is still unanswered, especially for multi-class situation. In this paper, we first prove that regularizing LDA is equivalent to augmenting the training set in a specific way and thereby propose an efficient model selection criterion based on the principle of maximum information preservation, extensive experiments prove the usefulness and efficiency of our method.