Optimal feature extraction for normally distributed data

In this paper, we propose an optimal feature extraction method for normally distributed data. The feature extraction algorithm is optimal in the sense that we search the whole feature space to find a set of features which give the smallest classification error for the Gaussian ML classifier. Initially, we start with an arbitrary feature vector. Assuming that the feature vector is used for classification, we compute the classification error. Then we move the feature vector slightly in the direction so that the classification error decreases most rapidly. This can be done by taking gradient. We propose two search methods, sequential search and global search. In the sequential search, if more features are needed, we try to find an additional feature which gives the best classification accuracy with the already chosen features. In the global search, we are not restricted to use the already chosen features. Experiment results show that the proposed method outperforms the conventional feature extraction algorithms.