Neural network learning for principal component analysis: A multistage decomposition approach

This paper presents a novel neural network model for finding the principal components of an N-dimensional data stream. This neural network consists of r (less than or equal to N) neurons, where the i-th neuron has only N - i + 1 weights and an N - i + 1 dimensional input vector, while each neuron in most of the relative classical neural networks includes N weights and an N dimensional input vector. All the neurons are trained by the NIC algorithm under the single component case([7]) so as to get a series of dimension-reducing principal components in which the dimension number of the i-th principal component is N - i + 1. In multistage dimension-reducing processing, the weight vector of i-th neuron is always orthogonal to the subspace constructed from the weight vectors of the first i - 1 neurons. By systematic reconstruction technique, we can recover all the principal components from a series of dimension-reducing ones. Its remarkable advantage is that its computational efficiency of the neural network learning based on the Novel information criterion (NIC) is improved and the weight storage is reduced, by the multistage dimension-reducing processing (multistage decomposition) for the covariance matrix or the input vector sequence. In addition, we study several important properties of the NIC learning algorithm.