Realtime learning algorithm for recurrent neural networks

In recent years, recurrent networks have been attracting attention as a processing mechanism of spatiotemporal patterns by neural networks. Although learning algorithms based on the steepest descent method have been proposed by Williams and Zipser and by Pearlmutter, that of Williams and Zipser has a problem of increased computational complexity for large-scale networks, and Pearlmutter's has a flaw in that learning cannot be completed in a real time. In this paper, gradients of objective functionals are derived by variational calculus and a real-time computation method of gradients is proposed. With this method, it is expected that the real-time learning, requiring less computational complexity compared to conventional learning algorithms, will become possible.