CHAOTIC DYNAMICS AND THE GEOMETRY OF THE ERROR SURFACE IN NEURAL NETWORKS

We have observed transient periodic and chaotic oscillations in the learning process of a class of multi-layered neural networks called perceptrons. Based on the geometric picture of the widening ravines in the error surface, we have derived a delayed logistic mapping describing observed complex oscillations from fast dynamics transverse to the ravine and have shown that complex dynamics arises through delayed period doubling bifurcations. This illustrates that transient dynamics can be used to extract information on the geometry of the error surface in neural networks.