LOCATION AND STABILITY OF THE HIGH-GAIN EQUILIBRIA OF NONLINEAR NEURAL NETWORKS

This paper analyzes the number, location, and stability behavior of the equilibria of arbitrary nonlinear neural networks without resorting to energy arguments based on assumptions of symmetric interactions or no self-interactions. The class of networks studied consists of very general continuous-time continuous-state (CTCS) networks that contain the standard Hopfield network as a special case. The emphasis is on the case where the slopes of the sigmoidal nonlinearities become larger and larger, i.e., the high-gain limit. The following results are proved: Let H = (0, 1)n and HBAR = [0, 1]n denote the open and closed n-dimensional hypercubes, respectively, on which the neural network evolves, and let I denote the (constant) vector of external inputs. Then, as the neural sigmoid characteristics become steeper and steeper, it is shown that the following statements are true for all I except for those belonging to a set of measure zero. 1) There are only finitely many equilibria in any compact subset of H. If there are no self-interactions, then these equilibria cannot be exponentially stable, and under mild conditions they are in fact unstable. If the network has symmetric (nonlinear) interactions, whether or not it has self-interactions, then the stable manifolds of all these equilibria have the same dimension, which can be computed explicitly. If the network also has no self-interactions, then all of these equilibria are unstable. 2) There are only finitely many equilibria in any face of H. If there are no self-interactions, then there are no equilibria in an edge of H. If the network has symmetric interactions, then the stable manifolds of equilibria in parallel faces of H have the same dimension, which can be computed explicitly. If the network also has no self-interactions, then all equilibria in the faces of H are unstable. 3) A systematic procedure is given for determining which corners of H contain equilibria, and it is shown that all equilibria in the corners of H are asymptotically stable. One corollary of the above results is that the standard Hopfield network can have asymptotically stable equilibria only in the corners of H, and trajectories starting at almost all initial conditions approach the corners of H. It is important to note that the proofs here are not based on energy arguments. As a result, these results are ''hardy'' in the sense that they continue to hold even if the network-dynamics are slightly perturbed.