Hierarchical neural networks with exponential storage

A hierarchical neural network model which is capable of storing and retrieving an exponential number of ultrametrically correlated state vectors is presented. Based on a spin glass analogy, the fully connected hierarchical model organizes neurons into a multitiered cluster hierarchy. Relaxation of the network is accomplished in the absence of external stochastic noise (i.e. at zero temperature) by what is effectively a tunneling process and can follow either a bottom-up or top-down updating procedure. An optimum cluster size is shown to exist which maximizes the total number of states stored by the network. The information capacity of the hierarchical model is also determined and contrasted with that of the standard Hopfield model. Numerical simulations of various hierarchical systems are presented.