Periodic Activation Functions in Memristor-based Analog Neural Networks

This work explores the use of periodic activation functions in memristor-based analog neural networks. We propose a hardware neuron based on a folding amplifier that produces a periodic output voltage. Furthermore, the amplifier's fold factor be adjusted to change the number of low-to-high or highto-low output voltage transitions. We also propose a memristorbased synapse circuit and training circuitry for realizing the Perceptron learning rule. Behavioral models of our circuits were developed for simulating a single-layer, single-output feedforward neural network. The network was trained to detect the edges of a grayscale image. Our results show that neurons with a single fold-with an activation function similar to a sigmoidal activation function-perform the worst for this application, since they are unable to learn functions with multiple decision boundaries. Conversely, the 4-fold neuron performs the best (up to similar to 65% better than the 1-fold neuron), as its activation function is periodic, and it is able to learn functions with four decision boundaries.