Sparsely Connected Neural Networks in FPGA for Handwritten Digit Recognition

Deep convolutional neural networks provide state-of-theart results for image classification tasks [1]. Due to the high amount of floating point operations, their implementation in embedded systems is still a challenge, but the rewards in case of success are significant. Embedded systems based on FPGA provide a much more efficient solution in terms of power, size and cost when compared with the alternatives (GPUs, workstations). This work presents an ongoing research aiming at developing new design methods capable of facilitating the integration of neural networks in image processing applications executing in FPGA. It has been shown [2] that L1 regularization can be used during the training phase of neural networks to reduce the number of floating point operations in multi-layer perceptrons. In this work we further analyze the impact of L1 regularization in other kinds of neural networks and conclude that pre-processing the data with convolutional layers in the FPGA improve not only the accuracy of the system but also allows for further reduction in floating point operations in the subsequent fully connected layers.