SELF-REPRESENTATION CONVOLUTIONAL NEURAL NETWORKS

The traditional convolutional neural networks (CNNs) learn numbers of fixed kernels(filters), which are used to obtain the representations of fixed patterns. Therefore, the knowledge representations of CNNs are limited to the number of kernels. In this paper, we present a Self-Representation Convolutional (SRC) layer to obtain richer knowledge representations of images by fully considering the self-similarity between adjacent pixels. SRC layer comprises a learnable local correlation measurement which measures the importance of adjacent pixels to the current pixel and two learnable linear parameters that perform linear projection on adjacent pixels and the weighted sum vectors, respectively. Compared with regular convolutional layers, the SRC layers can not only obtain comparable knowledge representations, but also reduce by a factor of 3x to 56x in the number of learnable parameters. Empirically, CNNs with SRC layers, called Self-Representation Convolutional Neural Networks (SRCNN), achieve strong performances on a range of visual datasets (SVHN, CIFAR-10 and CIFAR-100) while enjoying significant parameters and FLOPs savings.