Dict Layer: A Structured Dictionary Layer

Dictionary learning and deep learning both have played important roles in computer vision. Dictionary learning strives to learn best representative atoms to reconstruct the source images or signals. Alternating iterative methods are developed to solve such problems. On the other hand, deep learning performs in an end-to-end mode, where both feature extraction and classification are achieved simultaneously. Obviously, the scheme of deep learning is different from that of traditional dictionary learning which is shallow and focuses more on data reconstruction. However, studies on building a deep layer-stacked model imply that there could be a relationship between them. In this paper, the relationship between dictionary learning and deep learning is studied. Dictionary learning can be viewed as a special full connection layer (FC Layer) of deep learning. According to the relationship, we try to introduce those mature improvements from dictionary learning to deep learning. Hence, a new kind of layer named as Dict Layer is introduced in this paper, where the idea of structured dictionary is adopted. In Dict Layer, neural units (coefficients) are class specified, which means the activated neural units are encouraged to be the same class. The proposed method is evaluated on M-NIST, CIFAR-10 and SVHN as an improvement of FC Layer. Experiments on AR and Extended YaleB are conducted where Dict Layer is viewed as a special form of dictionary learning method. Results show that the outputs of Dict Layer are more discriminative and class specific than that of the traditional FC Layer.