Fisher vector for scene character recognition: A comprehensive evaluation

Fisher vector (FV), which could be seen as a bag of visual words (BOW) that encodes not only word counts but also higher-order statistics, works well with linear classifiers and has shown promising performance for image categorization. For character recognition, although standard BOW has been applied, the results are still not satisfactory. In this paper, we apply Fisher vector derived from Gaussian Mixture Models (GMM) based visual vocabularies on character recognition and integrate spatial information as well. We, give a comprehensive evaluation of Fisher vector with linear classifier on a series of challenging English and digits character recognition datasets, including both the handwritten and scene character recognition ones. Moreover, we also collect two Chinese scene character recognition datasets to evaluate the suitability of Fisher vector to represent Chinese characters. Through extensive experiments we make three contributions: (1) we demonstrate that FV with linear classifier could outperform most of the state-of-the-art methods for character recognition, even the CNN based ones and the superiority is more obvious when training samples are insufficient to train the networks; (2) we show that additional spatial information is very useful for character representation, especially for Chinese ones, which have more complex structures; and (3) the results also imply the potential of FV to represent new unseen categories, which is quite inspiring since it is quite difficult to collect enough training samples for large-category Chinese scene characters. (C) 2017 Elsevier Ltd. All rights reserved.