FCN based preprocessing for exemplar-based face sketch synthesis

Most of the current exemplar-based face sketch synthesis approaches directly synthesize face sketches from face photos. However, due to the great difference between face photos and sketches, as well as the cluttered backgrounds in photo images, there tends to be some noise, deformation and missing parts on the synthesized face sketches by most of the exemplar-based methods. Besides, most exemplar-based methods exist a common problem: they only produce satisfactory results when training and test samples originate from the same dataset. To address these issues, in this paper we propose a simple but effective method which consists of two stages: the preprocessing stage and the sketch synthesis stage. In the preprocessing stage, we first design a fully convolutional neural network for preprocessing (pFCN). To fit the preprocessing task, the pFCN is trained by an L1 based total loss function, which is simple yet could enhance the facial features. Then the full-size photo is fed to the well-trained pFCN to generate the feature map, which we call a semi sketch since it bridges the discrepancy between photo and sketch. At the sketch synthesis stage, the semi-sketches and an existing exemplar-based method are employed to synthesize the final sketches. Extensive experiments on public face sketch datasets verify that the proposed two-stage method improves the sketch synthesis quality of the state-of-the-art exemplar-based methods in terms of both recognition accuracy and perceptual quality. In addition, the experiments on cross-dataset indicate that the proposed method provides a new means for strengthening the generalization ability of the exemplar-based method. (C) 2019 Elsevier B.V. All rights reserved.