Body landmark detection with an extremely small dataset using transfer learning

We present a new landmark detection problem on the upper body of a clothed person for tailoring purposes. This is a landmark detection problem unknown in the literature, which is in the same domain as, but different to the 'fashion' landmark detection problem where the landmarks are for classifying clothing. An existing 'attentive fashion network' (AFN) was trained using 800,000 annotated images of the DeepFashion dataset, with a base network of VGG16 pre-trained on the ImageNet dataset, to provide initial weights. To train a network for 'body' landmark detection would require a similar sized dataset. We propose a deep neural network for body landmark detection where the knowledge from an existing network was transferred and trained with an extremely small dataset of just 99 images, annotated with body landmarks. A baseline model was tested where only the fashion landmark branch was used, but retrained for body landmarks. This produced a testing error of 0.068 (normalised mean distance between the predicted landmarks and ground-truth). The error was significantly reduced by adopting the fashion landmark branch and the attention unit of AFN, but substituting the classification branch with a new body landmark detection branch for the proposed Attention-based Fashion-to-Body landmark Network (AFBN). We tested 6 variants of the proposed AFBN model with different convolutional block designs and auto-encoders for enforcing landmark relations. The trained model had a low testing error ranging from 0.022 to 0.028 over these variants. The variant with an increased number of channels and inception units with residual connections, had the best overall performance. Although AFBN and its variants were trained with a limited dataset, the performance exceeds the state-of-the-art attentive fashion network AFN (0.0534). The principle of transfer learning demonstrated here is relevant where labelled domain data are scarce providing a low solution cost of faster training of a deep neural network with a significantly small dataset. [GRAPHICS] .