Crowded pose-guided multi-task learning for instance-level human parsing

Instance-level human parsing remains challenging due to the similarity between human instances and background, complex interactions, and various poses. Aiming at assigning each human-related pixel a semantic label and associate each label with the corresponding instance simultaneously, a new top-down method based on multi-task learning guided by crowded pose estimation is proposed to learn instance-level human semantic part information. Firstly, we introduce a path attention feature pyramid to learn more robust multi-scale shared semantic features by changing the feature propagation to concatenation and increasing channel attention at each layer in order to solve the problem of complex background. Secondly, by improving the learned shared features via spatial attention and RC-ASPP, we design an instance-agnostic human parsing module to learn body part segmentation and edge information. In addition, we design a Mask-RCNN-based crowded pose estimation module that uses D-SPPE and hierarchical association rules to obtain pose information. Finally, we define fusion strategy and multi-task learning loss to fuse different semantic features and instance features, which can learn the final instance-level human parsing results in an end-to-end manner. Extensive experimental results on PASCAL-Person-Part and MHPv2.0 dataset verify the effectiveness of our proposed method that outperforms most of state-of-the-art methods.