A Local-Global Estimator Based on Large Kernel CNN and Transformer for Human Pose Estimation and Running Pose Measurement

Running pose in the crowd can serve as an early warning of most abnormal events (e.g., chasing, fleeing, and robbing), which can be achieved by human behavior analysis based on human pose measurement. Although deep convolutional neural networks (CNNs) have achieved impressive progress on human pose estimation (HPE), how to further improve the trade-off between estimation accuracy and speed remains an open issue. In this work, we first propose an efficient local-global estimator for HPE (called LGPose). Then based on the keypoints estimated by our LGPose, a simple regression model is defined using the geometry of the joints to achieve fast and accurate running pose measurement. To model the relationships between the human keypoints, a visual transformer (ViT) encoder is adopted to learn the long-range interdependencies between them at the pixel level. However, the operation of the transformer encoder is based on sequence processing that linearly projects the 2-D image patches to 1-D tokens. It loses the important local information. Yet, locality is crucial since it has relevance to lines, edges, and shapes. To learn the locality, we design effective CNN modules, rather than the original fully-connected network (FCN), into the feedforward module of ViT. Experiments on the MPII and COCO Keypoint val2017 datasets show that the proposed LGPose achieves the best trade-off among the compared state-of-the-art methods. Moreover, we build a lightweight running movement dataset to verify the effectiveness of our LGPose. Based on the human pose estimated by our LGPose, we propose a regression model to measure running pose with an accuracy of 86.4% without training any other classifier. Our source codes and running dataset will be made publicly available.