A Deep Dilated Convolutional Self-attention Model for Multimodal Human Activity Recognition

Wearable-sensor-based Human Activity Recognition (HAR) has long been a hot topic in ubiquitous computing, which is benefit by the success of deep learning algorithms. The critical difficulties in multimodal sensing environments are how to represent the spatial-temporal dependencies while concurrently extracting features with high characterization. In this work, we propose a self-attention based deep dilated convolution network. Our method uses two channels, named temporal channel and spatial channel, respectively, to extract the readings-over-time and time-over-readings features from sensor signals. The self-attention mechanism helps directly capture the long time dependence of sensor signals. To extract local features while expanding the receptive field and avoiding information loss caused by pooling and upsampling, we use deep dilated convolution, which expanding the receptive field and avoiding information loss caused by pooling and upsampling. Extensive experiments on a self-built dataset and two available benchmark datasets (PAMAP2, OPPORTUNITY) reveal that the effectiveness of our proposed model is more competitive than the state-of-the-art methods in HAR tasks.