Attention-enabled hybrid convolutional neural network for enhancing human-robot collaboration through hand gesture recognition

Human-Robot Interaction serves as a framework that enables communication and collaboration between humans and robotic systems. Despite significant advances in this field, existing approaches often struggle with the complexities of hand gestures, including variability in spatial and temporal features. To overcome these limitations, we propose a novel approach integrating a hybrid Spectral-Polynomial Convolutional Neural Network with an Inception module and Long Short-Term Memory (LSTM) architecture. This hybrid model efficiently extracts spatial features, while the Inception module captures multi-scale information. Additionally, the LSTM layers identify temporal dependencies in gesture sequences. Furthermore, this work introduces a novel Context-Augmented Scaled Dot-Product Attention mechanism. This attention mechanism enhances the capability of the model to focus on relevant regions of the input data. In addition, a customizable robot architecture is designed to facilitate and enable these human- robot collaborative interactions. Our model demonstrates robust performance, achieving 99.77% accuracy on the Jochen-Triesch American Sign Language dataset, 99.23% on the Sebastien Marcel Static Hand Posture dataset, and 97.67% on a Custom Indian Sign Language dataset. The proposed model achieves state-of-the-art accuracy in hand gesture recognition for human-robot interaction.