Similar Finger Gesture Recognition using Triplet-loss Networks

We propose an efficient gesture recognition method for continuous finger gestures in untrimmed videos. We aim to discriminate similar finger gestures such as flicking. This type of gestures which are conducted only by the orientation and movement of the fingers tends to be similar, making them difficult for a correct classification since a clear temporal boundary of each target gesture is ambiguous. Thus, the recognition should focus on the accuracy to find the temporal boundaries of the target gestures. We proposed a framework based on a triplet-loss network which learns to decrease the distance of true positive boundaries while increasing that of false positive ones. Finally, we adopt a temporal representation of the segmented gesture using a stack of feature maps for gesture classification. Real-time processing and high performance are achieved with relatively compact deep learning models, which are evaluated on a new dataset of vehicle driver finger gestures. Our approach outperforms the results of previous works for online temporal segmentation and gesture classification, and it can run in real-time at 53 fps.