Real Time Interaction with Mobile Robots using Hand Gestures

We developed a robust real time hand gesture based interaction system to effectively communicate with a mobile robot which can operate in an outdoor environment. The system enables the user to operate a mobile robot using hand gesture based commands. In particular the system offers direct on site interaction providing better perception of environment to the user. To overcome the illumination challenges in outdoors, the system operates on depth images. Processed depth images are given as input to a convolutional neural network which is trained to detect static hand gestures. The system is evaluated in real world experiments on a mobile robot to show the operational efficiency in outdoor environment.