Understanding and learning of gestures through human-robot interaction

Humans can communicate each other by gestures. Even if they cannot recognize them at first, they can understand each other through interaction. This paper presents a robot system with such capability. The robot detects its user by recognizing his/her face. It can accept his/her commands given by gestures. The user may use unknown gestures to the robot. If the robot does not respond to his/her gesture, the user usually iterates the same gesture. The robot detects this repetitive pattern as an intentional gesture by which the user wants to give it some order. Then it shows a little action according to the meaning of the gesture that the robot guesses. It observes the user's reaction to its action. If he/she continues the same pattern gesture, the robot considers that its understanding is right, completing the action. It also registers the pattern as a gesture of the guessed meaning. Otherwise, it iterates the same procedure by taking another action as a candidate of the meaning. We have implemented such interactive capability on our intelligent wheelchair It is convenient that we can make it come or go by gestures when we are off. Experimental results confirm that the proposed interaction method is useful in actual complex environments where even registered gestures cannot always be recognized.