Visual feedback control of a robot in an unknown environment (learning control using neural networks)

In this paper, a visual feedback control approach based on neural networks is presented for a robot with a camera installed on its end-effector to trace an object in an unknown environment. First, the one-to-one mapping relations between the image feature domain of the object to the joint angle domain of the robot are derived. Second, a method is proposed to generate a desired trajectory of the robot by measuring the image feature parameters of the object. Third, a multilayer neural network is used for off-line learning of the mapping relations so as to produce on-line the reference inputs for the robot. Fourth, a learning controller based on a multilayer neural network is designed for realizing the visual feedback control of the robot. Last, the effectiveness of the present approach is verified by tracing a curved line using a 6-degrees-of-freedom robot with a CCD camera installed on its end-effector. The present approach does not necessitate the tedious calibration of the CCD camera and the complicated coordinate transformations.