Automatic grasping control of mobile robot based on monocular vision

Vision-based grasping control shows great flexibility and accuracy particularly in dynamic environment. This paper designs an automatic grasping control system with an eye-to-hand monocular camera and a mobile robot. The grasping control manipulation consists of coarse positioning stage and fine grasping stage. In coarse positioning stage, the mobile robot is guided to the location that placing the target based on the integrated vision navigation. In fine grasping stage, the target detection network and the target contour feature extraction strategy are elaborately designed to ensure the accuracy and efficiency of contour detection. In particular, the detection network based on improved Single Shot MultiBox Detector (SSD) is established to accurately detect the target in real-time. The obtained bounding box including the target from the detection network is set the region of interested (ROI). Then the color segmentation and morphological operation are well combined to extract the contour feature of the target, which improves the accuracy of target contour extraction. The pose of the target is estimated based on Perspective-n-Point (PNP) algorithm. Besides, a target tracking controller based on visual servoing is designed considering the movement of the mobile robot in grasping process. Numerous practical experiments are conducted to verify the effectiveness of the proposed dynamic grasping control method.