3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method

A workpiece recognition and picking system based on instance segmentation network and iterative optimization method is proposed for object detection and pose estimation of scattered and stacked texture-less industrial objects. This system consists of three modules, including image acquisition module, target detection module and pose estimation module. In image acquisition module, a dual RGB-D (RGB-depth) camera structure is designed to get higher quality depth data by merging three depth images. The target detection module modifies the instance segmentation network Mask R-CNN (regionbased convolutional neural network). The modified network takes RGB images and HHA (horizontal disparity, height above ground, angle with gravity) features containing three-dimensional information as input, and adds STN (spatial transformer network) modules inside to improve the segmentation performance of texture-less objects. Then the module can combine point cloud information to obtain the target point cloud. On this basis, the improved 4PCS (4-points congruent set) algorithm and ICP (iterative closest point) algorithm are used in pose estimation module to match the segmented point cloud with the target model and fine-tune the pose, and thus the final result of pose estimation is obtained. The robots accomplish picking action according to the estimated pose. The experiment results on our workpiece dataset and the actual picking system indicate that the proposed method can achieve fast target recognition and pose estimation for scattered and stacked objects with different shapes and less textures. Its performance can meet the requirements of practical applications with 1 mm position error and 1° angle error. © 2019, Science Press. All right reserved.