A YOLO-GGCNN based grasping framework for mobile robots in unknown environments

To grasp a desirable object in an unknown environment, both accurate map construction and reliable visual recognition are prerequisites for the collaboration of mobile robots. We propose a multirobot cooperative simultaneous localization and mapping (SLAM)-based map fusion algorithm which is developed to construct a map of an unknown environment through multiple exploring robots with high maneuverability. The resulting map is then transmitted to the mobile robotic arm through the Robot Operating System (ROS) for the upcoming navigation step. In the grasping module, a two-step cascaded system, i.e., YOLOv4 and a generative grasping convolutional neural network (YOLO-GGCNN), is proposed to grasp any given object via the mobile robotic arm. The capture accuracy of our algorithm is 86.0%, and the detection time required for a single frame is 0.11 s. In the recognition module, the grasping prediction is finally converted into an information flow that controls the operation of the robotic arm. The proposed framework is validated through applications in various environments and comparisons with existing approaches. The results indicate that our approach allows the mobile robotic arm to selectively and accurately grasp objects of interest in unknown environments, and it is more reliable than other methods.