Object affordance detection with boundary-preserving network for robotic manipulation tasks

Object affordance detection aims to identify, locate and segment the functional regions of objects, so that robots can understand and manipulate objects like humans. The affordance detection task has two main challenges: (1) Due to the need to provide accurate positioning information for the robot to manipulate objects, the affordance segmentation results are required to have high boundary quality. (2) Different kinds of objects have significant differences in appearances, but may have the same affordance. Correspondingly, parts with the same appearance may have different affordances. The existing methods regard affordance detection as an image segmentation problem, without focusing on the boundary quality of detection results. In addition, most of the existing methods do not consider the potential relationship between object categories and object affordances. Aiming at the above problems, we propose a boundary-preserving network (BPN) to provide affordance masks with better boundary quality for robots to manipulate objects. Our framework contains three new components: the IoU (Intersection-over-Union) branch, the affordance boundary branch and the relationship attention module. The IoU branch is used to predict the IoU score of each object bounding box. The affordance boundary branch is used to guide the network to learn the boundary features of objects. The relationship attention module is used to enhance the feature representation capability of the network by exploring the potential relationship between object categories and object affordances. Experiments show that our method is helpful to improve the boundary quality of the predicted affordance masks. On the IIT-AFF dataset, the performance of the proposed BPN is 2.32% (F-score) and 2.89% (F-score) higher than that of the strong baseline in terms of affordance masks and the boundaries of affordance masks, respectively. Furthermore, the real-world robot manipulation experiments show that the proposed BPN can provide accurate affordance information for robots to manipulate objects.