Point Set Registration With Semantic Region Association Using Cascaded Expectation Maximization

We introduce a new solution to point set registration, a fundamental geometric problem occurring in many computer vision and robotics applications. We consider the specific case in which the point sets are segmented into semantically annotated parts. Such information may for example come from object detection or instance-level semantic segmentation in a registered RGB image. Existing methods incorporate the additional information to restrict or re-weight the point-pair associations occurring throughout the registration process. We introduce a novel hierarchical association framework for a simultaneous inference of semantic region association likelihoods. The formulation is elegantly solved using cascaded expectation-maximization. We conclude by demonstrating a substantial improvement over existing alternatives on open RGBD datasets.