Joint Detection and Pose Tracking of Multi-Resolution Surfel Models in RGB-D

We propose a particle filter framework for the joint detection, pose estimation, and real-time tracking of objects in RGB-D video. We do not rely on the availability of CAD models, but employ multi-resolution surfel maps as a concise representation of object shape and texture that is acquired through SLAM. We propose to initialize the particle belief for tracking with pose votes cast from matching colored surfel-pair features at multiple resolutions. Multi-hypothesis tracking then finds the most consistent track over time. We utilize efficient registration of RGB-D images to the model to obtain improved proposals for particle filtering which greatly enhances tracking accuracy. We evaluate our approach on a publicly available RGB-D object tracking dataset, and show high rates of detection and good tracking performance with respect to various speeds of camera motion and occlusions.