A two-step approach to multiple facial feature tracking: Temporal particle filter and spatial belief propagation

It is challenging to track multiple facial features simultaneously when rich expressions are presented on a face. We propose a two-step solution. In the first step, several independent CONDENSATION-style particle filters are utilized to track each facial feature in temporal domain. Particle filters are very effective for visual tracking problems; however multiple independent trackers ignore the spatial constraints and the natural relationships among facial features. In the second step, we use Bayesian inference belief propagation to infer each facial feature's contour in spatial domain, in which we learn beforehand the relationships among contours of facial features with the help of a large facial expression database. The experimental results show that our algorithm can robustly track multiple facial features simultaneously, while there are large inter-frame motions with expression change.