Foreground prediction for bilayer segmentation of videos

Given a segmentation result (an alpha matte or a binary mask) of the former frame, foreground prediction is a process of estimating the probability that each pixel in the current frame belongs to the foreground. It plays a very important role in bilayer segmentation of videos, especially videos with non-static backgrounds. In this paper, a new foreground prediction algorithm which is called opacity propagation is proposed. It can propagate the opacity values of the former frame to the current frame by minimizing a cost function that is constructed by assuming the spatiotemporally local color smoothness of the video. Optical flow and probability density estimation based on a local color model are employed to find the corresponding pixels of two adjacent frames. An OPSIC (opacity propagation with sudden illumination changes) algorithm is also proposed which is an improvement of our proposed opacity propagation algorithm because it adds a simple color transformation model. As far as we know, this is the first algorithm that can predict the foreground accurately when the illumination changes suddenly. The opacity map (OM) generated by the opacity propagation algorithm is usually more accurate than the previously used probability map (PM). The experiments demonstrate the effectiveness of our algorithm. (C) 2011 Elsevier B.V. All rights reserved.