Object Segmentation in Multi-view Video via Color, Depth and Motion Cues

In the light of dense depth map estimation, motion estimation and object segmentation, the research on multi-view video (MVV) content has becoming increasingly popular due to its wide application areas in the near future. In this work, object segmentation problem is studied by additional cues due to depth and motion fields. Segmentation is achieved by modeling images as graphical models and performing popular Normalized Cuts method with some modifications. In the graphical models, each node is represented by a group of pixels, instead of individual pixels, which are obtained as a result of over-segmentation of the images. These over-segmented regions are also utilized in the dense depth map estimation step; in which 3D planar models are assigned for each of these sub-regions. Moreover, optical flow is estimated based on affine motion assumption for these regions. The links of the graphical models are weighted according to the depth, motion and color similarities of the pixel groups due to these regions. Once the links are obtained, segmentation is achieved by recursively bi-partitioning the graph via removing the weak links. Experiments indicate that the proposed framework achieves precise segmentation results for MVV sequences.