A novel moving object segmentation algorithm in the H.264 compressed domain

A new algorithm to segment the moving object in the H.264 compressed domain is proposed. This algorithm mainly utilizes motion vectors that are directly extracted from the H.264 bitstreams. In order to improve the robustness of the motion vector information, firstly the intra modes and intra prediction residual energy in I frames are used for region classification. The inter prediction residual energy of P frames is used to update the classification results. Some motion vectors are set to be zero based on the results. Secondly, an adaptive motion vector filter is then used according to inter partition modes. Finally, the corresponding Gibbs potential functions are defined based on these filtered motion vectors. The maximum a posteriori(MAP) is solved by iterated conditional mode(ICM). A robust moving object label is obtained finally. Experiment results are presented to verify the efficiency and the robustness of this algorithm.