Moving object segmentation and dynamic scene reconstruction using two frames

We present an iterative algorithm for segmenting independently moving objects and refining and updating a coarse depth map of the scene under unconstrained camera motion (translation and rotation) with the assumption that the independently moving objects undergoes pure translation. Given a coarse depth map acquired by a range-finder or extracted from a Digital Elevation Map (DEM), the ego-motion is estimated by combining a global ego-motion constraint and a local brightness constancy constraint using least median of squares (LMedS) which treats independently moving objects as outliers. Using the estimated camera motion and the available depth estimate, motion of the 3D points is compensated. We utilize the fact that the resulting surface parallax field is an epipolar field and use a corresponding parametric model to estimate the parallax vectors for all pixels. We use the previous motion estimate to get the epipolar direction and hence pixels where the parallax direction is not aligned towards the epipolar direction are segmented out as moving points. The depth map for static pixels is refined using the estimated parallax vectors. All segmented regions are removed for robustly estimating the ego-motion in subsequent iterations. A parametric flow model is fitted to the segmented regions and their 3D motion is estimated using subspace analysis. We present experimental results using both synthetic and real data to validate the effectiveness of the proposed algorithm.