Video indexing using MPEG motion compensation vectors

In the last years a lot of work has been done on color, textural, structural and semantic indexing of "content-based" video databases. Motion-based video indexing has been less explored, with approaches generally based on the analysis of optical flows. Compressed videos require the decompression of the sequences and the computation of optical flows, two steps computationally heavy. In this paper we propose some methods to index videos by motion features (mainly related to camera motion) and by motion-based spatial segmentation of frames, in a fully automatic way. Our idea is to use MPEG motion vectors as an alternative to optical flows. Their extraction is very simple and fast; it doesn't require a full decompression of the stream and saves us from computing optical flows. Additional computational economy comes from having one motion vector each 16x16 sub-image; this makes the algorithms faster than working with dense optical flows. Experimental results reported at the end of this paper show that MPEG motion compensation vectors are suitable for this kind of applications.