Unsupervised Learning of Categorical Segments in Image Collections

Which one comes first: segmentation or recognition? We propose a probabilistic framework for carrying out the two simultaneously. The framework combines an LDA 'bag of visual words' model for recognition, and a hybrid parametric-nonparametric model for segmentation. If applied to a collection of images, our framework can simultaneously discover the segments of each image, and the correspondence between such segments. Such segments may be thought of as the 'parts' of corresponding objects that appear in the image collection. Thus, the model may be used for learning new categories, detecting/classifying objects, and segmenting images.