Multiscale, multigranular statistical image segmentation

We consider a problem in image segmentation in which the goal is to determine and label a relatively small number of homogeneous subregions in an image scene, based on multivariate, pixelwise measurements. Motivated by current challenges in the field of remote sensing land cover characterization, we introduce a framework that allows for adaptive choice of both the spatial resolution of subregions and the categorical granularity of labels. Our framework is based on a class of models that we call mixlets, a blending of recursive dyadic partitions and finite mixture models. The first component of these models allows for the sparse representation of a spatial structure at multiple resolutions. The second component provides a natural mechanism for capturing the varying degrees of mixing of pure categories that accompany the use of different resolutions and for relating these to a user-specified hierarchy of labels at multiple granularities in a straightforward manner. A segmentation is produced in our framework by selecting an optimal mixlet model, through complexity-penalized maximum likelihood, and summarizing the information in that model with respect to the categorical hierarchy. Both theoretical and empirical evaluations of the proposed framework are presented.