Measuring the ecological validity of grouping and figure-ground cues

We begin with a systematic study of the information contained in locally computable cues about the presence and ownership of boundaries in natural scenes. A large, diverse set of images, each of which has been segmented into objects and surfaces by multiple human observers, captures the ecological statistics of boundary locations. Figure/ground relationships between neighboring regions have also been recorded for a subset of the segmented images. The degree of inter-subject consistency provides a target level of performance for our computational models. We describe algorithms for boundary detection and figure/ground assignment based on local cues and evaluate their ability to predict the human segmentations. To gauge whether these models are exploiting all the locally available information, additional human subjects provided boundary and figure/ground judgments on a set of image patches where context was synthetically limited by a circular aperture. We find that the lower-bound given by combinations of our local cues nearly achieves the "upper-bound" given by human observers with only local context but falls short of global human performance.; We also investigate simple mechanisms for integrating local grouping cues over an entire image. Gradient and patch based measurements are combined in a statistically optimal manner to yield an estimate of similarity between each pair of pixels in an image. We use the ground-truth segmentations in order to evaluate the relative power of different cues in predicting similarity. The set of pairwise similarities for an image may be combined by seeking a partition into segments that have large intra-segment similarity and small inter-segment similarity. We evaluate the extent to which spectral approximations for achieving such a partition are able to perform useful integration. By directly comparing both the input and output of spectral clustering to ground-truth segmentations, we give an empirical characterization of when such "globalization" works.