Perceptually-based texture and color features for image segmentation and retrieval

The rapid accumulation of large digital image collections has created the need for efficient and intelligent schemes for image retrieval, and especially Content Based Image Retrieval (CBIR). One of the most important and challenging components of CBIR systems is image segmentation. Since humans are the ultimate users of most CBIR systems, it is important to obtain segmentations that can be used to organize image contents according to categories that are meaningful to humans.; A new approach for color-texture image segmentation is proposed that is based on perceptual models and principles about the processing of texture and color information. First, new adaptive perceptual low-level color and texture features are derived. Then, image segmentation algorithms that combine these features are developed to obtain image segmentations that convey semantic information that can be used for content-based retrieval.; Two types of features are proposed. The first describes the local color composition in terms of spatially adaptive dominant colors, which on one hand, reflect the fact that the human visual system cannot simultaneously perceive a large number of colors, and on the other, the fact that image colors are spatially varying. The second is based on a multi-scale frequency decomposition, which approximates early processing in the human visual system. The local median energy of the subband coefficients is used as a simple but effective characterization of spatial texture. The median filter distinguishes the energy due to region boundaries from the energy of the textures themselves. Even though the estimation of the texture features requires a finite window, which limits spatial resolution, by appropriately combining the texture and color features, the proposed algorithms obtain robust, accurate, and precise segmentations.; The performance of the proposed algorithms is demonstrated in the domain of low resolution and compressed photographic images of natural scenes. Key parameters of the segmentation algorithms are determined by subjective experiments. It is shown that this perceptual tuning leads to significant improvements in performance. Comparisons with existing techniques demonstrate the advantages of the proposed approach.