Simultaneous object detection and segmentation using top-down and bottom-up processing

This thesis addresses the fundamental tasks of detecting objects in images, recovering their location, and determining their silhouette shape. Identifying objects of different categories, and estimating their location and physical extent, are basic, inter-related processes in the human visual system that enable us to interact with the world around us. However, computer vision algorithms typically approach each of these as independent processes, often providing information regarding either only object category, location, or shape.; In this thesis we focus on object detection techniques that (1) enable simultaneous recovery of object location and object shape, (2) require minimal manual supervision during training, and (3) are capable of consistent performance under varying imaging conditions found in real-world scenarios.; The work described here results in the development of a unified method for simultaneously acquiring both the location and the silhouette shape of specific object categories in outdoor scenes. The proposed algorithm integrates top-down and bottom-up processing, and combines cues from these processes in a balanced manner. The framework provides the capability to incorporate both appearance and motion information, making use of low-level contour-based features, mid-level perceptual cues, and higher-level statistical analysis. A novel Markov random field formulation is presented that effectively integrate the various cues from the top-down and bottom-up processes. The algorithm attempts to leverage the natural structure of the world, thereby requiring minimal user supervision during training. Extensive experimental evaluation shows that the approach is applicable to different object categories, and is robust to challenging conditions such as large occlusions and drastic changes in viewpoint.; For static camera scenarios, we present a contour-based background-subtraction technique that can be used as a preliminary step for category-specific object detection. Utilizing both intensity and gradient information, the algorithm constructs a fuzzy representation of foreground boundaries called a Contour Saliency Map. Combined with a low-level data-driven approach for contour completion and closure, the approach is able to accurately recover object shape.; We also present object detection and segmentation approaches that combine information from visible and thermal imagery. Exploiting the complementary nature of the constituent sensors enables such algorithms to be applicable under a wider variety of imaging conditions. For object detection, we present a simple contour-based fusion scheme that extends our background-subtraction algorithm. We also introduce a new feature-selection approach for object segmentation from multiple imaging modalities. Starting from an incomplete/partial segmentation from one sensor, the proposed approach is able to automatically extract relevant information from other sensors so as to generate a complete segmentation of the object. The algorithm utilizes criteria based on Mutual Information for defining feature relevance, and does not rely on a training phase.