Irregular-structure tree models for image interpretation

In this dissertation, we seek to accomplish the following related goals: (1) to find a unifying framework to address localization, detection, and recognition of objects, as three sub-tasks of image-interpretation, and (2) to find a computationally efficient and reliable solution to recognition of multiple, partially occluded, alike objects in a given single image. The second problem is to date an open problem in computer vision, eluding a satisfactory solution. For this purpose, we formulate object recognition as Bayesian estimation, whereby class labels with the maximum posterior distribution are assigned to each pixel. To efficiently estimate the posterior distribution of image classes, we propose to model images with graphical models known as irregular trees.; The irregular tree specifies probability distributions over both its structure and image classes. This means that, for each image, it is necessary to infer the optimal model structure, as well as the posterior distribution of image classes. We propose several inference algorithms as a solution to this NP-hard problem (nondeterministic polynomial time), which can be viewed as variants of the Expectation-Maximization (EM) algorithm.; After inference, the model represents a forest of subtrees, each of which segments the image. That is, inference of model structure provides a solution to object localization and detection.; With respect to our second goal, we hypothesize that for a successful occluded-object recognition it is critical to explicitly analyze visible object parts. Irregular trees are convenient for such analysis, because the treatment of object parts represents merely a particular interpretation of the tree/subtree structure. We analyze the significance of irregular-tree nodes, representing object parts, with respect to recognition of an object as a whole. This information is then exploited toward the ultimate object recognition.; Empirical results demonstrate that irregular trees more accurately model images than their fixed-structure counterparts quad-trees. Also, the experiments reported herein show that our explicit treatment of object parts results in an improved recognition performance, as compared to the strategies in which object components are not explicitly accounted for.