| Image appearance modeling is considered to be one of the fundamental problems in computer vision. Its successful solution has numerous applications in object tracking, recognition, surveillance, image and video processing, etc. However, the fact that the image appearance is determined by a large number of factors, including object shape, texture, pose, illumination and camera models, makes it to be a very challenging problem. In this thesis, we present a theory of analytical image appearance modeling, which is derived from fundamental physical laws, and show some applications of this theory in tracking and recognition. We rigorously prove that the image appearance space can be closely approximated to be multilinear, with the illumination and texture subspaces being trilinearly combined with the direct sum of the motion and deformation subspaces. This result allows us to understand theoretically many of the successes and limitations of the linear and multi-linear approaches existing in the computer vision literature (Principle Components Analysis, 3D Morphable Model, Active Appearance Model/Active Shape Model, Multilinear Model), and also identifies some of the conditions under which they are valid.;Starting from this theory, we show that it is possible to estimate low-dimensional manifolds that describe object appearance while retaining the geometrical information about the 3D structure of the object, termed as Geometry-Integrated Appearance Manifold (GAM). By using a combination of analytically derived geometrical models and statistical learning methods, this can be achieved using a much smaller training set than most of the existing approaches. We also show how to estimate, accurately and efficiently, the parameters of the GAM model through an inverse compositional (IC) tracking framework. We prove the theoretical convergence of this method and show that it leads to significant reduction in computational burden.;One of the most important applications of image appearance models is in object recognition. In this thesis, we present an analysis-by-synthesis framework for face recognition from video sequences that is robust to large changes in facial pose and lighting conditions. This method is based on the analytical image appearance model and the IC tracking framework. The method can handle situations where the pose and lighting conditions in the training and testing data are completely disjoint. We evaluate the algorithm on a face video dataset, and compare against image-based recognition algorithms. |
