Research On Outdoor Illumination Estimation Based On Basis Image Decomposition

In recent years, virtual reality grows rapidly, which is of great theoretical and applied significance. Augmented reality is developed on the basis of virtual reality. With the aid of computer graphics technology, interactive technology, sensor technology, three-dimensional display technology and computer vision technology, the augmented reality system can real-timely overlay computer-generated objects onto video images of real scenes, to enhance the user’s sensory experience. Along with the advance of mature concept and technology, augmented reality has been applied to research and development of sophisticated weapons and aircrafts, manufacture and repair of precision instruments, medical research and anatomical training, engineering and remote robot control, protection of cultural heritage and educational entertainment, and many other areas.Through the seamless integration of virtual objects and the real scene, augmented reality can enhance the display of the real world. However, many open problems still remain in augmented reality. In recent years, research on augmented reality mainly focused on tracking, registration and interactive technology, and research on illumination consistency was relatively less. But illumination consistency plays an important role in achieving high realism. So the realism of virtual objects was not high in many augmented reality systems. Due to the complexity of geometry and lighting, the illumination estimation of outdoor scenes is a difficult and hot problem at present. Also, it is one of research topics in computer vision. Varying illumination usually severely degrades the performance of algorithms proposed in object recognition and segmentation, video tracking, shadow detection, etc. Therefore, without the information of3D scene geometry, the real-time illumination estimation of outdoor scenes is of great importance for both computer graphics and computer vision.In the case that3D geometric information of the scene is unknown, the dissertation focuses on the illumination estimation of outdoor video images captured under a fixed view, and analyzes the property on solving illumination parameters based on the linear decomposition model of outdoor scene lighting. To get rid of the difficulties in reconstructing the3D geometry of large-scale outdoor scenes, we propose the new resolution strategy, which makes the outdoor illumination estimation more practical. The contributions of the dissertation are as follows:1) We prove that three images captured with the same sun position under different weather conditions show linear correlation, based on the theory of the basis image decomposition of the sunlight and the skylight for fixed outdoor scenes. Therefore, the basis image equations are systems of under-constraint so that they can not be solved automatically. Then, we present the algorithm of solving basis images using four images with two sun positions under two kinds of weather conditions. In the algorithm, the intensity of the skylight is estimated according to pixels in the shadow region, and the intensity of the sunlight is calculated based on the characteristic of the basis image decomposition. In addition, the hue consistency of pixels in the sun area between the basis images of the sunlight and the skylight is used to optimize the basis images and illumination parameters.2) On the basis of the global illumination model without any limitation on reflectance, we propose a linear decomposition equation for the images of static outdoor scenes. Then, we proved that each image of a static outdoor scene can be decomposed into a linear combination of basis images of the sunlight and skylight, which encapsulate the geometry and material reflectivity of the scene. As well as, it is proved that the resulted basis images are invariants of the scene, corresponding to the global illumination effects of the outdoor scene under a unit intensity of the sunlight and skylight. Based on the input images of an outdoor static scene, basis images can be obtained by minimizing a quadratic energy function.3) We investigate the constraints and priors of the basis images, and propose a novel decomposition method to solve for the sunlight and skylight basis images of static outdoor scenes from a time-lapse image sequence without any user interaction. During decomposition, we first detect shadowed pixels by analyzing the time-lapse curve of each pixel through k-means clustering, and then the basis images of sunlight and skylight are solved by an iterative procedure with the decomposition equation. The basis images are further optimized by exploiting their constraints and priors.The above approaches don’t request any information of scene geometry, avoiding the3D reconstruction of large-scale outdoor scenes. As well as, they are applicable to the general outdoor scene because any assumption on materials and textures of the scene is not required and there need not be a special object or special surface in the scene. And that, these approaches do not need any other input except video images of the outdoor scene and request no user interaction. So they can be easily used for the system of augmented reality.