Central Catadioptric Imaging System Modeling, Calibration And Application

Recently, modeling and perception of large range of environment is a prevailing proposition.The traditional cameras are able to provide a limited field of view, which cannot meet the satisfaction of researchers.In the ego-motion estimation from video or image sequence, when the translation lies outside the field of view of the camera, ambiguities of translation and rotation between frames may arise. As a special imag-ing system for environment perception, panoramic camera overcomes the problem of uncertainty of the estimation, caused by the limited field of view in the application with traditional cameras. Panoramic camera makes the estimation of ego-motion not be affected by the motion direction very much, thanks to the 360 degree field of view. Applications that benefit from imaging equipment possessing wide field of view, such as fisheye camera and panoramic camera, include video surveillance, teleconferencing, 3D reconstruction, robot navigation and virtual reality.Panoramic vision becomes an increasingly attractive sub-area in computer vision.As all known that geometric model is a preliminary for the rigid motion estimation in the applications such as 3D reconstruction. While calibration is a prerequisite in the applications related to the estimation of rigid motion, where it is an indispensa-ble procedure that to convert the point on image plane to the geometric model.Since calibration approaches for traditional camera is not suitable for panoramic camera, and the existing calibration approaches for panoramic camera is complicated and un-stable. Therefore, this paper focuses on the central catadioptric system with a single view point. In attempt to modeling, calibration and monocular 3D reconstruction, we merge the existing models, and design a novel calibration rig. Based on theoretical propositions, a new approach of calibration with concurrent lines is proposed. This approach is validated by synthetic data as well as real image, and further tested by 3D reconstruction from image sequence acquired by central catadioptric camera.Thanks to the single view point, the mathematical model for central catadioptric is well established. Many models are proposed, such as the general imaging model, the Taylor series model and the unifying model. On the basis of unifying model, the closed-form expressions of point and line are studied and algorithms for calibration as well as structure and ego-motion estimation are proposed in the following chapters.There are several steps of the projection from a 3D point to an image point in the unifying model.The projection of line in each step is prevailingly studied. This paper combines the projection of line on infinite plane, metric plane and image plane. The study of one line projection is extended to a set of lines, then parallel and concurrent lines where a set of lines have a common point are introduced.Calibration is a preliminary requirement of 3D reconstruction, video surveillance, ego-motion estimation and so on.The results of these applications highly depends the accuracy of calibration. In central catadioptric imaging system, the parameters related to mirror type is of significance in calibration.The calibration procedure may easily converge to a desirable result, when the type of the mirror is a priori, especially when the mirror is parabolic. However, parameters related to mirror and the intrinsic para-meters should be estimated simultaneously when the mirror type is not a priori.The uncertainty of the mirror type parameters may ruin the calibration algorithm or makes it unstable.As we know that, parallel lines have a common point at infinity while concurrent lines have a finite common point. A calibration approach with a single image of chessboard is proposed, when the mirror type is known. A calibration ap-proach using concurrent lines with concurrent lines pattern is proposed when it is un-known.The constraint on concurrent lines is introduced into calibration in order to robustify the estimation of conic fitting. Moreover, concurrent lines pattern provides linear equation for solving the projection of point and line, so that these nonlinear problem could be replace by several linear sub-problems, and the intrinsic parameters and mirror parameter are estimated simoultaneously.There are adequate geometric theorems as known as epipolar constraint between frames for structure from motion (SFM) or simultaneously localization and mapping (SLAM) with monocular conventional camera. Taking the nonlinearity of the projec-tion in catadioptric camera into account, the epipolar constraint for conventional cam-era is not suitable for catadioptric camera. In order to adopt epipolar geometry, the projection of sphere is mapped to cube so as to convert the catadioptric projection to a projection of multiple traditional cameras with a single common view point. The 8 point algorithm for calculating fundamental or essential matrix is extended from 2D to 3D consisting of multiple image planes.Therefore, epipolar geometry can be ex-ploited in monocular reconstruction from panoramic image sequence.In order to im-prove the robustness of reconstruction, camera trajectory and 3D structure are esti-mated and locally optimized in a given number of keyframes. When a number of new frames are imported, the motion of new frames and structure are calculated on the ba-sis of existing motion and structure iteratively while the outliers are removed, and fi-nally optimized by bundle adjustment.The proposed approach successfully solves the non-linear catadioptric system by linear algorithm such as 8-point algorithm and tri-angulation that are suitablt for traditional camera.