Calibration And 3D Reconstruction For Multi-mirror Catadioptric Systems

The omni-directional catadioptric camera composed of multiple mirrors is a powerful tool that can acquire multiple wide field-of-view images and reconstruct the scene in a single shot. Featuring a large field of view, they have gained much popularity in various fields, such as video surveillance and robotic navigation. Calibration for the catadioptric imaging system is a prerequisite step for most motion or 3D reconstruction-related applications. As existing models for multi-mirror catadioptric system are always very complex, and hard to calibrate or use. Therefore, it’s urgent to find a novel model which keeps the flexibility but works well for the multi-mirror catadioptric system.In this paper, we develop a flexible calibration method for general multi-mirror systems. We regard each combination of a mirror and its corresponding region in the imaging plane as a virtual sub-camera and convert the multi-mirror calibration problem into estimating the intrinsic parameters of each sub camera and the relative positions between them. To address the large amount of misalignment that exists in each sub-camera, a novel off-center spherical projection (OCSP) model is presented. OCSP greatly enhances the capability of dealing with large misalignment by inducing a movable virtual spherical projection center within the sphere.Based on the OCSP, we give out a method for calibration with planar grids. The simulation and real experiment verify the effectiveness of the model and the validity of calibration method.With the OCSP and multiple view geometry, we further propose a 3D-reconstruction using the multi-mirror catadioptric system. Using the single-shot view, we can get the sparse 3D reconstruction from the matched feature points. To make the dense 3D reconstruction available for multi-mirror catadioptric system, we first build the spherical image from the OCSP. Moreover, by transforming the rectified spherical image to latitude-longitude representation, the feature point matching of spherical stereo images can be sped up. In order to remove blunders and improve accuracy when dealing with 3D scenes, redundant measurements across spherical stereo images are linked and checked for geometric consistency. Both of the sparse and dense 3D reconstruction have demonstrate the accuracy of the calibration result and the effectiveness of the reconstruction method.