Panoramic SLAM System From A Multi-Camera Rig

This research presents a feature-based SLAM framework for panoramic image sequences captured by a multi-camera rig,which can handle with monocular fisheye image sequences,multi-fisheye image sequences and panoramic image sequences.Large Field-of-View(FoV)and small FoV fisheye cameras are proved to be advantageous to be applied into indoor SLAM and outdoor SLAM respectively,however the small FoV fisheye camera still suffer from the limited visual overlap between consecutive frames though it has its merit for urban canyon environments,the visual odometry could easily lose tracking when sudden change of view angle occurred.Multi-Camera Based Panoramic Camera utilize multiple small FoV fisheye cameras to capture a panoramic image which extended the FoV up to 360°while inherited the merit of small FoV fisheye cameras.To take fully advantage of the panoramic camera and explore the application of the SLAM system in the Mobile Mapping System(MMS),we adapt the state-of-art feature based SLAM system and extend it to a fully automatic SLAM for Multi-Camera Rig which could process multi-fisheye image sequences as well as the panoramic image sequences.Firstly,the geometry relationship between fisheye lens and panoramic camera along with its projection model would be introduced,then,this paper would elaborate the modification algorithm of combining the panoramic image with state-of-art SLAM system.Finally,the complete SLAM system pipeline for Multi-Camera Rig would be presented by details as blow:(1)Projection model of Mulit-Camera Based Panoramic Camera.Rigorous geometric relationships in a panoramic camera need to be clarified when using the panoramic images for surveying purpose.This paper first introduces the imaging model and distortion model of each fisheye lens in multi-camera based panoramic camera,and proposes an improved internal calibration model based on the general fisheye lens calibration model.Then we introduces the method of using the three-dimensional calibration field for external calibration,and the rigorous geometric relationship between fisheye lens,panoramic image and panoramic sphere is built.(2)The basic principles and methods of the SLAM system.In this part,we would describes the SLAM problem in a mathematical model,also the core algorithm for each module of existing SLAM systems.The performance and the limitation of existing SLAM systems are analyzed,as well as the applicable scenarios of the current SLAM system.The experiments on synthetic data were carried out,the generic calibration model and our modified model were both tested for calibration accuracy,plus,the localization experiments using the calibration parameters with EPnP algorithm were also carried out.The result shows that,our modified calibration model shows higher accuracy than the generic calibration model for the multi-camera rig,which eventually leads to higher localization accuracy than generic with same EPnP algorithm.(3)SLAM system based on panoramic image.Based on the wide-area outdoor wide-baseline image sequence acquisition mode,the defects and disadvantages of the current SLAM system are analyzed.The merit of 360-degree imaging of panoramic cameras is introduced into the SLAM system,combined with the framework of the state-of-art feature point method SLAM system.Each module,such as system initialization,pose tracking,local mapping,pose optimization,key frame selection,loop closure and other algorithms are improved for the purpose of developing a fully automatic SLAM system for multi-fisheye image sequences and panoramic image sequences,and enabled it to suitable for large-scale outdoor urban street scenes and scenes prone to GPS signal anomalies(eg,tunnels,overpasses,etc.).So that it could complement the ground motion mapping system(MMS)with more accurate localization information.Mean while,the experiment's analysis is carried out on both public dataset as well as our self-collected data.The results show that the panoramic SLAM solution proposed by this paper can successfully complete the pose tracking on the multi-fisheye image sequences as well as the panoramic image sequences while building the spars map for the surroundings,and is suitable for large-scale outdoor wide-baseline image sequences.The localization accuracy and robustness of our system is higher than other systems for fisheye or panoramic vision.