Research On Vehicle Panoramic Image Mosaic Technology Based On Fisheye Camera

Panoramic image is a new research hotspot in the fields of computer vision and virtual reality in recent years.It has high application value in remote sensing image processing,medical imaging,driving safety and other fields.Especially in driving safety,the onboard panoramic vision system based on fisheye camera has attracted more and more researchers' attention.In order to provide drivers with 360 degree road condition information around the body and improve driving safety factor,it is necessary to study on on-board panoramic image mosaic technology based on fisheye camera,which mainly includes preprocessing of fisheye image(corner detection,calibration of fisheye lens and distortion correction of fisheye image),vehicle panoramic image mosaic(registration and fusion).The main work of this thesis is as follows:Firstly,a sub-pixel level improved Harris corner detection method is proposed.The corners of checkerboard calibration board are detected by Harris corner detection,and the initial corner points are obtained.Before calculating the response function,the initial corner points are filtered by using the similarity of the gray values of the pixels in the image region to reduce the computational complexity.Then the response function of the filtered Harris corner is analyzed by binomial approximation to obtain the sub-pixel corner position.The experimental results show that this method can accurately detect the corners of checkerboard grid calibration board.Compared with Harris corner detection method based on gray difference and template,FAST corner detection method based on pixel search and improved SUSAN corner detection method,this method has the better detection efficiency.Then,the "Zhang Zheng You calibration" method is used to calibrate the fisheye camera.The internal parameter matrix and distortion coefficient of a large number of checkerboard calibration board pictures are solved,and the specific values of the parameters are given,which makes full preparations for the fisheye image correction in the next step.The method used in this thesis is simple,easy to implement and high precision,and is suitable for specific engineering applications.Next,the offset model based on spherical perspective projection is studied to correct the fisheye image.The relationship between incident angle and camera focal length is brought into the spherical model to solve the coordinate mapping relationship,and then the relationship between source image coordinates and target image is established for perspective projection.The experimental results show that the method proposed in this thesis is an effective method for correction of fisheye image distortion.Compared with the longitude and latitude correction method based on 2D space and the cylindrical perspective projection method based on 3D space,the average and maximum errors of the line segments after correction are very small.Subsequently,a vehicle panoramic image registration method based on improved Harris feature point detection and Franklin moment feature description is explored.The improved Harris corner method is used to detect image feature points,Franklin moment is used as rotation invariant descriptor of image features,and the initial point pairs are obtained by comparing the Euclidean distances of each feature point's circular neighborhood Franklin moment.Then,the initial point pairs are purified by random sample consensus(RANSAC)method to complete the correct registration of images.Using registration accuracy(CMR)as an evaluation index,three image registration methods,SIFT-based registration,SURF-based registration and Zernike moment-based registration,are compared in terms of anti-noise ability,anti-brightness change ability,anti-scale change ability and anti-rotation ability.The experimental results show that the method proposed in this thesis has higher registration accuracy,higher accuracy,better effect and shorter time consuming.It is suitable for engineering practice.Finally,a vehicle panoramic image fusion method based on projection gradient non-negative matrix factorization(NMF)and guided filtering is discussed.Contourlet transform is applied to four overlapping regions of vehicle panoramic image to obtain different frequency band information.Projection gradient NMF is used to fuse low pass subbands,absolute maximum rule of region is used to fuse bandpass subbands,and guided filtering is used to enhance details.Then contourlet inverse transform is added to the mosaic image to get the final vehicle panoramic mosaic results.Using information entropy,mutual information and average gradient of image as objective quantitative evaluation indexes,this thesis compares three fusion methods: method based on non-down-sampling shear wave transform,method based on wavelet transform and contourlet transform method based on improved threshold.The experimental results show that the fusion results of this method are clear,there is no obvious splicing gap,and the comprehensive performance is better.