Research On Surround View Panoramic Image Processing Algorithm In Assisted Driving System

With the popularization of automobiles and the public’s focus on driving safety,more and more new technologies are being used in driver assistant system.The research on advanced assisted driving system and driverless driving has also received widespread attention.Among them,assisted driving technology based on computer vision and image processing is widely used.This paper mainly researches the panoramic image processing algorithms of in assisted driving system.And the main research results are as follows:(1)The implementation process of the 2D surround view panoramic image processing algorithm is studied,and a 360-degree 2D surround view bird’s-eye panoramic view is realized based on four vehicle-mounted fisheye cameras.First,we use four fisheye cameras to capture the video data of the driving environment.And then the distortion correction algorithm is performed on the frames from the four fisheye cameras.After that the transformation matrix from the frame of each camera to the corresponding top view is calculated.Then we transform the four images into a unified top view image.Finally,a linear weighted fusion algorithm is used to process the stitching lines to obtain a seamless 360 ° bird’s view panorama.(2)Researched the realization principle of 3D panoramic view,and implemented the traditional 3D bowl projection model,then analyzed its characteristics,advantages and disadvantages,and we proposed a five-plane projection model and a semi-ellipsoid projection model.This paper first analyzes and implements the traditional bowl-shaped model.In a crowded environment,the objects on both sides of the traditional model will have a large deformation when projected on a curved surface model.So a five-plane projection model is proposed to reduce the bending deformation of the object by projecting the images on both sides on two vertical faces.At the same time,in order to observe a wider field of view and better suit the driver’s observation habits,a semi-ellipsoid projection model was proposed.(3)An adaptive 3D surround view panorama generation method based on motion estimation is proposed.Aiming at the shortcomings of traditional 3D panoramas which have fixed parameters and the projection vision effect cannot change with the environment,this paper proposes a vehicle-based 3D surround-view adaptive generation method based on motion estimation,which estimates the surrounding environment to change the projection model and adaptive generate a panorama.In this paper,first the distortion of fisheye frames are corrected,and the LK optical flow tracking algorithm is used to calculate the optical flow value of the feature points.By analyzing the change law of the scene optical flow,the feature points on the main plane of the obstacles on both sides are determined.Then we match these feature points of the main plane,and restore the relative depth of the points according to the epipolar geometry and triangulation,then transform the image to the top view,and calculate the rotation and translation relationship between the top views of the two frames.Since the parameters of the previously generated two-dimensional top view are known and the calibration cloth size is known,a scale factor can be calculated to estimate the relative depth to the true distance from the surrounding environment to the car.Finally,by searching a preset mapping table corresponding to the distance parameter,an on-vehicle 3D panoramic image corresponding to the parameter is adaptively generated.At the end of this article,relevant outdoor experiments were performed to verify the feasibility of this method.The advantage of our method is that there is no need to add additional sensor devices(such as lidar,etc.)to the system when sensing the environment except for a few cameras.And the implementation cost of our method is low.