Research On 3D Panoramic Vision Assistance System Of Vehicle

The panoramic vision assistance system of vehicle can provide drivers with more visual information by acquiring images around the vehicle through four fish-eye cameras installed in the front,rear,left and right of the vehicle.At present,the commonly used panoramic vision system is to form an overlooking panoramic 2D image after distortion correction,overhead transformation and image stitching.Although the blind area is greatly reduced,there are still some limitations such as stitching distortion,single view angle and poor flexibility.Therefore,the research of 3D panoramic vision assistance system has received extensive attention.This paper focuses on solving the key technologies of 3D panoramic image stitching,fusion,image enhancement,3D model building and texture mapping.In this paper,a 3D panoramic vision assistance system for automobile will be established to effectively reduce the occurrence of scratches,collisions,collapses and other accidents,so as to improve the safety of driving and the flexibility of operation.Firstly,based on the analysis of the realization principle of the 2D panoramic viewing system,the realization method and flow of the 3D panoramic vision assistance system of vehicle are proposed,and the key technologies to be solved are given.On this basis,a 3D panoramic vision system experimental platform consisting of fisheye cameras,image acquisition card and mounting bracket was established.Aiming at the higher requirement for camera calibration accuracy for 3D panoramic vision,the fisheye camera imaging model was deeply studied.The calibration of fisheye camera and the correction of fisheye image were realized by Visual Studio 2012 equipped with OpenCV library.The average error of fish-eye camera calibration can reach at least 0.04 pixels.Then,the image mosaic algorithm of automobile 3D panoramic system is studied.Seamless stitching of low dislocation distortion panoramic image is realized through the overlook transformation,image registration,fisheye image mosaic algorithm based on deviation transferring and splitting for vehicle panoramic system and weighted average fusion.A multi-scale homomorphic filtering algorithm is proposed for the problem of excessive darkness and uneven illumination.The filtered image is fused with three scales of ultra-high-speed exponential fuzzy operator to obtain different levels of detail information,and then the combined details are fused into the filtered image,which can increase the contrast of the filtered image and improve the clarity of the output images.The experimental results show that compared with the traditional homomorphic filtering algorithm,the multi-scale homomorphic filtering algorithm increases the average gradient(sharpness)by 1.72 times,the contrast ratio by 3.3 times,and the entropy value by 0.25 bits/pixel.To achieve 3D panoramic vision,the next step is to study the establishment of panoramic 3D model and texture mapping algorithm.A low distortion and uniform field of view vehicle panoramic 3D model composed of bottom plane,arc connection surface and tapered side is proposed to obtain better visual effect.Compared with spherical and cylindrical panoramic 3D model,it can be seen that the proposed 3D model can overcome the texture distortion caused by the spherical incompatibility.It can avoid the phenomenon that the three components of the cylindrical vehicle 3D model can't smoothly connected and the edge convergence occurs when viewed from above.Then,a texture mapping algorithm based on texture step is proposed.The simulation and experiment results show that the algorithm satisfies the three criteria of texture invariance and achieves a more realistic 3D panoramic image.Finally,research on target detection technology in 3D panoramic images is carried out.Based on Darknet,a target detection model is constructed and trained by using YOLOv3 convolutional neural network algorithm.The model is applied to spherical,cylindrical and low distortion uniform field of view in the vehicle 3D panoramic system to evaluate the performance of the target detection model.The experimental results show that under the same conditions,the bounding box class-related confidence of each target in the vehicle 3D panoramic system of low-distortion uniform field of view is over 95%.The target detection average time is 2.22ms faster than spherical 3D panoramic system and 2.44ms faster than cylindrical 3D panoramic system.