Research On Vehicle Panoramic Video Intellisense Technology Based On Image Stitching

With the development of information science and computer graphics,panoramic video stitching technology is used more and more widely,such as computer vision,virtual reality,automotive,etc..It is reflected in the automotive field,driverless,vehicle terminal monitoring,panoramic assisted parking and other systems.Video stitching is an extension of image stitching.It adds time to the original dimension.To output smooth panoramic video,the stitching of each frame must be fast enough.At the same time,it must have higher requirements for the registration and fusion algorithm.At this stage,the stitching technology of the vehicle panoramic video is still not mature.How to realize the real-time and high-quality panoramic video stitching based on the vehicle is still a major challenge.Therefore,the in-depth study of the panoramic video stitching technology has important theoretical significance and application value.The research and improvement of registration and fusion algorithms are proposed for the real-time performance of vehicle video stitching algorithm and the removing ghost of panoramic video,and a heterogeneous architecture based on CPU and GPU is used to design a system which can real-time perceive panoramic video.The specific research contents and main work of this paper are as follows:1.By improving SURF based on ORB,an image registration algorithm(FSURF)is proposed to solve the real-time problem of vehicle panoramic video stitching and improve the speed of image registration fusion.Firstly,the two images to be spliced are segmented according to the regional block idea.The region correlation verification is performed for each group of sub-block pairs to find the best sub-block pair,and then the feature points are obtained by the FAST feature point detection algorithm in the ORB.Simultaneously,the Hessian matrix is used to eliminate unstable edge response,and then the gray value difference is added while assigning descriptors to each feature point,which can enhance the robustness of the algorithm to illumination,and finally RANSAC algorithm is used to eliminate the mismatch points.The experimental results and data show that the proposed algorithm has obvious speed advantage over the traditional SURF algorithm,and since the feature point description makes full use of the gray level information,so the registration fusion result has better brightness for image blur and illumination variation.2.For the fusion phase,the existence of moving targets may lead to the ghost phenomenon in the stitched image.A multi-resolution fusion algorithm based on the best suture line is proposed in this paper.Firstly,the exposure correction technique is used to correct the transition phenomenon between frame images caused by the exposure problem,and then the best suture line is obtained from the color interpolation obtained by the pixel in the corrected image overlap region and the structural difference obtained by the Sobel operator.Finally,the fusion is performed on different resolution layers in combination with the multi-resolution fusion algorithm.The experimental results show that with the fusion algorithm,the fused image not only achieves a smooth transition,but also eliminates the ghost phenomenon,and obtains a complete and clear panoramic video image.3.Finally,a system that can sense vehicle panoramic video in real-time is designed.Although the traditional SURF algorithm is improved in this paper,and at the same time the regional block idea is utilized to accelerate the vehicle panoramic video stitching algorithm,but simply relying on the software has not been able to achieve the effect of real-time output smooth video.In order to solve this problem,the heterogeneous system based on CPU and GPU is proposed in the panoramic video stitching system in this paper.Through the high-performance graphics card,GPU's powerful floating-point computing power and CPU multi-threading technology,the system will be fully accelerated.The experiments verify that the scheme basically solves the real-time problem of video stitching mentioned at the beginning,and stitching of no less than 15 frames per second is achieved.