Research On UAV Aerial Image Stitching Based On Auto Stitch

With the development of UAV technology and camera technology,the camera carried by un-manned aerial vehicle(UAV)can obtain high-definition image.Latest civil UAV can even realize real-time transmission of 4K video.Due to UAV flying height and camera scope,it is difficult to obtain a large range of interested image from single UAV aerial images.UAV can obtain a wide range of scene information by flying and video aerial photography.In practi-cal application,through sampling the UAV video and then splicing the sampled images with image stitching technology,a high resolution image of large field of view can be obtained.Image stitching refers to form a panorama with wide viewing angle and high resolution based on a group of overlapping images through image transformation,image registration and im-age fusion.The basic process of image stitching includes image preprocessing,image regis-tration,camera parameter estimation,image fusion and so on.Among them,image registra-tion includes image feature point detection and feature point matching.Camera parameter estimation includes obtaining homography matrix,obtaining camera internal and external parameters,bundle adjustment method for camera internal and external parameters,wave-form correction,image projection and other steps.Image fusion includes the best suture line search and image fusion.The algorithm of image stitching has many steps and long process.In the case of UAV shooting for a long time and long range,the number of images sampled for UAV aerial video is large that may reaches 100 + in some scenarios.The number of sampled images to be stitched is too many,which leads to the time-consuming of existing image stitching and stitching failure due to large memory consuming.The existing research on image stitching mainly focuses on improving the effect of image stitching,but less on a large number of image stitching.A large number of images need to be stitched in UAV aerial scenes.For this application scenario,based on the existing Auto Stitch image stitching algorithm,this paper optimizes and improves the image stitching technology of large number(100 +)from the image feature point detection,bundle adjustment fine parameters,image fusion and Open MP multithread-ing technology,solve the problems of long time consuming and large memory consuming.The main innovations of this paper are as follows:1.This paper studies and improves the bundle adjustment algorithm.Bundle adjustment is one of the links of camera parameter estimation in image stitching.In the process of bun-dle adjustment,LM algorithm has many iterations,and its time-consuming increases rapidly with the increase of image stitching number.In this paper,based on the previous work,firstly,the confidence of matching point pairs is sorted and screened,and the best 60 pairs are selected as the final matching point pairs; then,according to the statistical information of ray error changes in the iterative process of LM algorithm,the variance information of ray error is counted,and the variance threshold of the last five ray errors is set.If it is less than this threshold,LM will jump out of LM algorithm iterative process earlier.Experimental results show that the improved bundle adjustment algorithm can greatly speed up the speed of parameter estimation while ensuring the effect of image stitching;2.This paper studies the problem of memory consumption in image stitching,and improves the memory optimization.In a large number of image stitching,there are millions of fea-ture points.Therefore,a lot of memory is consumed in the process of detecting and storing these feature points,which often leads to the interruption of the stitching process.In order to solve this problem,this paper proposes a stitching algorithm based on downsampling.In the feature point extraction stage,the image is downsampling first,and then the original im-age is used for image fusion in the image fusion stage.The algorithm greatly reduces the memory consumption in the process of feature point detection,and then reduces the memory consumption of the overall stitching process,and maintains the effect of image fusion;3.This paper studies the image stitching acceleration method based on Open MP multi-threading technology.According to the characteristics of independent and non-interference of some steps in the process of image stitching,based on the Open MP technology in C + +framework,the feature point extraction step and multi-layer fusion step of the image are pro-cessed in parallel,which further reduces the splicing time while ensuring the image stitching effect and the original image details;4.This paper studies the secondary stitching scheme based on the results of block stitching.For multi image stitching,firstly,the images are taken and stitched in groups,and then the stitching results are stitched twice.In this process,the phenomenon of ”cracks” will be pro-duced when the results of group stitching are directly stitched,which will affect the overall stitching effect.In this paper,a secondary stitching scheme based on image size normal-ization is proposed to solve this problem.In practical application,this scheme can greatly expand the aerial photo stitching area from one dimension to two dimensions.In this paper,the simulation experiment is carried out in Visual Studio 2017,and the soft-ware is written with QT framework.Through the optimization of algorithm and the design of stitching scheme,the fast and efficient aerial image stitching is realized.The research results of this paper have a good reference value for UAV aerial mapping and large number of image stitching optimization.