Research On Stitching Method Of UAV Multi-channel Aerial Video

Video stitching is a technique to stitch multiple narrow-view video frame sequences with overlapping areas into a wide-view video frame sequence through feature extraction,alignment,perspective transformation and fusion.In the UAV multi-camera aerial photography scenario,the video stitching technology fuses multiple frames with overlapping views transmitted to the ground station terminal into one large-view image,reducing the redundant information between video frames and improving the visual experience of the observer.Since UAVs acquire a larger number of images at the same moment and are disturbed by light,airflow and airframe jitter during flight,the traditional video stitching algorithm suffers from high algorithm complexity,poor robustness and obvious output video jitter.To address the above problems,the following research work is mainly carried out in this paper:(1)To address the problem of inconsistent exposure of multiple images that may exist during UAV operations,the exposure compensation algorithm is used to balance the exposure of multiple images in the pre-processing stage,the correction algorithm is used to correct the input image distortion,and then the geometric transformation of the processed image is done to reduce the impact of parallax.Through the above pre-processing methods,the accuracy of image stitching is effectively improved and the interference in the stitching process is reduced.(2)The video frame stitching algorithm based on the optimal stitching line is proposed to address the problems that the stitching line cannot effectively avoid the foreground objects and the stitching result has ghosting due to parallax.When finding the optimal stitching line,the foreground term constraint and alignment constraint are added to the energy function of the dynamic planning algorithm,so that the stitching line can avoid the moving objects effectively.After finding the best stitching line,the Laplace pyramid algorithm is used to fuse the stitching line part.Through experimental comparison,the method solves the problems of ghosting and obvious suture lines in the stitching results,and obtains higher quality stitching results.(3)To address the problem of jitter in the output video sequence after stitching,an adaptive Kalman stabilization algorithm using the cumulative displacement between frames to correct the Kalman gain is proposed.By finding the cumulative displacement between frames and correcting the Kalman gain coefficients,it can effectively de-jitter when the UAV camera has highfrequency jitter and can effectively follow the active camera motion when the active camera motion is strong.In addition,a method is proposed to effectively combine the video stitching algorithm and the video stabilization algorithm,which can calculate the stabilization parameters in parallel to obtain smooth and stable stitching results and reduce the time consuming of the stitching system.(4)To address the problem of time-consuming video stitching system,an optimization method is proposed to calculate the inter-camera geometric transformation model,columnar projection model and alignment model by multiple first-frame images.And we build a four-way video acquisition device by simulating the UAV multiple camera installation,and realize a semiphysical simulation splicing system for UAV multiple aerial video on Windows platform.The results show that the system effectively avoids foreground fragmentation and ghosting problems,and the stitching sequence output in real time is effectively stabilized.