Research On Video Stitching Algorithms In Different Application Scenarios

The video stitching technology stitches multiple small-view videos with overlap-ping regions into a complete large-view scene.This technology solves the limitation of the view field of single camera effectively,and is widely used in the fields of vir-tual reality,intelligent monitoring and so on.Therefore,video stitching technology is a hot topic in the field of computer vision and digital multimedia,and the related works are numerous.However,there are still many difficulties in the practice of video stitching,such as high real-time requirement,complex and varied application scenar-ios,and strict requirements for stitching effect.In this paper,two kinds of real-time video stitching algorithms are designed and implemented for different scenes,and a novel stitching method based on self-homography warping for large parallax scenes is proposed.This paper proposes two real-time video stitching algorithms for the actual scenes.One is based on seam and the other is based on cylindrical warping.The real-time video stitching algorithm based on seam is mainly used for stitching videos captured by parallel cameras,and is suitable for processing large scale scenes.In this algorithm,we realize video stitching by selecting the optimal seam.The content for each side of the seam is selected from a single frame,thereby effectively eliminating ghosting.In addition,we adopt multi-band blending in the fusion process of the algorithm,so the transition around the seam is natural.The real-time video stitching algorithm based on cylindrical warping is mainly used to stitch the videos captured by the ring-arranged cameras,and the stitching results of ultra wide or even 360-degree view can be ob-tained.In the process of video stitching,due to the different orientations of cameras,there is an angle between overlapping regions of videos,and the visual consistency is poor.By warping the videos to the cylinder,the algorithm overlaps the overlapping areas on the cylinder and achieves better stitching effect.In order to improve the speed of the above two video stitching algorithms,we extract the template frames to calculate the stitching template,and then apply the stitching template to the processing of subse-quent video frames,thereby simplifying the calculation.Moreover,we transplant the algorithms to the CUDA platform,and accelerate the algorithms through the powerful parallel computing capabilities of the GPU,so that the video stitching algorithms meet the real-time requirement.At present,the traditional stitching algorithms are not ideal for the stitching effect of large parallax scenes.In this paper,we propose a novel stitching algorithm based on self-homography warping,which is robust to large parallax.For image stitching of large parallax scenes,we first divide the foreground and the background of the images,and then stitch them separately,and finally blend the stitching results into a complete image.This algorithm stitching the foreground and the background separately avoids the situation that the traditional stitching methods can only align one plane of the image while ignoring other planes,so the image stitching quality is significantly improved.In addition to the image alignment and image blending in the traditional stitching al-gorithms,this algorithm also adds the steps of image segmentation,image repair and scenes fusion,which increase the complexity of the algorithm.Therefore,when the al-gorithm is applied to the video stitching,the stitching process is divided into two parts:stitching template initialization and stitching template update.In order to achieve the purpose of accelerating processing speed,the stitching template is updated only when foreground objects pass through the overlapping area,otherwise the initialized template is still used for stitching.Compared with image stitching,the video stitching algorithm of large parallax scenes also needs to consider the continuity of the stitching results.Therefore,we add the relative displacement between video frames in the algorithm to constrain the occurrence of discontinuity.The real-time video stitching algorithm based on seam has high speed and prac-ticability,and the stitching algorithm based on cylindrical warping can achieve 360-degree full coverage monitoring.The stitching algorithm based on self-homography warping can eliminate the ghosting in the stitching results of large parallax scenes ef-fectively,and the visual effect is remarkable.