Research On Real-time Video Stitching Technology Of Multi-camera

With the development of computer technology and multi-media technology,real-time video stitching is widely used in many fields,such as intelligent monitoring,driving assistance,video conferencing and so on.Video stitching is an extension of image stitching in the time dimension.Therefore,video stitching technology is based on image stitching,but different from it.This paper first introduces the process of image stitching,including image acquisition,image preprocessing,image registration and image blending.However,video stitching requires high real-time performance,therefore,the speed of stitching for each video frame should be fast enough.There are moving objects in the video,which puts forward higher requirements on the registration and blending algorithms of the video images.How to ensure that the moving objects in the video are clear without ghosting is very crucial.In this paper,SURF is selected as feature extraction algorithm in video registration and improvements are made.We divide the video overlap region into multiple sub-regions,and detecte and matche features in the sub-regions respectively,which improves the speed of feature matching and makes the distribution of features more uniform.According to the camera position,we add horizontal and vertical coordinate constraints in feature matching,which effectively eliminates many anomalous matching and accelerates the RANSAC algorithm.Because the cameras are stationary,the video registration in this paper is only carried out in the initialization stage of the video stitching.Moreover,the coordinate relationship is calculated according to the homography matrix and stored in the look-up table;during the stitching stage,the transformation of video images can be performed by only looking up the look-up table.In this paper,the optimal-seam-based offset compensation algorithm is used for video blending.The optimal seam selection algorithm can effectively eliminate the problem of parallax between multiple cameras and avoid the ghosting during the video blending.In this paper,an update strategy based on the optimal seam set is proposed.First,a set of optimal seams are selected in the initial stage of video stitching.Then,we detect the moving objects in the video scene frame by frame.When the object passes through the seam,we first select an optimal seam from the set and update the previous one.Furthermore,this paper presents an offset compensation algorithm based on look-up table to accelerate the video blending,which effectively eliminates the exposure and color difference on both sides of the seam and make the video blending more natural without any obvious stitching seam.This paper introduces the H.264 video codec and uses the coding information such as motion vector and macroblock type to speed up real-time video stitching.In the implementation of look-up table for image transformation and offset compensation,the introduction of motion vector and macroblock type can reduce the times of look-up.During the optimal seam selection,the introduction of a macroblock type allows the seam to pass through a flatter area but a foreground object;the same time,motion vector can be directly used to describe the size,position and speed of moving objects,greatly accelerating the detection of moving objects.Finally,this paper builds a dual camera platform based on HiSilicon Hi3518E and completes the design of video stitching software in Visual Studio 2015 IDE.The software adopts a modular,multi-threaded,pipelined design,making full use of CPU and accelerating the real-time video stitching.In this paper,two-way real-time video stitching experiments are carried out for indoor and outdoor scenes respectively.The experimental results achieve the expected stitching speed and effect.