The Research Of Key Frame Extraction Technology In Video Stitching

In the process of video stitching in special scenes,the color,brightness,texture and other image features between corresponding frames will change greatly.For example,when the platform is in motion,there is relative jitter between binocular cameras;When the illumination changes,the gray level between the front and back frames changes greatly.In the subsequent transformation,a common method is the projection matrix calculated only by the first frame or some frames.This will produce cumulative error,and the result is not satisfactory.Another method of frame-by-frame calculation will increase unnecessary computation,resulting in serious pause.Aiming at the problem of video stitching in specific scenes,this paper proposes a new key frame extraction method and similarity contrast strategy based on synchronous frame difference,which take into account the quality and speed of the video stitching.The main work of this paper is as follows:1.Before the image registration,we judge whether it is a key frame according to the set threshold,and then decide whether to update the projection matrix,which overcomes the cumulative error caused by jitter,and obtains a good video stitching effect.2.In key frame extraction,the method of comparing synchronous frame difference is used instead of the traditional method based on continuous frame difference,and the projection matrix is updated when the similarity between frames is less than the threshold value.3.binocular stereo vision is introduced,and a new key-frame extraction algorithm is designed.The color moment is weighted integrated with the gray-scale-based SAD(sum of difference)operator to construct a new image feature descriptor.4.When calculating the similarity between frames,a new search strategy is adopted to speed up: in the area outside the overlap of left and right frame images,the traditional template is replaced by cross feature straight lines,and the calculation is carried out by exhaustive method.In the case of camera jitter,the improved key frame extraction algorithm has solved the problem of large image deformation and excessive proportion of deformed frames in video stitching,and achieved a balance between stitching speed and time consuming.