Research On Video Frame Interpolation Methods Based On Complex Motion Types

Video frame interpolation studies how to generate and insert intermediate frames between consecutive frames in low frame rate video,to get a more coherent high frame rate video.As one of the hotspots in the field of computer vision,video frame interpolation is widely used in frame rate improvement,video compression and repair,slow motion generation and so on.There are a large number of complex motion types in the real scene,such as large-scale motion,non-uniform motion,fast motion that causes image blurring,which bring great challenges to the motion estimation in video frame interpolation and seriously affect the performance.Therefore,it has important theoretical value and practical significance to deeply study high-quality video frame interpolation methods for different complex motion types.This paper conducts in-depth research on video frame interpolation methods based on complex motion types.Starting from three common complex motion types — largescale motion,non-uniform motion,and fast motion that causes image blurring,the ability of complex motion modeling is improved by means of degree of freedom enhancement,long-term and short-term information optimization,and joint modeling of video frame interpolation and deblurring.The main research work of this paper is summarized as follows:(1)A video frame interpolation method based on enhanced spatial-temporal freedom is proposed.For the large-scale motion in video,by introducing higher spatialtemporal freedom in the feature extraction stage and frame fusion stage,the spatialtemporal feature information is effectively extracted in the Enhanced Spatial-Temporal Feature Extraction Module,and the frame fusion is adaptive in the Enhanced Freedom Fusion Module,so as to realize more accurate motion modeling under large-scale motion.The proposed method is verified by ablation and comparison experiments on public datasets such as Vimeo90 K and GOPRO.Compared with the base network,this method improves the evaluation index PSNR of frame interpolation results to 35.75 d B and 30.56 d B,and SSIM to 0.9611 and 0.9207,which is superior to mainstream algorithms with the same technical route,such as GDConv and CDFI.(2)A video frame interpolation method combining long-term and short-term motion information is proposed.Aiming at the non-uniform motion in complex motion scene,through the joint optimization of intermediate features by the Local Feature Alignment Module based on short-term motion information and the Global Feature Optimization Module based on long-term motion information,the more accurate modeling of non-uniform motion is realized.The proposed method is verified by ablation and comparison experiments on public datasets such as Vimeo90 K and DAVIS.Compared with the base network,this method improves the evaluation index PSNR by0.27 d B and 0.14 d B,and SSIM by 0.0056 and 0.0052,which is superior to mainstream algorithms with the same technical route,such as Feature Flow.(3)A joint modeling method of deblurring and video frame interpolation for lowquality video streams is proposed.Aiming at the blurry input problem caused by fast motion,based on the previous research,the Feature-based Deblurring Module and the Local Feature Fusion Module based on dual attention mechanism are added.Through the Local Feature Alignment and Fusion Module based on short-term motion information optimization and the Global Feature Collaborative Optimization Module based on long-term motion information optimization,the joint modeling of video frame interpolation and deblurring is realized.The proposed method is verified by ablation and comparison experiments on public datasets such as Adobe240 and GOPRO.Finally,the PSNR of the combined results of frame interpolation and deblurring is improved by0.43 d B and 0.11 d B,and the SSIM is improved by 0.0034 and 0.0041,which proves the effectiveness of the proposed method.This thesis contains 36 pictures,14 tables,and 76 references.