Video Super-resolution Reconstruction Algorithm Based On Optical Flow Estimation

Super resolution reconstruction is a classical and challenging problem in the field of computer vision,whose purpose is to restore the corresponding high-resolution images or videos from the input low-resolution images or videos.In recent years,with the development of deep learning,many video super-resolution reconstruction mod-els based on deep learning have been proposed.Although these methods have achieved good results,the existing methods only focus on the clarity of the results and ignore the time correlation between adjacent frames.Aiming at the shortcomings of existing models,this paper proposes two optimized network models,the specific contents are as follows:(1)Aiming at the problem that the current video super-resolution reconstruction algorithm transforms the task into dealing with a large number of separated multi-frame super-resolution reconstruction problems,which leads to the excessive amount of network calculation,a video super-resolution reconstruction network based on frame recurrent structure is proposed.By adopting frame recurrent structure,the current frame can directly access the reconstruction results of its previous frame.By avoiding the sliding window processing mode and processing each low resolution input only once,the model reduces the amount of network calculation.Firstly,the optical flow between adjacent low-resolution inputs is estimated and up sampled to obtain high-resolution optical flow.Then,the optical flow is used to warp the reconstruction result of the previous frame image to the current reconstruction target frame.Finally,the super-resolution fusion module with residual network structure is input for feature extraction and feature fusion,and the results are output.(2)Aiming at the lack of time correlation between adjacent frames in video super-resolution reconstruction,a video super-resolution reconstruction network based on dense residual blocks is proposed.The network improves the optical flow estimation module.Firstly,the three-layer optical flow estimation network based on dense residual block predicts the high-resolution optical flow between adjacent frames from coarse to fine,makes full use of the time dependence between adjacent frames,and then uses the high-resolution optical flow to compensate the motion of the reconstructed target frame,Finally,the compensated video frame is input into the super-resolution reconstruction network for feature extraction and feature fusion to obtain the high-resolution reconstruction results.The structure of dense residual blocks is introduced into the optical flow estimation network and super-resolution reconstruction network for feature extraction.In addition,the loss function is improved by combining the loss function of optical flow estimation module and super-resolution reconstruction module to obtain more detailed reconstruction results.The experimental results on Videoset4 show that the video super-resolution reconstruction network based on frame recurrent structure and the video super-resolution reconstruction network based on dense residual blocks pro-posed in this paper have improved the objective evaluation index and subjective visual perception,and the time coherence of the reconstruction results has also been enhanced.