Video Super-resolution Reconstruction Based On Learning Method

Video super-resolution is one of the fundamental problems in image processing and computer vision,which aims to recover high-resolution images from a series of low-resolution images.An ideal video super-resolution system should be able to properly extract image features and synthesize image details in multiple frames compared to a single image super-resolution.To achieve this goal,two important subproblems should be studied:(1)how to align multiple frames to build an accurate correspondence;and(2)how to effectively synthesize image details to achieve high quality output.Thus,this paper uses learning-based methods to study the above two problems and implement video sequence super-resolution.(1)A survey on current research about video super-resolution reconstruction is conducted.And then the theoretical basis of video super-resolution and learning-based methods are given.(2)Multi-frame super-resolution reconstruction based on global motion estimation is proposed which use Convolutional Neural Network(CNN)descriptor.The traditional feature descriptors need artificial design features and are sensitive to image noise.To avoid these shortcomings,a novel descriptor is designed based on Convolutional Neural Network,which is more stable in detecting feature points and can effectively improve the initial registration accuracy between video frames.In order to eliminate the mismatch's influence on the registration between video frames,a random sampling consistency algorithm with double-threshold iterative is designed whose loss function is based on spline curve.A fast reconstruction network for generating highresolution frames is proposed to improve information fusion process efficiency.The experimental results demonstrate that the proposed algorithm can synthesize image details more quickly and accurately.(3)A new adaptive video super-resolution reconstruction is proposed based on optical super-resolution.To amend frame distortion after motion compensation based on traditional bilinear coefficients,the method combining motion estimation and motion compensation is proposed to estimate motion vector based on CNN.An adaptive deformation layer based on optical flow and compensation filter is designed to synthesize new pixels in high-resolution grid more accurately.In order to reduce the influence of inaccurate inter-frame registration based on optical flow estimation,superresolution of optical flow is introduced.The proposed adaptive video super-resolution method can simultaneously perform super-resolution reconstruction on image and optical flow,which can provide accurate correspondence and better super-resolution results.The experimental results verify he proposed algorithm's superiority in video resolution.