Research On Super-Resolution Methods Of High-Magnification Image Reconstruct And Video Feature Mining

In the process of images and videos acquisition,it is difficult to display the information in the natural scene without distortion by the imaging results obtained due to the influence of objective factors such as the surrounding environment,imaging conditions and the performance of imaging equipment;In the transmission process,due to a series of operations such as channel compression and coding,the terminal receives degraded low-resolution images and videos.Super-resolution reconstruction technology can reconstruct and restore the prior information contained in low-resolution images or videos to obtain high-resolution images or videos.With the development of deep learning technology,the super-resolution reconstruction technology based on deep learning has greatly improved the reconstruction performance and efficiency.However,the current super-resolution reconstruction algorithm is limited by the limited information retained by the low-resolution image when performing image super-resolution reconstruction,so most of the research only focuses on ×2 times and ×4 times.There is little discussion on super-resolution reconstruction with high magnification(The reconstruction magnification is ×8 times and above).In video super-resolution reconstruction,the frame alignment method completes the alignment of adjacent video frames through the optical flow network.The research focuses on improving the accuracy of the optical flow calculation,while ignoring the deep mining of the information contained in the video frame sequence.In the frame non-alignment method,the feature information richness of the video frame sequence extracted by the network is limited by the number of feature channels,which affects the quality of the reconstruction results.In this dissertation,the above problems have been deeply studied,and a more efficient network structure has been designed to improve the quality of super-resolution reconstruction results.The main work and innovation of this dissertation are as follows:(1)Multi-task learning image high magnification super-resolution reconstruction.The high frequency information contained in the low resolution image degraded by high magnification is very serious,the detail texture is difficult to distinguish,the image contour is blurred,and the super-resolution reconstruction is difficult.Based on multi-task learning,this dissertation decomposes the image high magnification super-resolution reconstruction task into different subtasks,and trains each sub-task through the different magnification down-sampling data to obtain the super-resolution reconstruction model of different sub-tasks.The network model of each subtask is a part of the high magnification super-resolution reconstruction.The network models of different subtasks are connected in a cascade way to complete the high magnification superresolution reconstruction of the image.Since the network models of different sub-tasks are trained with the training data of the corresponding reduced sampling rate,the network models of different levels of sub-tasks can carry out targeted reconstruction according to the distribution characteristics of the image texture of the reduced sampling to this level.By cascading,the reconstruction advantages of different hierarchical network models are accumulated layer by layer,and the final high-quality image high magnification super-resolution reconstruction results are obtained.(2)Video super-resolution reconstruction based on multi-scale and non-local depth feature mining.In the optical flow super-resolution reconstruction network,the accuracy of optical flow calculation between adjacent video frames has a great impact on the results of super-resolution reconstruction.How to improve the accuracy of optical flow calculation is the research goal of most methods,but these methods ignore the depth mining of information contained in video frames.The optical flow super-resolution reconstruction network structure proposed in this dissertation includes a non-local module and a multi-scale feature fusion module.The non-local module can calculate the similarity relationship between the pixels in the video frame,extract the self-similar feature information of the video frame,and the multi-scale feature fusion module extracts the different scale features of the input network data in the optical flow network,fully mining the feature information of the video frame.The feature information is fully fused by different connection methods,and the optical flow calculation results and self-similar results are spliced and sent to the super-resolution reconstruction network.In the super-resolution reconstruction network,the multi-scale feature fusion module is also used to extract and fuse the multi-scale features of the input data until the result of video super-resolution reconstruction is obtained.The experimental results show that the proposed method has high quality for the reconstruction of video frame sequence.(3)Regional focus and feature aggregation enhance the feature richness of video superresolution reconstruction.In the network using motion estimation and motion compensation for video super-resolution reconstruction,inaccurate motion estimation will cause the quality of super-resolution reconstruction results to decline.At the same time,the number of features of the convolutional neural network is limited by the data of the feature channel,resulting in the limited richness of the feature information extracted by the network.In this dissertation,a region-focused recurrent recurrent recurrent neural network is proposed to perform video super-resolution reconstruction.The recursive network structure does not require the calculation of motion estimation and motion compensation for video frames,so as to avoid the impact of inaccurate motion estimation on super-resolution reconstruction results.After the video frame is input into the network,the regions with different reconstruction difficulties in the video frame are concerned through the region attention module,and the feature information of different regions is extracted from the shallow layer to the deep layer.The extracted feature information is connected to the last layer of the network through the feature aggregation structure to participate in the super-resolution reconstruction of the video frame,which increases the richness of features involved in the super-resolution reconstruction of the video frame and improves the quality of the reconstruction results.At the same time,in the process of network training,it is difficult to reconstruct the texture-rich areas in the video frame.The loss between the reconstruction results and the labels can be guided by the feature aggregation structure to focus on the study of the texture-complex areas in the video frame by the region focus module,and improve the reconstruction quality of the texture-complex areas.