Research On Image Super-Resolution Methods Based On Deep Learning

Resolution is an important measure to reflect the richness of image information.Images with high resolution(HR)are highly desired and can offer more meaningful details that would be critical in various applications.Single image super-resolution(SR)refers to enlarging a low resolution(LR)image to restore the corresponding high resolution image which is one of the research focuses in the field of image restoration.However,there are some deficiencies in traditional SR algorithms,such as great computational load,lack of finer details in results,blurry effect,block effect and etc.To solve those problems,we conduct the study of SR under the framework of deep learning.(1)We first make a thorough overview of the current SR algorithms,and then introduce the theoretical basis of super-resolution and deep learning(2)We propose conditional generative adversarial network(cGAN)as the solution to SR problem,where we add novel skip connections to the encoder-decoder generator of cGAN and design a PatchGAN discriminator based on image Markov random field.The skip connections could shuttle the shared low-level information directly across the net,and the PatchGAN discriminator only distinguishes real and fake at the scale of patches.This is beneficial because PatchGAN has fewer parameters,thus runs faster.In order to better maintain the low-frequency information and recover the high-frequency information,we propose a generator loss function combining the adversarial loss term and the L1 loss term.The former term is beneficial to the synthesis of fine-grained textures,while the latter is for learning the overall structure of LR input.Experiments demonstrate that the proposed method could generate HR images with richer details and less over-smoothness.(3)We also propose the pixel probabilistic model for super-resolution based on human visual saliency mechanism(HVSM).The deep SR architecture comprises of a PixelCNN and a residual network(ResNet),in which PixelCNN predicts the serial dependencies of the pixel sequence and ResNet for capturing the mapping.relationship between the LR input and HR output.Meanwhile,a region-specific reconstruction strategy is adopted to efficiently reduce the computational complexity.Firstly,a saliency map is calculated using a deep learning method,then pixels lying in salient regions are processed through the proposed deep network,while the non-salient pixels are generated in a fast interpolation-based manner.Additionally,we present a Bayesian optimization technique to automatically determine the optimal weight hyperparameter of loss function,avoiding costly manual parameter tuning.Furthermore,a modified image quality assessment based on HVSM is introduced,trying to align with the human visual perception.Experiments have validated the superiority of our method in SR of particular small inputs.