Reserch On Deep Learning Based Image Restoration Method

As an important carrier for the human to acquire information,a high-quality image typically contains rich texture details,appropriate brightness and contrast,as well as realistic and rich colors,which can give viewers a pleasing visual effect and help improve the performance of many computer vision systems.However,owing to various objective impact factors such as climatic conditions and hardware equipment,etc.,the acquired images could suffer from different degrees of degradation,resulting in producing low-quality images.This could affect the visual authenticity of human perception,and increase the difficulty of analyzing and processing image content in the later stage.Image restoration,aiming at recovering a high-quality image from its corresponding low-quality version,has essential research significance and application value in the image processing and computer vision fields.Currently,deep learning techniques,especially the deep Convolutional Neural Network(CNN)as a representative,have unique advantages for solving various vision-related issues,this can be attributed to the fact that CNN can automatically learn and integrate highly discriminative and representative features through multi-layer convolution operations.In recent years,deep learning has greatly promoted the development of image restoration,and a variety of deep learning-based methods have also emerged.Depending on how the network architectures are built,these deep learning-based methods can be divided into three categories:(1)image restoration method based on heuristic artificial network design;(2)image restoration method based on neural network architecture search;and(3)image restoration method based on network unrolling iterative algorithms.Based on deep learning theory,this dissertation carries out the research on the above three image restoration methods.The main contents and innovations are as follows:(1)In the research of image restoration method based on heuristic artificial network design,considering the diversity of image restoration tasks,this work takes the representative single image de-raining task as a research object,and designs a Joint Depth and Density Guided De-raining(JDDGD)method.Inspired by the relationship between the rain imaging and the scene depth,a Depth-Density Inference Network(DDINet)is manually designed to extract the scene depth and rain-streak density information from an input rainy image,followed by a manually-designed Density-Density-based Conditional Generative Adversarial Network(DD-CGAN)to exploit the depth and density information provided by the DDINet to direct the adaptive removal of rain streaks and fog in the input rainy images.In addition,to prevent the spatially-varying local artifacts,an effective global-local discriminators structure is introduced into the proposed DD-CGAN to globally and locally inspect the de-rained images.In order to achieve the best de-raining performance,multiple loss functions including multi-scale pixel loss,multi-scale perceptual loss,and global-local generative adversarial loss are jointly used to train the proposed JDDGD.Both quantitative and qualitative results show that the proposed JDDGD achieves superior performance to previous non-guided,density-guided,and depth-guided de-raining methods.(2)In the research of image restoration method based on neural network architecture search,this work takes the tasks of image de-noising and image de-raining as the research object,and proposes a Multi-scale Attentive Neural Architecture Search(MANAS)method,aiming to get rid of the daunting procedure of the artificial network design.In MANAS,a multi-scale attention search space is first formulated,which contains multiple flexible modules that are favorite to the image restoration task.Then under the search space,multi-scale attentive cells are built,which are further used to construct a powerful neural network for image restoration.The internal multi-scale attentive architecture of the network is searched automatically through a gradient-based search algorithm.Thus it can get rid of the daunting procedure of the artificial network design to some extent.Moreover,in order to obtain a robust model for image restoration,a practical and effective multi-to-one training strategy is also presented to allow the network to get sufficient background information from multiple degraded images with the same background scene,and meanwhile,multiple loss functions including external loss,internal loss,architecture regularization loss,and model complexity loss are introduced to jointly optimize the model parameters,aiming to achieve robust image restoration performance and controllable model complexity.Quantitative and qualitative results on both image de-noising and image de-raining tasks show that the proposed MANAS learned by multi-to-one training strategy outperforms multiple state-of-the-art deep learning-based image restoration methods.(3)In the research of image restoration method based on network unrolling iterative algorithms,this work takes the most representative image compressed sensing reconstruction task as a research object,and proposes a Compressed Sensing(CS)reconstruction method based on network unrolling iterative algorithms,named Proximal-Gen.The proposed Proximal-Gen first formulates a general domain of the recovered signals,which allows the subsequent reconstruction methods to recover the signals that deviate from the generative space.Then,based on the general domain,a generative model-based projection gradient descent algorithm is used to recover an intermediate signal lying in the generative range of a generator function,and a denoising model-based proximal gradient descent algorithm is used to recover a deviation signal free from the limitation of the generative space.It is worth noting that the input compressed measurements passing through the network of Proximal-Gen is equivalent to executing the iterative algorithms a finite number of times.Therefore,different from the JDDGD and MANAS which lack interpretability,both generative model-based projection gradient descent algorithm and denoising model-based proximal gradient descent algorithm in the proposed Proximal-Gen are expressed as a neural network,thus the whole unrolled neural network can be natually interpreted as a parameter optimization algorithm.Both quantitative and qualitative results show that,in comparison with multiple current generative model-based CS reconstruction methods,the proposed Proximal-Gen can achieve better reconstruction performance and higher reconstruction efficiency under most measurements.