Image Inpainting Based On Generative Adversarial Network

As an important branch in the field of digital image processing,digital image inpainting is mainly a technique that uses the computer to learn the feature information of the image,restore the missing information,and then automatically inpaint the missing regions of the image.Digital image inpainting has a broad application prospect,and has been applied in cultural relic restoration,satellite remote sensing,medical images and other fields.However,there are still many problems and challenges in today's image inpainting models.This thesis proposes two kinds of image inpainting models based on generative adversarial network(GAN).The main research content is as follows:Firstly,it is difficult for existing image inpainting models to inpaint an image as complete and detailed as a real image.To this end,an image inpainting model based on image patch similarity is proposed.First,by calculating the similarity between image patches in the real image and the inpainted image,we can establish an index to measure the texture details of the inpainted image,and a loss function based on the index.Then,by optimizing this loss function during the training process,we can inpaint realistic and detailed images.Finally,for the common convolution in the image inpainting models,the convolution operation is also carried out on the invalid pixels of the missing regions,leading to visual artifacts such as blurring,color discrepancy and checkerboard effect.To solve this problem,we propose an improved residual gated convolution.By making the model itself learn the mask update strategy to control the output value of convolution,we improve the utilization of valid pixels in the complete region and ignore invalid pixels in the missing region.Secondly,most of the existing image inpainting models are deterministic models,which can only get the same result for a single image to be inpainted,and cannot produce diverse results.To solve this problem,this thesis combines a variational autoencoder(VAE)with GAN and proposes a diverse image inpainting model based on VAE-GAN.First,we can obtain the normal distributions from the complete input image by the encoder in VAE and then reconstruct the input image by the decoder.Then,we can reduce the KL divergence between the normal distributions obtained from the image to be inpainted and the complete input image,so that during the inference process,we can obtain the diverse inpainting results by sampling from the normal distributions of the image to be inpainted.Further,a discriminator is used at the end of the network to to distinguish the reconstructed and inpainted images,so as to improve the authenticity of the inpainted image.After that,disentangled representations are adopted for the latent variables,which are sampled from normal distributions.So,wo can improve the latent variables' ability to represent factors in the images and further increase the diversity of inpainting results.Finally,Each region of the image has a different effect on the inpainting of a specific region.So it is necessary to increase the participation of key regions in the image inpainting process.By using self-attention mechanism,this inpainting model can learn an attention map,which can increase the weight of features that play a major role in the inpainting,and reduce the weight of secondary features,so as to improve the efficiency of the inpainting model and enhance the inpainting effect.Experimental results on image inpainting benchmark datasets such as CelebA,Places2 and PSV show that the proposed two image inpainting models solve the problems of lack of details and diversity in the results of existing image inpainting models.There are 38 figures,9 tables,and 98 references in this thesis.