Research On Image Inpainting Method Of Structure Reconstruction And Texture Synthesis Based On GAN

Recently,due to the vigorous development of computer technology,digital image processing technology has attracted more and more attention from experts and scholars.Nowadays,digital image processing technology plays an important role in people’s daily life and various work industries.With the generation,storage and transmission of massive digital images,it is inevitable that the image will be disturbed by external factors at a certain stage,resulting in the loss of partial information of the image.Thus,digital image inpainting technology came into being.Image inpainting refers to using the information of the known area in the image to fill the missing region of the image through certain restoration criteria,such as the known area and the damaged area have the same geometry structure or statistical characteristics,so as to make the repaired image have ideal visual effect.At present,most of the existing image inpainting methods are not satisfactory in reconstructing the image structure,especially when the important parts of the image are missing.Some methods focus on reconstructing a continuous and reasonable structure between the missing area and the undamaged area,but when restoring the image texture,they will generate fuzzy textures inconsistent with the surrounding area.In order to make the inpainted image have the continuous structure and vivid texture,this thesis propose an image inpainting method using edge prediction and appearance flow.Our image inpainting method mainly includes the following three stages.The first stage is the edge generation.Firstly,the RTV method is used to generate a smooth image,which can represent the global structure of the original image.Then the edge map of the smooth image is obtained by Canny detector,and the edge generator approximates the image structure information by predicting the edge data of the missing area of the smooth image representing the global structure,so as to obtain the whole edge structure of the image.The second stage is the smooth structure reconstruction.It takes the edge image and the damaged smooth structure image as the inputs,and predicts the smooth structure of the damaged area through the structure reconstructor to restore the global structure of the smooth image.The third stage is the texture generation.The texture generator takes the damaged image and the smooth structure image generated in the previous stage as the inputs,and obtains the corresponding feature image through encoding.The appearance flow is used to generate the matrix representing the correlation between the regions of the image,that is,the appearance flow field,which makes the pixels from the source region “ flow ” to the damaged region to generate a vivid texture.Then the features of the undamaged region are transferred to the damaged region by Gaussian sampling,and the image finally repaired is obtained by decoding.The texture generator with appearance flow operation can generate vivid texture after the reconstructed structure image obtained.We conducted experiments on two datasets: Places2 and Celeb A.Compared with the existing methods,the image structures repaired by this method are more reasonable,the textures are more vivid and the performance of this method better.