Semantic Feature Fusion For Large Hole Image Inpainting

Image inpainting aims to fill holes in images,which originated from the crack filling of fine art works in the Middle Ages.With the development of electronic technology,image inpainting is widely used in image compression,information security,cultural relics protection,film and television production and other fields,which puts forward higher requirements for inpainting technology.Holes may contain complex structures or different layouts.Image inpainting needs to make reasonable assumptions about the hole information and then construct a suitable model.When the hole is large,a lot of image information will be lost,so large hole image inpainting is a challenging task.Traditional inpainting methods can only satisfy the consistency of low-level features,but can’t satisfy the rationality of high-level features,so they can’t guarantee the texture clarity and structural consistency when applied to large hole inpainting.Recently,image inpainting methods based on deep learning have made remarkable progress on rectangles and irregular holes,and generative adversarial networks have been able to produce more realistic and semantically consistent.However,most image inpainting methods can’t achieve the desired results in the image inpainting of large holes,especially in the preservation of texture and structure.In order to solve these problems,this paper proposes a large hole image inpainting method based on semantic feature fusion.First,in the decoder stage,the input layer is used to modulate the deep information of the network,which improves the distribution of the deep features of the network and effectively guides the loss gradient propagation.Then,the self-attention method is used to solve the long-distance dependence of effective information to holes,which effectively improves the texture details of the image.Finally,the structural consistency and color consistency of the image are improved by the joint loss supervision mechanism with higher receptive field.Experimental results show that the proposed inpainting network performs better than other state-of-the-art inpainting methods on CelebA-HQ and Places.