Research On Image Inpainting Based On Attention Mechanism

Digital image inpainting is a process using computer programs/software to automatically infer and correct missing information or removed objects in images.The process involves inferring and reconstructing the missing portions of images from known data.This process has extensive applications which include but are not limited to restoring old photographs,artworks,removing undesirable objects and video editing.Presently,image inpainting methods rooted in traditional techniques,specifically those based on partial differential equations,sparse representation and texture information,are primarily implemented to fill in large image areas or correct for missing sample resources.Nonetheless,image inpainting using deep learning is predominantly performed through self-encoding networks,generative adversarial networks and Transformer techniques,which can semantically restore a location even when sample resources are absent.Notably,there still remain some issues that require attention and remedy,such as mismatches between the filled areas and the surrounding information in images featuring large missing areas,complex structural information and textures.Hence,the purpose of this paper is to propose a more effective image inpainting model,while also detailing the principal functions of the proposed model and its contributions.(1)Currently,image inpainting methods that attempt to consider both texture and structure still lack in consistent semantic restoration.To address this issue,we propose an AU-GAN(Attention U-shaped convolutional neural network-Generative Adversarial Networks)based image inpainting method that uses mask position encoding.Our main contributions include: 1)incorporating mask position encoding into the generator to accelerate the convergence speed and improve the accuracy of content filling,2)improving the texture quality of the image by introducing an attention mechanism into the generator network,and 3)enhancing the adversarial loss to better restrict the content of generated images and promote high-quality image inpainting.(2)In comparison to the Attention Mechanism,the Transformer model has more effectively accomplished global modeling and perception capabilities,enabling better control of global content and more accurate semantic filling.Therefore,we propose a transformer-based image inpainting technique that replaces the Attention Mechanism to enhance the image inpainting quality.We performed the following main tasks: 1)We modified the Transformer model to create two branches and reduce computation time,2)we extracted texture and detail features of the image to be restored by the Transformer model and added these features to the U-Net network to improve the image generation quality,and 3)we added style loss to make the restored images more coherent.Our restoration methods were validated by conducting subjective and objective quantitative analysis on CelebA and Paris databases,thereby demonstrating their effectiveness.