| With the rapid development of computer vision and identity authentication technologies,the application value of face images is increasing in today’s society,and the incomplete face image restoration technology also has great research significance.Different from other natural scenes,face images have strong semantic and highly structured characteristics.Therefore,it is difficult for traditional image inpainting algorithms to achieve satisfactory results in face images.In recent years,deep learning technology has achieved outstanding performance in the field of computer vision,and also gradually occupied the mainstream position in image inpainting tasks.In order to improve the visual authenticity and content consistency of the repaired face image,this paper further investigated the deep learning based inpainting method for mutilated face images under the conditions of rule holes and irregular holes respectively.The main research contents are as follows:(1)The idea of multi-scale feature extraction is introduced into the two-stage image inpainting network model,and the generator network structure and contextual attention module are improved to propose a face image inpainting algorithm based on multi-scale convolution and contextual attention.In the coarse inpainting stage of the proposed algorithm generator,multi-scale facial feature information is extracted in parallel through convolution of different scales,which improves the content rationality of the coarse repair results.In the fine inpainting stage,the multi-scale contextual attention module is further used to refine the semantic and texture features of the face image to enhance the consistency of the image context.The algorithm produces better inpainting results compared with the selected other image inpainting algorithms in the case of face images with regular shape of missing regions.(2)This paper further considered the irregularity of the incomplete area of the face image,a face image inpainting algorithm based on contour prior and multi-scale feature aggregation is proposed for the characteristics of independent structure and rich semantics of the face image.The proposed algorithm consists of two parts: contour inpainting model and image inpainting model.The face contour is first repaired by the contour inpainting model,and then the repaired contour is used as a structural prior to guide the reconstruction of the face image content.The generator adopts gated convolution instead of ordinary convolution,and the discriminator adopts the PatchGAN structure of spectral normalization.The gated convolution mechanism and PatchGAN enable the model to learn the effective information in the irregular incomplete image,and the spectral normalization can play a role in stabilizing the generative adversarial networks training.This paper also designed the dense multi-scale fusion residual module and the attention weighted aggregation module to improve the network feature extraction ability,enhance the semantic consistency of face images and repair texture details.In this paper,the proposed two algorithms are fully experimentally validated on CelebA and CelebA HQ face datasets,and the repair inpainting is compared with other related algorithms,proving that the proposed face image inpainting algorithm has excellent performance in both visual perception and quantitative evaluation. |
