| In recent years,deep learning has achieved a great success in the field of computer version due to the rapid development and extensive applications.Image restoration based on convolutional neural network becomes a hot topic in CV community and many excellent results have been achieved.As one of the research hotspots,image restoration has wide application scenarios including restoring fragmentary photos,removing head devices such as glasses,and removing debris from scenic photos.However,existing facial image completion algorithm results still lack fine facial details due to the large occlusions and complex facial texture.In order to solve the above problems,we proposed a brand new reference image guided algorithm which restores more vivid facial features.We firstly implement the Occluded facial landmark detection algorithm for making better use of the mentioned guidance image.The landmark detection algorithm uses the multi-layer convolution neural network to extract the feature information of the guidance image and the occluded image respectively,and then the Spatial Transformer Network which make use of the guidance feature is introduced as a affine transformation network.Experiments validate the effectiveness of our network in generating satisfying 68 landmarks on occluded images across variation in pose and expression.We use Moving Least Square algorithm to warp the guidance image through setting the facial landmarks of the occluded image as the target coordinates while the landmarks of the clear reference image as the source coordinates.Experiments validate the effectiveness of providing more meaningful guided information form the warped image which has the same pose with the occluded face,and the facial features of the character can also be maintained.In addition,we proposed a new global and multi-local discriminator algorithm based on Generative Adversarial Network(GAN).The introduction of multiple discriminant networks makes the algorithm pay more attention to the details of facial features.Through the experiments on CASIA-Web Face dataset,the proposed inpainting method outperforms the existing algorithms,and generates satisfying results on occluded facial images across variation in pose and expression. |
