Research On Face Image Relighting Technology Based On Decomposition Optimization

Lighting is crucial for the visual quality and information conveyance of face images.Difference in lighting conditions affect the visual effects of an image and information conveyed by the image.The face image relighting technology aims to keep the original face in an image unchanged and re-render its image under arbitrary given lighting condition.This technology can change the lighting condition in an image after which is acquired,so as to achieve the purpose of changing the visual effect of the image,which can be applied in various fields such as digital image editing and face detection.Existing face image relighting methods mainly utilize the deep neural network models to first decompose the input image into face feature and illumination information,and re-render the relit image after replacing the illumination information.During the process,the decomposition of the original image largely determines the relighting results.Therefore,this paper focuses on the challenges in the decomposition process for the face image relighting task and conduct the following research:(1)Aiming at the problem of generating spots in the relit images due to the lack of geometric structure features in face information decomposed by the relighting model from a single input image,a depth feature fused face image relighting method is proposed.A deep neural network model with a dual-branch encoder structure is designed to extract both features from the input image and the corresponding depth map,thus enhancing the model’s understanding ability of the face geometric structure through integrating image feature and depth feature extracted by the two encoder branches.Additionally,a feature fusion module based on the Vision Transformer structure is designed to perform feature fusion between the different levels of the two encoder branches.This fusion module compensates the geometric information in the image features of different scales,so as to further improve the model’s ability to grasp geometric information of the face in the image.Comparative experimental results on multiple datasets show that the proposed method can avoid to generate spots in the relit images.(2)A perceptual optimized self-supervised face image relighting method is proposed to address the challenge of high dependence on paired datasets of fully supervised face image relighting methods.The method transfers the illumination of a reference image to the original face image.To ensure the feasibility of self-supervised training,a strategy of multiple model reuse is adopted to reconstruct the original image,facial features,and lighting information,and a bidirectional reconstruction loss is designed to supervise this reconstruction process.Moreover,to address the issue of the inability to directly supervise the relit images in the self-supervised training strategy,a multiscale bidirectional perceptual loss is designed to enhance the facial feature consistency between the relit image and the original image at different scales,as well as the global lighting consistency between the relit image and the reference image,thus improving the visual quality of the relit images.Comparative experiments demonstrate that the proposed method can achieve high visual quality while completing the relighting task using a self-supervised strategy.