Research On Accurate Face Makeup Transfer Method Based On Generative Adversarial Network

As an important branch in the image style transfer field,makeup transfer has received extensive attention in recent years due to its high application value in beauty cameras,webcasting,cosmetics retail,and other fields.The purpose of face makeup transfer is to transfer the makeup components(such as foundation,eye shadow,lipstick,etc.)in the reference image to the corresponding area of the source image.However,due to the variety and complexity of makeup styles,there are still many challenges to be overcome in precise makeup transfer.This paper firstly concludes the technical development route of makeup transfer in recent years from two levels based on traditional image processing and deep learning.We found that in addition to the difficulties of rich makeup styles,various combinations,and the lack of paired image datasets,the implementation of accurate makeup transfer has the following three problems:(1)The face identity of the source image will change during the makeup transfer process;(2)When the background is complicated,the transfer process may modify the background information;(3)There are obvious artifacts at transfer combined positions.Although deep learning techniques have achieved good results in makeup transfer,most of them only concentrate on one or two requirements,and it is difficult to realize facial identity preservation,background preservation,and accurate makeup transfer at the same time.To solve the above issues,this paper proposes a accurate face makeup transfer method:RAMT-GAN(Realistic and Accurate Makeup Transfer with Generative Adversarial Network)to achieve realistic and accurate makeup style transfer.Specifically,we utilize a dual input/output network that builds on the BeautyGAN architecture:(1)We introduce an identity preservation loss,which leverages the foreground parsing mask to obtain identity loss to maintain the face identity;(2)The background invariant loss is proposed to maintain the background details,which minimizes the background difference between generated results and reference images;(3)By introducing the makeup loss of four local regions(face,eyes,lips,eyebrows)and combining with histogram matching to realize an accurate makeup transfer.Finally,this paper conducts qualitative and quantitative experimental evaluation of RAMT-GAN.The former is mainly from the visual level,including the research and comparison of results generated in normal poses and different poses.The latter is based on a user study,beauty prediction scores and facematch distance to proves RAMT-GAN is superior to the existing methods.Furthermore,we conduct ablation experiments on the identity preservation loss,background invariant loss,and makeup loss in the RAMT-GAN.Extensive experiments demonstrate that the proposed makeup transfer model can synthesize makeup faces with accurate reference style as well as maintaining the identity feature and the background information.