Research On Specular Object Surface Highlight Removal Based On Improved Dual Generative Adversarial Network

Highlight on the optical images of high-gloss leather,glass,plastic,metal parts and other mirror-reflective objects make it difficult to directly apply optical measurement techniques such as object detection,intrinsic image decomposition and tracking suitable for objects with diffuse reflection characteristics.Therefore,highlight detection and removal has become a hot issue in the research field of computer vision.In recent years,the deep convolutional neural network represented by CNN as a generalized method of function mapping relationship fitter has shown wide superiority in the field of image data processing research.Deep convolutional neural networks can perform supervised learning on a large number of paired specular-diffuse datasets,using the specular-to-diffuse domain mapping to remove surface highlights.However,The acquisition of paired datasets has certain difficulties in reality,but we can easily obtain a large amount of unpaired data.The dual generative adversarial network led by CycleGAN is an unsupervised learning framework that makes full use of the unpaired dataset to learn the mapping relationship between two domains.This paper establishes an improved dual generative adversarial network framework based on the classic CycleGAN,which mainly solves the following two problems:(1)A confidence map based on independent averages is proposed on the basis of the dual generative adversarial network CycleGAN.The CycleGAN + confidence map can guide the network to search for initial values quickly,thus solving the current problem of slow network convergence due to the lack of a very strict mathematical definition to distinguish between specular and diffuse reflective components.(2)For a typical high-gloss metal sphere,the diffuse component in the off-peak part of the measured value is only around 0.1,while the specular component in the peak can reach100.In order to solve the problem of anisotropy in the optimization process caused by the peak specular reflection of a high-gloss specular-reflective object being much larger than the diffuse reflection value when it is not peak,we propose a logarithm-based metric that makes the specular and diffuse reflection components comparable.At the same time,the combination of Dense Net and U-Net is used in the generator of the improved network instead of the initial Res Net architecture,which allows the network to have fewer parameters and a deeper hierarchy,thus allowing the network to converge faster and to extract richer feature information.In addition,this paper introduces perceptual loss based on the improved network to avoid the loss of relevant feature information in convolutional downsampling to ensure the stability of the generated images.