Research On Metal Surface Highlight Removal Method Based On Dynamic Adaptive Mask

Detection algorithms based on computer vision rely heavily on digital images,where the quality and content of the images often affect the algorithm’s results.However,some objects,due to their material properties(such as metallic objects),are prone to producing specular reflections,which can cause the loss of valuable information in the image.This interference is particularly severe in the presence of external light sources.Moreover,in certain scenarios,specular reflections occur more frequently than in others,such as in industrial scenes,where the objects being detected are often made of metal and are therefore more likely to produce specular reflections that can affect detection results.Additionally,vehicles on the streets in outdoor scenes are prone to producing specular reflections under sunlight,which can damage the information of the vehicles in images or videos,thus affecting the accuracy of the resulting 3D information and matching algorithms.Therefore,research on specular reflection removal is of great practical significance.In this thesis,we conducted research on specular reflection removal using deep learning methods,focusing on scenarios where specular reflections occur frequently.The main contributions of this thesis are as follows:(1)This thesis constructed two high-gloss region pixel-level classification datasets and a binocular visual stereo matching dataset for nine types of high-gloss influence matching problems.The two high-gloss pixel classification datasets are based on a metal grayscale synthesis dataset and a real-world metal high-gloss dataset,which include objects with high-gloss and non-high-gloss states under different materials,light sources,and surface shapes.The binocular visual stereo matching dataset is based on the KITTI 2012 and KITTI 2015 datasets,and by adjusting the brightness and contrast of objects,it is divided into the following cases according to the degree of glossiness: large-area high-gloss regions that exist in the same area of the left and right views of the same object,high-gloss regions that exist in different areas of the left and right views of the same object,and objects with high-gloss regions in only a single view,which fill the gap in this type of dataset.(2)In 2D image scenes,the use of computer vision methods for measurement and reconstruction of metal objects is affected by surface specular highlights,leading to erroneous results.Moreover,due to the variability of metal objects and the complexity of the environment,various kinds of highlight removal is a challenge.To address this issue,this thesis proposes a widely applicable two-stage encoding-decoding network architecture,which solves the highlight problem on metal surfaces of industrial scenes without changing the original lighting configuration,and is suitable for situations where ground truth data of industrial metal highlights are not available.Firstly,a pixel-level classification highlight detection module is used to divide the original image into highlight and non-highlight regions using an interest mask,and a pixel-level classification dataset of highlight regions is created for testing and training.Then,a highlight removal module combined with partial convolution is used to remove the highlight regions of objects in the image.Finally,a synthetic highlight dataset and an industrial highlight dataset are used to validate the effectiveness of the method.(3)In 3D scenes,the binocular visual detection method is effective for reconstructing objects such as human faces and animals.However,the surface texture features of metal materials are relatively limited,and they are easily affected by lighting interference during the reconstruction process,which affects point matching and reconstruction,especially for complex specular highlights,such as large specular highlight areas,different specular highlight areas in left and right views,and only one image contains specular highlight areas.In this thesis,we propose a disparity repair method based on a dynamic mask fusion convolution offset specular highlight removal method to solve such problems.Firstly,we use a pre-training method to generate an adaptive dynamic specular highlight mask that preserves some of the original features of the metal surface.Then,based on the dynamic specular highlight mask,we calculate the offset of the convolution kernel,adjust the shape of the kernel to better match the structure of the metal surface.Finally,we use a two-level neural architecture search method to derive the feature network and matching network.