Depth Map Completion Of Indoor Scenes Guided By Color Image Feature

In the field of computer vision and computer graphics,scene depth map completion is a very important research topic.Depth information captured when shooting with RGB-D cameras has a wide variety of applications in fields such as autonomous driving,3D completion,virtual reality and augmented reality.However,due to the defects of indoor environment with cluttered objects and mutual occlusion,depth sensor device light-sensitive devices and other things,the depth information collected by them is not complete.Therefore,how to effectively complete the depth map of indoor scenes is a difficult problem.Existing methods are often based on deep convolution neural networks to repair the complementary scene depth map.However,such methods cannot effectively repair local high-frequency information such as object edges and surface details within the scene.In this paper,we propose two kind of methods to guide the scene depth map completion using color images of the scene,by introducing attention mechanism in the deep convolution neural network.The main research contents and findings of this paper are as bellow:(1)Depth completion based on a channel attention mechanism.In order to effectively use the color map of indoor scenes to complete the depth map completion task,a depth map completion network based on the channel attention mechanism is proposed.The network takes a depth map and a corresponding color image of the scene as input,and uses the color image to guide the task of the depth map completion.The joint features of the scene color map and depth map are first extracted,and the extracted joint features are decoded according to the channel attention mechanism to obtain the initial predicted depth map;then the predicted information of the scene depth is gradually optimized with the help of the propagation algorithm on the non-local region to obtain the complete scene depth map;finally,experiments are conducted on public datasets such as Matterport3D,and the method is analyzed in comparison with typical methods.The method achieves 92.3%accuracy at a threshold value of 1.25,95.9%accuracy at a threshold value of 1.25~2,and 97.4%accuracy at a threshold value of1.25~3.Among them,the channel attention mechanism module can effectively enhance the channel weights of high-frequency components in the feature map.The method fuses the scene color map and depth map feature information,improves the performance of the depth map completion network through the attention mechanism,and effectively completes the missing depth information when the depth camera captures indoor scenes.The raw incomplete scene depth map and recover its structural information.(2)Depth completion based on pyramid squeezed attention mechanism.In order to fully extract the multi-scale joint features of the depth map and color map of indoor scenes,a depth map completion network with a pyramid squeezed attention mechanism is proposed.Using the scene depth map and the corresponding color image as input,the network first extracts and fuses the multi scale features of the color image and the depth map,and decodes the extracted joint features by convoluted based on the pyramid squeezed attention mechanism to obtain the initial predicted depth map;then,alliterative optimizes the predicted scene depth information with the help of a canonical neighborhood propagation algorithm;finally,the network is used to obtain a complete and accurate scene depth map completion using this network.Among them,the pyramid squeezing attention mechanism module can fully extract the multi-scale features of the depth map and color images,and assign appropriate weights to different feature channels.The experimental validation is carried out on public datasets such as Matterport3D.The method achieves 93.8%accuracy at a threshold of 1.25,96.7%accuracy at a threshold of 1.25~2,and 97.9%accuracy at a threshold of 1.25~3.The experiments show that the method can fully extract the features of color image and depth map and effectively complement the depth map of indoor scenes,and effectively improve the overall accuracy of the depth map of scenes,and the edges of the complemented depth image are clearer and the structure is completed.