Single Depth Map Super-Resolution Reconstruction Based On Convolutional Neural Network

With the development of computer vision,more and more attention has been paid to the acquisition and processing of depth information.Depth information is widely used in robot navigation,public safety,medical treatment and other fields,it can be obtained by depth camera.Due to hardware constraints,depth maps tend to have low resolution.In order to meet the needs of practical application,without changing the hardware system,it is very desirable to develop an effective and efficient super-resolution reconstruction technology of depth maps.To improve the reconstruction quality of depth map,based on convolutional neural network,the dissertation studies the single depth map super-resolution reconstruction methods.The main work is as follows:(1)According to the different classification of reconstruction methods,the research status of depth map super-resolution reconstruction is studied.We analyze the characteristics of depth map and degradation process,introduce the basic concept of convolutional neural network and the evaluation criteria of reconstruction.Moreover,we compare several classical depth map super-resolution reconstruction networks.(2)On the one hand,using the up-sampling depth map as the input of convolutional neural network will produce inaccurate initial pixel value and will slow down the speed of network training,and increasing the number of training pixels will make the network training time longer;On the other hand,multi-channel training can not calibrate the feature channel weights autonomously.For both,we study and implement a single depth map super-resolution reconstruction convolutional neural network combined with sub-pixel assembly and channel attention mechanism.The low-resolution depth map input in this algorithm does not need to be preprocessed,the original low-resolution depth map is directly trained by multi-channel feature channel after being input into the network,at the same time,the response of the feature channel is self-adaptive calibrated by adding channel attention mechanism module.Finally,the sub-pixel assembly is used to arrange and combine the learned multi-channel nonlinear mapping to obtain the reconstructed depth map at the end of the network,and an end-to-end depth map super-resolution reconstruction network is established.The experimental results show that the algorithm performs have a good performance on objective evaluation index and good reconstruction quality.Not only the training speed is accelerated,but also the algorithm results on the test set are better than most of the existing image super-resolution reconstruction algorithms.(3)In view of the existing depth map super-resolution reconstruction algorithm based on convolutional neural network,the learning of mapping relationship is limited in the localreceptive field,we study and implement a single depth map super-resolution reconstruction convolutional neural network based on non-local means constraint.The low-resolution depth map will be up-sampled through sub-pixel assembly layer after training in the sub-pixel convolutional neural network.Then based on the classical non-local means algorithm,we use the self-similarity of depth map to impose the non-local means constraint on the upsampling result to get the final reconstruction result,and an end-to-end depth map super-resolution reconstruction network is established.The experimental results show that the generated high-resolution depth map has clear details and good visual effect,the algorithm results on the test set are better than the existing image super-resolution reconstruction algorithm.