Depth Estimation On 4D Light Field Based Convolutional Neural Network

Traditional cameras can only record a two-dimensional plane,they can’t get scene depth information.With the continuous development of imaging theory,the technology of light field came into being which can obtain richer light field information in stereo space.Using light field camera to capture light information to obtain scene depth has gradually become a research hot spot,which is of great significance to the development of 3D reconstruction.In recent years,deep learning has developed rapidly and has made breakthrough progress in areas such as speech recognition,image segmentation,object detection,image super-resolution reconstruction,and text comprehension.In this paper,we applied the convolution neural network to the depth extraction of light field images.A method of depth estimation of light field image based on convolution neural network is proposed.The main work and innovation of this thesis are listed as follows.(1)Building the datasets.At present,the number of existing public data sets is small,and the four-dimensional light field images are difficult to applied directly to convolutional neural networks.Based on the existing datasets,a new data set construction method is proposed.Firstly,the light field image is converted into an Epipolar Image(EPI),and the four-dimensional light field image is mapped into a two-dimensional space.By using the relation that the slope of the line is proportional to the depth of the scene,the horizontal and vertical EPI block areas corresponding to the pixel points in the center view image are extracted.Invalid data in the data set is removed,and then the data set is balanced to finally build the experimental data set.(2)The construction and training of convolutional neural networks.We transform the depth computation into a classification problem.We know the depth range of the scene of the light field and determine the classification quantity by defining the calculation precision.Based on the classical convolutional neural network model,we propose a network model with a twin sub-network structure,which can fully learn the regional characteristics of horizontal and vertical EPI blocks and make full use of the information provided by the data sets.Finally,we use the already constructed data sets to train the network.(3)Testing the algorithm we proposed.The test results are compared with the current algorithms with good performance.The results show that our algorithm has higher accuracy in scene depth calculation and its network prediction results are closer to the true depth.