Light Field Depth Estimation Of Combining Sub-aperture And Focal Stack Images

Light field depth estimation is critical to the applications such as 3D reconstruction,automatic driving and object tracking.A micro-lens array is inserted between the main lens and the senor in the light field camera,which can simultaneously record the intensity information and direction information of light,and significantly improving the accuracy of depth estimation.Although the special geometric structure of the light field provides useful angle information,it is still challenging in the case of complex regions such as occlusion,edge and texture,which resulting in the loss of depth image details.Therefore,based on the characteristics of light field image data,this paper fully exploits the spatial and angular information of sub-aperture image array.At the same time,considering the light field can freely change the focal distance according to the refocusing characteristic,from which rich depth semantic features and focal structure information can be obtained,and the depth information contained in the focal stack image can be further fused to improve the accuracy of depth estimation.The main research contents include:Addressing the challenge of complex regions in light field depth estimation.firstly,based on the spatial and angular information of light field sub-aperture images,a semantic guidance-based light field depth estimation network is proposed,which utilizes contextual information of light field images to solve ill posed problems in complex regions.In the process of feature extraction of sub-aperture images,in order to obtain effective edge features,the encoder-decoder structure is combined with the spatial pyramid pooling structure to increase the receptive field and capture the multi-scale contextual information.Then,an adaptive local cross-channel interaction feature attention module without dimensionality reduction is used to eliminate information redundancy,and multi-channels are effectively fused.Finally,the stacked hourglass is introduced to connect multiple hourglass modules in series,and more rich context information is obtained by using the encoder-decoder structure.The experimental results on 4D light field dataset new HCI demonstrate that the proposed method has higher accuracy and generalization ability,which is superior to the depth estimation method compared,and retains better edge details.On this basis,using the characteristics of sub-aperture images containing matching clues and focal stack images containing defocus clues,a light depth depth estimation network combining matching clues and defocusing clues is proposed,which utilizes multi-information fusion and multi-layer attention mechanism to solve ill posed problems of complex areas.Considering the consistency principle of light field,the channel attention mechanism is used to solve the occlusion problems in depth estimation for the matching cost volume obtained after the feature extraction of sub-aperture images.In order to explore the spatial correlation in the focus stack images,the defocus cost volume obtained after the feature extraction of the focus stack images uses the spatial attention mechanism to solve the problems of repeated texture and noise in depth estimation.In addition,in order to better obtain effective depth features,the branch attention mechanism is used for multiple branches to improve the ability of information identification while performing information fusion.The model effect has been verified on HCI 4D light field dataset,and the algorithm can obtain more accurate depth results in occlusion,edge and complex texture areas.In summary,based on the characteristics of the light field,this article proposes effective solutions to the problems in complex areas in depth estimation,and has potential application prospects in related fields such as 3D reconstruction and saliency detection.