Depth-Guided View Synthesis For Light Field Reconstruction

Light field images can record the two-dimensional spatial and angle information of natural scenes at the same time,reconstructing the three-dimensional structure of real scenes with high accuracy,which has great research value and application prospect.However,there are still many problems when obtaining light field images of real scenes.For example,light field images obtained with light field cameras generally have low resolution and poor image quality.Besides,due to the large amount of redundant information of light field images,huge storage resources and network resources are consumed during storage and transmission of light field images.In order to solve these problems,this dissertation conducts in-depth research on light field synthesis,which aims to synthesize dense and high-quality light field images using monocular images or sparse multi-view images.The method of light field synthesis based on depth maps has fast computation and high synthesis quality.However,the depth maps required by this method are difficult to obtain in practice,and the depth estimation error due to occlusion affects the final synthesis result.Therefore,how to accurately estimate the high-precision depth maps and how to repair the erroneous areas in the composite image due to occlusion are the focuses of light field synthesis research based on depth maps.The main research contributions of this dissertation are as follows:1.Aiming at the problem that the actual depth maps are difficult to obtain during the light field synthesis based on depth maps,this dissertation proposes an end-to-end convolutional neural network for unsupervised depth estimation.The image can be an unsupervised depth map of all views(7 × 7)in the light field,and then a dense light field image is synthesized according to the depth field-based light field synthesis method.2.Aiming at the problem of image optimization of light field synthesis results based on depth maps,this dissertation defines a multi-loss function based on the characteristics of light field images to optimize the synthesis results.First,the depth map is optimized based on the symmetrical and complementary properties of multiple sub-aperture views of the light field image.Then the digital field refocusing technology is used to constrain the structural relationship between the views to optimize the quality of the synthesized view as a whole.3.Aiming at the error problem caused by occlusion during the synthesis of light field based on depth maps,this dissertation uses disparity map as a guide to treat the erroneous synthesis area as "image noise".By constructing a full convolutional residual neural network,noise features in the synthesized image can be learned in this learningbased method.Based on this learning-based method,these noise features can be learned and further corrected to a large extent.Through the above-mentioned three innovations,compared with current research results on the two public datasets "flowers" and "30 scenes",whether based on monocular light field synthesis or multi-eye light field synthesis,both the peak signalto-noise ratio(PSNR)and the structural similarity(SSIM)have been improved significantly.