Light Enhancement And Super-resolution Methods For Light Field Images

Unlike traditional 2D images that only record the intensity information of light,light field images(LFIs)also record the angle information of light.This characteristic makes LFIs are widely used in many computer vision tasks.However,LFIs sacrifice image quality(dynamic range,contrast,and spatial resolution)to record the angle information of light,which greatly limits their applications.This paper proposes a light field image light enhancement method fusing the exposures of LF-DSLR image pairs,and a light field image spatial super-resolution method via a hybrid imaging system.The main content includes:1)A light field image light enhancement method is proposed via the exposure fusion of a hybrid imaging system.By exploiting the exposure difference between digital single-lens reflex(DSLR)cameras and plenoptic cameras,the proposed method addresses the low dynamic range and low brightness of LFIs by fusing DSLR images and LFIs.Firstly,the RANSAC-Flow is utilized to align the LF-DSLR image pair.Then aligned DSLR image and LFI are input into our feature fusion network to generate high-quality LFI.We extract the features of the LFI and the aligned DSLR image through the proposed histogram equalization attention module(HEAM)and 3D Swin Transformers.The extracted features are fused through a dense residual network to generate enhanced LFI.Finally,the enhanced LFI is separated into a reconstructed LF-DSLR image pair,which makes the enhanced LFI retain the information of the input LF-DSLR image pair as much as possible.Additionally,a dataset consisting of real-world LF-DSLR image pairs is proposed to train and validate the proposed method.Extensive experiments on real-world LFIs demonstrate the effectiveness of our method.2)A light field image spatial super-resolution method is proposed via a hybrid imaging system.Existing LFI spatial super-resolution(SR)methods have performed well in synthetic scenarios,but their performance significantly diminishes in real-world scenarios.The hybrid imaging system consists of a plenoptic camera and a digital single-lens reflex(DSLR)camera to capture LF-DSLR image pairs.The high-resolution DSLR images are used to help reconstruct high-resolution LFIs in real-world scenarios.First,the DSLR image is aligned with the center sub-aperture image of the LFI using RANSAC and homography estimation.Then spatial transformers and angle transformers are adopted to extract features from the LFI and DSLR image.Next,the LF feature enters the proposed PABB module to improve spatial resolution.During the reconstruction phase,the upsampled LF feature and the DSLR feature are added and further reconstructed through spatial transformers and angle transformers.Finally,the reconstructed feature is added with the input LFI,and enters a convolution layer to generate the final high-resolution LFI.Extensive experiments demonstrate that our method outperforms other methods in LFI spatial super-resolution tasks in real scenarios.