Research On Light-Field Super-Resolution Via Dense Interaction Between Spatial And Angular Domain

Different from the traditional two-dimensional imaging,the light field camera can capture the light intensity of different angles and positions at the same time.The captured light field image contains rich structural geometric information of the real-world scene.Therefore,it greatly promotes the development of various potential vision based applications,such as image refocusing,saliency detection,depth estimation,de occlusion,and 3D reconstruction.Limited by the sensor resolution,the spatial resolution and angular resolution of the light field image are also greatly limited.Therefore,using the light field image super-resolution method to improve the light field image resolution has become a research hotspot.The research difficulty of light field image super-resolution is how to effectively use the rich redundant information between views to recover the missing details,and epipolar information and spatial-angular information are two important factors to obtain effective information.The epipolar information contains the pixel offset relationship of the same scenic spot in different views,which can be used for the extraction and restoration of detail information.The extraction and interaction of spatial-angular information can supplement the missing detail information between spatial domain and angle domain.In addition,for large parallax scenes,due to the limited receptive field of the network model,the baseline of the light field can be reduced by parallax offset method,so that more relevant information can be captured in the limited receptive field,and further combined with the attention mechanism to dynamically select the area for effective fusion of relevant information.Based on the extraction of epipolar information and spatial-angular information,this paper studies the super-resolution of light field images in different parallax scenes,and all of them have been effectively improved.Firstly,aiming at the spatial super-resolution of light field,this paper realizes the effective and dense interaction of redundant information of light field from two aspects:epipolar information and spatial-angular information.Therefore,Epipolar-Aware Grid(EAG)is proposed to gradually interact different epipolar information in the way of grid connection.In addition,a Parallel Spatial Angular Integration Block(PSAIB)is proposed to extract spatial and angular features in two branches in parallel,and realize the effective interaction and fusion of spatial domain and angular domain information through cross domain connection.The spatial super-resolution network model of light field with dense interaction between spatial and angular proposed in this paper is mainly composed of EAG and some PSAIBs.A large number of experiments on public data sets show that this model surpasses the existing methods in 2x and 4x tasks.Secondly,aiming at the light field angular super-resolution,a light field angular super-resolution network model with dense interaction between spatial and angular based on EAG and PSAIB is proposed in this paper.Firstly,EAG is applied to the input feature extraction,and then the intermediate results are obtained by feature up sampling.Then,EAG and several PSAIBs are used for feature fusion,and finally the fused features are reconstructed into high angular resolution results.A large number of experimental results on public data sets show that this model has achieved superior results in a variety of tasks.In addition,for the light field spatial and angular super-resolution task,recovering the sub-pixel details in the light field large parallax scene is a challenging problem.Therefore,a network model of attention mechanism based on optical parallax shift is proposed in this paper.This model dynamically fuses the results of different offsets through the attention mechanism,which can more effectively explore the complementary information of light field in large parallax scenes,and can be flexibly applied to any spatial or angular super-resolution model.Experiments show that the performance of this model is greatly improved under a variety of complex tasks of light field super-resolution.