Multi-Information Fusion-Based Super-Resolution For Light Field Image

Light field images contain rich spatial information and angle information,which is widely used in the field of 3D reconstruction and virtual reality.However,due to the internal limitations of the light field camera,the low spatial resolution of the light field images hinder the development of various applications,which is specifically manifested as the blur of the edge area of the image,the loss of high frequency information and the occlusion problems.Most existing super-resolution methods for light field images do not fully utilize the spatial and angular information of light fields,and most deep learning-based networks choose to learn iteratively in one light field or through different networks,and the structural consistency between each view is difficult to retain.In order to solve the problem of light field image blur,considering that light field has multiple information manifestations,this paper conducts relevant research from multiple information fusion methods,namely spatial angle information decoupling fusion and multiplex sub-pixel information fusion.The main contents include:1.Considering that the spatial information in the light field sub-aperture images contain rich texture and high-frequency details,while the angle information shows the correlation between different views,a super-resolution network based on feature interaction fusion and attention is proposed,which fully integrates the spatial-angle information of the light field through feature extraction and feature interaction fusion module;the feature channel attention module is designed to adaptively learn effective information,suppress redundant information and refine the high-frequency details of the image.Finally,the light field structure consistency module is designed to maintain the disparity structure between the light field images.Experiments on five optical field datasets show that the super-resolution results obtained from the proposed network effectively solve the region blur problem and are clearer in the details.2.A super-resolution network of light field image based on sub-pixel information assistance is proposed.The light field sub-aperture image is formed into stacks in four directions as input,and the 3D convolution block is used to extract EPI in different directions.In order to solve the problems of information redundancy and regional occlusion,the attention is focused on the effective and unblocked sub-pixel information through the feature fusion module;in order to solve the information asymmetry in the spatial and angle fields of the light field,the features are refined by the asymmetric spatial domain convolution and angle domain of the feature refinement module,and the spatial angle information is integrated through dense jump connection to maintain the structural consistency of the light field.The experimental results show that the reconstructed images achieve better results in the edge blurred areas caused by occlusion.In conclusion,both methods in this paper can improve the image blur problem and obtain sharper and more perfectly detailed high spatial resolution light field images.