Research On Remote Sensing Image Fusion Model Based On Convolutional Neural Network

Remote sensing images can quickly acquire large-scale ground information and play an important role in many fields such as resource and environment investigation,feature classification and crop acreage extraction.In practical applications,it is difficult to meet the demand with only a single panchromatic or multispectral image,and fusion of remote sensing images is needed.Remote sensing image fusion is the process of combining information from two different images together to generate a new image.In recent years,with the rapid development of deep learning technology,the application of convolutional neural network to the direction of remote sensing image fusion has become a current research hotspot.However,the existing fusion methods often cannot simultaneously take into account the spectral and spatial information of remote sensing images in the fusion process,and are prone to the distortion of spectral and spatial structures.In response to the above problems,two fusion models based on convolutional neural networks are built in this paper around the research of remote sensing image fusion methods,and the work done in this paper is as follows.1.Dataset production.In this paper,19 Gao Fen 1 images,50 Gao Fen 2 images,20 Gao Fen6 images and 4 Super View1 images from October 2020 to December 2021 in Shandong were preprocessed for radiometric calibration,orthorectification,atmospheric correction and image alignment,and the multispectral images of Super View1 and their corresponding Gao Fen 6images obtained from the preprocessing were made into a dataset with reference.The Gao Fen1,Gao Fen 2 and Gao Fen 6 images were made into datasets without reference respectively.2.For the spectral and spatial structure distortion that is easy to occur in the fusion process of remote sensing images,the normalized vegetation index and normalized water index,which are unique in remote sensing images,are used as a priori knowledge to constrain the fusion process of panchromatic and multispectral images to reduce the spectral loss in the fusion process according to the characteristics of remote sensing images;the squeeze and excitation module in the attention mechanism is combined with convolution units of different scales to retain useful feature information and eliminate useless noise and redundant information.Based on this,the knowledge-guided remote sensing image fusion method（RSFuse Net）is constructed.3.Aiming at the problem that high-resolution reference images are difficult to obtain in remote sensing image fusion and the information in the original panchromatic and multispectral is not fully utilized in the fusion process,the spectral loss function is designed to calculate the loss between the fused image and the original multispectral image,and the texture loss function is designed to calculate the loss between the fused image and the original panchromatic image,so as to achieve the effect of constraining the fusion process;meanwhile,the The improved VGG network is used to extract the features of panchromatic and multispectral images,and the features are fused based on the residual network and squeezed excitation structure,on which the self-supervised remote sensing image fusion model（SSRSFuse Net）is constructed.The model can fuse panchromatic and multispectral images without introducing additional reference images to further enhance the utilization of the original images.The experimental results show that the Quality with No-reference Index（QNR）value of the SSRSFuse Net model is 0.8838,the spectral distortion index Dλvalue is 0.1113,and the spatial distortion index D_s is 0.0055,all of which are better than the comparison model,indicating that the method can achieve image fusion without introducing additional reference images and effectively improve the fusion effect.It shows that the method can achieve image fusion without introducing additional reference images,effectively improving the fusion effect and reducing the labor and material resources spent on acquiring reference images.