Research On Spatial Spectrum Fusion Of Remote Sensing Images Based On Deep Learning

Remote sensing image fusion refers to the fusion of the same scene images obtained by the same or different sensors to obtain high quality remote sensing images suitable for many research fields.The fusion of panchromatic(PAN)image and multispectral(MS)image is a research hotspot in the field of remote sensing image fusion,that aims at generating MS images with high spatial and spectral resolution by fusing the high spatial resolution PAN images and the high spectral resolution MS images with low spatial resolution,so as to achieve the spatial-spectrum fusion of remote sensing images.A large number of remote sensing image fusion methods have achieved rich results.The traditional linear remote sensing image fusion method usually causes spectral distortion.The current nonlinear fusion method based on deep learning is easy to ignore making full use of the information of each convolution layer,and the deep-level network is prone to the gradient disappearance and gradient explosion.Based on this,this thesis proposes two fusion methods based on deep learning,combining the characteristics of densely connected convolutional networks(DenseNet)and residual networks(ResNet).The main research contents include:(1)A spatial-spectrum fusion method for remote sensing images based on the residual network with dense convolution(DCRNet)is proposed.In this method,multiple densely convolutional blocks are built to make full use of the hierarchical features of the convolution layer.The short path connection between the layers in the block is conducive to the transmission of both features and gradients,and the burden of repeated learning of redundant features is reduced by reusing features.Meanwhile,the information flow is accelerated by the transition layer between every two blocks.These maximize the use of features and extract rich features.In addition,the network uses residual learning to fit the residual between deep and shallow features,which can accelerate the convergence speed of the network and alleviate the gradient disappearance and gradient explosion.To verify the effectiveness of the method,the simulated and real data experiments on the 4-band GaoFen-1(GF-1)and 8-band WorldView-2(WV-2)satellites are used.From both subjective visual evaluation and objective quantitative evaluation,the fusion results obtained by the method in this thesis are superior to the results produced by the compared traditional methods and deep learning methods.Moreover,the network in this thesis is robust and can be generalized well to other satellite images without pre-training.The high fidelity of spectral information is realized and the resolution of spatial details is improved by reusing features.A good balance between fusion performance and operation efficiency is achieved,which is conducive to the application research of remote sensing images.(2)A spatial-spectrum fusion method for remote sensing images based on the residual network with joint dense convolution(JDCRNet)is proposed.This method is mainly aimed at how to improve the operation efficiency of network.Based on the network structure of the first method,the densely convolutional blocks are redesigned,and the number of convolutional layers in the block is changed.At the same time,the input of the densely convolutional block is only used as the input of the first layer in the block,while the output of the block is only the output of the last layer in the block.Then,the adjacent blocks are directly connected in series,which is conducive to the transmission of information.Finally,the outputs of the multiple densely convolutional blocks are concatenated jointly together for dense feature fusion.The subjective visual effect and the objective quantitative evaluation show that the proposed method improves the spatial information while maintaining the spectral information,and effectively improves the operation efficiency while having certain generalization ability.