Remote Sensing Image Fusion Based On Multi-morphological Convolutional Neural Network

The unique complexity and the inherent uncertainty of remote sensing images cause computers to face many ambiguity and fuzzy problems when processing remote sensing images.Compared with natural images,remote sensing images cover more complex information about features,atmosphere and noise,and so on.The variability and complexity of feature information in different images present new challenges to tasks such as image classification and recognition in the later stage.How to effectively pre-process the remote sensing images,regulate their uncertainties and enhance the inherent features of the images is of great significance for the subsequent practicable operation.Because the hardware technology limits the satellite sensor,the earth observation satellite can only acquire panchromatic images with high spatial resolution and low spectral resolution and multispectral images with low spatial resolution and specific spectral resolution.Remote sensing image fusion is one of the effective means to improve the recognizability of remote sensing images and their operability in other subsequent applications.This thesis addresses the problem that existing remote sensing image fusion algorithms are challenging to obtain spectral and spatial resolution.From the perspective of sparse decomposition of morphological image components,the sparse representation is introduced to remote sensing image fusion based on deep learning using the powerful nonlinear feature representation capability of current deep learning.The morphological component analysis is used to decompose the remote sensing images sparsely.The dense connectivity is introduced into the remote sensing image fusion algorithm by improving the convolutional neural network model to further enhance the quality of the fused images.Aiming to make full use of the spatial information of remote sensing images and improve the spatial resolution of the fused multispectral images,the remote sensing image fusion method with multi-morphological convolutional neural networks is studied.The main work is summarized as follows.（1）Since sparse representation can effectively describe features’ spectral and spatial information,a remote sensing image fusion method based on multi-morphological sparse decomposition is studied using the convolutional neural network framework.The morphological component analysis method is introduced into remote sensing image fusion by combining local discrete cosine transform basis and curvilinear wave transform basis to form a decomposition dictionary.The sparse decomposition is used to accurately extract the texture components and cartoon components of panchromatic and multispectral images,quickly obtain the original information contained in the images,and reduce the complexity of image processing.The different morphological components of the sparse decomposition are input to the convolutional neural network.The adjacent pixels are calculated so that the network extracts its key features and fuses them effectively,and outputs a multispectral remote sensing image containing rich spatial information.（2）Based on the previous work,a remote sensing image fusion method based on a multi-morphological dense network is proposed by introducing an improved dense connection into the convolutional neural network.Based on the sparse decomposition of the input image,the number of channels in the network is changed by establishing dense connections between the front-layer network and the back-layer network in the neural network with the same scale of the feature layers,enabling the network to retain the input image features to a greater extent with fewer parameters and computational cost,which significantly reduces the complexity of the network computation.Instead of single-path feature extraction,dual-path feature extraction is used in dense blocks to connect the features of texture components to the feature blocks of cartoon components to capture the complex relationships between the input images.The algorithm achieves better fusion results than existing popular models such as residual networks,and obtains fused images with richer spatial details and spectral information.This thesis effectively improves the existing remote sensing image fusion method based on the convolutional neural network by using the multi-morphological components in the images so that the final fused remote sensing images retain the spectral information while incorporating more representational detail information.The experimental results on different satellite datasets show that the method in this thesis achieves better fusion results in terms of both subjective visual effects and objective evaluation metrics compared with classical remote sensing image fusion methods and popular deep learning methods.