Remote Sensing Image Fusion Based On Multi-Morphology Attention Mechanism

Remote sensing images play an important role in environmental monitoring,precision agriculture and national defense security.However,due to the limitation of satellite sensor imaging technology,usually only multispectral images containing mainly spectral information or panchromatic images possessing high spatial resolution can be obtained.Therefore,in order to make full use of multi-source data,the goal of remote sensing image fusion in this thesis is to fuse multispectral images and panchromatic images together to generate multispectral images with high spatial resolution.An image is composed of multiple morphological components,and most popular deep learning-based remote sensing image fusion methods currently treat the input source image as a single component,thus ignoring the diversity of image components contained in remote sensing images.To address the above problems,this thesis takes convolutional neural network as a framework to study how to effectively extract richer spectral and spatial features and achieve a balance between both spectral and spatial to obtain high-quality fused remote sensing images from three perspectives: sparse decomposition morphological component analysis,multi-resolution multi-scale and attention mechanism,respectively.The main research work of this thesis is summarized as follows:(1)In this thesis,we propose a remote sensing image fusion method based on optimal scale morphological convolutional neural network using the principle of entropy from information theory.We use an attentional convolutional neural network to fuse the optimal cartoon and texture components of the original images to obtain a high-resolution multispectral image.We obtain the cartoon and texture components using sparse decomposition-morphological component analysis with an optimal threshold value determined by calculating the information entropy of the fused image.In the sparse decomposition process,the local discrete cosine transform dictionary and the curvelet transform dictionary compose the morphological component analysis dictionary.We sparsely decompose the original remote sensing images into a texture component and a cartoon component at an optimal scale using the information entropy to control the dictionary parameter.Experimental results show that the remote sensing image fusion method proposed in this thesis can effectively retain the information of the original image,improve the spatial resolution and spectral fidelity.(2)Based on the previous work,continue to utilize the idea of multi-morphology,and improves and optimizes the network structure to propose a remote sensing image fusion method based on multi-morphology attention mechanism.The spectral information and spatial information are obtained through a dual-branch feature extraction network,and then the weight of key information to be fused is enhanced based on multiple attention mechanisms to make the network more flexible to handle different feature regions without increasing the computational cost,and finally a fused image with uniform spectral distribution and rich spatial details is obtained after the feature fusion network.The experimental results show that the remote sensing image fusion method based on the multimorphology attention mechanism can effectively retain the information of the original image,improve the spatial resolution and spectral fidelity,outperform the existing fusion methods in terms of both subjective visual quality and objective evaluation indexes,and provide a new idea for image fusion from the perspective of multi-morphology deep learning.In the work of this thesis,multi-scale morphological component analysis and information entropy can obtain the cartoon component and texture component of the input source image,and effectively retain the spectral information and spatial information of the input source image.Multiple attention mechanisms focus on effective spatial and spectral information without increasing the computational load of the network,thus generating highquality fused remote sensing images.