Study On Pansharpening Algorithm For The Fusion Of Panchromatic And Multispectral Remote Sensing Images

With the rapid development of remote sensing technology,multispectral image can provide richer information about ground objects,and is widely used in agricultural production,mineral monitoring,and environmental protection.However,high-resolution multispectral image can not be directly obtained due to the performance limitations of remote sensing equipments.Satellite sensors usually can only capture a high-resolution(HR)panchromatic image(PAN)and a corresponding low-resolution(LR)multispectral images(MS).Pansharpening,as an image fusion technology,aims to sharpen the original multispectral images by using the spatial details of panchromatic images.That is to say,by fusing the spatial and spectral information from these two kinds of images,multispectral images with high resolution in both spatial and spectral domains can be generated.Since the quality of the fusion MS image will play an important role in deciding the accuracy of subsequent recognition tasks,such as classification and change detection,how to design a fusion algorithm with good performance has become an attractived topic to the scholars who are in the domestic and overseas.This thesis takes panchromatic and multispectral remote sensing images as the research objects,and conducts the research work from two aspects: traditional methods and deep learning methods.It mainly focuses on spectral distortion existing in the traditional component substitution(CS)fusion algorithm,and insufficient network representation ability and the lack of true labeling information involved in the deep learning-based fusion models.To deal with these problems,this thesis mainly studies on how to alleviate the spectral distortion caused by the replacement of spatial components,how to improve the feature learning ability of the fusion network,and how to train te fusion network in an unsupervised manner without any lable information.The research contents and innovations involved in this thesis mainly include the following aspects:1.Aiming at the spectral distortion involved in the traditional component subsititution fusion algorithm,an adaptive fast IHS transform fusion algorithm driven by regional spectral characteristics is proposed.It takes gaofen-2 image with four bands as the main research object,which relies on fast intensity-hue-saturation(IHS)transformation fusion framework,and constructs a vegetation spectral characteristics-based regional-level fusion rule.Such rule can realize the adaptive generation of replacement component,so as to reduce the spectral distortion during fusion.Specifically,considering that different regions have different requrements for spatial and spectral resolutions,the algorithm first proposes an adaptive super-pixel merging strategy based on the terrain correlation between panchromatic and multispectral images,so that the algorithm can carry out specific fusion processing according to the internal attributes of different regions.Then,by analyzing the influence of vegetation spectral characrestics on resolution requirements,a fusion strategy guided by the regional ratio vegetation index is proposed to produce the replacement component.Such strategy can adaptively adjust the proportion of spatial information extracted from the original panchromatic and multispectral images in each region,so as to balance the effect of spatial enhancement and spectral fidelity.The experiment proves that the algorithm can provide better results in improving spatial detail and preserving spectral information.2.Aiming at the indiscriminate utilization of diverse features and insufficient combination of multi-level features involved in the CNN-based fusion models,a supervised fusion network based on channel similarity attention(CSA)is proposed,which can realize adaptively feature learning by using a channel attention mechanism,thus improving the fusion performance in spatial enhancement and spectral fidelity.Firstly,a channel attention mechanism based on interchannel dot-product similarity is designed,which learns the associated weight of each channel according to the inherent relationships between channel features,thus obtaining a better feature correlation learning;secondly,a channel attention residual dense block(CARDB)based on CSA is constructed for the deep feature extraction,which can fully mine local features and suppress redundant information;then,a multilevel feature fusion(MFF)block is proposed to fully combine the shallow and high-level features,in order to further improve the representation ability of the network.Remote sensing images with different numbers of multi-spectral bands as used to conduct comparision experiments,which proves the effectiveness and applicability of the proposed model.3.Aiming at the insufficient information flow and lack of real labeling information in the common CNN-based fusion network,a fusion algorithm based on unsupervised convolutional neural network is proposed.Firstly,an interactive fusion network structure is designed,which uses HR PAN-guided feature fusion and adopts a share source skip connetion(SSC)based on LR multispectral image,so that the spatial and spectral features from the input images can be fully transmitted and reused;secondly,by analyzing the conversion relationship between the network input and output,and mining the coupling relationship between the image quality index and the loss function,A loss function without reference image is proposed,which employs spatial constraints,spectral consistency,and a adjustment component based on the quality index with no reference to adaptively exploring the potential characteristics of the fused multispectral image,so that the network can be learnt on full-resolution images via unsupervised training,and the effectiveness of the proposed model has been verified on different types of remote sensing images.