Multispectral And Hyperspectral Fusion Method And System Based On Attention Network

Hyperspectral(HS)image is a three-dimensional data cube with hundreds of continuous spectral bands.It provides rich spectral information and explores the attribute features of the land-covers deeply.However,the HS image has low spatial resolution,which limits its further applications.Multispectral(MS)image has rich spatial information and can provide fine geometric features of the image.Therefore,HS-MS fusion can achieve information complementarity and increase the accuracy of information.HS-MS fusion is one of the top research topics of remote sensing.In the reconstruction process,the restoration of high frequency details and the reduction of information loss are the difficulties of HS-MS fusion.In recent years,deep learning technology has made outstanding achievements in various fields and received widespread attention.Inspired by spatial-spectral attention mechanism,this paper proposes two HS-MS fusion methods.One is a method based on spatial attention network,and the other is a method based on spatial-spectral-joint attention and residual network.The main work and innovations of this paper are as follows:1)This paper proposes a HS-MS image fusion method based on spatial attention network.It is difficult to recover the high frequency information when reconstructing the high resolution(HR)HS images.In order to solve this problem,a spatial attention network based on convolution blocks is added to the convolution layer based fusion network.The entire network is composed of a fusion subnet and an attention subnet.The role of the fusion subnet is to fuse the HS and MS images.The attention subnet is designed to generate the attention which can segment high frequency regions and smooth regions accurately.The attention further enhances the spatial details of the reconstructed image.The spatial attention network effectively enhances the spatial structure of the HS image and characterizes the spatial details of the image.2)This paper proposes a HS-MS image fusion method based on the spatial-spectral-joint attention and residual network.The model consists of two parts.One is the fusion subnet,which is based on channel attention and residual.The other part is the spatial attention subnet based on bottleneck blocks.The channel attention enhances the key information of the channel direction and suppresses non-critical information.In order to reduce information loss,a residual network structure is utilized.The spatial attention network can improve network performance.On the other hand,it can reduce training time.This method makes full use of spatial and spectral information and maintains the spatial and spectral structure of HS image better.Experimental results compared with some state-of-the-art methods illustrate that our method is outstanding in both visual and numerical results.3)We design a HS-MS image fusion system based on attention network,which integrates multiple fusion algorithms.The inputs of the system are low resolution(LR)HS image and HR MS image.The system can reconstruct the HR HS image through the processing of the fusion algorithm.