Research On Remote Sensing Image Fusion Based On Dynamic Convolutional Neural Network

With the wide application and development of remote sensing technology,remote sensing images with higher quality are necessary.The target images obtained by different sensors contain different information.Multi-spectral imaging sensor needs to image multiple bands at the same time,which leads to its low spatial resolution.Panchromatic imaging sensor only needs to image one band,the image formed by it contains rich spatial information and has strong spatial structure,but its spectral information is seriously lacking.Pansharpening is a method to obtain multispectral images with high spatial resolution by fusing panchromatic images with multispectral images.Nowadays,many methods for remote sensing image fusion based on convolutional neural network have been proposed,although they have better fusion performance,further research and exploration are needed.In this paper,two methods based on dynamic convolutional neural network are proposed to realize pansharpening.One method is a multi-scale dynamic filtering neural network for pansharpening.Different from the general methods based on convolutional neural network,the filter in this method is automatically generated by the network,which can be learned during training and changed dynamically with the input in the test.In order to improve the spatial resolution of the image while maintaining the spectral resolution of the multispectral image,to obtain fusion image with higher information content and clearer texture,this paper propose a multi-scale dynamic filter to extract multi-scale details by combining deep learning and multi-scale idea.In order to better protect the spectrum,the spectral loss function is added to train the network.Another method is a spatial dynamic selection neural network for pansahrpening.The dynamic feature extraction module is designed to dynamically select the features of each pixel according to the input image,which is composed of multiple spatial dynamic blocks and cross-scale context connection blocks.Since the spatial structure and spectral characteristics of each pixel are different,two complementary branches are designed in the spatial dynamic block to extract different features,and then the gate module is used to select spatial features adaptively,so as to obtain better and more accurate spatial details.Multi-scale network structure is designed to obtain more abundant information and the cross-scale context connection block is used to integrate the information of different scales.The above two methods increase the adaptability of the network and improve the fusion performance of the network.Experiments on GeoEye-1,QuickBird and WorldView-3 satellite images show that the fusion results of the proposed methods have better fusion performance and adaptability compared with the traditional convolutional neural network methods.