Research On Multi-source Remote Sensing Image Fusion Algorithm Based On Deep Learnin

Raw multi-source remote sensing data including multi-channel low resolution multi-spectral image(LRMS)and single-channel panchromatic image(PAN).Raw multi-spectral image is rich in spectral detail but has low resolution,while raw panchromatic image has high resolution and is rich in spatial detail information but lack the necessary spectral information.Pansharpening is the process of using raw panchromatic image and multi-spectral image to obtain highly accurate remote sensing image.Based on the powerful feature extraction capability of deep neural networks,this paper proposed two unsupervised deep neural network-based algorithms for pansharpening.The first pansharpening algorithm is named as “Multi-Scale Detail Injection-Based Improved Generative Adversarial Network for Pansharpening(MSDI-GAN)”.The network structure of the algorithm has two discriminators and one generator,where the spatial discriminator and spectral discriminator are used to retain the spatial and spectral information of the original multi-source remote sensing data respectively.Dense Net is used to densely connect the various convolutional layers of the generator,with the aim of fully extracting feature information from the original multi-source remote sensing data.Each convolutional layer of the generator is used to extract multi-scale feature information from the original data and inject it into the corresponding discriminant layers of the two discriminators.This approach deepens the exchange of gradient information between the generator and the two discriminators,with the aim to optimize the actual network training process and speed up network convergence.Three sets of experiments and three different datasets have been conducted to evaluate the performance of the first fusion algorithm and to compare it with other classical and deep learning pansharpening algorithms.Based on the qualitative visual effect evaluation and quantitative objective index evaluation of the experimental results,it is proved that the first pansharpening algorithm in this paper has achieved the expected results.At the same time,the first fusion algorithm has problems such as complex network model structure,high hardware requirements,long training and testing time.The second pansharpening algorithm is named as “Pansharpening Algorithm based on Original Multi-Scale Improved Generative Adversarial Network(OMS-GAN)”.This algorithm is based on the consideration of simplifying the network model and speeding up the training and testing time of the neural network,which uses the single-generator-single-discriminator network framework of the original GAN,optimizes the single adversarial of the original network to the multi-scale multiple adversarial of this algorithm network,and designs appropriate generator and discriminator loss function for the neural network.After extensive experimental validation,the fusion results of the second algorithm OMS-GAN are slightly worse than the first algorithm MSDI-GAN,but the training and testing time is reduced,making it ideal for application scenarios with strict time and cost requirements.Of course,if the goal is high accuracy and the training and testing time requirements are not particularly stringent,then MSDI-GAN would be a more suitable choice.After experimental verification,both fusion algorithms proposed in this paper(MSDI-GAN,OMS-GAN)have achieved the high accuracy.