Remote Sensing Image Fusion Algorithm Based On Generative Adversarial Network

With the popularization of remote sensing technology,high resolution multi-spectral images are required in many fields such as disaster early warning and military detection.However,remote sensing images obtained by a single sensor have limitations in spectral information and spatial information.The fusion of multi-spectral images and panchromatic images to obtain high resolution multi-spectral images has become a research hotspot in recent years.In recent years,with the development of artificial intelligence technology,kinds of deep learning models perform well in the field of image processing.The generative adversarial network has received extensive attention in the study of computer vision such as image fusion and image super-resolution,due to its strong ability of generating high quality images.This thesis mainly focuses on the fusion of multi-spectral images and panchromatic images by using generative adversarial networks,and how to improve the spatial resolution and spectral quality of fused images by increasing the special domain knowledge in remote sensing images.The main contents are as follows:1.The research of generative adversarial network framework suitable for feature level fusion of remote sensing images.This multi-stream fusion architecture is an end-to-end network framework.Firstly,the features of initial multi-spectral image and the panchromatic image are extracted according to specific rules.The generator uses the subnetwork to extract and purify the input multi-source images respectively.Then the generator’s main network acquires the fused images by learning the features superimposed and the entire process is performed in the feature domain.According to the characteristics on the bands of the multi-spectral image and panchromatic image in the remote sensing field,MTF coefficients are introduced into the discriminator to discriminate the spectral information and spatial structure information of the fused image simultaneously.The experiments show that the multi-stream fusion generative adversarial network framework can effectively improve the quality of the fused image.2.The research of remote sensing image fusion based on multi-stream generative adversarial network combines with variational model.Firstly,according to the prior assumption of the variational model,the gradient operator is used to extract the spatial structural information of the panchromatic image,and then the extracted spatial structure information is used to replace the panchromatic image input in the generator to reduce the panchromatic images’ interference with spectral information during the fusion.Then,the initial low resolution multi-spectral image is added to the reasonable convolution layer in the generator as a supplement to the spectral information.Improve the objective functions of the generator and discriminator by the energy function of the variational model.Finally,the least squares generation adversarial network is used to train the model to reduce the loss of spectral information and spatial structure information.The LSGAN has a higher stability than the original GAN during the training process.Simulation experiments and real data experiments demonstrate that the proposed method can generate high quality fused images,which is better than the most of contrast remote sensing image fusion methods in both subjective visual and objective evaluation indicators.