Pan-sharpening Methods For Remote Sensing Image Based On Convolutional Neural Network

The technology of Remote sensing has become one of the indispensable methods for human to obtain the ground information of earth.High-resolution multi-spectral remote sensing images have been widely used in many fields such as geomorphological surveying and military detection.However,due to the limitations of hardware and other factors,the remote sensing image obtained by a single sensor cannot have both high-quality spatial and spectral information.Therefore,fusion of multi-spectral images and panchromatic images captured by remote sensing satellites to obtain high-resolution multi-spectral remote sensing images has become a key issue in remote sensing applications.In recent years,convolution neural networks have attracted the attention of scholars in computer vision research such as image recognition,image fusion,and super-resolution reconstruction due to its excellect image processing effects.This thesis mainly studies the use of convolutional neural networks to fuse multi-spectral images and panchromatic images to obtain a fusion image with both high-quality spatial information and multispectral information（pan-sharpening）,the main contents of this thesis are as follows:1.A generative adversarial network with joint multi-stream architecture and spectral compensation for pan-sharpening has been proposed.Combine the method based on the variational framework and deep learning,with it has complementary advantages.The pansharpening method based on the variational framework can well balance the fidelity of spatial information and the fidelity of spectral information,but there are problems such as high time complexity and difficulty in solving.The pan-sharpening method based on deep learning can effectively fit the non-linear mapping function of panchromatic image and multi-spectral image to fusion image,and obtain the fusion result.The method we proposed combines the variational model with the conditional generative adversarial network,and refers to the energy function of variational model to design loss function of the generator with spectral constraints and spatial structure constraints.At same time,the generator network of the multi-stream fusion convolutional neural network architecture is designed according to the characteristics of the multi-source image,and in order to supplement the spectral information,a spectral compensation structure is designed.Finally,the fusion image is obtained through the confrontation between the generator and the discriminator.Experiments express that the method we proposed can attain very encouraging results,largely gain over the most contrast methods.2.A multi-stream architecture and multi-scale convolution neural network for pansharpening has been proposed.Compared with most methods based on deep learning,which often ignore the importance of the up-sampling process of multi-spectral images,and extract spectral information from the up-sampled multi-spectral image.The method we proposed directly extracts spectral information from the original multi-spectral image by constructing a spectral feature extraction subnet,which reduces the influence of noise caused by the up-sampling process.At same time,considering that the existing deep learning methods do not make full use of the feature information of different levels of the network,this method builds a pyramid module composed of multiple backbone networks,which can perform feature extraction under different spatial receptive fields,and learns image information at multiple scales.Finally,we construct a spatial spectrum prediction subnet to fuse the high-level features output by the pyramid module and the low-level features of the network front end to obtain multispectral images with high spatial resolution.Experiments show that the fusion image generated by the proposed method is superior to the most of advanced pan-sharpening methods in both subjective visual and objective evaluation indicators.