Research On Pan-sharpening Method Based On Convolutional Neural Network

As an important image processing technology,remote sensing image fusion has received widespread attention in recent years.Pan-sharpening combines a high spatial resolution panchromatic(PAN)image with a low spatial resolution multispectral(MS)image to obtain a high spatial resolution MS image.It can improve the image quality and obtain a more detailed description of the scene.Recently,deep learning technology has become a hot research topic.As one of its representative networks,convolutional neural network has achieved remarkable results in the field of image processing and is also used to deal with the pan-sharpening problem.The thesis focuses on the pan-sharpening methods based on convolutional neural network.The main research contents of the thesis are as follows:(1).Inspired by the ARSIS(Amélioration de la Résolution Spatiale par Injection de Structures)concept,a shallow-deep convolutional network and detail injection-based pan-sharpening method is proposed.Compared with other methods based on deep learning,the method combines domain knowledge,and extracts different levels of spatial detail information from the PAN image by designing a shallow-deep convolutional network;at the same time,based on the spectral differences of MS bands,a detail injection model based on spectral discrimination is proposed,which injects the detail information of different bands into the MS image separately,reducing the spatial and spectral distortion.Experiments show that the proposed method can effectively extract the richer and more comprehensive texture detail information that needs to be injected into each MS band,and can maintain the spectral characteristics while improving the spatial resolution of the MS image as much as possible,obtaining better fusion results in visual and index evaluation.(2).Inspired by the idea of image super-resolution,a pan-sharpening method based on multiscale dense network is proposed.Considering the high similarity between the source images and the fused image,the method introduces dense connections and designs a new end-to-end pan-sharpening network,which can fully utilize the spatial and spectral features of the source images to reconstruct the high-resolution MS image.The proposed network constructs a new multiscale dense block,which uses multiscale feature extraction and dense connections to extract rich layered features;on this basis,a global dense connection structure is designed to achieve continuous sharing and using of features to reduce distortion;and the global residual learning is proposed to make the network pay more attention to the changing part of the images,improving the fusion performance.The effectiveness of the proposed multiscale dense block,global dense connection and global residual learning is verified by experiments,and the experimental results show that the fusion effect of the method is better than other comparison methods,e.g.the peak signal-to-noise ratio(PSNR)is increased over 10% on World View2 dataset.