A Study On Convolution Neural Network Based Hyperspectral Pansharpening

Over recent years,with the development of space technology and the popularization of remotely sensed industry in China,satellite data processing and application have become a research hotspot,especially for hyperspectral images which jointly combine scene-based spatial imaging and pixel-based spectral perception.However,the observed hyperspectral images always suffer from low spatial resolution since a trade-off between manufacture payment and imaging signal-to-noise.In this case,accurate object recognition and target tracking turn hard to achieve.Therefore,experts attempt to boost the spatial quality of hyperspectral images with the help of high spatial resolution single-band panchromatic images in a data fusion way,also termed pansharpening.In this thesis,we focus on the hyperspectral pansharpening task and convolutional neural network framework.We first study the problem of hyperspectral pansharpening,such as,the different spectral range between hyperspectral and panchromatic images,the challenge of hundreds-bands detail reconstruction solely by single-band panchromatic image,and the output spatial resolution restricted by that of panchromatic image.To tackle the aforementioned issues,we propose novel convolutional neural networks with the consideration of data characteristics and imaging theorem of hyperspectral images,respectively called spectrally predictive convolutional neural network,hierarchical detail injection based convolutional neural network,and adaptive interpretation based convolutional neural network.The spectrally predictive model takes the original spectral characteristics of objects as input and outputs its recombination by pixel-wise spectral convolution,which aims at guaranteeing spectral fidelity.The hierarchical detail injection model decomposes the lacking spatial details into pieces,for the sake of high efficiency in detail reconstruction by convolutional neural networks.Inspired by bicubic interpretation,the adaptive interpretation model learns a mapping between position offset and weights of adjacent pixels by neural network so that hyperspectral images with arbitrary spatial resolution are generated in a fixed model.Experimental results on public hyperspectral datasets approve our proposed methods with regards to other state-of-the-art pansharpening methods,in terms of spectral fidelity and detail reconstruction.The value of the research in this thesis is not just about a novel method with better pansharpening quality and more efficient implementation.More importantly,we make an attempt to address the issues of low interpretation and weak generalization in deep learning by designing network architecture and loss function with prior knowledge from hyperspectral imaging characteristics and pansharpening problems.In addition,we jointly take pansharpening method design and the demand of the following hyperspectral employment into account,which contributes to accelerating the practical application and industrialization of pansharpening.