Super-resolution Enhancement Of Hyperspectral Data Through Deep Learning Methods

Hyperspectral image(HSI)contains rich spatial and spectral information,which is beneficial for identifying different materials.HSI has been applied in many fields,including land-cover classification and target detection.However,due to the limited photonic energy,there should be trade-off between spatial resolution,bandwidth,swath width,and signalnoise-ratio.So the spatial resolution of HSI is often moderate,which may lead to spectral mixture of different endmembers.In addition,some earth observation applications,such as urban mapping and fine mineral exploration,requires high spatial resolution image.Since it is expensive to enhance the resolution by updating imaging hardware,enhancing the resolution of HSI via image processing method is more realistic.Deep learning has strong capacity in extracting feature and representing mapping function,it could effectively represent the mapping function between low-and high-resolution HSI,and super-resolve HSI.In this thesis,we focus on HSI spatial resolution enhancement,and propose several deep learning based HSI enhancement methods.The main contributions are as follows:1.HSI contains information in both spatial and spectral domains,to jointly extract features from the spectral and spatial domains in deep learning model,we propose a convolutional neural network with two-channel architecture.There are two channels of convolutional neural networks,each of which is designed for features in spectral and spatial domains.The spectrum of each pixel is the input of the spectral channel,its neighborhood window is the input of the spatial channel.Features extracted from the spectral and spatial domains by the two channels are then concatenated and jointly fed to fully connected layers,spatial-spectral joint deep features can be obtained in the output of the fully connected layers.We validate the discriminative ability of the features using land-cover classification experiment,it is shown that the extracted features can lead to competitive performance compared with other methods on several datasets.2.We propose a convolutional neural network with two branches architecture.There are two branches of convolutional neural networks.One branch extracts features from the spectrum of each pixel in the low resolution HSI,the corresponding neighborhood window of the pixel in the multispectral image is used as input of the other branch.The features extracted from the hyperspectral and multispectral image are then fused by fully connected layers,the spectrum of high resolution HSI would be generated on the output of fully connected layers.The experimental results demonstrate that the proposed fusion method is competitive with other state-of-the-art methods on several testing data.3.To address the issue of how to reconstruct image details and textures in single-frame HSI super-resolution,we propose a multi-scale wavelet 3D convolution neural network for single-frame HSI super-resolution.Wavelets could describe image details and textures in different scales and orientations,focusing on reconstructing the wavelet coefficients in super-resolution is beneficial for preserving the image details and textures.The proposed network is designed to predict the wavelet coefficients of high resolution HSI.It is composed of an embedding subnet and a predicting subnet,both of which are built on 3D convolutional layers.The embedding subnet projects the HSI into deep feature space and extracts feature cubes from it.The feature cubes are then fed to the predicting subnet,which consists of multiple output branches to predict wavelet coefficients of different wavelet sub-bands.The high resolution HSI can be obtained via inverse wavelet transformation.The experimental results demonstrate that the proposed method achieves competitive performance compared with other single-frame hyperspectral super-resolution methods.4.To address the issue of how to assess the super-resolved HSI without any reference in real remote sensing applications,we propose a no-reference HSI quality assessment method based on quality-sensitive feature learning.We collect pristine hyperspectral datasets,analyze the statistics under different quality distortion in both spectral and spatial domains,and extract several statistical features that are sensitive to image quality,then learn the distribution of these features.The learned distribution can be treated as benchmark,when quality distortion exists,the distribution would deviate.The distance between the distribution of super-resolved image and the benchmark is regarded as quality score.The lower score means the better image quality.The experimental results demonstrate that the proposed quality score is consistent with indices such as peak-signal-noise-ratio on several testing data.