Hyperspectral Image Classification With Structured Convolutions And Graph Neural Networks

Remote sensing hyperspectral image(HSI)is a kind of 3D cube data containing abundant radiation,spectral and spatial information.Owing to the ability in distinguishing categories of land covers,HSI technology has been widely used in the fields of environmental monitoring and geological exploration.Classifying each pixel in HSIs accurately is the basis of such applications.However,due to the high cost of labeling pixels,the available supervised samples in HSI classification tasks are usually insufficient;meanwhile,affected by the illumination environment,mixed pixels,high-dimensional and redundant information,hyperspectral data often possess obviously small inter-class distance and large intra-class distance,which seriously limits the accuracy of HSI classification under limited samples.To make full use of the inherent structure information of HSIs to further improve the ability of earth observation,based on the deep learning theory and combined with the structure prior knowledge of land covers,this thesis explores a series of deep structured spectral-spatial modeling and classification applications of HSIs,mainly including the following five points:(1)To alleviate the serious performance degradation of deep networks when training samples are insufficient,a spectral-spatial convolutional dense network is proposed.HSIs are generally of high dimension while training samples are often limited,which makes it difficult for a deep network to learn spectral-spatial structures of different land covers efficiently,resulting in serious performance degradation.To alleviate this problem,we propose to separate the regular convolution into a spectral1 D convolution and a spatial 2D convolution to decouple“the spectral-spatial structures”of HSIs,which can reduce network parameters sharply without weakening discriminability of features and help train network with limited samples.The usage of dense connections enables the network to generate dense hierarchical features from shallow to deep layers,which can alleviate the vanishing gradient problem and reuse features simultaneously.In addition,different from the traditional networks that base on local HSI patches,the proposed network takes large HSIs as input,which avoids the loss of spatial information caused by patch segmentation and enables the network to classify all pixels in parallel.The experimental results show that the proposed method has satisfactory classification performance under limited samples,and its classification time is much shorter than other methods.(2)To suppress the high classification errors in cross-class edge regions caused by fixed kernel shape,a shape-adaptive convolutional neural network(CNN)is proposed.Due to the lack of shape adaptability,regular convolutional kernels are usually difficult to model complex structures of different land covers,leading to over-smoothing in the classification of cross-class edge regions.To tackle this issue,we propose to dynamically adjust the shape of convolutional kernels according to the distribution pattern of land covers.The distribution pattern is extracted from HSIs automatically using a sub-network,which can characterize the structures of land covers at different spatial locations.Then,the shape of convolutional kernels can be guided and adapted to different land-cover structures,thereby effectively suppressing irregular and unexpected features and improving feature learning in cross-class edge regions.The experimental results show that the proposed method can dynamically adjust the shape of convolutional kernels and let them adapt to different land covers,which brings excellent detail-preserving classification ability.(3)To deal with the limitations of CNNs in modeling the dependencies between different land covers,a hybrid deep network combining CNN and graph convolutional network(GCN)is proposed.Traditional CNNs are generally used to process Euclidean structured data,while GCNs are adept in modeling and characterizing non-Euclidean structured data,which means there is data structure incompatibility between the two different network architectures.To utilize spatial information of HSIs more comprehensively,we propose to integrate a CNN and GCN into a single network framework by establishing mapping relationship between graph nodes and image pixels.In this framework,the CNN and GCN branches are used to model Euclidean and non-Euclidean spatial structures of land covers and extract their pixel-and superpixel-level complementary features,respectively.Finally,by fusing these two different structured features,the proposed method can learn the long-and short-range spatial topological dependencies between different objects at the same time,thus promoting the structure characterization and recognition of various land covers.The experimental results show that the Euclidean pixel-level features and the non-Euclidean superpixel-level features have satisfactory complementarity,and the proposed method possesses excellent classification performance and great inference speed.(4)To break the limitation that single-scale superpixel-based GCNs cannot adequately characterize the complex spatial topological structures of HSIs,a hierarchical superpixel structured graph U-Net is proposed.To reduce the calculations of GCNs,an HSI is usually pretreated into a graph based on a specific superpixel segmentation,which limits the modeling of spatial topological structure to the same scale,leading to inadequate utilization of spatial information.To break this limitation,we propose to build a series of hierarchical segmentations from fine to coarse by progressively merging adjacent regions in HSI.By converting hierarchical segmentations into different-level graphs,the topological structures of HSI can be modeled in a multi-scale hierarchical manner.Furthermore,based on the merging relationship between hierarchical superpixels,we establish a pooling and unpooling function to transfer features freely between any two adjacent-level graphs,enabling different-level graphs to work together in a single framework and learn complementary features at different scales on different-level graphs.The experimental results show that by extracting hierarchical multiscale features from fine to coarse and then fusing them from coarse to fine,the proposed method can achieve higher classification performance and robustness than a single-scale GCN.(5)To solve the problem that GCNs are hard to be applied to large HSIs,a multiscale aggregated graph convolutional discriminative network is proposed.Generally,to capture the long-range dependencies between different land covers,GCNs usually work on large graphs based on the superpixel segmentation of a full image scene.It means that the requirements for computer hardware increase with the increase of HSI size.When facing very large HSIs,too large graphs make GCNs difficult to handle.To deal with this issue,we propose to decompose a large graph into many subgraphs by sampling the related nodes and edges of the pixels to be classified from the original large graph,thus any size of HSI can be processed by batch computing.Moreover,to fully use topological information in subgraphs,we also propose the multiscale aggregated graph convolution that can extract and fuse different-scale features in each convolutional layer.In addition,a discriminative classifier is proposed to enhance the compactness of feature expression.By calculating class-level feature embeddings,the intra-class and inter-class distances can be measured explicitly,which can be used to enhance feature learning ability when training the network.Experimental results show that the proposed method has great batch classification accuracies,and the discriminative classifier can promote the compactness of feature representation.