Distributed Parallel Classification Of Hyperspectral Images Based On Spark

The hyperspectral image features wide coverage,high dimensional bands and a huge amount of data,which contains abundant spatial and spectral information.Hyperspctral image classification is one of the very important measure of ground-objects identification,and having a wide range of applications in areas such as resource exploration and environmental monitoring.With the continues improvement of hyperspectral image detection technology,the hyperspectral image data is getting larger and larger,leading to time consuming computation when processing hyperspectral data.Spark is a distributed big data processing framework,integrated in-memory computation.Spark supports intermediate task results caching,and is suitable for solving complex iterative computing problems of massive data.Based on Spark platform,this paper studies the distributed parallel computing of hyperspectral image classification algorithm,and proposes two kinds of distributed parallel classification method of hyperspectral image.The main work is as follows:(1)Based on the Spark cloud computing platform,this paper proposes the Distributed parallel Spatial Correlation Regularized Sparse Representation Classification(DP-SCSRC)algorithm for distributed Hyperspectral image classification.In DP-SCSRC,firstly,adjacent hyperspectral image indexes are stored in the same partition of Spark’s RDDs to preserve spatial correlation information.Secondly,Joint Distributed Matrix(JDM)is created to reduce overhead data synchronization between computing nodes.Experimental results on real hyperspectral data demonstrate that DP-SCSRC achieves a remarkable speedup and is scalable with larger data size.(2)Base on TensorFlowOnSpark cloud computing platform,this paper proposes Spatial-spectral Convolutional Neural Network for distributed hyperspectral classification(SS-CNN).In SS-CNN,the residual network layer is innovatively utilized as the feature combination,which combines the spectral information of the input layer and the spatial features obtained after the convolution layer to make full use of the joint spatial-spectral features,to enhance the classification accuracy.The experimental results show that the SS-CNN implemented on the TensorFlowOnSpark is faster compared with training on single machine.