Mutiscale Spectral-spatial Fusion Network For Hyperspectral Images Classification

The influence and rapid development of hyperspectral imaging not only arouse great interest in remote sensing,but also receive high attention from military,agriculture,environmental science,medicine and other fields,where feature extraction and classification of hyperspectral images have become one of the most active topics in recent years.With the advantages of high resolution spectral information and ‘image-spectrum merging' technology,as well as the shortcoming of nonlinearities,‘same object with different spectrum' and ‘foreign objects with same spectra' phenomena that caused by plethora of factors,learning representative and discriminative features that make full use of spectral and spatial information is of cardinal significance for hyperspectral imagery interpretation,and will strongly promote its application in various research fields.Based on in-depth analysis of hyperspectral data and systematically review of current hyperspectral processing methods,this dissertation focus on proposing more effective approaches for discriminative feature extraction and precise classification.The main research contents are summarized as follows:1)Based on transfer learning and joint sparse representation,a deep multiscale feature based multitask joint sparse representation approach is proposed to combine the local and global spatial information in hyperspectral image and achieve classification.By transferring the filter parameters of all the convolution modules in VGG-16,the local spatial information and global semantic features of hyperspectral images at different scales are extracted in this method.With visual display,the inter-class separability of features increases significantly as the scale deepens,but the aggregation within the same class also be deteriorated gradually.Thus in this paper,combining the advantages and disadvantages of each scale feature,we constructs multi-task joint sparse representation under different dictionaries to obtain the final classification results.Compared with existing feature extraction and sparse representation methods,the classification accuracy and time complexity of this method are significantly improved.2)In view of adaptive multiscale feature extraction,this paper further put forward a fully convolution network-based deep multiscale spatial distribution prediction method,in which the spatial features with higher discriminability are studied meanwhile the computational complexity is greatly reduced.In this approach,all the filter parameters in VGG-16-based deep fully convolution network are transferred to complete multiscale spatial distribution prediction of hyperspectral images firstly.Then,weighted fusion is carried out to fuse the raw spectral information from the deep multiscale spatial features,so as to compensate for the insensitivity of the deep network to high resolution spectral features.Finally,three classifiers are utilized to verify the validity and universality of the extracted features.From the comparison,our method shows certain advantages over the existing methods,especially for hyperspectral data with highly nonlinear distribution and spatial diversity.Besides,the real-time performance of this method is considerably improved.3)In face of the different response at various scales from natural images and hyperspectral images,a collaborative multitask autoencoder method is proposed for multiscale spectralspatial fusion.Firstly,multiscale spatial features of hyperspectral images are extracted by transferring filter parameters of VGG-16.Then,on each scale,the deep spatial feature cooperated with raw spectral information are projected to a common low dimensional space by collaborative sparse autoencoder method with an unsupervised manner,and meanwhile retain as much spectral-spatial information as possible.Finally,the multiscale spectral-spatial features are obtained by upsampling and weighted fusion of all scale features.Support vector machine classifier is used to evaluate the proposed method qualitatively.Compared with existing feature learning methods,our method is more efficient for hyperspectral image classification,especially for datasets with fewer labeled samples,complex semantic information and large geometric space heterogeneity.4)Deep spectral-spatial fusion strategy is significantly effective for discriminative hyperspectral image feature extraction.However,collaborative autoencoder only consider learning the commonality between different features,but ignore the interaction and potential manifold relations between the objects within the same feature set,especially for feature extraction at pixel level.Therefore,we further puts forward a kind of superpixel-based relation autoencoder method to reduce the dimension of deep spatial feature.In this work,representational consistency constraint is added in the objective function with the consideration of the embedded image manifolds in the original spectrum,and enhance the cohesiveness of hidden layer features by iterative supervised learning.In order to reduce the computational complexity of relational constraint and choose the neighborhood relationship adaptively,consistency constraint in each superpixel is further introduced to optimize and improve relational autoencoder.The final feature extraction is accomplished by collaborative encoder of spectral-spatial features and weighted fusion of multiscale features.A large number of experimental results demonstrate that our proposed method achieves expected results in discriminant feature extraction and has certain advantages over the existing methods.In conclusion,this paper focus on the condition of small sample data,and consider how to combine the local and global information for hyperspectral image classification,how to combine the transfer Learning,deep learning and traditional machine learning method to extract more discriminative features which is more conducive to understanding and recognition of hyperspectral images,and thus put forward four methods including multi-task classification and discriminant semantic feature extraction.At the same time,a large number of experimental analyses are conducted to verify the feasibility of all the methods proposed in this paper and the advantages compared with the existing methods.