Research On Extreme Learning Machine And Application To Hyperspectral Remote Sensing Image Analysis

In machine learning community,there are many effective methods,which are used to tackle different real-world application problems.Commonly used methods are support vector machine,artificial neural network,K-nearnest neighbor,navie Bayes and decision tree.Artificial neural network is one of the most popular method for its changeable network structure and powerfull simulation capability.Being used to train single-hidden layer neural networks,extreme learning machine(ELM)shows the characteristics of simple implementation,fast learning speed and effectiveness.Due to these characteristics,ELM has been applied to solve many application problems and achieved promising results.However,ELM is challenged by the following two factors: randomly generated network parameters and small-size data analysis.For the former,in ELM,the input network weights and hidden biases are manually selected,and the output weights are analytically determined.Randomly generated network parameters will result in inferior performance when ELM is applied to solve real-world applications.For the latter,it is very difficult to acquire enough data from some real-world applications.To tackle such small-size data,ELM may lead to overfitting problem.The above two potential problems are considerated from the algorithm perspective of ELM.In reality,ELM is well-accepted in hyperspectral remote sensing image(HSI)application.For the application perspective,traditional ELM only uses the spectral information for HSI analysis,and ignores the potential spatial information.Therefore,ELM can not achieve superior performance in HSI application.In this thesis,we study the improved ELM algorithms and application to HSI analysis.The major contributions are summarized as follows:(1)Supervised network parameter learning: memteic extreme learning machineTo solve the randomly generated network parameter problem,we incorporate memetic algorithm into ELM to adaptively learn suitable network parameters in a supervised manner.In our proposed method,the memetic algorithm is comprised of adaptive differential evolution algorithm and simulated annealing algorithm.During the evolutionary learning process,network parameters are regarded as individual in the population,which go through the fitness calculation,global optimization and global search to survive the superior and eliminate the inferior.The optimal network parameters are finally used for the construction of the proposed model.Experimental results show the superiority of the proposed method.(2)Unsupervised network parameter learning: sparse pre-trained extreme learning machineThe existing network parameter learning methods for ELM are supervised learning methods.In general,it is very difficult and expensive to acquire label for each data sample.Motivated by this,we use a sparse autoencoder to learn suitable network parameters for ELM in an unsupervised manner.In the proposed method,the ELM-based sparse autoencoder with L1 norm regularization is used to determine its output weight network parameters,and encodes the usefull information into them.By using the matrix transpose and average operator,the network parameters of the proposed model can be determined by the output weight network parameters learned by the ELM-based sparse autoencoder.Experimental results demonstrate that the proposed method outperforms the existing supervised netwotk parameter learning methods for ELM.(3)Small-size data analysis: instance cloned extreme leaning machineSmall-size data is common in many real-world applications,and overfitting problem may occur in such suitation.To this end,we introduce instane cloning technique into ELM to locally alleviate the unbalanced data distribution and mitigate the overfitting problem of the learning model.Relying on the idea of K-nearest neighbor,instance cloning technique choose nearest samples from training data for each testing sample,and analytically determine their clones.Finally,the proposed model is trained on the combination of the original training data and the clones of nearest training samples for testing data.Experimental results claim that the proposed method is superior to the compared methods.(4)Spectral-spatial HSI analysis: superpixel-based kernel extreme learning machineHSI data consists of a large number of spectral features.In addition to the basic spectral information,HSI also contains plentiful potential spatial information.To take full advantage of the potential spatial information hidden in HSI,we propose an ELM-based spectral-spatial hyperspectral remote sensing image classification method to improve the accuracy of identifying land cover of pixel in HSI.The proposed method employs superpixel segmentation algorithm and principal component analysis to extract spatial features in an unsupervised manner,and adopts a kernel ELM to be the classifier to learn the combination of spectral features and spatial features and identify the land cover for each pixel.Experimental results show that the proposed method is skilled in indentifying land covers for HSI.In summary,we first analyze the deficiency of ELM,and study ELM from the perspectives of algorithm and application for in-depth research.For the algorithm perspective,we propose two suitable network parameter learning method for ELM in supervised and unsupervised manner and one effective learning method for ELM to tackle small-size data problem.For the application perspective,we propose an ELM-based spectral-spatial hyperspectral remote sensing image classification learning method.Experiments and comparisons in our research demonstrate the effectiveness of these proposed algorithms.