Research On Multi-classification Ensemble Algorithm Based On Stochastic Configuration Network

Classification problems are common in life.For the two-class problem,many mature algorithms have been widely used.However,many machine learning algorithms fail to achieve satisfactory classification results when dealing with multi-classification problems,especially when there are many categories and there is a large amount of complex data.Therefore,there is still a lot of research space for multi-classification problems.The neural network method is widely used in classification problems due to its excellent nonlinear processing capability and self-learning ability.In dealing with multi-classification problems,the common neural network algorithm is BP neural network algorithm with Softmax function.However,for massive complex data,BP neural network algorithm has the dis-advantages of low convergence efficiency and easy local optimal solution.Therefore,some researchers have added random thoughts to the neural network and proposed the random weight neural network.The random weighted neural networks have been widely used because of their extremely fast learning speed,not easy to fall into local optimal solutions,and the training process is simple and easy to implement.The Stochastic config-uration network is a special single hidden layer random weight neural network.Under a specific supervision mechanism,the hidden layer nodes are gradually generated according to the training error.The randomly configured network solves the difficulty in selecting the number of nodes in the neural network method and is highly flexible.We attempts to apply a randomly configured network to a multi-classification problem.In order to better handle the multi-classification problem,the following improvements have been made:(1)In order to prevent over-fitting,an L2 regularization term is added to the error function;(2)Using a Gaussian radial basis function(RBF)that is more effective on the classification problem instead of the Sigmoid function;(3)In order to prevent the class imbalance in the multi-classification problem,according to the cost-sensitive learning method(CSL)we proposed the weighted SCN algorithm.When there are many categories in the multi-classification problem and the difference between some categories is small,the neural network algorithm has a high probability of mis-classification.Therefore,after using the SCN algorithm to classify the original data for the first time,for each sample data selected two classes with the highest probability score to divide the data into m data subsets.Then,m data subsets are reclassified using a random forest algorithm with stable classification effect.Finally,according to the classification result of the random forest,determine which of the two classes is the final class of the sample data.Finally,10 real multi-class data sets were selected on the UCI website for experimental verification.First,the validity of the weighted SCN algorithm(WSCN)is verified;Then analyze the existence of mis-classification and a necessity to reclassify using other algorithms;Finally,the WSCN-RF algorithm is compared with the BP neural network multi-classification algorithm and the random forest algorithm.The results show that the proposed WSCN-RF algorithm is superior to other multi-classification algorithms in classification accuracy.