Research On Least Squares Support Vector Machine For Sparse Large-scale Datasets

Support Vector Machine(SVM)is one popular algorithm in the field of machine learning,which has been proved to have good performance in a large number of applications.Least Squares Support Vector Machine(LSSVM)is an improved model of the SVM,which simplifies the complex process of the SVM algorithm in solving convex quadratic programming problems.In the past decades,the LSSVM model had advantages over other machine learning algorithms on small sample problems,making it develop rapidly and be widely used in various fields.At present,the amount of large-scale data information is growing rapidly in the form of a power exponential curve,so LSSVM is facing the dual challenges of increasing data dimensions and volume.Due to the lack of sparsity of LSSVM,it is more difficult to solve large-scale datasets.Therefore,this paper focuses on the LSSVM classification and regression of large-scale datasets from the aspects of sparse data samples and feature sparseness,including the following main tasks:(1)To solve the problem of LSSVM lacks of sparsity,a sparse LSSVM algorithm based on the quadratic Renyi entropy theory is proposed.The initial dataset is divided into several subsets,and the samples corresponding to the maximum entropy are selected as support vectors(Support Vectors,SVs)in each subset.These SVs can approximately represent the initial dataset and serve as model input for training and test.The experimental results show that the quadratic Renyi entropy is an effective method to solve the sparse problem of LSSVM,the trained model can be applied to the classification problem of large-scale datasets and has a good classification ability.(2)For regression problems,a least squares support direction based on sparse samples and mixed kernel function learning is proposed.The algorithm uses meanshift clustering method to sparse the initial dataset,and gets the reduced sub-sample set for model training.In the training process of LSSVM,the single kernel function is transformed into a mixed kernel function,and the improved Artificial Bee Colony intelligent algorithm is utlized to optimize all the parameters in the model,and the trained model is used for regression prediction.The effectiveness of the proposed algorithm is verified in the standard UCI public dataset,and compared with the related algorithms based on BP,RVM,ELM and LSSVM.Experimental results show that the algorithm improves the prediction accuracy of LSSVM and obtains better fitting degree.(3)Compared with deep learning,LSSVM is one of the typical shallow network models.For large-scale datasets,a classification algorithm based on deep structure LSSVM is proposed.This algorithm is a multi-layer LSSVM structure.In each layer,the dataset is compressed into a small sample subset by using edge detection and K-means technology,which is used for singlelayer LSSVM model training.After multi-layer superposition,the final classification function is obtained.The proposed algorithm is verified in the UCI dataset,and the results show that the algorithm has good performance in solving the classification problem of large-scale datasets.(4)For the problem of large number and high dimension of large-scale datasets,the LSSVM model for the double sparse data sets is established.The subset of the initial dataset obtained by double of sample sparsity and feature sparseness is used for LSSVM model training and prediction.Through the verification and analysis of the UCI dataset,the LSSVM model based on double sparse dataset can effectively solve the problem of sparsity,and when encountering largescale datasets,it can obtain better prediction performance.(5)In the cold rolling production process,to obtain accurate rolling force prediction values,a multi-layer sparse sample LSSVM rolling force prediction model is established.The algorithm uses the edge detection method reduce the initial dataset to a smaller dataset and participates in the training of the LSSVM model.Experiments show that compared with other algorithms,this algorithm merely solves the problem of the lack of solution sparsity of LSSVM and ensures that the algorithm has higher rolling force prediction accuracy.