Research On Multiple Birth Support Vector Machines With Different Loss Functions

Twin support vector machine(TWSVM)is an extension of well-known support vector machine(SVM).In addition to keeping the advantages of SVM,the training speed of TWSVM is also higer than that of SVM.TWSVM is originally designed for binary classification.Many pattern recognition problems in real world are multi-class classification problems.In order to extend TWSVM to multi-class classification problem,the study of multi-class twin support vector machines has been paid more and more attentions.Multiple birth support vector machine is one of the recently proposed multi-class twin support vector machines.Multiple birth support vector machine has drawn extensive attentions of scholars because it has higher training speed than most of other multi-class twin support vector machines for the classification problems that contain large numbers of classes.However,compared with traditional multi-class twin support vector machines such as one versus one multi-class twin support vector machine,multiple birth support vector machine needs to be improved with regard to classification accuracy.Loss function directly affects the performance and training speed of machine learning algorithms.It is important to study the multiple birth support vector machines with different loss functions.This thesis aims to improve multiple birth support vector machine by starting from studying the loss item.In order to futher improve the approach,the idea of granular computing is introduced into multiple birth support vector machine.The main contents of this thesis are as follows.Study on multiple birth support vector machine based on weighted linear loss.Weighted linear loss multiple birth support vector machine uses weighted linear loss function to modify the objective function of multiple birth support vector machine.The model of weighted linear loss multiple birth support vector machine uses weighted linear loss function instead of Hinge loss function to estimate the empirical risk.This thesis transforms the optimization model of weighted linear loss multiple birth support vector machine to systems of linear equations.Weighted linear loss multiple birth support vector machine only needs to solve systems of linear equations while the Hinge loss multiple birth support vector machine has to solve quadratic programming problem,so the training speed of weighted linear loss multiple birth support vector machine is always higher.The proposed algorithm is tested by experiments on UCI datasets.Study on multiple birth support vector machine based on Ramp loss.Because the Hinge loss function used in the initial multiple birth support vector machine does not have upper bound,the algorithm is sensitive to the presence of outliers.In order to solve the problem,the thesis combines Ramp loss function that has upper bound and lower bound with multiple birth support vector machine and proposes Ramp loss multiple birth support vector machine.The model of Ramp loss multiple birth support vector machine is non-convex optimization problem.Common optimization tool can hardly find exact solution of the non-convex optimization problem.This thesis designs a concave-convex procedure method to solve the non-convex optimization problem.The classification performance of the proposed Ramp loss multiple birth support vector machine is tested and verified by experiments and compared with multiple birth support vector machine,multiple birth least squares support vector machine and weighted linear loss multiple birth support vector machine.In order to further improve multiple birth support vector machine,following the idea of granular support vector machine,the granular computing is introduced into multiple birth support vector machine,multiple birth least squares support vector machine,weighted linear loss multiple birth support vector machine and Ramp loss multiple birth support vector machine to get four granular multiple birth support vector machines.These algorithms firstly divide the feature space into a sequence of information granules,then build sub-classifier in each information granule and lastly obtain the decision function by combining all sub-classifiers.The experimental results indicate that the proposed four granular multiple birth support vector machines based on different loss functions have better performances than the original methods.