Support Vector Machine Theory, Algorithm And Implementation

The support vector machine(SVM) , a sort of machine learning method based on the statistical learning theory, was proposed by Vladimir N. Vapnik during the middle of the 1990s. Now SVM has become a new hot research field in the machine learning area, for the sake of it' s excellent performance in solving the small sample problems in the machine learning area. However, some problems of it, such as the expenditure problems in training SVM, the SVM's generalization function, the selection of the kernel function etal, have not been well resolved in the long run. Therefore, the theoretical superiorities of SVM could not be displayed when it' s used to solve the substantiate problems.To widen the SVM' s application area and make a deeper research into SVM, in the thesis it is aimed to solve the expenditure problems for training SVM, the SVM' s generalization function, the selection of the kernel function. The study focuses are put on the following aspects:1) The theoretical basement of SVM is analyzed, the flaws of the empirical risk minimization and the superiorities of the structure risk minimization is researched, the current usage and research station of SVM is summarized, and the superiorities and some aspects to be improved of SVM are pointed out.2) The implementation method of convex quadratic programming is investigated. When used to solve the convex quadratic programming problems with super large scale of training samples, the traditional method has many shortcomings such as the computation time, the memory expenditure and the computation accuracy. When used to solve the convex quadratic programming problems with super large scale of training samples(11000 training samples), the algorithm designed in this paper works better.3) Some of the SVM algorithms are compared, based on the structure risk minimization a common method for improving the SVM algorithm is proposed.4) Some important factors which have influence on SVM' s training function and SVM' s learning function are analyzed, and it is pointed out that accompanied with the augment of the training samples number the SVM' s training time will be prolonged in a linear way, and that the trust span will become narrower and the classification correct ratio will be improved, and that the penalty factor will change in a small range when the classification correct ratio comes to the maximum.5) The bit map method is used to describe the textures when classifying the Brodatz and KTH TIPS texture images, which helps to cut the expenditure of texture feature extraction and transforms the problem into a machine leaning problem, fit for the SVM method. From the experiments in the paper(under the acceptable expenditure), high classification correct ratio was achieved.