Sufficient Dimension Reduction Based On Kernel Function In Kernelized Support Vector Machine

In this thesis,we focus on improving the kernelized support vector machine via sufficient dimension reduction.In the field of machine learning,the support vector machine is an important classifier.The kernelized support vector machine is an extension of the support vector machine to solve the issues of more complicated classification,of which the main idea is to change or increase the dimension of the original feature through the mapping function.In the kernelized support vector machine,the kernel function plays a key role.From the kernel function,the gram matrix can be obtained,which provides the information of the relative positions of the augmented data points without specifying their values.That means the augmented feature vector can be infinite-dimensional if necessary.However,if too many useless new features are included,the contribution of the useful ones may be masked,and that may be quite harmful.Hence,we try to conduct dimension reduction of the augmented feature vector,so as to reduce redundant information,collect effective feature information and optimize the model.As what we need here is a nonparametric dimension reduction,we start from a method of sufficient dimension reduction.Specifically,we start from the cumulative mean estimation(CUME).We choose CUME because its basic idea of distributionweight has a great advantage on the efficient use of the information and makes it suitable to handle the dimension reduction of the augmented feature vector,which is probably infinite-dimensional.A method called the kernel cumulative mean estimation(K-CUME)is proposed.The basic idea of K-CUME is to transform the CUME dimension reduction of the augmented feature vector into an equation involving gram matrix,which makes the direct handling of the infinite-dimensional augmented feature vector avoided.This transformation is based on the inner product and the relationship between the kernel function and the augmented feature vector.A specific algorithm of K-CUME is designed.The proposed K-CUME method is illustrated by the analyses of a simulated annular high-noise dataset and a real dataset(wine data).