Research On Several Learning Problems Based On Kernel Alignment

Kernel method is a commonly used method for pattern analysis.The principle is to transform linear indivisible problems into linear separable problems in a high-dimensional feature space through a nonlinear mapping,so that problems can be processed using linear algorithms.Moreover,inner product in the feature space can be directly calculated utlizing a kernel.Due to different kernels will generate different structures in the mapped high dimensional space,the performance of kernel methods largely depend on the correct choice of kernel.Furthermore,different parameters for the same kernel also will affect the performance of kernel methods.Consequently,it is a hot topic to study kernel and kernel parameter selection in the field of machine learning.Kernel alignment,which aims at measuring the consistency between two kernels,is a commonly used mehtod to select appropriate kernel for a particular learning problem.The advantage of utilizing kernel alignment to select kernel is that only the alignment values need to be calculated to make the kernel suitable for the learning problem,and not related to the specific classifier training process.After the concept of kernel alignment was proposed,kernel alignment has been improved,expanded and applied by lots of scholars.Based on kernel alignment,this paper mainly studies the selection of fuzzy kernels and its application in the attribute reduction of heterogeneous data,as well as the kernel selection and feature selection for multi-labeled data.The main work includes the following aspects:(1)A new method for selecting fuzzy kernels based on kernel alignemnt is proposed.In fuzzy decision system,a new ideal kernel is defined,based on which a novel fuzzy kernel alignment model is constructed.Then,the fuzzy kernel is selected through minimizing the fuzzy alignment value between the defined ideal kernel and a kernel for the learning problem at hand.In order to verify its effectiveness,we prove that the upper bound of classification error of a support vector machine will decrease with the decrease of the fuzzy alignment value.Furthermore,the proposed fuzzy kernel selection method is also applied to the attribute reduction of heterogeneous data.Experimental results show that the proposed attribute reduction of heterogeneous data based on fuzzy kernel alignment is effective.(2)A novel classification algorithm is proposed based on kernelized fuzzy rough set.Transform the positive domain of kernelized fuzzy rough set into the sum of the distances of samples to the classification hyperplane,and then an optimization problem for solving classification hyperplane is constructed by maximizing the positive domain.The experimental results show that the proposed classification algorithm based on kernelized fuzzy rough set is effective.(3)Based on kernel alignment,a kernel selection method for multi-label learning and an improved classifier chain multi-label learning algorithm are proposed.Firstly,an appropriate ideal kernel is presented for multi-label datasets.Then kernel is selected by determaining weights of the linear combined kernel in feature space through maximizing the alignment value between the linear combined kernel and ideal kernel.Moreover,this proposed kernel selection method is further improved by considering the local kernel alignment criterion.Secondly,given the kernel in feature space,the order of classifier chain can be obtained respectively by minimizing the alignment value between this kernel and the convex combination of ideal kernels defined by each label,and directly calculating the alignment values between kernel and each label.Experimental results demonstrate the effectiveness of the two proposed algorithms based on kernel alignment.(4)Propose an alignment based feature selection method for multi-label learning.Firstly we define an ideal kernel in label space as a convex combination of ideal kernels defined by each label,and a linear combination of kernels where each kernel corresponds to a feature.Secondly,through maximizing the kernel alignment value between linear combined kernel and ideal kernel,both weights in the two defined kernels are learned simultaneously.And the learned weights of labels is employed as the degree of labeling importance.Finally,features are ranked according to their weights in linear combined kernel,and features with small weights are deleted.The proposed feature selection method can learn importance degree of labels automatically,and effectiveness of this method is demonstrated by experimental comparisons.