Multitask Multiclass Privilege Information Support Vector Machines

This article introduces multitask multiclass privileged information support vector machines. This paper also presents support vector machines, multiclass classification problems, multitask learning, privileged information learning in classification problems in detail.Support vector machines are important implementations of the statistical learning theory. Support vector machines have very strong capabilities to learn and to gener-alize in classification and regression fields. Support vector machines can find the best hyperplane for linearly separable problems. For linearly un-separable problems, uti-lizing slack variables, support vector machines can find the compromise of the error correction and generalization capability. With kernel learning tricks, support vector machines can solve un-linearly classification problems by the projection of the orig-inal data into high dimensional spaces. After using kernel functions, support vector machines can easily find a separable hyperplane in high dimensional spaces. Support vector machines obtain good results in real world applications.In classification field, support vector machines are a kind of binary classifier. The traditional support vector machines can not solve multiclass problems directly. Usually, there are three strategies such as one-against-one, one-against-all, the directed acyclic graph to extend binary support vector machines to solve multiclass problems. But these three kinds of multiclass strategies have different shortcomings. The one-against-one strategy can not using information between different classes, because it just classify two different classes one time. For a classification problem with K classes, the one-against-one strategy have to train K(K-1)/2classifiers in the learning procedure. Al-though the one-against-all strategy put all classes in the training procedure, it takes all the other different classes as one classes. Then it ignores differences between different classes. Although the directed acyclic graph can utilize the information between differ-ent classes to build up an directed acyclic graph for classification. This method has one important shortcoming which names the collection of classification mistakes creating by front classifiers.Multiclass support vector machines are an important extension of traditional bi-nary support vector machines. Considering multiclass classification problems, the strat-egy used by multiclass support vector machines is very different with ones used by traditional binary support vector machines. It cast multiclass categorization problems as a constrained optimization problem with a quadratic objective function which yields a direct method for training multiclass predictors. Because this strategy can capture the information between different classes, it can get comparable results with front three strategies in the classification accuracy, the training time, the number of support vector machines.As another important extension of traditional single task learning support vector machines, multitask support vector machines can improve the generalization perfor-mance by leveraging the domain-specific information contained in the related tasks. The past empirical work has shown that, when there are relations between tasks to learn, it is beneficial to learn them simultaneously instead of learning each task inde-pendently.In the traditional learning procedure of support vector machines, the privileged information (hidden information) is not considered. The privileged information stands for the prior information which can been captured from the training data, while this information can not gathered from testing data. After training with the privileged infor-mation, support vector machines can improve classification accuracy.The multitask multiclass support vector machines introduced in this paper not only can learn multitask multiclass problems, but also can use the privileged information in training procedure. In this paper, the multitask learning, the multiclass problems and the privileged information are presented in detail. The experimental results also show effective results of our learning model.