Probabilistic Outputs For Bounded Support Vector Ordinal Regression

Ordinal regression is a supervised learning problem,where training samples are marked by a group of ordinal numbers.Its goal is to learn a classifier from a discrete or continuous set of labeled samples to predict the label of a new sample.The ordering relation and nonmetric property of the label set distinguish it from the multiclass classification and metric regression.In fact,ordinal regression has been widely used in areas where human preferences play an important role.For examples,credit rating,face recognition,information retrieval,medical research,picture processing,and so on.Due to the wide applicability of the ordinal regression,there have been many algorithms to solve the ordinal regression in the field of machine learning.The support vector machine(SVM)has become the most widely used model in the ordinal regression because of its good generalization performance,and its structure is especially suitable for the model with threshold parameters.The support vector ordinal regression machine with explicit constraints(SVOREX)considers using the empirical errors from the samples of adjacent classes to determine the thresholds,and defines parallel discriminant hyperplanes for the ordinal ranks by optimizing multiple thresholds,ensuring that thresholds are ordered at the optimal solution.However,the algorithm ignores distribution characteristics of the ordered data sets,especially when the samples that are from the first category and the last category are in an unbounded range,it has no advantage in solving the problem.In the view of this,we propose a novel ordinal regression model,which is named as bounded support vector ordinal regression with explicit constraints(BSVOREX).By adding boundary constraints to the ends of thresholds,the new model can describe the distribution characteristics of data sets better.Numerical experiments show that the model has good generalization performance.In addition,constructing a classifier which can produce a posterior probability is very useful in the practical recognition problems.Posterior probabilities are also required when a classifier is making a small part of an overall decision,and the classification outputs must be combined for the overall decisions.However,the output of common support vector ordinal regression machine is a real number,instead of a probability.Thus,to establish proper algorithms that can give a probabilistic outputs for ordinal regression is also worth studying.Focus on this problem,a posterior probabilistic output model based on Bayesian criterion is proposed,which is used to obtain the probabilistic outputs for the bounded support vector ordinal regression machines,so that the model can not only have the qualitative interpretation,but also have the quantitative evaluation when classifying on the ordered datasets.Finally,numerical experiments on are carried out on several data sets to verify the validity and feasibility of the proposed algorithm.