Online Ordinal Regression Based Ramp Loss And Its Application

Online ordinal regression plays a key role in the learning task of tag sorting of data samples.However,due to the impact of data noise,many noise elimination algorithms have been carried out.Based on the RAMP loss function,we propose PA-RAMP,PRIL-RAMP and other noise tolerance algorithms based on PA and PRIL class,and study its anti-noise performance and application in online ordinal regression,mainly from the following three aspects:Firstly,based on the basic framework of online ordinal regression PA algorithm,the noise sensitivity of Hinge loss was analyzed.The Ramp loss function was introduced,and the non-convex loss objective function was transformed into a convex optimization problem by CCCP method.The PA-RAMP and SCA-RAMP noise tolerance algorithms were proposed to reduce the impact of noise data.Based on the online convex optimization theory,the threshold protection of the algorithm is proposed,and the prediction accuracy,statistical properties and noise tolerance sensitivity of the algorithm are studied on the actual datasets.The results show that,with the increase of noise level,the proposed PA-RAMP algorithm has higher prediction accuracy than the benchmark algorithm PA-I,and the anti-noise performance of PA-RAMP is better than that of other benchmark methods(PA,PA-I and PRIL)on data sets with different noise levels.This indicates that RAMP loss can effectively reduce the impact of noise data to a certain extent.Secondly,in view of the PA-II algorithm which minimizes the loss of Squared Ramp loss,the noise tolerance format of the algorithm in the online ordinal regression is improved.Based on the Ramp loss of the Squared error,the PA-II-SRAMP and SCA-II-SRAMP algorithms are proposed,and the rationality of the algorithms is analyzed.The experimental results show that the anti-noise performance of PA-II-SRAMP algorithm is obviously better than PA-II and PRIL algorithm,and is always better than PA and PA-I algorithm.Compared with the noise-resistant PA-RAMP algorithm,the computational efficiency is significantly enhanced.It is proved that the proposed algorithm can effectively improve the noise sensitivity of online ordinal regression.Finally,to solve the problem of complex parameter updates of noise resistant algorithm in big data background,a new PRIL-RAMP online ordinal regression algorithm based on fixed-step update rule and K-PRIL-RAMP algorithm in nonlinear format are proposed.The OGD method is used to iteratively solve the parameter optimal solution,and the threshold protection of the proposed PRIL-RAMP algorithm is proved theoretically.Experimental results show that compared with the PRIL-RAMP and K-PRIL-RAMP proposed by the benchmark algorithm,the anti-noise performance is significantly enhanced.In addition,compared with PA-RAMP algorithm of automatic selection of step size,PRIL-RAMP algorithm is more stable against noise,and it is a robust algorithm that is easy to process noisy data streams.