The Convergence Analysis And Application Of Kernel Regularized Regression Learning And Vector Ranking

In the economic and social problems,many practical tasks can be transformed into regression problems,such as market trend prediction,population forecast,economic development factor analysis,stock price forecasting and so on.There are many applications in ranking problem.For example,ranking learning has many practical values as a powerful tool in information retrieval,quality control,survival analysis and computational biology.In the regression analysis,we often encountered nonlinear problems,the commonly used regression methods,such as multiple linear regression,stepwise regression,linear quantile regression methods are no longer suitable for data fitting and forecasting.By defining a kernel function as a nonlinear transform,it can map the nonlinear input data space to a high-dimensional feature space.Then,the regression function can be constructed in the linear feature space.Moreover,it is only necessary to calculate the inner product of mutual data in the feature space.So it is an effective method to solve the nonlinear regression problem by using kernel function.In Regression problem,the algorithm based on empirical risk minimization belongs to an ill posed problem,which is prone to over-fitting phenomenon in the case of small sample size.It has a high fitting degree to the training data,but the ability to predict the unknown data is poor,and the complexity of the model is too large.For this reason,Tikhonov presents a regularization method that adds a regularization term to represent the model complexity after the expected risk.A regression learning algorithm is constructed under the structural risk minimization criterion.As a new method used in learning theory,the kernel regularization technique combines the advantages of kernel function and regularization method.In this paper,based on kernel regularization method,the convergence rate of regression learning and vector ranking learning is studied under the least square loss.The upper bound of the algorithm error is analyzed,in order to measure the ability of the learning algorithm based on the training samples to predict unknown data.We investigate whether the convergence speed of the algorithm is affected by the convexity,approximation performance and capacity of the reproducing kernel space.This is a focus problem in learning theory.The main contents and innovations of this paper are embodied in the following three aspects:1.Recently,many scholars have investigated the convergence rate for the regularization regression learning algorithm based on reproducing kernel Hilbert spaces.However,the simplicity of structure of Hilbert spaces also firmly marks its limitation since many data do not come with the distance induced from an inner-product.Therefore,it is necessary to expand the hypothesis function space,Banach space is a reasonal selection.The investigation on the convergence performance analysis of the regularized regression algorithm based on reproducing kernel Banach spaces is a new research field.A key theoretical problem is how the convergence rate is influenced by the geometry property of the Banach spaces.In the third chapter,on the basis of existing research results,under the assumption that Banach space B has q-uniform convexity(q>1)and a uniformly continuous reproducing kernel,we provide the learning rate for the kernel regularized regression based on reproducing kernel Banach spaces.The learning rate is provided in both expected mean and empirical mean.The results show that the uniform convexity influences the learning rate.We improve the learning speed obtained in the existing literature.Then we give an example satisfying the conditions to explain the reasonableness of the derived theorem based on reproducing kernel Banach spaces.2.For the case of Banach space without convexity requirements(q=1),it has not been seen the convergence analysis of regularization regression algorithm based on reproducing kernel Banach space.In the fourth chapter,we study this issue and give the probability bounds for the generalization error in terms of the expected mean.The results show that the upper bound of the expected error is related to the sample size,the complexity of the reproducing kernel Banach space,the approximate error,the upper bound M of the output space Y,and the the reproducing kernel.3.Ranking learning can be regarded as a special regression problem,but it also has its differences.In ranking,one learns a real-valued function that assigns scores to instances,but the scores themselves do not matter.Instead,the key is the relative ranking of instances induced by those scores.Recently,ranking theory is unified with the machine learning.So the regularized kernel ranking is formed.It is a new research topic to extend the general rankling setting to vector rankling problem.In the fifth chapter of this paper,by using the covering number and the reproducing property of the hypothesis space,we give an investigation on the quantitative convergence analysis of the kernel regularized vector ranking based on reproducing kernel Hlibert spaces with least square loss.We use the tool of Gateaux derivative to study the qualitative relation between the solution and the hiding distribution and quantitatively show the robustness for the solution.Finally,we provide a learning rate of the vector-valued ranking algorithm based on the approximation ability and capacity of the involved reproducing kernel Hlibert spaces.In addition,both the results of Monte Carlo numerical simulation and the empirical analysis of economic prediction indicate that the kernel regularized regression method is an effective way to deal with nonlinear regression problems.