Study Of Distributed Regularized Regression Learning Algorithm Based On Multi-scale Kernels

A distributed regularized regression learning algorithm based on multi-scale kernels is proposed in this paper,which improves the least square regularized regression learning algorithm in a reproducing kernel of Hilbert space.For some complex data sets with non-flat distribution and large sample size in regression problems,based on multiple kernels method and distributed learning method,this paper randomly partitions them into multiple disjoint data subsets,and considers different degree of fluctuation in each one,multi-scale Gaussian kernel functions with a linear combination of different coefficients are adopted.The sum space of the reproducing kernel of Hilbert spaces generated by Gaussian kernels of different scales is used as the hypothesis space.According to the least square regularized regression algorithm,a local estimator is learned from each subset independently at the same time.Finally,a total estimator is obtained by weighting all the local estimators.The multiple kernels method composed with Gaussian kernels of different scales can combine the characteristics of each single kernel well.Furthermore,it can fit the various degrees and the trend of fluctuation as much as possible.By a divide-and-conquer approach,distributed learning method can reduce running time and memory cost of kernel matrix inversion,which resulting in improving running efficiency of the algorithm.The combination coefficients of the different scale kernel functions are set on different data subsets to make them change dynamically.It can make the multi-scale kernel functions have a certain adaptive ability.The performance of the proposed algorithm and other three existing ones is evaluated on two simulated data sets and four real data sets.The experimental results show that the proposed algorithm can achieve better fitting effect in shorter periods of time.And it not only can guarantee the better fitting ability,but also can reduce the running cost required for normal implementation successfully.