Regularization Parameter Selection Methods For Kernel Ridge Regression

Kernel ridge regression is a simple and powerful nonparametric regression method that combines model complexity and prediction accuracy.Despite the existing literature has established a theoretical framework for kernel ridge regression estimation and hypothesis testing,the theoretical properties of regularized parameter estimators still largely unclear.This thesis is the first to systematically study the choice of regularization parameters for kernel-ridge regression and its theoretical properties.Two types of regularization parameter estimation,Frequentist-based and Bayesian-based estimates,are proposed in this thesis,and their theoretical properties are studied respectively.The frequentist approaches minimize asymptotically unbiased estimators of average mean-squared error of the estimated function.The Bayesian approaches by assuming that the original function comes from a Gaussian process,are to formulate the kernel ridge regression as an equivalent linear mixed effects model and then use the maximum likelihood method to estimate the regularization parameter.Under two different data generation mechanisms,this thesis studies the asymptotic properties of the two types of regularization parameter estimates.The consistency and asymptotic normality of the regularization parameter estimators are proved in this thesis under weaker assumptions.At the same time,under the linear mixed effects model,it is proved that the two types of regularization parameter estimates are asymptotically consistent.Finally,numerical simulations aer used to examine the theoretical results and the excellent properties of the parameter estimators.