Universality Of Kernel-based Deep Learning Method

Obtaining more effective information from limited empirical data is the basic starting point for solving many theoretical and practical problems.As people enter the era of cloud database,the development of various effective data processing methods has become an urgent need for the development of science and technology.The research of data processing method needs suitable ”soil”,that is,hypothesis function space or function class,in order to achieve effective approximation to the objective function.An ideal hypothetical function space or function class requires consistent approximation of the data processing method,that is,continuous functions on any compact subset of the input space can be uniformly approximated by the solution function generated by the data processing method as the input data increases.Reproducing kernel Hilbert space is an ideal background space for point value data processing.The problem of uniform approximation has attracted great attention in the field of machine learning,and has been extensively studied in theory and application.This paper mainly studies the uniform approximation of deep learning method based on regenerated kernel,which was first proposed by Bohn et al.in 2019.Firstly,we review the uniform approximation of continuous functions to several typical function spaces or function classes,including polynomial function spaces,Sigmoid function classes,and reproducing kernel Hilbert spaces.Secondly,we study the double-layer regularized network based on reproducing kernel Hilbert space composition,establish the corresponding representer theorem,and study the uniform approximation of the composition of uniformly approximated kernel to the composite continuous function.Finally,inspired by the idea of sparse approximation,we study a double-layer regularized network based on reproducing kernel Banach space composite.On the basis of establishing the representer theorem of regularized networks,we give the uniform approximation of the composition of uniformly approximated kernel to the compound continuous function.