Research On Model Selection Of Deep Belief Network

As deep learning knowledge is widely applied to all fields of life,the study of deep learning is particularly important.In recent years,models for deep learning have emerged in an endless stream.There are also many variant models for each model.How to choose the right model to apply to the corresponding fields has become the focus of attention of the majority of researchers.However,due to the peculiar nature of the deep learning model and the complex computational problems,the choice of the model and the evaluation model have become obstacles to people's practice.Deep belief network is a network model proposed for the first time in the field of deep learning.Its sub-model RBM model has many variant models,and its model selection problem is also a hot topic.Because of the many types of variant models of RBM model,this paper first analyzes the binary RBM,Gaussian RBM,mean covariance RBM and classification RBM from a statistical point of view.For each model visible layer variables and hidden layer variables,The conditional distribution and the conditional distribution of each visible variable and each hidden layer variable were deduced in detail.Secondly,the probabilistic distribution of the deep belief network model and the probability distribution of its variables in each layer are analyzed by the RBM model and the model training algorithms CD and PCD are simply analyzed.For the problem of model selection in deep belief network,its essence lies in the choice of RBM model.At present,the selection method for this model is mainly based on the reconstruction error method and the annealing importance sampling method.The method of reconstructing the error is computationally simple but lacks objectivity and is not entirely reliable.The main idea of importance sampling of annealing is to use importance sampling and simulated annealing techniques to estimate the log-likelihood of the RBM model for the data,and to evaluate the merits of the model by the magnitude of the log-likelihood.However,there is only a detailed analysis of the binary RBM model that is evaluated using this method.Therefore,in this paper,we propose a method for selecting the Gaussian RBM model and the mean-covariance RBM model using the importance sampling method for annealing,and give specific steps.In the selection process of the mean covariance RBM model,because the samples distributed in the middle are difficult to collect,samples are obtained by using the HMC sampling method.Finally,the ISOLET data set in UCI data is selected for empirical analysis.Two different training algorithms(CD and PCD)training models are used for RBM models with different numbers and different hidden layer variables,and are selected using the models introduced in the article.The method estimates the log-likelihood value,selects the appropriate model,then analyzes the data in the deep belief network model,and analyzes the results obtained by the model.The results show that the deep belief network consists of RBM models with large log-likelihood values.The classification effect on data is good,which proves that the method proposed in the article has certain feasibility.