Stochastic Variational Inference For Probabilistic Model And Its Application

Probabilistic model is one of the most widely used density estimation and clustering tools because of its flexible expression.At present,with the rapid development of Internet technology and the explosive growth of data,how to apply the probabilistic model to large-scale data sets has become an urgent problem.To tackle this problem,this paper mainly discusses how to solve the problem of parameter estimation and model selection in the mixture model on large-scale data sets.First of all,this paper introduces the current research situation and the meaning of the probabilistic model,raises the definition of the mixture model.We introduce two main methods of parameter estimation and model selection: deterministic methods and non-deterministic methods.Among them,the deterministic method is represented by EM algorithm.However,the EM algorithm is sensitive to initial value and can't automatically determine the number of mixture weight.Therefore,the non-deterministic method represented by Bayesian method is adopted.However,Bayes theory can not be directly used to solve the problem so that Variational Inference is adopted.Next,the Inverted Dirichlet Mixture Model(IDMM)is modeled in this paper.The IDMM is modeled with the traditional Bayesian framework to approximate the true posterior distribution by continuously maximizing the lower bound of the variational function.Based on the traditional Bayesian Variational Inference,the Stochastic Variational Inference is introduced to solve the problem of low efficiency.When updating local parameters,the traditional Variational Inference needs to update the local variational parameters with all sample points.When the local parameters are updated in each iteration,the Stochastic Variational Inference extracts some samples to calculate the local variational parameters,then updates the intermediate global variational parameters with local parameters.Finally the current global variational parameters are obtained by weighted average of the intermediate variational parameters and the previous global variational parameters.Finally,experimental comparison between the two methods shows that the Stochastic Variational Inference reduces the computational burden of the algorithm and greatly improves the efficiency of the algorithm.Experiments are carried out on the synthetic data and real data set Scene13 and Caltech4 to prove the efficiency of the Stochastic Variational Inference.