An Effective EM Algorithm For Estimating The Parameters Of The Negative Binomial Distribution And Its Application

With the change of natural environment and the popularization of science and culture,people pay more and more attention to the topics and data that affect human life and health,such as gene sequencing,seismic frequency and air pollution.Among them,count data is an indispensable part of information mining in the age of big data and artificial intelligence.The phenomenon of over-dispersion,say the variance exceeds the mean commonly arises in most count data sets.This fact leads to that the flexible negative binomial distribution has attracted the attention of more and more researchers in comprison with the Poisson distribution.However,the problem of parameter estimation for negative binomial distribution has always been a challenging research topic.Therefore,this paper studies an effective expectation maximization(EM)algorithm for estimating the parameters in the negative binomial distribution and investigates it in the unsupervised classification model,that is to say,the generalization and application of mixed negative binomial distribution model and hidden Markov model with negative binomial output distribution.Not only does this work enrich the parameter estimation methods for the classification model,but also it extends the classification model based on the negative binomial distribution.For the mixture of negative binomial distribution model with wide application background,an effective EM algorithm for parameter estimation is proposed in this paper.This algorithm avoids the step of nesting iterative numerical solutions in the M-step in the EM algorithm and is an optimization of the traditional algorithm.The results of numerical simulation show that the new algorithm significantly improves the operation speed and is consistent with the traditional method in classification accuracy.In specific applications,the algorithm performs well in fitting seismic data and classification problems.For a more complex hidden Markov model whose output distribution is a negative binomial distribution,a similar EM algorithm that can effectively avoid nested iterations is generalized.The results of numerical simulation show that the proposed algorithm performs well in both parameter estimation and classification problems.The application of negative binomial distribution and hidden Markov model in air pollution classification,not only realizes the creation of air pollution classification and application model based on PM2.5 data,but also the accuracy of the new parameter estimation algorithm and its great advantages in operation speed are verified in practice.