EM Algorithm And Applications

EM (expectation-maximization) algorithm which proposed in1977by Dempster, Laind and Rubin is an iterative maximum likelihood estimation of the optimization strategy for seeking parameters. It can focus on the non-complete data from the parameters of maximum likelihood estimation, which is a very simple and practical learning algorithm. This method can be wide-ly used in missing data, incomplete data and censored data with noise. EM algorithm is an effective method in the missing data and incomplete data of maximum likelihood estimation.At first, the article describes the background and significance of the EM algorithm, research status, and the theoretical of EM algorithm optimization iterations and its general steps. Then gives an instructive example so that we can better understand the EM algorithm, In the next, the convergence of the EM algorithm has been proved, and each iteration of the EM algorithm can always improve the likelihood function until it converges to a stable point, then study the convergence rate of the EM algorithm.As can be seen from the previous section, the advantages of the EM al-gorithm is obvious, just like its simplicity, convergence, steady increase, but it also has many shortcomings, then the paper describes the various improve-ments of the EM algorithm for the shortcomings. Including Monte Carlo EM algorithm which improved E-step and its principle, the general steps iterative; ECM, ECME algorithm which improved M-step and its theory, algorithm, convergence rate analysis, and PX-EM algorithm which improved convergence rate for the EM algorithm.The last part of the article gives three very important applications of EM algorithm, including the application of bivariate normal distribution parameter estimation, Gaussian mixture distribution parameter estimation and hidden Markov model (HMM) parameter estimation, which is known as the Baum-Welch algorithm, a special case of the EM algorithm. Finally, programming the algorithms by MATLAB and the obtained parameter estimation results is quite satisfactory.In this paper, a comprehensive study on the EM algorithm, the EM al-gorithm can be seen that there are obvious advantages in dealing with the problem of missing data. Of course, as described in Chapter4, EM algorithm also has many shortcomings (such as slow convergence; difficulty calculation of E step, M-step), etc., but I believe that with more scholars to conduct in-depth research EM algorithm, EM algorithm will have a greater promotion and improvement, these problems will be gradually solved.