EM Algorithm And Its Variable Metric Acceleration

The EM algorithm is a general method of finding the maximum-likelihood estimate of the parameters of an underlying distribution from a given data set when the data is incomplete or has missing values. It has some important applications in dealing with incomplete data. Its biggest advantages are simple, numerical stability and small storage. Particularly, eachiteration can ensure that the log-likelihood function of the observation data is monotonous but not reduction. But the biggest drawback of the EM algorithm is the slow pace of convergence and sub-linear convergence rate which prevent the application of the EM algorithm.We use symmetry in 2-rank corrected formula of non-linear optimization to improve the EM algorithm and show EMD, EMB and EMDB three new accelerations. They are all to improve the M-step of EM algorithm. They share in the monotonous increase the value of the likelihood function and stability of the convergence on the basis of the EM algorithm to improve the convergence rate.Thesis studies the improved EM algorithm and its applications in the experiment.Firstly, we mainly outline the background of the EM algorithm, historical significance and current domestic and foreign scholars of research, so that readers understand the status and influence of the EM algorithm and the importance of the paper topic. We give the major theoretical basis for the followings.Secondly, we introduce the formal definition of the EM algorithm and show the derivation for the nature of the EM algorithm (convergence and monotonicity), so that the reader can understand the simplistic and universal theory of the EM algorithm. Finaly, we use the correction formula (DFP formula, BFGS formula, a joint formula which exit in variable metric method of the non-linear optimization) for the conditional expectation maximization of the M-step to improve the EM algorithm.We show EMD, EMB, EMDB three new algorithm, so as to achieve the purpose of speeding up convergence we show a comparative study for the advantages and disadvantages of the three algorithm. We give a proof for the good nature of them and numerical algorithm experiment to verify the new algorithm's effectiveness and feasibility.