The Restricted EM Algorithm Under Linear Inequality Restrictions

Incomplete data and order restriction arc two very important fields in statistics, which have been widely applied in cngincering, biology, medicine, cconomics or social scienccs, etc. They oftcn occur together in many situations. Therefore, research on the two problcms is very important in practice.EM algorithm is a powerful method for maximum likelihood estimation in incomplete data problems. The basic idea of the EM algorithm is in "gene-counting method" used by geneticists in the estimation of ABO blood group gene frequencies and other genetic problems. The name EM algorithm is given by Dempster, Laird and Rubin in a celerated paper read before the Royal Statistical Society in 1976 and published in its journal in 1977. It contains two steps: E-step and M-step, that is, Expectation Step and Maximization Step.A number of modifications and extensions of EM algorithm have been developed from 1977 (Mclachlan, 1997). Due to difficulties in E-step in somesituations, Walkcr(1996) suggests to use Monte Carlo in E step, for example, MCMC or importance sampling. Due to difficulties in M step in some situations, Meng and Rubin(1993) propose ECM algorithm that replaces the M-step by a number of conditional maximization (CM) steps. Liu and Rubin (1994) propose ECME algorithm that replaces some CM-steps by the actual incomplete log-likelihood rather than a current approximation to it as given by the Q-function with the EM and ECM algorithms. A further development to be considered is the AECM algorithm of Meng and van Dyk (1995), which is obtained by combining the ECME algorithm with the Space-Alternating Generalized EM (SAGE) algorithm of Fessler and Hero (1994).The estimation under order restrictions has been developed rapidly since the 1950s. Now many people are still devoted to the research. There are many kinds of order restrictions and corresponding statistical methods, please see the two books of Barlow, Bartholomew, Brenner and Brunk (1972) and Robertson, Wright and Dykstra (1988) A special issue of Journal of Statistical Planning and Inference (2002) discusses the topic. Gao and Shi (2003) and Shi and Zheng (2004) discuss the maximum likelihood estimation under order restrictions in contingency tables. But these order restrictions have a simple expression where A0 is a given matrix and is a restricted vector. An cxtenstion is Aa where a may be a nonzero vector. Call Aa the linear inequality restrictions and call a linear equality restrictions.Kim and Taylor (1995) discuss the estiamtion under linear equality restric-tions in Journal of the American Statistical Association. Razzaghi and Kodell (2000) discuss the estimation under nonlinear equality restrictions and solve the problem of risk assessment of quantitative responses of a new drug by the method. But the research of estimation under restrictions a has extensive practical backgrounds. But the method of Kim and Taylor(1995) is not suitable for the general problems. Modifications of M-step of Meng and Rubin (1993) and Liu and Rubin (1994) are also not suitable for the problems.Based on these reasons, the paper proposes the restricted EM algorithm under linear inequality restrictions and discusses linear models, mixed linear models and generalized linear models. The paper has five chapters. Chaper 1 reviews some necessary or related knowledge; Chapter 2 introduces linear models. Let linear models bewhere Y is partly missing and is restricted by a. This chapter discusses maximum likelihood estimation and a related testing problem under linear inequality restrictions when E is known. The chapter proposes the restricted EM algorithm under linear inequality restrictions and proves the convergence of the algorithm. Chapter 3 considers maximum likelihood estimation under linear inequality restrictions in a linear model (1) with missing data and nuisance parameters E. Chapter 4 proposes the restricted EM algorithm under linear inequality restrictions in mixed linear model. Chapter5 proposes the restricted EM algorithm under linear inequality restrict...