Order selection in classical finite mixture models and variable selection and inference in finite mixture of regression models

The model selection problem is often a very important first step in statistical analysis. In this thesis, we have focused on model selection problems in mixture models. These models are typically used to analyze data that arise from heterogeneous populations. Mixture models are ubiquitous in virtually every facet of statistical analysis, machine learning and data mining.; In some applications, the scientific background may not be sufficient to determine the appropriate order of a finite mixture model in data analysis. That is, the number of components of the model, called the order of the mixture, is often unknown and has to be estimated from the data. The order selection problem, in finite mixture models is a fundamental and yet difficult problem that has received great attention in the past few decades.; The first problem we focus on, in this thesis, is the problem of order selection. A literature review of the major developments for the problem of order selection in finite mixture models is given. A new penalized likelihood approach is then proposed for the problem of order selection. The new method deviates from the well-known classical information-based approaches such as AIC (Akaike 1973) and BIC (Schwarz 1978) because the penalty function is directly a continuous function of the parameters of the mixture model. This makes the new method more stable than AIC and BIC. The new method is also less computationally intensive than many existing methods since the order of the model is determined through a single optimization procedure. The performance of the method has been investigated theoretically and via extensive simulations.; The second problem we focus on, in this thesis, is the problem of variable selection in FMR models, when the order of mixture model is given based on some prior information. We first review the problem of variable selection in the context of multivariate linear regression models. A short survey of the methods used to deal with this problem is given. In light of new developments in variable selection methods such as LASSO by Tibshirani (1996) and SCAD by Fan and Li (2001), a new penalized likelihood approach for variable selection in FMR models is then developed. In the new method, we particularly follow the method of Fan and Li (2001). The new method deviates from AIC and BIC by introducing a penalty which depends on the sizes of the regression coefficients, and moreover takes into account the mixture structure of the model. (Abstract shortened by UMI.)...