Estimation Of Priors In Bayesian Nonparametrics

Bayesian nonparametrics is a relatively young, yet fast growing field of statistics, in which it not only produces a large number of theoretical achievements but also widely applies itself in various substantive fields and directions. However, its classical research spots focus on a pure Bayes hierarchical structure, with the priors specified presumably. This framework might be challenged once the prior could not be assigned easily, especially in view of the fact that Bayesian nonparametrics requests a rather complicated process prior. This disadvantage of traditional Bayesian nonparametric analysis stimulates us to employ a more flexible and robust framework—empirical Bayes—to proceed statistical inference and modeling. This is due to in empirical Bayes analysis, the prior parameters are estimated based on observations, rather than pre-specified. On the other hand, as is well-known, estimating parameters via observations would not make sense if without identifiability, and identifiability is one of the preconditions to prove asymptotic conver-gence properties of parameter estimates or posterior distributions. Many statisticians tried to figure out under what conditions identifiability will naturally holds and to our knowledge, there exist plenty of fruits on identifiability of finite mixtures while identifia-bility for countably infinite mixture models is still rarely studied thus an open problem. For example, as a special kind of countably infinite mixture, Ferguson (1983) pointed out the identifiability of Dirichlet process mixture model was yet unsolved and ambiguous. Motivated by these challenges in Bayesian nonparametrics, we try to estimate the prior parameters of several process priors under an empirical Bayes framework with different data structures:univariate data, multivariate data, even monotone missing data. We also try to generate some conveniently-applied sufficient conditions for identifiability of countably infinite mixtures. This dissertation is organized as the following chapters.Firstly, we provide a comprehensive review on Bayesian nonparametrics in Chapter1, including why we apply Bayesian nonparametrics, a brief history of Bayesian non-parametrics, its abundant theoretical achievements and applications. We also discuss the computational issues, future research directions and potential challenges to Bayesian nonparametrics, via recalling a series of literatures from numerous statisticians. Also, we introduce several types of data structures, accompanied with a familiar empirical Bayes assumption. We believe these data structures are so general and representative that they would recapitulate our efforts on modeling various practical datasets.In chapter2, it presents a sufficient condition for identifiability of countably infinite mixtures, which is expressed by means of well-ordering on the sets of distributions and uniform convergence of series. We think this sufficient condition is easier to be checked than the infinite-dimensional matrix conditions generated by Tallis (1969). Then this suf-ficiency is applied to revisit some examples for which the identifiability is well-established and further explore the identifiability for several novel distribution families, including normal, gamma, cauchy, noncentral χ2, generalized logistic distributions.The next chapter is concerned with estimating the prior parameters in Dirichlet pro-cess priors when the data are monotonically missing. Our goal is to estimate the unknown precision parameter a and the probability measure a of DP(α, α), with partially observed data in the empirical Bays framework. We find that, under Dirichlet process priors, data missing has no impact on the estimation of the precision parameter a, which can be effec-tively estimated by maximizing certain likelihood function. For the probability measure a, on the contrary, one has to resort to nonparametric density estimation methodologies for missing data when it assumes a has density. The strong consistency and asymp-totic normality of the estimates of a are given under very weak conditions, then the L1convergence of a’s density estimate is also proved. Besides, the optimal selection of the bandwidths via minimizing the asymptotic mean integrated squared errors involved in this density estimate is examined for2-dimensional missing data. We find that our density estimate behaves better than some existing approaches under monotone missing data.Chapter4is related to the estimation of prior parameters in Polya tree priors with univariate observations. Specifically, we try to obtain the empirical Bayes estimates of prior parameters in (?) of Polya tree prior PT(Ⅱ,(?)) with presumably specified partition II based on data. Firstly, we overview the basic model and theoretical properties of Polya tree priors, then define several kinds of Polya tree priors and give sufficiencies to take absolutely continuous distributions as their supports. Two types of estimates to prior parameters in Polya tree priors—the maximum likelihood estimate and the moment estimate—are given, where associated properties are also illustrated, including the rela-tionship between this model and Beta-binomial distribution. We also present numerical simulations on various estimates to validate their respective theoretical performances.In the last chapter, we estimate the prior parameters of multivariate Polya tree prior based on multivariate observed samples within an empirical Bayes framework. This part could be regarded as a multivariate generalization to the univariate Polya tree, and the corresponding empirical Bayes analyses indeed are similar to the ones of univariate case. At the commencement, we give the definition and prove several theoretical properties of multivariate Polya tree priors, accompanied with data structure and model assumptions. Then we provide the moment estimate and maximum likelihood estimate to the prior parameters of multivariate Polya tree priors, and discuss the relationship between this model and the Dirichlet-multinomial distribution. Numerical simulations are also given to illustrate the performance of the empirical Bayes estimates, in forms of several tables and graphs.