Nigh Dimensional Feature Screening And Model Selection In Time Series

With the development of scientific techniques, ultra-high dimensional data sets have appeared in diverse areas of the sciences, engineering and humanities. However all the existing variable selection methods (e.g., LASSO, LARS, SCAD) may not perform well when the dimension of predictor variables p is much larger than sample size n. An alternative approach that has been advocated in the literature is to first perform variable screening to reduce the dimensionality p to some moderate scale, and then apply variable selection methods in the second stage. Since the seminal work of Fan and Lv (2008) on sure independence screening, there has been a recent surge of interest on (ultra)-high dimensional variable screening(or feature screening).In our chapter2, we propose a new metric, the so-called martingale difference correlation, to measure the departure of conditional mean independence between a scalar response variable Y and a vector predictor variable X. Our metric is a natu-ral extension of distance correlation proposed by Szekely, Rizzo and Bahirov (2007, Annals of Statistics), which is used to measure the dependence between Y and X. The martingale difference correlation and its empirical counterpart inherit a number of desirable features of distance correlation and sample distance correlation, such as algebraic simplicity and elegant theoretical properties. We further use martingale dif-ference correlation as a marginal utility to do high dimensional variable screening to screen out variables that do not contribute to conditional mean of the response given the covariates. Further extension to conditional quantile screening is also described in detail, i.e., screening out variables that do not contribute to conditional quantiles of the response given the covariates, which is useful for analyzing high dimensional heterogeneous data. The sure screening properties of conditional moan screening and conditional quantile screening are rigorously justified. Both simulation results and real data illustrations demonstrate the effectiveness of martingale difference correla-tion based screening procedures in comparison with the correlation ba.sed and distance correlation based counterparts.Jiang and Liu (2004) consider model selection based on estimators that are asymp-totically normal. They construct a cost function for the models in consideration, and show that the minimizer of the cost function is a consistent estimator of the under-lying true model. Despite the absence of a likelihood function, the cost function is shown to be related to an approximate posterior probability conditional on the pa-rameter estimates, which enables a Bayesian-type evaluation of all candidate models and not just to present one best choice. In addition, the method is associated with a Bayesian-type interpretation similar to BIC. Inspired by this idea, we try to solve; two time series related problems under model selection framework in chapter3and4. As for weakly dependent data, many estimators of parameters of interest are following asymptotic normal distribution due to the central limit theory or functional central limit theory. That’s why the model selection method based on asymptotic distribution is really appealing under time series context.Empirical analyses in economics often face the difficulty that the data is correlated and heterogeneous in some unknown fashion. Many estimators of parameters of interest remain valid and interesting even under the presence of correlation and heterogeneity, but it becomes considerably more challenging to correctly estimate their sampling variability. The typical approach is to invoke a law of large numbers to justify inference based on consistent variance estimators. But with the existence of heteroscedasticity and autocorrelation of unknown form, estimating asymptotic variance consistently can be pretty complicated due to the bandwidth selection and the nuisance parameter estimation. An alternative method is to construct robust test statistics based on self-normalization, which lead to an asymptotic pivotal statistics. However, as far as we’re aware, the previous literatures have all focused on classical hypothesis tests, and none of them have considered the more general problem of a Bayesian flavored approach to model selection and testing, where possibly more than two models are of interest. Thus, we develop a model selection method based on estimators obtained from subsamples in chapter3. The use of subsample estimators helps to bypass the consistent estimation of the asymptotic variances of the parameter estimates. There are many ways of forming a composite dataset based on these subsample estimates, and a unified way exists to construct an approximate likelihood from the asymptotic distribution of these composite data, based on which we can perform Bayesian model selection. Such a method can be applied to many situations when a full specification of the probability model is not required. We show that there exist simple formulas for the Bayes factor and Schwarz’s Bayesian Information Criterion (BIC) based on our construct, and that they select the true model consistently in the frequentist sense. A simulation study is performed to confirm the consistent property for model selection.Testing the martingale difference sequences (MDS) is an interesting problem that has received a lot of attentions in economics. In the last2or3decades, people tried various ways to test MDS, using spectral density, neural network and variance ratio, to name a few. In our chapter4, we proposed an Bayesian-type martingale difference test with the model selection technique. The development of this methodology is based on the following bold speculation that the sample covariance of the sequence’s fourier transform is asymptotically normal. Our method is constructed by evaluating spectral density function f(λ,u) on[N/2] frequencies and using Bayesian model selection to decide whether these [N/2] estimators f(λl,u) from a straight line or not, where N is the sample size, we consider the non-MDS model as a full model which contains the MDS model as a submodel. Following a similar manner suggested in Jiang and Liu (2004), the possibility of each candidate model is evaluated based on partial posterior probability. The expression of bayesian factor between two models has been shown, as well as its consistency property. The simulation study is carried out with BIC constructed from asymptotic normal distribution. Some pretty encouraging results have been presented among several MDC and non-MDC data generating processes. But this part of work is still in an unfinished status, improvement will be expected with a better prior and the posterior sampling technique applied.