| Regression analysis is an important tool for the study of natural science, engineering, and social and economic development. Regression model which study the relationship between several random variable is an important statistical method. In general, the regression models include parametric regression model, nonparametric regression model and semiparametric regression model, etc.Nonparametric regression methods have become very popular in the last decades be-cause of the fact that employing a mis-specified parametric method will typically result in inconsistent estimates and as a consequence invalid statistical inference. In recent years many authors have developed nonparametric regression estimates, which provide an at-tractive supplement to parametric regression estimates. Though nonparametric model has great adaptability, and it has greater advantages in terms of reduce the modeling error. However, the price of this flexibility can be high for several reasons. First, esti-mation precision decreases rapidly as the dimension of the predictors increases, i.e., the curse of dimensionality, which is an unavoidable problem in nonparametric estimation. Second, it is difficult to integrate discrete predictors into the nonparametric specification. And third, it is a sophisticated task to graph and interpret the resulting function in the multidimensional case. In order to overcome these shortcomings partially, it is impor-tant to know the structure information behind the data. However, how to sufficiently use the structure information behind the data is still a challenging issue. Chapter2a local linear-additive estimation and its relevant version are proposed to automatically capture the additive information for general multiple nonparametric regressions, specifically, we consider the following nonparametric regression: where Y is the real-valued response variable, X=(X(1),…,X(p))’is the p-dimensional covariate and ε is the error with conditional mean zero and variance σ2(X) given X. In the full model,we only assume that the unknown regression function is smooth.It is known that local linear estimation for nonparametric regression function r(x)in full model(??)is defined asru(x)=β(0)(x)withβ(0)(x)being the first component of the following solution: whereβ=(β(0),β(1),…β(p))’,and(Xi,Yi),i=1,…,n,are i.i.d.observations from the model(??).The local additive estimator can be obtained by minimizing with respect to β(0)(u)and β(j)(u(j)),j=1,…,p,respectively,where nx is the number of data Ui in [-1,1]p and Y(Ui)are the corresponding responses. We combine the local linear estimator and the local additive estimator to define the local linear-additive estimator by minimizing with respect to β(0) and β(j),=1,…,p.In(0.4),hj and,hj(u)may be different； the former is global bandwidth used for local linear estimator and the latter is local bandwidth used for local additive estimator. Note that here β(j),j=1,…,p, are functions of the vector u,thus they are different from those used in(??),in which each β(j)x(u(j)) depends only on the corresponding component u(j). In(0.3),λs≥0and η(x)≥0are the global and the local penalty parameters,respectively.We use them to penalize the global and the local nonadditivity of the local linear estimator.Solving the above optimalization problem yields the local linear-additive estimator where β(x) is the local linear estimator of β, γad,x(u) is the local additive estimator, W1(x;λ,η)) and W2(x;λ,η) are corresponding weights respectively. Such an estimator can be expressed as a weighted sum of the local linear estimator and the local additive estimator. Thus, our method connects two types of local estimators, the local linear (or the local constant) estimator and the local additive estimator. Thus the new esti-mators can be achieve an adaptive fitting between the full model and global additivity. On the other hand, like the local linear estimator, the new estimators can obtain the optimators have closed representations and thus make the computation easy and accu-rate. The theoretical results and simulation studies show that the new approach has a low computational complexity and can significantly improve the estimation accuracy. Also a new theoretical framework is introduced as a foundation of locally and globally connected statistical inference. Based on this framework, the newly defined estimator can be regarded as a projection of the response variable onto full function space with respect to the locally and globally connected norms.Semiparameteric regression models have been well researched and popularly used for their flexibility and interpretability. Among semiparametric models, single-index varying-coefficient model has the features of both the single-index model and the varying-coefficient model. Thus it effectively avoids the "curse of dimensionality" of nonpara-metric model and has the explanatory power of the linear regression model. So much work has been done on parameter estimation and hypothesis, but mostly in the mean regression setting. To our knowledge, there still lacks of study in the quantile regression setting. Single-index varying-coefficient models in quantile regression are konwn to pro-vide a more complete description of the response distribution than in mean regression. In Chap3, we consider the single-index varying-coefficient model in quantile regression where (X, Z)∈Rp Rq are covariates, Y is the response variable, g(·) is an q-dimensional vector of unknown functions, β=(β1,…,βp)T is a p-dimensional unknown parameter vector, and the model error ε satisfy P(ε≤0)=T for some known constant T∈(0,1). Under this model, gT(βTX)Z is the conditional T th-quantile of Y given X and Z. We impose no conditions on the heaviness of the tail probability or homoscedasticity of ε. For the sake of identifiability, we assume that||β||=1, the first component of β is positive, and g(x) cannot be the form as g(x)=αTxβTx+-γTx+c, where||·||denotes the Euclidean metric, α,γ∈Rp, c∈R are constants, and α and β are not parallel to each other. We develops a variable selection method for single-index varying-coefficient models in quantile regression using a shrinkage idea. The proposed procedure simultaneously selects significant covariates with functional coefficients and local significant variable with paramtric coefficients. Specifically, by a series of transformation, we get the objective function where Wi(φ)=Wi(β). Let φ and γ=(γT1,…,γTq)T be the solution by minimizing (0.7). Then, the penalized robust regression estimator of β based on the check loss is and the estimator of gk(u) can be obtained by Under defined regularity conditions, with appropriate selection of tuning parameters, the consistency of the variable selection procedure and the oracle property of the estimators are established. Moreover, due to the robustness of the check loss function to outliers in the finite samples, our proposed variable selection method is more robust than the ones based on the least squares criterion. The proposed method can naturally be applied to deal with pure single-index model and varying-coefficient model. Finally, the mothod is illustrated with numerical simulations.With the development of scientific techniques, ultra-high dimensional data sets have appeared in diverse areas. However all the existing variable selection methods (e.g., LAS-SO, Dangtzig,SCAD) may not perform well when the dimension of predictor variables p is much large than sample size n. An alternative approach that has been advocated in the literature is to first perform variable screening to reduced the dimensionality p to some moderate scale, and then apply variable selection methods in the second stage. Since the seminal work of Fan and Lv (2008) on sure independence screening, there has been a recent surge of interest on (ultra)-high dimensional variable screening (or feature screening). However, the most existing feature selection methods such as SIS and its relevant versions heavily depend on the specified model structure. Furthermore, feature interactions are usually not taken into account in the existing literature. Chap4presents a novel feature selection method for the model with variable interactions, without the use of structure assumption. Specifically, when a model contains the interaction terms and the interaction terms are only constituted by the first power of constitutive terms, each term can be written as the uniform form Xm11Xm22…Xmpp, where m1,…,mp∈{0,1} and1≤m1+m2+…+mp≤p.For ease of expression, we first define for m1,…mp,∈{0,1}. Then, we defineIn this paper, ωm1,…,mp is used to serve as the population quality of the marginal utility measure between Y and Xm11Xm22,…,Xmpp Consequently, we can employ the sample estimate of ωm1,…,mp to rank all the same-order interaction terms. Thus, the new ranking criterion is flexible and can deal with the models that contain interactions. Moreover, the new screening procedures are not complex,consequently,they are computationally efficient and the theoretical properties such as the ranking consistency and sure screening properties can be easily obtained. Several real and simulation explamles are presented to illustrate the methodology.Testing for heteroscedasticity has long been a standard practice in regression anal-ysis. It is well known that the statistical methods designed for homoscedastic models may result in substantial loss of efficiency when the errors are actually heteroscedastic (Dette and Munk1998). Thus, it is important to check whether the model under study has heteroscedasticity before further statistical inferences. It is known that when there are multiple heteroscedasticity, classical heteroscedasticity diagnosis may be imprecise and computationally intensive. In Chap5, we study the multiple heteroscedasticity test from a new standpoint via penalized likelihood approaches. We consider the following linear regression modelwhere Y is an n x1vector of observable responses, X∈Rn×p is a fixed design matrix of predictors, β∈Rp is a fixed (unknown) coefficient vector and ε∈Rn is a random error vector. For convenience, we assume that the matrix X has rank p, and ε follows a multivariate normal distribution with mean0and covariance matrix E, where E is a diagonal matrix with diagonal entries σ21,σ22,…,σ2n. For ease of presentation, we denote σ2=(σ21,σ22,…,σ2n)T.The i-th case of model (0.11) can be written as We can set Xi1=1to allow the model to include an intercept function. Here we want to look for a feasible method to detect the underlying heteroscedasticity in linear model (0.12). For ease of expression, we denote σ=(σ1,σ2,…,σn)T. Without loss of generality, we assume that most of the components of a are1and only a few components are not1. It implies that vector σ-11is sparse, because heteroscedasticity should not be the norm, where1is a n-dimensional vector with all elements1. If σi=1, then the ith case is regular; otherwise, it is an heteroscedasticity. We aim to find a robust estimate of σ and thereby to identify which cases are heteroscedastic and which are not. The sparsity of σ-1motivates us to use a penalized likelihood method to minimize the following objective function: over β and σ, where Pλ(·) is a positive penalty function defined on [0,∞) and λ∈[0,∞) is the tuning parameter. To take advantage of the existing algorithm and program, we make a series of transformation for the above objective function, and then we can get the new objective functionIt is similar in form to the objective function of penalized SCAD regression. We know that σi=1(i=1,…,n) is equivalent to γi=0(i=1,…, n) by transformation γi=1-1/σi. Therefore, by the duality relation between σ and γ (γ1,…,γn)T, we can check the heteroscedasticity. Our method has the following distinguishing features:(1)By our method, multiple heteroscedasticity test can be established, with no need to construct test statistics and compute distributions of these test statistics. So our approaches avoid a lot of complicated calculation such as the calculation of maximum likelihood estimation.(2) Unlike other methods such as the likelihood ratio test and the score test, which perform heteroscedasticity test in a stepwise manner, our approach can test the heteroscedasticity of all observations simultaneously.(3) Furthermore, our new method is similar in form to the classical penalized likelihood methods such as the Lasso and the SCAD. Therefore, it can be easily implemented via existing algorithms and related softwares. |
PDF Full Text Request