Second-order Least Squares Estimation For Linear Regression Model

Linear regression models occupy a very important position in the theory and appli-cation of mathematical statistics. More and more research areas are related to the pa-rameter estimation problem in linear regression models. Parameter estimation method has been discussed a lot by the domestic and foreign intellectuals. A few of estima-tion methods have been put forward:least squares estimation method, ridge regression estimation method, principal components estimation method and generalized shrunken least squares estimation method are commonly used. The ordinary least squares esti-mation is based on minimization of the squared distance of the response variable to its conditional mean given the predictor variable. In this paper, we extend this method by including in the criterion function the distance of the squared response variable to its second conditional moment. Due to the wide application of the linear regression model, the paper mainly studies the second-order least squares estimation of the linear model with error of asymmetrical distribution. Moreover, we derive the strong consistency and asymptotic normality for this "second-order" least squares (SLS) estimator under gener-al regularity conditions. What’s more, it is shown that this "second-order" least squares estimator is asymptotically more efficient than the ordinary least squares estimator if the third moment of the random error is nonzero, and both estimators have the same asymptotic covariance matrix if the error distribution is symmetric. More important, the result apply to the linear regression models, where the random error distributions are not necessarily known. Simulation studies show that the variance reduction of the second order estimator can be as high as 50% for sample sizes lower than 50.