Improvement Of Parameter Estimation And Outlier Detection Method In Linear Regression Model

Linear regression model as the most basic and widely used mathematical model in statistical disciplines is a powerful tool for explore the relationships between variables, and analyzing data validity. This paper mainly focus on two parts:linear regression model parameter estimation method, linear regression model outlier detection method. Based on these research methods, this paper came up with theoretical refinements with program implementation.The purpose of regression model is to get relationship between the vari-ables, i.e. parameter estimation. Classical parameter estimation idea is to minimize the square of error estimation, called Least-Square Estimation. Un-der Gauss — Markov assumptions, Least-Squares Estimation is an unbiased estimate of the minimum variance estimation methods. When there is some kind of approximate linear relationship between the independent variables, the Mean Square Error of Least-Squares Estimate is large. For the previous short-comings of traditional methods, Principal Component Analysis Method and Ridge Estimation Method were proposed.In this paper, the above methods were integrated and improved with K Integrate Principal Estimation. This method separate eigenvalue matrix A into two parts,A1 and A2, according to the values of the matrix XTX. Then, plus K= diag(kq+1,…,kp), constants kj≤0,j= q+1,...,p on A2, which contains small part of the information. Then this paper gave the condition that K Integrate Principal Estimation Method is better than Least-Squares Estimation as well as the General Ridge Method. The traditional Ridge Method is the case that adding same constant k on eigenvalues. However, different constants’condition was not considered. Thus, this article raised the Functional Ridge Estimation, adding function form matrix F(K)= diag(f1,f2(k2), …, fq(kq)), where the known function fi(ki),i=1,...,q are different functions satisfying some nonnegativity and d-ifferentiability conditions. Theoretically proved the condition that Functional Ridge Estimation Method better than Principal Components Estimation and the Least-Squares Estimation in MSE sense, as well as Least-Squares Estima- tion in gerneral MSE sense.When setting the regression model, we need to move out the large devia-tion data from original model,i.e. outlier detection. In this paper, the details of the common outlier detection methods were discussed and researched. In Data Deletion Model, deleting one or more rows of sample data may encounter design matrix X to be non-full rank. The Least-Squares Estimation is based on the assumption that design matrix is full rank to get the estimation. If it not satisfied, the matrix XTX is unable to inverse. To overcome this situa-tion, the paper considered the idea of using the Generalized Inverse Method on matrix XTX and making singular value decomposition method to deal with; In the MSOM,t statistic is used to determine the outlier. This paper used MATLAB software to generate the random variables and error of the sample data. Depending on the given model, we can calculate the dependent variable to obtain the parameters of the Least-Square Estimation. And by applying "disturbance" thinking of sample points, the simulated evaluation was used to test the sensitivity of t statistics; In the Heteroscedastic Model, this pa-per introduced a classical methods, named Lagrange Multiplier method. The assumption of homogeneous variance was used as the restriction condition of Weighted Least Squares Estimation. According to the core idea of Lagrange Multiplier method, this constrained optimization problem was turned into an unconstrained optimization problem, no longer need to construct test statistics and computing distribution function. Finally, we achieved these methods on examples.