Font Size: a A A

RIDGE REGRESSION: ESTIMATION AND PREDICTIO

Posted on:1982-08-28Degree:Ph.DType:Dissertation
University:University of MinnesotaCandidate:MCGUE, MATTHEW KERRYFull Text:PDF
GTID:1470390017465316Subject:Statistics
Abstract/Summary:
It is well known that the least squares estimates of the regression coefficients in a multiple regression study are highly unstable when the data are multicollinear. Ridge regression is an alternative to least squares developed by applied statisticians in an attempt to define more stable estimates of the parameters. Previous research has shown that ridge regression can produce estimates which are more efficient (i.e., have smaller mean squared error of estimation) than the least squares estimates. The present research investigated whether use of the ridge regression estimates would lead to more accurate prediction (i.e., smaller mean squared error of prediction) than use of the least squares estimates.;The performance of least squares was compared to the performance of five ridge regression variants using a seven factor simulation experiment. Independent variables in the experiment included the sample size, number of independent variables, level of multicollinearity, squared multiple correlation, validity concentration, and two additional factors which served to define the design points at which a prediction was to be made. Performance was measured in terms of estimates of the mean squared error of estimation, the mean squared error of prediction, and the squared validity coefficient for each of the procedures.;Consistent with previous research, ridge regression was found to produce more efficient estimates of the regression coefficients than least squares. This superiority was especially marked when the sample size was small, the number of independent variables large, the level of multicollinearity high, the squared multiple correlation low, and the validity concentrated along the first principal component of the independent variables.;In terms of the mean squared error of prediction, those ridge regression variants studied consistently improved upon least squares. The magnitude of this improvement was smaller, in absolute terms, than the magnitude of improvement observed in the mean squared error of estimation measure.;Least squares performance on the squared validity coefficient measure was little affected by manipulation of the independent variables studied. Consequently, ridge regression could not greatly improve upon least squares on this measure. The results showed a consistent, although slight, improvement over least squares possible with ridge on the squared validity coefficient measure.;It was concluded that ridge regression could greatly improve upon least squares when the purpose of the multiple regression study is to obtain efficient point estimates of the regression coefficients. If the purpose of the study is to define a function of the independent variables which could be used to generate predictions in future samples, then ridge regression would lead to a consistent improvement over least squares in those situations studied. If the purpose of the study is to test hypotheses on the parameters or define interval estimates of the parameters, then present procedures are to be preferred.
Keywords/Search Tags:Regression, Least squares, Estimates, Mean squared error, Independent variables, Estimation, Define, Multiple
Related items