Dimension Reduction of the Explanatory Variables in Multiple Linear Regression

Abstract

In classical multiple linear regression analysis problems will occur if the regressors are either multicollinear or if the number of regressors is larger than the number of observations. In this note a new method is introduced which constructs orthogonal predictor variables in a way to have a maximal correlation with the dependent variable. The predictor variables are linear combinations of the original regressors. This method allows a major reduction of the number of predictors in the model, compared to other standard methods like principal component regression. Its computation is simple and quite fast. Moreover, it can easily be robustified using a robust regression technique and a robust measure of correlation.