The term multicollinearity is due to Ragnar Frisch.3 Originally it meant the existence of a "perfect," or exact, linear relationship among some or all explanatory variables of a regression model.4 For the k-variable regression involving explanatory variable Xi, X2,..., Xk (where Xi = 1 for all observations to allow for the intercept term), an exact linear relationship is said to exist if the following condition is satisfied:
where X1, X2,..., Xk are constants such that not all of them are zero simultaneously.5
Today, however, the term multicollinearity is used in a broader sense to include the case of perfect multicollinearity, as shown by (10.1.1), as well as the case where the X variables are intercorrelated but not perfectly so, as follows6:
A.1 X1 + X2 X2 + ••• + X2 Xk + vi = 0 (10.1.2)
where vi is a stochastic error term.
To see the difference between perfect and less than perfect multicollinear-ity, assume, for example, that X2 = 0. Then, (10.1.1) can be written as
A2 A2 A2
2See his A Course in Econometrics, Harvard University Press, Cambridge, Mass., 1991, p. 249.
3Ragnar Frisch, Statistical Confluence Analysis by Means of Complete Regression Systems, Institute of Economics, Oslo University, publ. no. 5, 1934.
4Strictly speaking, multicollinearity refers to the existence of more than one exact linear relationship, and collinearity refers to the existence of a single linear relationship. But this distinction is rarely maintained in practice, and multicollinearity refers to both cases.
5The chances of one's obtaining a sample of values where the regressors are related in this fashion are indeed very small in practice except by design when, for example, the number of observations is smaller than the number of regressors or if one falls into the "dummy variable trap" as discussed in Chap. 9. See exercise 10.2.
6If there are only two explanatory variables, intercorrelation can be measured by the zero-order or simple correlation coefficient. But if there are more than two X variables, intercorre-lation can be measured by the partial correlation coefficients or by the multiple correlation coefficient R of one X variable with all other X variables taken together.
CHAPTER TEN: MULTICOLLINEARITY 343
which shows how X2 is exactly linearly related to other variables or how it can be derived from a linear combination of other X variables. In this situation, the coefficient of correlation between the variable X2 and the linear combination on the right side of (10.1.3) is bound to be unity. Similarly, if k2 = 0, Eq. (10.1.2) can be written as
A2 A2 A2 A2
which shows that X2 is not an exact linear combination of other X's because it is also determined by the stochastic error term Vj.
As a numerical example, consider the following hypothetical data:
X2 X3 X3
10 50 52
15 75 75
18 90 97
24 120 129
It is apparent that X3i = 5X2i. Therefore, there is perfect collinearity between X2 and X3 since the coefficient of correlation r23 is unity. The variable X* was created from X3 by simply adding to it the following numbers, which were taken from a table of random numbers: 2, 0, 7, 9, 2. Now there is no longer perfect collinearity between X2 and X*. However, the two variables are highly correlated because calculations will show that the coefficient of correlation between them is 0.9959.
The preceding algebraic approach to multicollinearity can be portrayed succinctly by the Ballentine (recall Figure 3.9, reproduced in Figure 10.1). In this figure the circles Y, X2, and X3 represent, respectively, the variations in Y (the dependent variable) and X2 and X3 (the explanatory variables). The degree of collinearity can be measured by the extent of the overlap (shaded area) of the X2 and X3 circles. In Figure 10.1a there is no overlap betweenX2 and X3, and hence no collinearity. In Figure 10.1& through 10.1e there is a "low" to "high" degree of collinearity—the greater the overlap between X2 and X3 (i.e., the larger the shaded area), the higher the degree of collinearity. In the extreme, if X2 and X3 were to overlap completely (or if X2 were completely inside X3, or vice versa), collinearity would be perfect.
In passing, note that multicollinearity, as we have defined it, refers only to linear relationships among the X variables. It does not rule out nonlinear relationships among them. For example, consider the following regression model:
where, say, Y = total cost of production and X = output. The variables X2 (output squared) and Xi3 (output cubed) are obviously functionally related
344 PART TWO: RELAXING THE ASSUMPTIONS OF THE CLASSICAL MODEL
(a) No coUinearity (b) Low coUinearity
FIGURE 10.1 The Ballentine view of multicollinearity.
(a) No coUinearity (b) Low coUinearity
(c) Moderate collinearity (d ) High collinearity (e) Very high collinearity
FIGURE 10.1 The Ballentine view of multicollinearity.
to Xi, but the relationship is nonlinear. Strictly, therefore, models such as (10.1.5) do not violate the assumption of no multicollinearity. However, in concrete applications, the conventionally measured correlation coefficient will show Xi, X2, and X3 to be highly correlated, which, as we shall show, will make it difficult to estimate the parameters of (10.1.5) with greater precision (i.e., with smaller standard errors).
Why does the classical linear regression model assume that there is no multicollinearity among the X's? The reasoning is this: If multicollinearity is perfect in the sense of (10.1.1), the regression coefficients of the X variables are indeterminate and their standard errors are infinite. If multicollinearity is less than perfect, as in (10.1.2), the regression coefficients, although determinate, possess large standard errors (in relation to the coefficients themselves), which means the coefficients cannot be estimated with great precision or accuracy. The proofs of these statements are given in the following sections.
CHAPTER TEN: MULTICOLLINEARITY 345
There are several sources of multicollinearity. As Montgomery and Peck note, multicollinearity may be due to the following factors7:
1. The data collection method employed, for example, sampling over a limited range of the values taken by the regressors in the population.
2. Constraints on the model or in the population being sampled. For example, in the regression of electricity consumption on income (X2) and house size (X3) there is a physical constraint in the population in that families with higher incomes generally have larger homes than families with lower incomes.
3. Model specification, for example, adding polynomial terms to a regression model, especially when the range of the X variable is small.
4. An overdetermined model. This happens when the model has more explanatory variables than the number of observations. This could happen in medical research where there may be a small number of patients about whom information is collected on a large number of variables.
An additional reason for multicollinearity, especially in time series data, may be that the regressors included in the model share a common trend, that is, they all increase or decrease over time. Thus, in the regression of consumption expenditure on income, wealth, and population, the regres-sors income, wealth, and population may all be growing over time at more or less the same rate, leading to collinearity among these variables.
Was this article helpful?
Learning About The Rules Of The Rich And Wealthy Can Have Amazing Benefits For Your Life And Success. Discover the hidden rules and beat the rich at their own game. The general population has a love / hate kinship with riches. They resent those who have it, but spend their total lives attempting to get it for themselves. The reason an immense majority of individuals never accumulate a substantial savings is because they don't comprehend the nature of money or how it works.