where VIF (a measure of collinearity) is the variance inflation factor [ = 1/(1 — r|3)] discussed in Chapter 10 and r23 is the correlation coefficient between variables X2 and X3; Eqs. (13.3.4) and (13.3.5) are familiar to us from Chapters 3 and 7.

As formulas (13.3.4) and (13.3.5) are not the same, in general, var (a 2) will be different from var (fa). But we know that var (fa) is unbiased (why?). Therefore, var (a2) is biased, thus substantiating the statement made in point 4 earlier. Since 0 < r|3 < 1, it would seem that in the present case var (a2) < var (fa). Now we face a dilemma: Although <52 is biased, its variance is smaller than the variance of the unbiased estimator fa (of course, we are ruling out the case where r23 = 0, since in practice there is some correlation between regressors). So, there is a tradeoff involved here.10

The story is not complete yet, however, for the a2 estimated from model (13.3.2) and that estimated from the true model (13.3.1) are not the same because the RSS of the two models as well as their degrees of freedom (df) are different. You may recall that we obtain an estimate of a2 as a2 = RSS/df, which depends on the number of regressors included in the model as well as the df (= n, number of parameters estimated). Now if we add variables to the model, the RSS generally decreases (recall that as more variables are added to the model, the R2 increases), but the degrees of freedom also decrease because more parameters are estimated. The net outcome depends on whether the RSS decreases sufficiently to offset the loss of degrees of freedom due to the addition of regressors. It is quite possible that if a re-gressor has a strong impact on the regressand—for example, it may reduce RSS more than the loss in degrees of freedom as a result of its addition to the model—inclusion of such variables will not only reduce the bias but will also increase precision (i.e., reduce standard errors) of the estimators.

On the other hand, if the relevant variables have only a marginal impact on the regressand, and if they are highly correlated (i.e., VIF is larger), we may reduce the bias in the coefficients of the variables already included in the model, but increase their standard errors (i.e., make them less efficient). Indeed, the tradeoff in this situation between bias and precision can be substantial. As you can see from this discussion, the tradeoff will depend on the relative importance of the various regressors.

10To bypass the tradeoff between bias and efficiency, one could choose to minimize the mean square error (MSE), since it accounts for both bias and efficiency. On MSE, see the statistical appendix, App. A. See also exercise 13.6.


To conclude this discussion, let us consider the special case where r23 = 0, that is, X2 andX3 are uncorrelated. This will result in b32 being zero (why?). Therefore, it can be seen from (13.3.3) that a2 is now unbiased.11 Also, it seems from (13.3.4) and (13.3.5) that the variances of a2 and ¡2 are the same. Is there no harm in dropping the variable X3 from the model even though it may be relevant theoretically? The answer generally is no, for in this case, as noted earlier, var(a2) estimated from (13.3.4) is still biased and therefore our hypothesis-testing procedures are likely to remain suspect.12 Besides, in most economic research X2 and X3 will be correlated, thus creating the problems discussed previously. The point is clear: Once a model is formulated on the basis of the relevant theory, one is ill-advised to drop a variable from such a model.

Inclusion of an Irrelevant Variable (Overfitting a Model)

Now let us assume that

Yi = ¡31 + ¡32 X2i + ui (13.3.6) is the truth, but we fit the following model:

and thus commit the specification error of including an unnecessary variable in the model.

The consequences of this specification error are as follows:

1. The OLS estimators of the parameters of the "incorrect" model are all unbiased and consistent, that is, E(a\) = ¡1, E(a2) = ¡2, and E(a3) = ¡3 = 0.

2. The error variance a2 is correctly estimated.

3. The usual confidence interval and hypothesis-testing procedures remain valid.

4. However, the estimated as will be generally inefficient, that is, their variances will be generally larger than those of the ¡3 s of the true model. The proofs of some of these statements can be found in Appendix 13A, Section 13A.2. The point of interest here is the relative inefficiency of the a's. This can be shown easily.

From the usual OLS formula we know that

Was this article helpful?

0 0
Rules Of The Rich And Wealthy

Rules Of The Rich And Wealthy

Learning About The Rules Of The Rich And Wealthy Can Have Amazing Benefits For Your Life And Success. Discover the hidden rules and beat the rich at their own game. The general population has a love / hate kinship with riches. They resent those who have it, but spend their total lives attempting to get it for themselves. The reason an immense majority of individuals never accumulate a substantial savings is because they don't comprehend the nature of money or how it works.

Get My Free Ebook

Post a comment