The method of ordinary least squares is attributed to Carl Friedrich Gauss, a German mathematician. Under certain assumptions (discussed in Section 3.2), the method of least squares has some very attractive statistical properties that have made it one of the most powerful and popular methods of regression analysis. To understand this method, we first explain the least-squares principle.
Recall the two-variable PRF:
However, as we noted in Chapter 2, the PRF is not directly observable. We
CHAPTER THREE: TWO-VARIABLE REGRESSION MODEL 59
estimate it from the SRF:
where Yi is the estimated (conditional mean) value of Yi.
But how is the SRF itself determined? To see this, let us proceed as follows. First, express (2.6.3) as
which shows that the Ui (the residuals) are simply the differences between the actual and estimated Y values.
Now given n pairs of observations on Y and X, we would like to determine the SRF in such a manner that it is as close as possible to the actual Y. To this end, we may adopt the following criterion: Choose the SRF in such a way that the sum of the residuals J2 Ui = J](Yi — Yi) is as small as possible. Although intuitively appealing, this is not a very good criterion, as can be seen in the hypothetical scattergram shown in Figure 3.1.
If we adopt the criterion of minimizing YI Ui, Figure 3.1 shows that the residuals U2 and U3 as well as the residuals U1 and U4 receive the same weight in the sum (U1 + U2 + U3 + U4), although the first two residuals are much closer to the SRF than the latter two. In other words, all the residuals receive
Was this article helpful?
Learning About The Rules Of The Rich And Wealthy Can Have Amazing Benefits For Your Life And Success. Discover the hidden rules and beat the rich at their own game. The general population has a love / hate kinship with riches. They resent those who have it, but spend their total lives attempting to get it for themselves. The reason an immense majority of individuals never accumulate a substantial savings is because they don't comprehend the nature of money or how it works.