Stochastic Processes


the first quarter of 1970 could have been any number, depending on the economic and political climate then prevailing. The figure of 2872.8 is a particular realization of all such possibilities.5 Therefore, we can say that GDP is a stochastic process and the actual values we observed for the period 1970-I to 1991-IV are a particular realization of that process (i.e., sample). The distinction between the stochastic process and its realization is akin to the distinction between population and sample in cross-sectional data. Just as we use sample data to draw inferences about a population, in time series we use the realization to draw inferences about the underlying stochastic process.

Stationary Stochastic Processes

A type of stochastic process that has received a great deal of attention and scrutiny by time series analysts is the so-called stationary stochastic process. Broadly speaking, a stochastic process is said to be stationary if its mean and variance are constant over time and the value of the covariance between the two time periods depends only on the distance or gap or lag between the two time periods and not the actual time at which the covariance is computed. In the time series literature, such a stochastic process is known as a weakly stationary, or covariance stationary, or second-order stationary, or wide sense, stochastic process. For the purpose of this chapter, and in most practical situations, this type of stationarity often suffices.6

To explain weak stationarity, let Yt be a stochastic time series with these properties:

Covariance: yk = E[(Yt — f)(Yt+k — f)] (21.3.3)

where yk, the covariance (or autocovariance) at lag k, is the covariance between the values of Yt and Yt+k, that is, between two Y values k periods apart. If k = 0, we obtain y0, which is simply the variance of Y(= a2); if k = 1, y1 is the covariance between two adjacent values of Y, the type of co-variance we encountered in Chapter 12 (recall the Markov first-order autoregressive scheme).

Suppose we shift the origin of Y from Yt to Yt+m (say, from the first quarter of 1970 to the first quarter of 1975 for our GDP data). Now if Yt is to be stationary, the mean, variance, and autocovariances of Yt+m must be the

5You can think of the value of $2872.8 billion as the mean value of all possible values of GDP for the first quarter of 1970.

6A time series is strictly stationary if all the moments of its probability distribution and not just the first two (i.e., mean and variance) are invariant over time. If, however, the stationary process is normal, the weakly stationary stochastic process is also strictly stationary, for the normal stochastic process is fully specified by its two moments, the mean and the variance.


same as those of Yt. In short, if a time series is stationary, its mean, variance, and autocovariance (at various lags) remain the same no matter at what point we measure them; that is, they are time invariant. Such a time series will tend to return to its mean (called mean reversion) and fluctuations around this mean (measured by its variance) will have a broadly constant amplitude.7

If a time series is not stationary in the sense just defined, it is called a nonstationary time series (keep in mind we are talking only about weak stationarity). In other words, a nonstationary time series will have a time-varying mean or a time-varying variance or both.

Why are stationary time series so important? Because if a time series is nonstationary, we can study its behavior only for the time period under consideration. Each set of time series data will therefore be for a particular episode. As a consequence, it is not possible to generalize it to other time periods. Therefore, for the purpose of forecasting, such (nonstationary) time series may be of little practical value.

How do we know that a particular time series is stationary? In particular, are the time series shown in Figures 21.1 and 21.2 stationary? We will take this important topic up in Sections 21.8 and 21.9, where we will consider several tests of stationarity. But if we depend on common sense, it would seem that the time series depicted in Figures 21.1 and 21.2 are nonstationary, at least in the mean values. But more on this later.

Before we move on, we mention a special type of stochastic process (or time series), namely, a purely random, or white noise, process. We call a stochastic process purely random if it has zero mean, constant variance a2, and is serially uncorrelated.8 You may recall that the error term ut, entering the classical normal linear regression model that we discussed in Part I of this book was assumed to be a white noise process, which we denoted as ut ~ iidn(0, a2); that is, ut is independently and identically distributed as a normal distribution with zero mean and constant variance.

Nonstationary Stochastic Processes

Although our interest is in stationary time series, one often encounters non-stationary time series, the classic example being the random walk model (RWM).9 It is often said that asset prices, such as stock prices or exchange rates, follow a random walk; that is, they are nonstationary. We distinguish two types of random walks: (1) random walk without drift (i.e., no constant or intercept term) and (2) random walk with drift (i.e., a constant term is present).

7This point has been made by Keith Cuthbertson, Stephen G. Hall, and Mark P. Taylor, Applied Econometric Techniques, The University of Michigan Press, 1995, p. 130.

8If it is also independent, such a process is called strictly white noise.

9The term random walk is often compared with a drunkard's walk. Leaving a bar, the drunkard moves a random distance ut at time t, and, continuing to walk indefinitely, will eventually drift farther and farther away from the bar. The same is said about stock prices. Today's stock price is equal to yesterday's stock price plus a random shock.


Random Walk without Drift. Suppose ut is a white noise error term with mean 0 and variance a 2. Then the series Yt is said to be a random walk if

In the random walk model, as (21.3.4) shows, the value of Yat time t is equal to its value at time (t — 1) plus a random shock; thus it is an AR(1) model in the language of Chapters 12 and 17. We can think of (21.3.4) as a regression of Y at time t on its value lagged one period. Believers in the efficient capital market hypothesis argue that stock prices are essentially random and therefore there is no scope for profitable speculation in the stock market: If one could predict tomorrow's price on the basis of today's price, we would all be millionaires.

Now from (21.3.4) we can write

Y2 = Y1 + U2 = Yo + U1 + U2 Y3 = Y2 + U3 = Yo + U1 + U2 + U3

In general, if the process started at some time 0 with a value of Y0, we have


In like fashion, it can be shown that var(Yt) = ta 2 (21.3.7)

As the preceding expression shows, the mean of Y is equal to its initial, or starting, value, which is constant, but as t increases, its variance increases indefinitely, thus violating a condition of stationarity. In short, the RWM without drift is a nonstationary stochastic process. In practice Yo is often set at zero, in which case E(Yt) = o.

An interesting feature of RWM is the persistence of random shocks (i.e., random errors), which is clear from (21.3.5): Yt is the sum of initial Yo plus the sum of random shocks. As a result, the impact of a particular shock does not die away. For example, if u2 = 2 rather than u2 = o, then all Yt's from Y2 onward will be 2 units higher and the effect of this shock never dies out. That is why random walk is said to have an infinite memory. As Kerry Patterson notes, random walk remembers the shock forever™; that is, it has infinite memory.

1oKerry Patterson, op cit., Chap. 6.


Interestingly, if you write (21.3.4) as

where A is the first difference operator that we discussed in Chapter 12. It is easy to show that, while Yt is nonstationary, its first difference is stationary. In other words, the first differences of a random walk time series are stationary. But we will have more to say about this later.

Random Walk with Drift. Let us modify (21.3.4) as follows:

where 8 is known as the drift parameter. The name drift comes from the fact that if we write the preceding equation as

it shows that Yt drifts upward or downward, depending on 8 being positive or negative. Note that model (21.3.9) is also an AR(1) model.

Following the procedure discussed for random walk without drift, it can be shown that for the random walk with drift model (21.3.9),

As you can see, for RWM with drift the mean as well as the variance increases over time, again violating the conditions of (weak) stationarity. In short, RWM, with or without drift, is a nonstationary stochastic process.

To give a glimpse of the random walk with and without drift, we conducted two simulations as follows:

where ut are white noise error terms such that each ut ~ N(0, 1); that is, each ut follows the standard normal distribution. From a random number generator, we obtained 500 values of u and generated Yt as shown in (21.3.13). We assumed Yo = 0. Thus, (21.3.13) is an RWM without drift. Now consider

which is RWM with drift. We assumed ut and Y0 as in (21.3.13) and assumed that 8 = 2.


The graphs of models (21.3.13) and (21.3.14), respectively, are in Figures 21.3 and 21.4. The reader can compare these two diagrams in light of our discussion of the RWM with and without drift.

The random walk model is an example of what is known in the literature as a unit root process. Since this term has gained tremendous currency in the time series literature, we next explain what a unit root process is.

FIGURE 21.3 A random walk without drift.

FIGURE 21.3 A random walk without drift.

FIGURE 21.4 A random walk with drift.

FIGURE 21.4 A random walk with drift.


Econometrics: Some Basic Concepts


Was this article helpful?

0 0
Budgeting Basics

Budgeting Basics

Get All The Support And Guidance You Need To Be A Success At Budgeting. This Book Is One Of The Most Valuable Resources In The World When It Comes To Get Your Finances In Order Once And For All.

Get My Free Ebook

Post a comment