Welcome back to practical
time series analysis. We're looking at stochastic processes and
their realizations time series. In trying to gain traction
on them by developing some properties that'll allow
us to get work done. One of those properties is
the concept of weak stationarity. We've seen that noise
is weakly stationary. We've seen that random walks
are not weakly stationary. And in this lecture, we try to show in a formal way that moving
average processes are weakly stationary. In this lecture, we'll look at the ACF. Here we're looking at the autocovariance
function of a moving average process and then we could get
the autocorrelation function. Start with some building blocks,
some IID random variables. We'll start with Z of t as our
building blocks, iid with mean 0, that'll be important to us and
constant variance, sigma squared. Recall that we build moving
average process of order q by starting at Z sub t and
looking back in time to Z of t- q. And, just taking a sum
with some weightings. The weightings are given by the betas. In order to develop our result, we'll
remind you of a basic idea on covariance. We saw this in elementary statistics,
by looking at summations and coming up with the so-called
computing formulas. Here, I'm thinking about random
variables and using operator notation. The covariance of two random variables is
going to look like the expected value of the product minus the product
of the expected values. So immediately, perhaps trivially,
if your random variables have zero mean, you can get their covariance
just by looking at the expected value of the product,
that'll really help us moving forward. If we're going get the covariance
of random variables Xt and Xt + k then we'll be able to use
this idea to get a very nice result. Let's not make more of
this than we need to. X of t is built on a bunch of independent
identically distributed Z sub t's. Same think with the Xt + k. K is just telling you how far away from
each other random variables X of t and X of tk are along our stochastic process. If k is big, bigger than Q,
then the random variables Xt and Xt + k are being built from different
independent underlying random variables. They're not being built by anybody in
common, any of the random variables in common and so we wouldn't expect them
to have any relationship to each other. They're each just being built from
different sets of noise variables. Working a little bit more rigorously,
the expected value of each of these is 0. So we can obtain the covariance just
by looking at the expected value of the product. There are more elegant perhaps, more sophisticated ways to do since our
mathematical preliminaries in our course are trying to be kept as sort of basic as
possible, though we expect many people on the course have very sophisticated
mathematical backgrounds. But, just to make sure we keep
everything as basic as possible, we'll look at the covariance of two
random variables along our moving average process as the expected value of the
product that you see in these brackets. Again, if the underlying
Z of t are independent, we won't get contributions to
the covariance unless X sub t and Xt + k are close enough together. If we expand the product,
this looks a little bit tedious but we can just work it term by term. If we expand the product by multiplying
out in the most simple minded way possible, we see that to get the
covariance, we need the expected value or this term in a square brackets. Now the Z sub t's are independent and
since the expected value of the product is the same as the product to the expected
values for independent random variables, when the subscripts disagree, like t and
c + k, if k is not equal to zero, when the subscripts disagree,
you don't get any contribution. This, the term,
the individual term will evaluate to zero. So what we'll look for in the sum would be terms where the
subscripts on the Zs actually do agree. That's our basic idea here in coming
up with the covariance function. For little bit of intuition,
this is how this works. When k is = to 0, we'll come down to
basically just the variance of X sub t. When k is = to 0, the only place we're
going to get contributions to our sum will be along the main diagonal of the
sort of implied, is not exactly matrix but this sort of implied diagonal here
in the matrix in our bracket. That means that when k = 0, the expected
value works out as sigma squared. And then we'll just take the sum of
the individual coefficients squared. When k is = to 1, again,
we're going to look for pairs of Zs in our brackets
where the subscripts agree. When k is = to 1, it isn't along the main
diagonal where we get agreement now, but we look up 1. So, up 1 in terms of the diagonal and perhaps you could say to the right
on each one of the terms. So, I'm seeing Z sub t,
Z sub t agreeing here. Down in this row, we will have
Zt- 1 Zt- 1 agreeing, continue all the way down until you wind up at
the very last contribution down here. When you work out what
the formula look like, you will factor out your sigma squared and come up with the sum i = 0
to q- 1 beta i beta i + 1. When k is = to 0, we had q terms and
now, we have one fewer term. Actually, we had q + 1 terms that
are main diagonal and now we had q terms. When k is = to q, they'll be fewer
terms still that are involved. Again, we're just looking to see
where the subscripts might agree. And we're only going to catch that
up in this, upper right hand corner. Every other term will evaluate as 0,
giving a sigma squared beta 0 beta q. One last slide and then we're done. When k is less than or equal to q, obviously if k is greater
than q no contribution. When k is less than or equal to q, this chart here is meant to show you how
the contributions are going to behave. I have x sub t in my head up here, I have x sub t + k and we move through and we look at the underline Zs and
the coefficients involve on those Zs. And we do this for both of these. Here's your Zt + k. Beta nought through beta q. The only place where you're going
to get a contribution is where the subscripts agree and
that's on these terms here. If you look to see how are we
going to build the total term then, we'll have beta nought and beta k. Beta 1 beta k + 1 and we'll keep on going until we get at beta q- k and beta q. When you form the sum then,
sigma squared comes out from Z sub t,
Z sub t, Z sub t -1, etc. We pulled that out. And we get a sum on our beta terms. The important things to note
in this very important formula is that there is no t dependence. It doesn't matter where along your
stochastic process you're looking. The covariance of X sub t and X sub t + k do not depend upon
location along the random, along the stochastic process but only on
the separation of the random variables. That's the very idea of weak stationarity. So, in this lecture, we've shown
that moving average processes or weakly stationary. And we've also come up with an explicit
formula for the covariance of two different random variable
along our moving average process. In the next lecture, we'll look at another
canonical stochastic process or a very, very important basic stochastic process,
the autoregressive process.