In this lecture we'll talk
about Yule-Walker equations. Our objective is to introduce Yule-Walker
equations and obtain the correlation function of autoregressive processes
using Yule-Walker equations. So here is the procedure for finding out
the correlation function of an AR process. We're going to first assume,
a priori assumption is that we're going to assume that the process,
AR process, is actually stationary. Then we're going to take the product
of the model with Xn minus k, or Xt minus k and
take expectation of both sides. And then those [INAUDIBLE] expectations
will give us an equation for gamma k. But if you divide it by gamma 0, you're going to get a difference
equation for rho k. The rho being
the autocorrelation function. And then we obtain these difference
equations, which we call Yule-Walker equations, and we're going
to solve those difference equations. Let's look at an example. Let's say we have an AR(2)
process as following. And if I look at its polynomial operator,
phi B, it is 1 minus 1 over 3 B
minus 1 over 2 B squared. And if I look at the solutions of this
polynomial, as B is a complex number, we obtain real roots here, both of
which has magnitude greater than 1. So both roots are actually
outside of the unit circle in R2. So this AR(2) process is exactly
stationary, so what are we going to do? You're going to look at
the expectation of it. If I take the expectation of Xt, then expectation of Xt is one-third
of expectation of X t minus 1, expectation of X t minus 2,
expectation of Z0, this is a random noise. We obtain that mu,
the expectation is actually 0. Now we multiply both
sides of the equations, the model star, with X t minus k. So Xt is multiplied by Xt minus k,
and then you take the expectation. Every term is multiplied by Xt minus k,
and we take the expectation. This left-hand side is going to
be basically gamma k, all right? This is the autocovariance function gamma
k, this is gamma k-1, and so forth. So assume that, this is mu 0 and assume
the expectation Zt and Xt minus k is 0. There is no correlation between the Zt and
Xt minus k in the previous steps. Then gamma minus k, this is minus k,
one-third gamma negative k plus 1, 1 over 2 gamma negative k plus
2 has to equal to each other. Since gamma k is even function for
any k, we can rewrite this difference equation as gamma k,
gamma k minus 1, and gamma k minus 2. We divide by gamma 0,
which is sigma squared. And we obtain the set of equations,
difference equations for rho k, which is called Yule-Walker equations. Now we know how to, we're going to
review the difference equation. We know how to solve these
Yule-Walker equations. We're going to look at the solution,
the format of lambda k. We'll write the characteristic equation. Lambda squared minus one-third
lambda minus 1 over 2 equal to 0 is the characteristic equation of
the Yule-Walker equations in this case. The roots are the following. And if I put them back into the lambda, rho k is the linear combination of
lambda 1 to the k, lambda 2 to the k. The thing is, we have to find c1 and
c2, using some constraints, right. The first constraint is at rho 0,
the first, the autocorrelation with itself is always 1, which would tell me
that c1 plus c2 is actually 1. But also we know that for
k equal to p minus 1, p is 2 in this case. k equal to 1, rho 1 has to
equal to rho negative 1, right. This is also true when rho
2 equal to rho negative, rho 3 equal to rho negative 3./ But we're
only going to use it in this case for rho 1 equal to rho negative 1, right? Let's put k equal to 1. So rho 1 is one-third rho 0
plus 1 over 2 rho negative 1. This is the Yule-Walker
equation at k equal to 1. But since rho negative 1 and rho 1
are equal to each other and rho 0 is 1, we obtain that rho 1 is actually 2 over 3. And this is going to give
us another constraint. So if I put 1 into the k, in my solution, c1 times lambda 1 plus c2 times
lambda 2 has to equal to 2 over 3. In order words, we have this system
of equations c1 plus c2 equal to 1, some other constraint,
which will tell me that c1 and c2 are specific numbers,
these numbers here. Okay, then for any k, we found rho k, autocorrelation function of AR(2) process. And it is the following expression. And for native guys, of course,
rho k has to equal to rho negative k. Now, I have done some simulations in r. So what we did here is the following. This first graph is obtained
using ACF function in R. So we simulated AR(2) model,
according to this specific AR(2) model, and we found ACF. And ACF is basically, slowly decreasing. And eventually,
you have no significant correlations. But then we also obtained rho(k), right. This green plot is basically,
literally the plot of rho(k). And I put autocorrelation because
this is rho(k) and we obtain a very, very similar picture. So what have we learned? We have learned that Yule-Walker equations
is set of difference equations that governs autocorrelation function of
underlying stationary autoregressive process. And we also learned how to find
autocorrelation function of stationary autoregressive process
using Yule-Walker equations, which is basically a difference equations.