Hello everybody, in this lecture we'll talk about
Yule-Walker equations in matrix form. Our objective is to rewrite the
YULE-Walker equations we have already seen In a matrix form for
auto regressive processes of order p. So let me remind you ARP
processes are where the Xt is regressed on the t previous values
starting from t- 1 until t- p. Right, so that phi determines
the order of the process, phi 0 is a constant that might
come there in our process. And Zt is our innovations,
random disturbance, or white noise which is normal with some sigma
standard deviation. So what we would like to do is
to somehow use your equation in matrix form to estimate this
parameters but this comes a little later. For that reason,
we need your equations in matrix form. Let me just note the following, if you actually take expectation of
this model right here, this P model. If you take expectation,
assuming the model is stationary, so this is stationary RP process. Expectation of XT is mu. Expectation of XP is mu. Expectation of X minus 2 is mu. An expectation of xt minus p is mu,
expectation of the random noise is zero from this,
you can actually find mu, right? Mu can be calculated in terms of these
coefficients, but that's not what I would like to do, I would like to subtract
these two equations side by side. For example, if I subtract xt minus mu
this phi 0 will go away and then I will plot view 1 that will plot view 2 and this
is basically the expression we obtain. Realize the following if I define Xt
minus mu as a new random variable, call it Xt/t, it's basically
the previous random variable shifted. To the right or
left depending on the sign of mu but interesting thing is that expectation
of tilda t now is actually 0. In other words,
if I replaced this by x tilda t, then I get this ARP processes without
a constant, which would have 0 mean. So in other words, we're going to basically work
on AR[p] process with mu = 0. And in examples wherein mu
is not [INAUDIBLE] Xt and we will try to fit the model AR(p)
model with expectations here. Okay, so this is our model,
our model is Xt regressed on the p values, previous p lags, plus some random noise. Remember Yule-Walker equations. Yule-Walker equations questions for
this process will be 0K equal to a different equation for
0k minus 1 unto RK minus P and this is always true for
K greater or equal to 1. And then remember that rho 0
at correlation is always 1 or the itself right every random available
auto correlation with the self is one. And for the negative values we can
basically write one negative K and calculate negative K. So let's write this the correct way for different values of K from one to P,
all of them. So we're going to have P equations here,
sSo this is our Yule Walker equation. I plug k equal to 1,
this becomes row 1, this becomes row 0, this becomes row -1,
this becomes row 1- p, right? -1, -2, 1- p,
all these negative values here. If I plug 2 in,
This is rho 1 this is rho 0, then I will have all seven negative
values, rho 3 and so forth. So we basically rewrite this Yule-Walker
equation for every value of k from 1 to p. So we get p equations but then we remember
the following, rho k is rho -k, right? So rho -1 is rho 1, rho -2 is rho 2, so every rows with negative indices,
row negative 1, row negative 2. We can replace them with row 1, row 2,
and so forth, so let's do that. So if I put row negative k to row k,
for every possible value of k in that system of equations,
we will obtain the following system. In other words, row negative 1
now is gone, we have row 1 for when -2 is gone we have row 2 and
rho -1 here is going to be we have rho 1. This one was rho 1- p, now it is rho p- 1,
so we get this new system of equations. And we write it by realizing that they're all 0 the auto
correlation of lack 0 is always 1. So basically, rho 0 is 1, so
this is just phi 1, this is just phi 2, this is just phi 3. Basically the diagonal we
get rid of our rho 0s and we get these system of equations and
we can write this in a matrix form. Realized this is the r coefficient matrix,
so this is our coefficient matrix, we'll call that r coefficient matrix and this is our coefficients of the process,
phi 1, phi 2, phi p. And on the left-hand side,
we have rows from 1 to p. Okay, so in other words, I can write
this as b = R f R being A p by p matrix. Phi is p by 1 matrix, b is p by 1 matrix. So if I would like to solve this,
remember at the end of today, I would like to get this coefficient. If I would like to write this, I Trying to find inverse of R if it
exists, it turns out that it exists. R is a very nice matrix
inverse of R exists. If we multiply inverse from both sides, we get phi which is the coefficients of
my system equal to R inverse times B. Now when we are actually trying to fit ARP process to actual time series data set,
of course we do not have. We do not have autocorrelation function, we actually have sample
autocorrelation function. So how do we update the whole process? Well, every time you see roles,
we replace them with Rs and we try to find phi's, right? So instead of role rows here I have r1,
r2. I call this b hat, instead of r matrix,
we call this r hat. This is approximation and instead of phi, all of these should
have hats in here, we get phi hat. And then we estimate phi hat by finding
inverse of not the original r but r hat. Okay, so let me just mention
that these Matrices R and R Hat--both of these
are symmetric matrices. They are positive semidefinite matrices. In other words, all eigenvalues are nonnegative and
their inverses actually exist. In other words, if you look at
this equation, b equal to r or b had equal to r had, had, we can actually
solve this, there's a unique solution. So let me give you an example, so
we will estimate of the following model. Assume that we have this AR2
process with the no mean, zero mean and
we would like to find first r1 and r2. This is example autocorrelation,
we can find them using acf working in R. And then all we have to do is basically
solve the system of equations for phi 1 hat and phi 2 hat both of these will
be an estimation for phi 1 and phi 2. If we have AR(3) process,
basically if we are doing the same thing, we have three coefficients now. We're going to have three by
three systems, this is our r hat, this is phi hat, and this our r, b hat
matrix and we were trying to solve this. And this matrix has an inverse,
we'll be able to find phi 1, phi 2, phi 3 hats here, so what have we learned? We have learned the matrix form of the
Yule-Walker equations, which is b = r phi. And we have learned how to
estimate the coefficients of AR processes using
Yule-Walket equations