1 00:00:00,440 --> 00:00:04,280 In this lecture we'll talk about Yule-Walker equations. 2 00:00:04,280 --> 00:00:10,076 Our objective is to introduce Yule-Walker equations and obtain the correlation 3 00:00:10,076 --> 00:00:15,200 function of autoregressive processes using Yule-Walker equations. 4 00:00:15,200 --> 00:00:20,786 So here is the procedure for finding out the correlation function of an AR process. 5 00:00:20,786 --> 00:00:25,598 We're going to first assume, a priori assumption is that we're going to 6 00:00:25,598 --> 00:00:30,550 assume that the process, AR process, is actually stationary. 7 00:00:30,550 --> 00:00:35,940 Then we're going to take the product of the model with Xn minus k, 8 00:00:35,940 --> 00:00:39,966 or Xt minus k and take expectation of both sides. 9 00:00:39,966 --> 00:00:45,182 And then those [INAUDIBLE] expectations will give us an equation for gamma k. 10 00:00:45,182 --> 00:00:47,090 But if you divide it by gamma 0, 11 00:00:47,090 --> 00:00:50,285 you're going to get a difference equation for rho k. 12 00:00:50,285 --> 00:00:52,655 The rho being the autocorrelation function. 13 00:00:52,655 --> 00:00:55,868 And then we obtain these difference equations, which we call 14 00:00:55,868 --> 00:01:00,210 Yule-Walker equations, and we're going to solve those difference equations. 15 00:01:00,210 --> 00:01:02,421 Let's look at an example. 16 00:01:02,421 --> 00:01:06,250 Let's say we have an AR(2) process as following. 17 00:01:06,250 --> 00:01:09,910 And if I look at its polynomial operator, phi B, 18 00:01:09,910 --> 00:01:14,170 it is 1 minus 1 over 3 B minus 1 over 2 B squared. 19 00:01:14,170 --> 00:01:20,283 And if I look at the solutions of this polynomial, as B is a complex number, 20 00:01:20,283 --> 00:01:26,212 we obtain real roots here, both of which has magnitude greater than 1. 21 00:01:26,212 --> 00:01:30,842 So both roots are actually outside of the unit circle in R2. 22 00:01:30,842 --> 00:01:36,096 So this AR(2) process is exactly stationary, so what are we going to do? 23 00:01:36,096 --> 00:01:39,220 You're going to look at the expectation of it. 24 00:01:39,220 --> 00:01:41,328 If I take the expectation of Xt, 25 00:01:41,328 --> 00:01:45,781 then expectation of Xt is one-third of expectation of X t minus 1, 26 00:01:45,781 --> 00:01:50,665 expectation of X t minus 2, expectation of Z0, this is a random noise. 27 00:01:50,665 --> 00:01:54,492 We obtain that mu, the expectation is actually 0. 28 00:01:54,492 --> 00:01:58,480 Now we multiply both sides of the equations, 29 00:01:58,480 --> 00:02:02,090 the model star, with X t minus k. 30 00:02:02,090 --> 00:02:06,810 So Xt is multiplied by Xt minus k, and then you take the expectation. 31 00:02:06,810 --> 00:02:11,590 Every term is multiplied by Xt minus k, and we take the expectation. 32 00:02:11,590 --> 00:02:14,359 This left-hand side is going to be basically gamma k, all right? 33 00:02:14,359 --> 00:02:19,227 This is the autocovariance function gamma k, this is gamma k-1, and so forth. 34 00:02:19,227 --> 00:02:26,040 So assume that, this is mu 0 and assume the expectation Zt and Xt minus k is 0. 35 00:02:26,040 --> 00:02:31,240 There is no correlation between the Zt and Xt minus k in the previous steps. 36 00:02:31,240 --> 00:02:36,093 Then gamma minus k, this is minus k, one-third gamma negative k plus 1, 37 00:02:36,093 --> 00:02:40,300 1 over 2 gamma negative k plus 2 has to equal to each other. 38 00:02:40,300 --> 00:02:45,010 Since gamma k is even function for any k, we can rewrite 39 00:02:45,010 --> 00:02:49,610 this difference equation as gamma k, gamma k minus 1, and gamma k minus 2. 40 00:02:49,610 --> 00:02:53,160 We divide by gamma 0, which is sigma squared. 41 00:02:53,160 --> 00:02:57,264 And we obtain the set of equations, difference equations for rho k, 42 00:02:57,264 --> 00:02:59,830 which is called Yule-Walker equations. 43 00:03:01,360 --> 00:03:04,260 Now we know how to, we're going to review the difference equation. 44 00:03:04,260 --> 00:03:06,915 We know how to solve these Yule-Walker equations. 45 00:03:06,915 --> 00:03:10,480 We're going to look at the solution, the format of lambda k. 46 00:03:10,480 --> 00:03:12,500 We'll write the characteristic equation. 47 00:03:12,500 --> 00:03:15,775 Lambda squared minus one-third lambda minus 1 over 2 equal to 0 48 00:03:15,775 --> 00:03:20,100 is the characteristic equation of the Yule-Walker equations in this case. 49 00:03:20,100 --> 00:03:21,142 The roots are the following. 50 00:03:21,142 --> 00:03:24,036 And if I put them back into the lambda, 51 00:03:24,036 --> 00:03:30,070 rho k is the linear combination of lambda 1 to the k, lambda 2 to the k. 52 00:03:30,070 --> 00:03:36,130 The thing is, we have to find c1 and c2, using some constraints, right. 53 00:03:36,130 --> 00:03:40,740 The first constraint is at rho 0, the first, the autocorrelation with itself 54 00:03:40,740 --> 00:03:44,910 is always 1, which would tell me that c1 plus c2 is actually 1. 55 00:03:44,910 --> 00:03:50,220 But also we know that for k equal to p minus 1, p is 2 in this case. 56 00:03:50,220 --> 00:03:52,851 k equal to 1, rho 1 has to equal to rho negative 1, right. 57 00:03:52,851 --> 00:03:55,840 This is also true when rho 2 equal to rho negative, 58 00:03:55,840 --> 00:04:00,357 rho 3 equal to rho negative 3./ But we're only going to use it in this case for 59 00:04:00,357 --> 00:04:02,960 rho 1 equal to rho negative 1, right? 60 00:04:02,960 --> 00:04:06,631 Let's put k equal to 1. 61 00:04:06,631 --> 00:04:09,734 So rho 1 is one-third rho 0 plus 1 over 2 rho negative 1. 62 00:04:09,734 --> 00:04:12,960 This is the Yule-Walker equation at k equal to 1. 63 00:04:12,960 --> 00:04:17,281 But since rho negative 1 and rho 1 are equal to each other and rho 0 is 1, 64 00:04:17,281 --> 00:04:19,881 we obtain that rho 1 is actually 2 over 3. 65 00:04:19,881 --> 00:04:23,839 And this is going to give us another constraint. 66 00:04:23,839 --> 00:04:27,109 So if I put 1 into the k, in my solution, 67 00:04:27,109 --> 00:04:32,710 c1 times lambda 1 plus c2 times lambda 2 has to equal to 2 over 3. 68 00:04:32,710 --> 00:04:37,559 In order words, we have this system of equations c1 plus c2 equal to 1, 69 00:04:37,559 --> 00:04:41,158 some other constraint, which will tell me that c1 and 70 00:04:41,158 --> 00:04:44,390 c2 are specific numbers, these numbers here. 71 00:04:45,750 --> 00:04:49,302 Okay, then for any k, we found rho k, 72 00:04:49,302 --> 00:04:53,966 autocorrelation function of AR(2) process. 73 00:04:53,966 --> 00:04:56,323 And it is the following expression. 74 00:04:56,323 --> 00:05:01,650 And for native guys, of course, rho k has to equal to rho negative k. 75 00:05:01,650 --> 00:05:05,740 Now, I have done some simulations in r. 76 00:05:05,740 --> 00:05:07,210 So what we did here is the following. 77 00:05:07,210 --> 00:05:13,155 This first graph is obtained using ACF function in R. 78 00:05:13,155 --> 00:05:18,145 So we simulated AR(2) model, according to this 79 00:05:18,145 --> 00:05:22,850 specific AR(2) model, and we found ACF. 80 00:05:22,850 --> 00:05:25,280 And ACF is basically, slowly decreasing. 81 00:05:25,280 --> 00:05:28,820 And eventually, you have no significant correlations. 82 00:05:30,090 --> 00:05:32,840 But then we also obtained rho(k), right. 83 00:05:32,840 --> 00:05:36,570 This green plot is basically, literally the plot of rho(k). 84 00:05:36,570 --> 00:05:40,759 And I put autocorrelation because this is rho(k) and we obtain a very, 85 00:05:40,759 --> 00:05:42,140 very similar picture. 86 00:05:43,370 --> 00:05:44,150 So what have we learned? 87 00:05:44,150 --> 00:05:49,028 We have learned that Yule-Walker equations is set of difference equations that 88 00:05:49,028 --> 00:05:53,765 governs autocorrelation function of underlying stationary autoregressive 89 00:05:53,765 --> 00:05:54,419 process. 90 00:05:54,419 --> 00:05:58,754 And we also learned how to find autocorrelation function of 91 00:05:58,754 --> 00:06:03,871 stationary autoregressive process using Yule-Walker equations, 92 00:06:03,871 --> 00:06:07,177 which is basically a difference equations.