1 00:00:11,100 --> 00:00:16,680 So in this lecture, we are going to discuss a very important topic when it comes to Financial Times 2 00:00:16,680 --> 00:00:23,190 series, this is the random walk in the corresponding random walk hypothesis to give you a very brief 3 00:00:23,190 --> 00:00:27,660 summary, a random walk is what we implemented when we did price simulations. 4 00:00:28,140 --> 00:00:33,180 This lecture will expand on what we did by taking a more theoretical look at what we've already done 5 00:00:33,180 --> 00:00:33,990 in practice. 6 00:00:34,740 --> 00:00:39,990 The practical part is useful, but the theoretical part is critical for providing you with necessary 7 00:00:39,990 --> 00:00:40,660 insights. 8 00:00:41,190 --> 00:00:46,620 In fact, we'll learn later in this course that the random walk is a special case of a Arima, a very 9 00:00:46,620 --> 00:00:48,150 important time series model. 10 00:00:49,640 --> 00:00:51,980 So what is the random hypothesis? 11 00:00:52,910 --> 00:00:56,890 Well, put simply, it says that stock prices follow a random walk. 12 00:00:57,380 --> 00:01:01,850 Now, of course, you may not know exactly what a random walk is yet, but that's what this lecture 13 00:01:01,850 --> 00:01:05,260 is about now because of the nature of random walks. 14 00:01:05,270 --> 00:01:09,890 If stock prices do, in fact, follow a random walk, then they are unpredictable. 15 00:01:10,310 --> 00:01:12,380 The rest of this lecture will show you how. 16 00:01:17,080 --> 00:01:21,390 But first, let's discuss some of the history behind the random walk hypothesis. 17 00:01:22,120 --> 00:01:27,100 Firstly, the mathematical concept of random walks has existed for a long time. 18 00:01:27,610 --> 00:01:29,770 As you'll see, it's just a mark of process. 19 00:01:29,770 --> 00:01:32,740 And so it's something you would normally learn in probability class. 20 00:01:33,220 --> 00:01:38,410 The random walk hypothesis is specific to finance and stock prices in particular. 21 00:01:39,070 --> 00:01:44,460 It was popularized in the 70s when a book called A Random Walk Down Wall Street was released. 22 00:01:45,040 --> 00:01:49,400 In fact, this was the book that also popularized the efficient market hypothesis. 23 00:01:50,230 --> 00:01:55,870 Note that both the random walk hypothesis and the efficient market hypothesis lead to the same conclusion, 24 00:01:56,110 --> 00:01:58,070 which is that you can't beat the market. 25 00:01:58,720 --> 00:02:02,200 Now, of course, there are people who don't believe in the random hypothesis. 26 00:02:02,350 --> 00:02:06,760 And so another book has come out called A Non Random Walk down Wall Street. 27 00:02:07,540 --> 00:02:12,320 Interestingly, this book came out almost 30 years later after a random walk down Wall Street. 28 00:02:12,610 --> 00:02:18,490 So it's not as if the random hypothesis and the efficient market hypothesis are ideas which are easily 29 00:02:18,490 --> 00:02:19,090 debunked. 30 00:02:19,630 --> 00:02:25,180 In this course, we're actually going to fit models to stock prices and we'll find that sometimes the 31 00:02:25,180 --> 00:02:27,940 best fitting model is, in fact, a random walk. 32 00:02:32,740 --> 00:02:34,300 So what is a random walk? 33 00:02:34,900 --> 00:02:39,550 Well, probably the simplest random walk works like this start at any price. 34 00:02:40,070 --> 00:02:46,120 Then in order to generate the next price, simply pick either plus one or minus one with equal probability. 35 00:02:46,720 --> 00:02:54,290 So P one is equal to P0 plus E one where E one, it can be either minus one or plus one, then generate 36 00:02:54,290 --> 00:02:59,920 P two from P one in the same way by picking either plus one or minus one with equal probability and 37 00:02:59,920 --> 00:03:01,150 then adding it to P1. 38 00:03:01,840 --> 00:03:05,110 Then we find P three and then we find P four and so on. 39 00:03:05,590 --> 00:03:07,150 So this is a random walk. 40 00:03:07,870 --> 00:03:14,200 Basically you can imagine yourself walking on the sidewalk in one dimension at every step you either 41 00:03:14,200 --> 00:03:19,480 decide to take one step to the left or one step to the right based on the result of a coin flip. 42 00:03:19,990 --> 00:03:22,030 Your walk is then a random walk. 43 00:03:22,780 --> 00:03:25,300 Notice one important property of the random walk. 44 00:03:25,660 --> 00:03:27,960 It's impossible to predict the next value. 45 00:03:28,360 --> 00:03:30,820 You only have a 50 percent chance of getting it right. 46 00:03:31,630 --> 00:03:36,940 In other words, your ability to predict the result of your walk is the same as your ability to predict 47 00:03:36,940 --> 00:03:38,920 the result of a series of coin flips. 48 00:03:43,870 --> 00:03:48,570 Now, we know that changes in stock price aren't just minus one and plus one, but can be real valued. 49 00:03:48,970 --> 00:03:54,250 In fact, we spent a lot of time in the previous section of this course trying to figure out what is 50 00:03:54,250 --> 00:03:56,330 the distribution that stock returns follow. 51 00:03:57,040 --> 00:04:00,260 Let's assume for now that the noise term is Gaussian. 52 00:04:01,030 --> 00:04:03,820 What would our algorithm be for generating stock prices? 53 00:04:04,360 --> 00:04:07,750 Again, we start at p0 equal to some arbitrary value. 54 00:04:08,260 --> 00:04:15,040 To find the next price, we first sample E from our Gaussian, then we add a P zero plus one to find 55 00:04:15,040 --> 00:04:16,500 P one the next price. 56 00:04:17,020 --> 00:04:20,590 We do the same thing to generate P2 and P3 and so forth. 57 00:04:21,130 --> 00:04:26,740 This should sound familiar because it's exactly what we did in our price simulation exercise from the 58 00:04:26,740 --> 00:04:27,780 previous lecture. 59 00:04:28,480 --> 00:04:31,090 In fact that was exactly a random walk. 60 00:04:31,780 --> 00:04:38,740 Notice again how we can't predict P one from P zero or equivalently we can't predict P one minus P zero, 61 00:04:38,770 --> 00:04:41,490 which is just E one, which is Gaussian noise. 62 00:04:41,980 --> 00:04:45,550 We can only predict one insofar as we know its expected value. 63 00:04:50,520 --> 00:04:55,760 Here's something interesting we can do that helps us understand why working with log prices is valuable, 64 00:04:56,460 --> 00:05:00,690 the general formula for a random walk with a drift is as follows. 65 00:05:01,680 --> 00:05:04,660 Muse called the drifter, and it's considered to be constant. 66 00:05:05,040 --> 00:05:08,880 If you're thinking of a time series, this would control the trend of the Time series. 67 00:05:09,690 --> 00:05:13,710 E of T is a Gaussian with mean zero and some variance sigma squared. 68 00:05:14,640 --> 00:05:22,170 In this case, time T and part time T minus one are the log prices at time T and time T minus one respectively. 69 00:05:23,100 --> 00:05:30,960 Note that if I take time T minus one to the left hand side, I get a time T minus time T minus one, 70 00:05:31,260 --> 00:05:32,610 which is the log return. 71 00:05:33,890 --> 00:05:38,330 If we were working with nonlawyers returns, this wouldn't be as convenient, since we would need a 72 00:05:38,330 --> 00:05:41,910 P of T minus one in the denominator to represent the return. 73 00:05:42,800 --> 00:05:48,200 What this says is that the log return is just the thing on the right hand side, which is just the Gaussian 74 00:05:48,200 --> 00:05:50,250 with Meenu and Variance Sigma squared. 75 00:05:50,840 --> 00:05:55,320 So the random walk model goes hand in hand with log prices and log returns. 76 00:05:55,840 --> 00:06:01,610 In fact, this model is the basis for the Black-Scholes formula which earned the Nobel Prize in economics. 77 00:06:06,420 --> 00:06:10,320 Now, the big question is, of course, is the random walk hypothesis correct? 78 00:06:10,890 --> 00:06:14,800 Well, let's recognize that there are some hidden assumptions in the random walk model. 79 00:06:15,570 --> 00:06:20,460 First is that the log returns are ID independent and identically distributed. 80 00:06:21,000 --> 00:06:25,340 We have seen that this may not be true because we have observed volatility clustering. 81 00:06:26,040 --> 00:06:31,620 If the volatility changes over time, then by definition it's not identically distributed. 82 00:06:32,250 --> 00:06:38,790 Furthermore, if the volatility in one period has some relationship to nearby periods, that is high. 83 00:06:38,790 --> 00:06:43,620 Volatility is clustered with other high volatility, then it's also not independent. 84 00:06:48,480 --> 00:06:52,690 At the same time, the random walk model is convenient and easy to work with. 85 00:06:53,220 --> 00:06:59,580 We will find that when we fit Arima models to stock prices, sometimes the best fitting model will be 86 00:06:59,580 --> 00:07:00,420 a random walk. 87 00:07:00,840 --> 00:07:07,230 So it wouldn't be wrong to say that sometimes for certain periods of time, stock prices do look like 88 00:07:07,230 --> 00:07:08,580 they follow a random walk. 89 00:07:09,210 --> 00:07:15,030 As with the efficient market hypothesis, it's possible to use statistical tests to determine whether 90 00:07:15,030 --> 00:07:17,340 or not stock prices follow a random walk. 91 00:07:22,030 --> 00:07:27,610 Now, since this is, of course, on Time series, we're going to do some time series analysis on random 92 00:07:27,610 --> 00:07:28,250 walks. 93 00:07:29,200 --> 00:07:33,750 Let's recognize that a random walk is just a specific instance of a Markov chain. 94 00:07:34,300 --> 00:07:39,400 If you've ever taken any of my courses on NLP or reinforcement learning, you should be familiar with 95 00:07:39,400 --> 00:07:40,260 this concept. 96 00:07:40,870 --> 00:07:42,310 The basic idea is this. 97 00:07:42,910 --> 00:07:44,110 Consider the sentence. 98 00:07:44,110 --> 00:07:46,930 The quick brown fox jumps over the lazy dog. 99 00:07:47,350 --> 00:07:52,960 If I gave you the sequence, the quick brown fox jumps over the lazy, how can you predict the next 100 00:07:52,960 --> 00:07:54,520 word of this sentence? 101 00:07:55,240 --> 00:08:00,610 Well, one solution is to build a probability distribution so you have the probability of the word a 102 00:08:00,610 --> 00:08:05,830 time t given the word a time, T minus one, given the word of times you minus two and so on. 103 00:08:06,370 --> 00:08:08,410 We call such a model a language model. 104 00:08:13,370 --> 00:08:18,800 Well, to get to the point, the mark of assumption says this, it says that instead of the word a time 105 00:08:18,800 --> 00:08:24,950 t, depending on all previous words, it only depends on the most immediate preceding word. 106 00:08:25,580 --> 00:08:31,580 That is P of word a time T given word, a time T minus one word at time, T minus two and so on is equal 107 00:08:31,580 --> 00:08:35,690 to P of word a time T given word of time, T minus one. 108 00:08:36,500 --> 00:08:40,910 Now you might think, OK, that's fine, but let's make this a little less abstract. 109 00:08:45,720 --> 00:08:50,350 Suppose I give you the word lazy and I ask you to predict the next word in my sentence. 110 00:08:50,760 --> 00:08:52,750 Of course, there are many possibilities. 111 00:08:53,190 --> 00:08:58,520 It could be lazy dog, but you'd probably be cheating because that's the sentence I gave you earlier. 112 00:08:59,160 --> 00:09:04,920 It might be lazy programmer, who is the author of this course, but again, you're going to use exogenous 113 00:09:04,920 --> 00:09:06,250 data to make your prediction. 114 00:09:07,020 --> 00:09:08,290 How about lazy student? 115 00:09:09,150 --> 00:09:14,190 In fact, it's quite difficult to know with any certainty exactly what the next word will be, given 116 00:09:14,190 --> 00:09:15,230 only a single word. 117 00:09:16,050 --> 00:09:16,830 Consider the word. 118 00:09:16,830 --> 00:09:19,590 The the next word could be practically anything. 119 00:09:20,190 --> 00:09:25,080 So the lesson here is that the mark of assumption is an extremely strong modeling assumption. 120 00:09:25,470 --> 00:09:27,180 At the same time, it's quite useful. 121 00:09:32,170 --> 00:09:37,690 So let's assume we have a Gaussian random, OK, this is excessive T equals to X of T minus one plus 122 00:09:37,690 --> 00:09:42,290 F.T. Where it is Gaussian distributed with mean zero and variance sigma squared. 123 00:09:42,910 --> 00:09:49,810 In this case, we can see that X of T is completely determined by a Gaussian distribution center, that 124 00:09:49,810 --> 00:09:52,600 X of T minus one with a variance sigma squared. 125 00:09:53,200 --> 00:09:59,320 That is, it does not depend on any previous values in the series, not X, a T minus two, not actually 126 00:09:59,350 --> 00:10:00,720 T minus three and so on. 127 00:10:01,390 --> 00:10:04,840 Therefore the Gaussian random walk forms a Markov chain. 128 00:10:09,830 --> 00:10:15,970 So if stock prices follow a Gaussian random walk, then the next obvious question is how do we forecast 129 00:10:16,610 --> 00:10:19,440 remember that because the next step is essentially random. 130 00:10:19,580 --> 00:10:22,130 The best we can do is find the expected value. 131 00:10:22,820 --> 00:10:28,850 Well, the expected value of a Gaussian with mean X of T minus one is just the mean X of T minus one. 132 00:10:29,510 --> 00:10:30,560 So what does this say? 133 00:10:31,220 --> 00:10:36,740 It's saying that if your stock price follows a random walk, then your best guess for the next stock 134 00:10:36,740 --> 00:10:39,540 price in the series is just the previous value. 135 00:10:39,860 --> 00:10:41,970 We cannot do any better than this. 136 00:10:42,680 --> 00:10:47,690 Notice that this justifies our method of filling in missing data, which is to copy the previous stock 137 00:10:47,690 --> 00:10:49,040 price forward in time. 138 00:10:53,980 --> 00:11:00,130 Now, as you know, often when we make estimates and statistics, we also want to quantify how confident 139 00:11:00,130 --> 00:11:01,620 we are in those estimates. 140 00:11:02,380 --> 00:11:08,380 Let's suppose we start at X of T and we want to forecast tall steps into the future to find X of T, 141 00:11:08,380 --> 00:11:15,370 plus how we already know the expected value of X 50 plus how it's just X of T, the same value we started 142 00:11:15,370 --> 00:11:15,740 with. 143 00:11:16,360 --> 00:11:17,680 But what does this variance? 144 00:11:18,340 --> 00:11:20,820 Well, we can use our price simulation formula. 145 00:11:21,430 --> 00:11:25,930 We know that X 50 plus one is equal to acts of T plus T plus one. 146 00:11:26,560 --> 00:11:33,040 Based on that, we also know that X 50 plus to zero to x 50 plus one plus the two plus two, which is 147 00:11:33,040 --> 00:11:34,860 added one to all the time indices. 148 00:11:35,650 --> 00:11:42,610 However, we can substitute X 50 plus one and then we would get X of T plus E plus one plus eight plus 149 00:11:42,610 --> 00:11:43,030 two. 150 00:11:43,780 --> 00:11:47,980 And then we keep following this pattern until we get to 50 plus tau. 151 00:11:48,430 --> 00:11:55,240 So X of T plus tau Ziko to X of T plus F.T. plus one policy of T plus two all the way up to 80 plus 152 00:11:55,240 --> 00:11:55,780 tão. 153 00:11:56,560 --> 00:12:00,400 Now luckily we did something exactly like this in the previous section. 154 00:12:01,150 --> 00:12:07,330 If all the E's are Gaussian with mean zero and variance sigma squared, then there's some as mean zero 155 00:12:07,330 --> 00:12:09,790 and variance tau time sigma squared. 156 00:12:10,420 --> 00:12:15,340 Therefore we can say that the variance in our estimate increases linearly with tau. 157 00:12:16,030 --> 00:12:21,580 More commonly we work with the standard deviation so we can see that the standard deviation of our forecast 158 00:12:21,790 --> 00:12:25,810 increases with the square root of the number of forecasting steps. 159 00:12:30,470 --> 00:12:34,560 Let's consider a well-known theorem from statistics, the central limit theorem. 160 00:12:35,390 --> 00:12:41,540 We know that our forecast, the Time T plus tau is the last known price of T plus the sum of a bunch 161 00:12:41,540 --> 00:12:42,640 of noise terms. 162 00:12:43,280 --> 00:12:48,030 Recall that the central limit theorem says that this sum tends to a Gaussian distribution. 163 00:12:48,740 --> 00:12:54,740 And so even if your returns do not necessarily follow a Gaussian distribution in the short term, what 164 00:12:54,740 --> 00:12:55,880 happens in the long term? 165 00:12:56,480 --> 00:12:59,720 Well, in the long term, you're just adding up a bunch of random variables. 166 00:12:59,900 --> 00:13:03,830 And due to the central limit theorem, their distribution approaches a Gaussian. 167 00:13:08,730 --> 00:13:13,920 I want to end this lecture with a tale about a famous experiment run by The Wall Street Journal in nineteen 168 00:13:13,920 --> 00:13:14,470 eighty eight. 169 00:13:15,360 --> 00:13:20,640 And this experiment called the Dart Throwing Investment Contest, professional stock traders from the 170 00:13:20,640 --> 00:13:26,550 New York Stock Exchange competed against dummy investors who simply threw darts on a board to choose 171 00:13:26,550 --> 00:13:27,390 stocks randomly. 172 00:13:28,110 --> 00:13:32,850 Now, granted, one might argue that throwing darts is not actually random and there may have been better 173 00:13:32,850 --> 00:13:34,400 ways to make random choices. 174 00:13:35,010 --> 00:13:41,190 In any case, they found that professional investors beat the dummy investors sixty one out of one hundred 175 00:13:41,190 --> 00:13:45,660 times and the dummy investors won only 39 out of 100 times. 176 00:13:46,290 --> 00:13:51,780 So you might think it's better to go with a professional investor rather than just picking stocks randomly. 177 00:13:52,500 --> 00:13:58,470 However, the professional investors only beat the market 51 out of 100 times. 178 00:13:59,100 --> 00:14:04,610 This is why it's often advised not to use active investing, although your bank will tell you otherwise. 179 00:14:04,980 --> 00:14:10,440 Just don't forget your bank is there to sell you things, not to give you good advice if you buy into 180 00:14:10,440 --> 00:14:11,910 an actively managed fund. 181 00:14:11,940 --> 00:14:17,520 First of all, you may only have a 50 percent chance of beating the market and on average you will match 182 00:14:17,520 --> 00:14:18,030 the market. 183 00:14:18,570 --> 00:14:23,860 However, the fees for actively managed funds are much higher than passively managed funds. 184 00:14:24,300 --> 00:14:29,100 Therefore, if you invest in the market itself, your fees will be much lower and you will have the 185 00:14:29,100 --> 00:14:30,780 same expected return anyway.