1
00:00:11,150 --> 00:00:17,210
So in this lecture, we are going to continue our script to forecast stocks using Amazon forecast and

2
00:00:17,210 --> 00:00:19,450
industrial strength forecasting solution.

3
00:00:20,210 --> 00:00:26,120
So far we've downloaded our data, converted it into the correct format and uploaded that data to us.

4
00:00:26,120 --> 00:00:33,650
Three, we've imported our data into Adewusi forecast by creating a data set group and data sets for

5
00:00:33,650 --> 00:00:34,780
that data set group.

6
00:00:35,360 --> 00:00:38,730
We've added our files from S3 to those data sets.

7
00:00:39,410 --> 00:00:42,980
The next step is the fun part where we actually get to build our predictor.

8
00:00:44,000 --> 00:00:49,460
You learn terminology, you can think of this like calling model does fit, although as you'll soon

9
00:00:49,460 --> 00:00:51,800
see this actually does quite a bit more.

10
00:00:53,030 --> 00:00:59,570
So we start by defining our algorithm and you can get these from the AWADI documentation, so don't

11
00:00:59,570 --> 00:01:01,340
worry about having to memorize this.

12
00:01:01,970 --> 00:01:05,720
Basically, Amazon forecast supports six different algorithms.

13
00:01:06,140 --> 00:01:11,570
We'll be using an algorithm called DPR Plus, which I've chosen because I think that's what most people

14
00:01:11,570 --> 00:01:13,460
taking this course would be curious about.

15
00:01:14,060 --> 00:01:15,680
So what is DPR plus?

16
00:01:16,230 --> 00:01:20,170
Essentially, it's an auto regressive RNA that predicts a scalar.

17
00:01:20,810 --> 00:01:26,090
However, it's able to learn from a time series of multiple input features, which is exactly what we

18
00:01:26,090 --> 00:01:26,600
have.

19
00:01:27,680 --> 00:01:29,990
There's another algorithm called CNN QR.

20
00:01:30,170 --> 00:01:34,340
So you might be interested in that if you want to try other deep learning techniques.

21
00:01:35,000 --> 00:01:41,120
Interestingly, HWC forecast also lets you use profit, which was invented by Facebook, for which we

22
00:01:41,120 --> 00:01:42,840
already have a Python library.

23
00:01:43,550 --> 00:01:48,330
There's an algorithm called NPT's, which stands for non Parametric Time series.

24
00:01:48,770 --> 00:01:50,900
This looks like it was invented by Amazon.

25
00:01:51,080 --> 00:01:53,710
So there's not a lot of literature about how it works.

26
00:01:54,980 --> 00:02:00,800
And finally, we have classic algorithms like Arima and exponential smoothing, which is called ETS

27
00:02:00,800 --> 00:02:07,340
in the docs, note that other algorithms like Arima and ETS do not support related data sets.

28
00:02:07,610 --> 00:02:11,330
So if you want to use those, then you can't have any related time series.

29
00:02:12,680 --> 00:02:17,960
However, you might also want to consider whether there's any benefit of using a service like this if

30
00:02:17,960 --> 00:02:19,190
all you want to do is Arima.

31
00:02:20,030 --> 00:02:25,970
So overall, HWC forecast lets you choose from six different algorithms out of these.

32
00:02:25,970 --> 00:02:28,820
I thought DPR plus would be the most interesting.

33
00:02:29,090 --> 00:02:30,590
So that's the one we'll focus on.

34
00:02:33,910 --> 00:02:38,210
So the next step is to create our predictor by calling the create predictive function.

35
00:02:38,860 --> 00:02:41,240
You'll notice this takes in quite a few arguments.

36
00:02:41,860 --> 00:02:45,830
First, we pass in the predictable name as well as the algorithm are in.

37
00:02:46,630 --> 00:02:52,300
Next, we have the forecast horizon, which tells us how far into the future that we would like to forecast.

38
00:02:53,280 --> 00:02:55,690
Next, we have a flag to choose whether or not to use.

39
00:02:55,690 --> 00:03:01,450
Automatable in this case, auto will automatically select which algorithm to use.

40
00:03:01,930 --> 00:03:05,830
However, since we already chose DPR plus this is not necessary.

41
00:03:07,190 --> 00:03:13,040
As an exercise, you might want to try setting this to true to see if Amazon forecast to find that some

42
00:03:13,040 --> 00:03:14,390
other algorithm works better.

43
00:03:16,120 --> 00:03:18,290
The next option is perform HPO.

44
00:03:18,790 --> 00:03:23,950
This will automatically perform hyper parameter optimization, which everyone in machine learning hates

45
00:03:23,950 --> 00:03:24,380
to do.

46
00:03:24,940 --> 00:03:29,530
I'm going to set this to false, but I encourage you to try this out for yourself to see how well it

47
00:03:29,530 --> 00:03:30,140
works.

48
00:03:31,270 --> 00:03:33,300
Do note that you do have to pay for this.

49
00:03:33,520 --> 00:03:36,640
So keep in mind that this will cost both time and money.

50
00:03:37,780 --> 00:03:43,720
In my experience, training with HBO on took about four hours, whereas it only took about one hour

51
00:03:43,870 --> 00:03:45,010
with this setting off.

52
00:03:45,460 --> 00:03:47,620
However, that was with a different data set.

53
00:03:47,650 --> 00:03:48,910
So keep that in mind.

54
00:03:50,520 --> 00:03:56,580
Also, recall that wall clock time is not equal to compute hours in Amazon, since Amazon runs its training

55
00:03:56,580 --> 00:03:57,240
in parallel.

56
00:03:57,930 --> 00:04:02,960
So if Amazon tries 100 different hyper parameter settings, you have to pay for all of them.

57
00:04:03,990 --> 00:04:08,100
The next option, which I've come into now, is for choosing the forecast type.

58
00:04:08,700 --> 00:04:13,230
You'll see this show up later in our forecast, but basically they represent the kind of prediction

59
00:04:13,230 --> 00:04:14,170
we want to make.

60
00:04:14,670 --> 00:04:19,060
By default, we get zero point one zero point five and zero point nine.

61
00:04:19,860 --> 00:04:21,050
So what does this mean?

62
00:04:21,840 --> 00:04:24,630
Essentially, these are quintiles or percentiles.

63
00:04:25,020 --> 00:04:31,620
So zero point one means the true value is expected to be lower than the 10 percent line, 10 percent

64
00:04:31,620 --> 00:04:37,620
of the time, zero point nine means the true value is expected to be lower than the 90 percent line

65
00:04:37,890 --> 00:04:39,340
90 percent of the time.

66
00:04:40,500 --> 00:04:43,140
This means that zero point five is the median prediction.

67
00:04:43,980 --> 00:04:50,100
Also note that by default, the mean is not returned, which is what people usually expect when performing

68
00:04:50,100 --> 00:04:50,820
a prediction.

69
00:04:51,390 --> 00:04:55,680
So if you want the mean, you can add this to the list of return types as shown.

70
00:04:57,420 --> 00:05:03,030
Lastly, we pass in our data set group Aaren, so our predictor knows where the training data can be

71
00:05:03,030 --> 00:05:05,550
found and also the data set frequency.

72
00:05:09,680 --> 00:05:16,010
As expected, creating the predictor launches a job for which we can check the job status, so the next

73
00:05:16,010 --> 00:05:18,770
step is to grab the R.M. for our predictor.

74
00:05:21,670 --> 00:05:27,700
Next, we call the function described predictor passing in the predictor, and again, you just keep

75
00:05:27,700 --> 00:05:30,930
running this until it's as active as you can see.

76
00:05:30,940 --> 00:05:34,080
The last time I ran this, it took less than 30 minutes.

77
00:05:37,910 --> 00:05:44,120
OK, so next, we can check how accurate our model is by calling the function to get accuracy metrics.

78
00:05:44,810 --> 00:05:50,290
So this will give us a big dictionary telling us various metrics about how our model performs.

79
00:05:50,960 --> 00:05:56,300
As you can see, this gives us metrics like the root mean squared error, weighted, absolute percentage

80
00:05:56,300 --> 00:05:57,570
error and so forth.

81
00:05:58,310 --> 00:06:03,080
Note that this is on the test set, which is chosen to be the last bit of the data set that we passed

82
00:06:03,080 --> 00:06:03,380
in.

83
00:06:05,820 --> 00:06:11,490
OK, so to summarize this lecture, here's what we've done so far, we've learned how to take our data

84
00:06:11,490 --> 00:06:16,040
and create a predictor which effectively fits a model to the data we've provided.

85
00:06:16,560 --> 00:06:20,850
We've enumerated all the possible models that can be used in Adewusi forecasts.

86
00:06:21,060 --> 00:06:24,100
And I think we've chosen the most interesting one DPB.

87
00:06:24,120 --> 00:06:30,330
Plus, we've seen the possibility of several nice options, such as Auto Amelle and hyper parameter

88
00:06:30,330 --> 00:06:33,450
optimization, which I encourage you to try it on your own.

89
00:06:33,600 --> 00:06:36,570
If you're serious about training the best model possible.

90
00:06:37,590 --> 00:06:42,540
In the next few lectures, we'll look at how to create a forecast from the model we've just trained

91
00:06:42,690 --> 00:06:47,460
and we'll compare how close it is to the actual future prices that we've tried to predict.