1
00:00:00,000 --> 00:00:03,300
In the previous video, you
looked at RNNs and how

2
00:00:03,300 --> 00:00:05,985
you could use RNNs to do
some simple prediction.

3
00:00:05,985 --> 00:00:07,380
You then improved on them,

4
00:00:07,380 --> 00:00:09,090
using LSTMs instead of

5
00:00:09,090 --> 00:00:12,105
RNNs to get a better fit
with our predicted data.

6
00:00:12,105 --> 00:00:13,980
In this video, you'll
take a look at

7
00:00:13,980 --> 00:00:15,810
the LSTM notebook after

8
00:00:15,810 --> 00:00:18,285
which you can try that notebook
out for yourself.

9
00:00:18,285 --> 00:00:20,430
As always, let's
ensure that we're

10
00:00:20,430 --> 00:00:22,485
using TensorFlow too,
before starting.

11
00:00:22,485 --> 00:00:24,075
And now that we see that we are,

12
00:00:24,075 --> 00:00:26,220
let's run this code
to create the data,

13
00:00:26,220 --> 00:00:27,540
and then the helper function

14
00:00:27,540 --> 00:00:29,205
that we'll turn it
into a dataset.

15
00:00:29,205 --> 00:00:30,525
Will then run our model,

16
00:00:30,525 --> 00:00:32,295
which has two LSTM layers.

17
00:00:32,295 --> 00:00:33,690
You can of course change this,

18
00:00:33,690 --> 00:00:37,090
but we'll run it to find
our optimum learning rate first.

19
00:00:37,090 --> 00:00:39,150
Once it's done, we can plot

20
00:00:39,150 --> 00:00:41,410
the loss versus
the learning rates,

21
00:00:41,410 --> 00:00:42,965
and we'll find
a good learning rate

22
00:00:42,965 --> 00:00:45,155
is about ten to the minus five.

23
00:00:45,155 --> 00:00:47,735
So, we'll update
our learning rates of that.

24
00:00:47,735 --> 00:00:50,500
I will train it for
about 500 epochs.

25
00:00:50,500 --> 00:00:53,805
When it's done, we'll
plot the forecast.

26
00:00:53,805 --> 00:00:56,540
As we can see, it's
definitely improved.

27
00:00:56,540 --> 00:00:59,945
Changing to LSTMs gets rid
of that plateau that we saw,

28
00:00:59,945 --> 00:01:02,030
and the curve is
closer to the data,

29
00:01:02,030 --> 00:01:04,210
but it's still not quite right,

30
00:01:04,210 --> 00:01:08,310
and it gives us an MAE
of a little over 8.5.

31
00:01:08,690 --> 00:01:11,210
So, that's a good start in using

32
00:01:11,210 --> 00:01:13,450
recurrent neural networks
to fix the problem.

33
00:01:13,450 --> 00:01:14,705
Have a play with a notebook

34
00:01:14,705 --> 00:01:16,385
and see if you can improve on it.

35
00:01:16,385 --> 00:01:18,515
Next week, we're going
to do one more thing,

36
00:01:18,515 --> 00:01:20,990
and that's adding
a convolutional layer before

37
00:01:20,990 --> 00:01:23,785
the LSTMs to see
the impact on training.

38
00:01:23,785 --> 00:01:25,910
After that, you'll
start moving away from

39
00:01:25,910 --> 00:01:27,350
synthetic data and you'll

40
00:01:27,350 --> 00:01:29,790
start using the real-world stuff.