1
00:00:00,000 --> 00:00:02,190
In the last video,
we saw how to use

2
00:00:02,190 --> 00:00:04,275
RNNs for prediction of sequences,

3
00:00:04,275 --> 00:00:05,790
and we learned a
little about how to

4
00:00:05,790 --> 00:00:07,740
construct a neural
network with them.

5
00:00:07,740 --> 00:00:09,540
In this video,
you'll work through

6
00:00:09,540 --> 00:00:11,180
a notebook with all of the code,

7
00:00:11,180 --> 00:00:12,360
and when you're done
watching, you can

8
00:00:12,360 --> 00:00:14,340
try out the notebook
for yourself.

9
00:00:14,340 --> 00:00:17,880
As usual, let's first ensure
that we're running 2.0,

10
00:00:17,880 --> 00:00:19,230
and then we'll run
the code to set up

11
00:00:19,230 --> 00:00:21,405
the data and create the dataset.

12
00:00:21,405 --> 00:00:24,690
Then we'll run the neural
network code but adjusting

13
00:00:24,690 --> 00:00:26,220
the learning rate per epoch in

14
00:00:26,220 --> 00:00:29,200
order to find the best one
for the full training.

15
00:00:29,900 --> 00:00:32,460
We'll plot the results to try to

16
00:00:32,460 --> 00:00:34,470
find the optimal learning rate.

17
00:00:34,470 --> 00:00:37,800
You can see here, it's between
10_minus 6 and 10_minus 5.

18
00:00:37,800 --> 00:00:39,680
So I pick a value halfway say,

19
00:00:39,680 --> 00:00:41,920
5 times 10_minus 5.

20
00:00:41,920 --> 00:00:45,050
Here's the code to train
the neural network.

21
00:00:45,050 --> 00:00:47,120
I've set the optimal
learning rates and

22
00:00:47,120 --> 00:00:49,805
I've picked 400 epochs
for which to train.

23
00:00:49,805 --> 00:00:53,470
I'll now train the neural
network for the 400 epochs.

24
00:00:53,470 --> 00:00:55,640
Once it's trained,
I can use it to

25
00:00:55,640 --> 00:00:59,255
forecast for the validation
range and plot the results.

26
00:00:59,255 --> 00:01:01,985
On all my plot, I can
see that my prediction

27
00:01:01,985 --> 00:01:04,595
isn't too bad other
than this plateau,

28
00:01:04,595 --> 00:01:08,215
which is definitely going to
impact my MA in a bad way.

29
00:01:08,215 --> 00:01:10,610
But despite that my MA is only

30
00:01:10,610 --> 00:01:13,575
about 6.41, so it's not too bad.

31
00:01:13,575 --> 00:01:18,065
If I plot the MA and loss
for training, I'll see this.

32
00:01:18,065 --> 00:01:19,760
Showing that while it looks quite

33
00:01:19,760 --> 00:01:22,010
flat after about epoch 50,

34
00:01:22,010 --> 00:01:24,409
it was actually still
reducing slowly,

35
00:01:24,409 --> 00:01:25,610
and when we zoom in,

36
00:01:25,610 --> 00:01:26,845
we can see that.

37
00:01:26,845 --> 00:01:30,545
We saw instability after
about epoch 400 earlier,

38
00:01:30,545 --> 00:01:33,120
so I just stopped
training at that point.