1
00:00:00,000 --> 00:00:02,820
So let's now take
a look at some code for

2
00:00:02,820 --> 00:00:04,830
LSTMs and how they can work

3
00:00:04,830 --> 00:00:07,840
with the data that we've
been playing with all week.

4
00:00:08,690 --> 00:00:12,750
So here's the update to
our code to use LSTMs.

5
00:00:12,750 --> 00:00:16,335
Let's unpack it and take a look
at the interesting parts.

6
00:00:16,335 --> 00:00:21,030
First of all is the
tf.keras.backend.clear_session,

7
00:00:21,030 --> 00:00:23,970
and this clears
any internal variables.

8
00:00:23,970 --> 00:00:26,550
That makes it easy for
us to experiment without

9
00:00:26,550 --> 00:00:29,805
models impacting
later versions of themselves.

10
00:00:29,805 --> 00:00:33,570
After the Lambda layer that
expands the dimensions for us

11
00:00:33,570 --> 00:00:37,125
I've added a single LSTM
layer with 32 cells.

12
00:00:37,125 --> 00:00:38,940
I've also made a bidirectional to

13
00:00:38,940 --> 00:00:41,550
see the impact of
that on a prediction.

14
00:00:41,550 --> 00:00:45,320
The output neuron will give
us our prediction value.

15
00:00:45,320 --> 00:00:48,220
I'm also using a learning rate
of one times 10 _

16
00:00:48,220 --> 00:00:51,920
minus six and that might be
worth experimenting with too.

17
00:00:51,920 --> 00:00:55,070
So here's the results
of running this LSTM on

18
00:00:55,070 --> 00:00:56,330
the synthetic data that we've

19
00:00:56,330 --> 00:00:58,145
been using throughout the course.

20
00:00:58,145 --> 00:01:00,680
The plateau under
the big spike is still

21
00:01:00,680 --> 00:01:03,410
there and are MAE is
in the low sixes.

22
00:01:03,410 --> 00:01:06,215
It's not bad, it's not
great, but it's not bad.

23
00:01:06,215 --> 00:01:07,850
The predictions look
like there might be

24
00:01:07,850 --> 00:01:09,830
a little bit on the low side too.

25
00:01:09,830 --> 00:01:12,260
So let's edit our code to add

26
00:01:12,260 --> 00:01:15,070
another LSTM to see the impact.

27
00:01:15,070 --> 00:01:17,760
Now you can see the second layer

28
00:01:17,760 --> 00:01:19,310
and note that we had to set

29
00:01:19,310 --> 00:01:21,560
return sequences equal to true on

30
00:01:21,560 --> 00:01:24,170
the first one in order
for this to work.

31
00:01:24,170 --> 00:01:25,730
We train on this and

32
00:01:25,730 --> 00:01:27,725
now we will see
the following results.

33
00:01:27,725 --> 00:01:29,990
Here's the chart.
Now it's tracking

34
00:01:29,990 --> 00:01:32,195
much better and closer
to the original data.

35
00:01:32,195 --> 00:01:33,410
Maybe not keeping up with

36
00:01:33,410 --> 00:01:36,425
the sharp increase but at
least it's tracking close.

37
00:01:36,425 --> 00:01:39,590
It also gives us a mean
average error that's a lot

38
00:01:39,590 --> 00:01:41,000
better and it's showing

39
00:01:41,000 --> 00:01:43,280
that we're heading in
the right direction.

40
00:01:43,280 --> 00:01:45,680
If we edit our source to add

41
00:01:45,680 --> 00:01:48,904
a third LSTM layer like this,

42
00:01:48,904 --> 00:01:50,840
by adding the layer and having

43
00:01:50,840 --> 00:01:52,850
the second layer
return sequences is

44
00:01:52,850 --> 00:01:55,695
true we can then
train and run it,

45
00:01:55,695 --> 00:01:58,190
and we'll get
the following output.

46
00:01:58,190 --> 00:01:59,420
There's really not that much of

47
00:01:59,420 --> 00:02:02,275
a difference and are MAE
has actually gone down.

48
00:02:02,275 --> 00:02:04,020
So that's it for looking at

49
00:02:04,020 --> 00:02:06,750
LSTMs and predicting
our sequences.

50
00:02:06,750 --> 00:02:08,690
In the next video
you'll take a look at

51
00:02:08,690 --> 00:02:10,790
using convolutions to see what

52
00:02:10,790 --> 00:02:12,260
the impact will be on using

53
00:02:12,260 --> 00:02:15,720
them for predicting
time-series content.