1
00:00:00,000 --> 00:00:03,120
But I'd like to add a couple
of new layers to this,

2
00:00:03,120 --> 00:00:05,520
layers that use the Lambda type.

3
00:00:05,520 --> 00:00:08,655
This type of layer is one
that allows us to perform

4
00:00:08,655 --> 00:00:11,099
arbitrary operations
to effectively

5
00:00:11,099 --> 00:00:14,739
expand the functionality
of TensorFlow's kares,

6
00:00:14,739 --> 00:00:18,375
and we can do this within
the model definition itself.

7
00:00:18,375 --> 00:00:20,640
So the first Lambda layer will be

8
00:00:20,640 --> 00:00:22,995
used to help us with
our dimensionality.

9
00:00:22,995 --> 00:00:24,960
If you recall when we wrote

10
00:00:24,960 --> 00:00:27,480
the window dataset
helper function,

11
00:00:27,480 --> 00:00:29,460
it returned
two-dimensional batches

12
00:00:29,460 --> 00:00:30,915
of Windows on the data,

13
00:00:30,915 --> 00:00:32,900
with the first being
the batch size

14
00:00:32,900 --> 00:00:35,270
and the second the number
of timestamps.

15
00:00:35,270 --> 00:00:37,915
But an RNN expects
three-dimensions;

16
00:00:37,915 --> 00:00:40,165
batch size, the number
of timestamps,

17
00:00:40,165 --> 00:00:42,155
and the series dimensionality.

18
00:00:42,155 --> 00:00:43,640
With the Lambda layer,

19
00:00:43,640 --> 00:00:45,380
we can fix this without

20
00:00:45,380 --> 00:00:48,275
rewriting our Window
dataset helper function.

21
00:00:48,275 --> 00:00:50,060
Using the Lambda, we just

22
00:00:50,060 --> 00:00:52,025
expand the array
by one dimension.

23
00:00:52,025 --> 00:00:54,050
By setting input shape to none,

24
00:00:54,050 --> 00:00:55,580
we're saying that
the model can take

25
00:00:55,580 --> 00:00:57,725
sequences of any length.

26
00:00:57,725 --> 00:01:02,015
Similarly, if we scale
up the outputs by 100,

27
00:01:02,015 --> 00:01:03,550
we can help training.

28
00:01:03,550 --> 00:01:05,840
The default activation
function in

29
00:01:05,840 --> 00:01:07,760
the RNN layers is tan H

30
00:01:07,760 --> 00:01:10,175
which is the hyperbolic
tangent activation.

31
00:01:10,175 --> 00:01:13,030
This outputs values between
negative one and one.

32
00:01:13,030 --> 00:01:15,260
Since the time
series values are in

33
00:01:15,260 --> 00:01:18,230
that order usually in
the 10s like 40s, 50s,

34
00:01:18,230 --> 00:01:20,780
60s, and 70s, then scaling up

35
00:01:20,780 --> 00:01:22,520
the outputs to the same ballpark

36
00:01:22,520 --> 00:01:23,915
can help us with learning.

37
00:01:23,915 --> 00:01:26,065
We can do that in
a Lambda layer too,

38
00:01:26,065 --> 00:01:28,550
we just simply multiply
that by a 100.

39
00:01:28,550 --> 00:01:31,910
So let's now take a look at
what it takes to build out

40
00:01:31,910 --> 00:01:33,590
the full RNN so we

41
00:01:33,590 --> 00:01:35,630
can start doing
some predictions with it.

42
00:01:35,630 --> 00:01:37,980
You'll see that in
the next video.