1
00:00:00,652 --> 00:00:04,359
So consider this RNN,
these has two recovered layers, and

2
00:00:04,359 --> 00:00:07,416
the first has
return_sequences=True set up.

3
00:00:07,416 --> 00:00:10,796
It will output a sequence which
is fed to the next layer.

4
00:00:10,796 --> 00:00:14,573
The next layer does not have
return_sequence that's set to True, so

5
00:00:14,573 --> 00:00:16,631
it will only output to the final step.

6
00:00:16,631 --> 00:00:20,260
But notice the input_shape,
it's set to None and 1.

7
00:00:20,260 --> 00:00:24,406
TensorFlow assumes that the first
dimension is the batch size, and

8
00:00:24,406 --> 00:00:28,343
that it can have any size at all,
so you don't need to define it.

9
00:00:28,343 --> 00:00:33,390
Then the next dimension is the number of
timestamps, which we can set to none,

10
00:00:33,390 --> 00:00:37,304
which means that the RNN can
handle sequences of any length.

11
00:00:37,304 --> 00:00:42,742
The last dimension is just one because
we're using a unit vary of time series.

12
00:00:42,742 --> 00:00:47,813
If we set return_sequences to true and
all recurrent layers, then they will

13
00:00:47,813 --> 00:00:52,736
all output sequences and the dense layer
will get a sequence as its inputs.

14
00:00:52,736 --> 00:00:58,075
Keras handles this by using the same dense
layer independently at each time stamp.

15
00:00:58,075 --> 00:01:00,513
It might look like multiple ones here but

16
00:01:00,513 --> 00:01:04,031
it's the same one that's being
reused at each time step.

17
00:01:04,031 --> 00:01:07,539
This gives us what is called
a sequence to sequence RNN.

18
00:01:07,539 --> 00:01:09,511
It's fed a batch of sequences and

19
00:01:09,511 --> 00:01:12,551
it returns a batch of
sequences of the same length.

20
00:01:12,551 --> 00:01:15,349
The dimensionality may not always match.

21
00:01:15,349 --> 00:01:18,961
It depends on the number of
units in the memory sale.

22
00:01:18,961 --> 00:01:23,770
So let's now return to a two-layer
RNN that has the second one not

23
00:01:23,770 --> 00:01:25,266
return sequences.

24
00:01:25,266 --> 00:01:28,120
This will give us an output
to a single dense.