1
00:00:00,000 --> 00:00:02,100
Here are a few phrases that

2
00:00:02,100 --> 00:00:03,900
were generated when I gave the

3
00:00:03,900 --> 00:00:07,500
neural network the sentence
Lawrence went to Dublin,

4
00:00:07,500 --> 00:00:10,215
and I asked it to predict
the next 10 words.

5
00:00:10,215 --> 00:00:12,255
We'll see the code
for that in a moment,

6
00:00:12,255 --> 00:00:16,185
but notice that there's a lot
of repetition of words.

7
00:00:16,185 --> 00:00:21,090
In this sentence, three of
the last five words are wall,

8
00:00:21,090 --> 00:00:25,170
and here three of the
last four are ball,

9
00:00:25,170 --> 00:00:30,150
and even in this one the word
relations gets repeated.

10
00:00:30,150 --> 00:00:32,925
This is because our LSTM was

11
00:00:32,925 --> 00:00:35,355
only carrying context forward.

12
00:00:35,355 --> 00:00:37,280
Let's take a look at
what happens if we

13
00:00:37,280 --> 00:00:40,100
change the code to
be bidirectional.

14
00:00:40,100 --> 00:00:42,920
By adding this line simply

15
00:00:42,920 --> 00:00:45,125
defining the LSTM
is bidirectional,

16
00:00:45,125 --> 00:00:47,400
and then retraining,

17
00:00:47,860 --> 00:00:50,600
I can see that I do converge a

18
00:00:50,600 --> 00:00:53,670
bit quicker as you'll
see in this chart.

19
00:00:53,690 --> 00:00:56,245
After training and testing,

20
00:00:56,245 --> 00:00:58,145
I now get these sentences.

21
00:00:58,145 --> 00:00:59,900
They make a little
bit more sense,

22
00:00:59,900 --> 00:01:01,780
but there's still
some repetition.

23
00:01:01,780 --> 00:01:03,860
That being said, remember this is

24
00:01:03,860 --> 00:01:06,740
a song where words
rhyme such as ball,

25
00:01:06,740 --> 00:01:08,405
all and wall, et cetera,

26
00:01:08,405 --> 00:01:11,730
and as such many of them
are going to show up.