1
00:00:00,000 --> 00:00:02,340
Welcome back. In this week,

2
00:00:02,340 --> 00:00:04,650
you'll learn how to
use the Keras layer in

3
00:00:04,650 --> 00:00:07,350
TensorFlow to implement
word embeddings,

4
00:00:07,350 --> 00:00:09,120
which is one of
the most important ideas

5
00:00:09,120 --> 00:00:11,035
in Natural Language Processing.

6
00:00:11,035 --> 00:00:14,575
If you have a vocabulary size
of say 10,000 words,

7
00:00:14,575 --> 00:00:16,470
rather than using the numbers

8
00:00:16,470 --> 00:00:19,170
from 1-10,000 to
represent these words,

9
00:00:19,170 --> 00:00:22,340
can you have a better way
to represent those numbers?

10
00:00:22,340 --> 00:00:23,850
So this week you'll learn about

11
00:00:23,850 --> 00:00:25,410
word embeddings and also get to

12
00:00:25,410 --> 00:00:28,410
train your own neural network
for text classification.

13
00:00:28,410 --> 00:00:31,740
It's really interesting
that I really liked

14
00:00:31,740 --> 00:00:33,750
how embeddings work for

15
00:00:33,750 --> 00:00:35,925
representing
the semantics of a word.

16
00:00:35,925 --> 00:00:38,150
So now instead of the word
just being a number,

17
00:00:38,150 --> 00:00:40,610
it's like a vector in
n-dimensional space.

18
00:00:40,610 --> 00:00:43,625
We're using 16 and 32 dimensions
in different examples.

19
00:00:43,625 --> 00:00:45,450
So for example the word dog,

20
00:00:45,450 --> 00:00:46,820
might be a vector pointing in

21
00:00:46,820 --> 00:00:49,489
a particular direction
and then the word K9,

22
00:00:49,489 --> 00:00:51,170
could be learned as
a vector pointing

23
00:00:51,170 --> 00:00:52,549
in a very similar direction,

24
00:00:52,549 --> 00:00:54,020
and we know they
have very similar

25
00:00:54,020 --> 00:00:56,145
semantic meaning off of that.

26
00:00:56,145 --> 00:00:59,015
All of this is actually
done for us in embeddings.

27
00:00:59,015 --> 00:01:01,060
So as we train like our datasets,

28
00:01:01,060 --> 00:01:04,339
we'll train for example with
IMDB which is movie reviews,

29
00:01:04,339 --> 00:01:06,140
and there's a set
of positive reviews

30
00:01:06,140 --> 00:01:07,620
and a set of negative reviews,

31
00:01:07,620 --> 00:01:09,260
and what will actually happen is

32
00:01:09,260 --> 00:01:11,779
the embeddings for
the words in these reviews,

33
00:01:11,779 --> 00:01:14,165
and we can actually
plot them on a chart.

34
00:01:14,165 --> 00:01:15,740
So we'll see something
that looks like

35
00:01:15,740 --> 00:01:17,390
a globe and one of the poles on

36
00:01:17,390 --> 00:01:18,740
the globe or
all the words that are

37
00:01:18,740 --> 00:01:20,660
clustered around for
the positive review,

38
00:01:20,660 --> 00:01:22,520
and all the words
on the other pole

39
00:01:22,520 --> 00:01:24,490
of the globe are clustered
around the negative review,

40
00:01:24,490 --> 00:01:26,090
and it really helps
us to start to see

41
00:01:26,090 --> 00:01:27,895
the semantics behind these words.

42
00:01:27,895 --> 00:01:29,420
I think one of the coolest things

43
00:01:29,420 --> 00:01:30,650
about word embedding is you can

44
00:01:30,650 --> 00:01:32,000
download the pretrained word

45
00:01:32,000 --> 00:01:33,800
embedding that maybe
someone else has trained.

46
00:01:33,800 --> 00:01:35,705
This gives your
learning algorithm

47
00:01:35,705 --> 00:01:38,420
a hint into the meaning
of these words.

48
00:01:38,420 --> 00:01:40,145
So when you see
another vocabulary word,

49
00:01:40,145 --> 00:01:42,065
maybe you see K9
for the first time,

50
00:01:42,065 --> 00:01:44,060
it's as if you can let
the algorithm know

51
00:01:44,060 --> 00:01:45,980
that K9 means
something a little bit

52
00:01:45,980 --> 00:01:47,900
like dog related to
dog even if you've

53
00:01:47,900 --> 00:01:51,215
never seen the word K9 in
your specific training.

54
00:01:51,215 --> 00:01:52,525
That's amazing.

55
00:01:52,525 --> 00:01:55,764
Today in the world
of applied NLP,

56
00:01:55,764 --> 00:01:57,830
word embeddings has
proved to be one of

57
00:01:57,830 --> 00:01:59,660
the most powerful
and useful ideas to

58
00:01:59,660 --> 00:02:01,670
help teams get good performance.

59
00:02:01,670 --> 00:02:02,730
So in this week,

60
00:02:02,730 --> 00:02:05,660
you'll learn how to use word
embeddings in TensorFlow.

61
00:02:05,660 --> 00:02:07,980
Let's go on to the next video.