1
00:00:00,000 --> 00:00:02,760
We could take character encodings

2
00:00:02,760 --> 00:00:04,530
for each character in a set.

3
00:00:04,530 --> 00:00:06,615
For example, the ASCII values.

4
00:00:06,615 --> 00:00:09,735
But will that help us understand
the meaning of a word?

5
00:00:09,735 --> 00:00:13,935
So for example, consider the
word 'LISTEN' as shown here.

6
00:00:13,935 --> 00:00:17,440
A common simple character
encoding is ASCII,

7
00:00:17,440 --> 00:00:19,020
the American Standard Code for

8
00:00:19,020 --> 00:00:22,440
Information Interchange with
the values as shown here.

9
00:00:22,440 --> 00:00:25,010
So you might think
you could have a word

10
00:00:25,010 --> 00:00:27,890
like LISTEN encoded
using these values.

11
00:00:27,890 --> 00:00:29,510
But the problem with
this of course,

12
00:00:29,510 --> 00:00:30,740
is that the semantics of

13
00:00:30,740 --> 00:00:33,565
the word aren't encoded
in the letters.

14
00:00:33,565 --> 00:00:37,110
This could be demonstrated
using the word 'SILENT

15
00:00:37,110 --> 00:00:40,610
' which has a very different
and almost opposite meaning,

16
00:00:40,610 --> 00:00:42,650
but with exactly
the same letters.

17
00:00:42,650 --> 00:00:45,275
So it seems that training
a neural network

18
00:00:45,275 --> 00:00:48,680
with just the letters
could be a daunting task.

19
00:00:48,680 --> 00:00:51,530
So how about if we
consider words?

20
00:00:51,530 --> 00:00:53,930
What if we could give
words a value and

21
00:00:53,930 --> 00:00:56,510
have those values used
in training a network?

22
00:00:56,510 --> 00:00:58,250
Now we could be
getting somewhere.

23
00:00:58,250 --> 00:01:00,590
So for example,
consider this sentence,

24
00:01:00,590 --> 00:01:02,060
I Love my dog.

25
00:01:02,060 --> 00:01:04,810
How about giving
a value to each word?

26
00:01:04,810 --> 00:01:06,935
What that value is
doesn't matter.

27
00:01:06,935 --> 00:01:09,230
It's just that we have
a value per word,

28
00:01:09,230 --> 00:01:12,830
and the value is the same for
the same word every time.

29
00:01:12,830 --> 00:01:15,650
So a simple encoding for
the sentence would be

30
00:01:15,650 --> 00:01:19,445
for example to give
word 'I' the value one.

31
00:01:19,445 --> 00:01:22,635
Following on, we could
give the words 'Love',

32
00:01:22,635 --> 00:01:24,750
'my' and 'dog' the values 2,

33
00:01:24,750 --> 00:01:26,340
3, and 4 respectively.

34
00:01:26,340 --> 00:01:27,744
So then the sentence,

35
00:01:27,744 --> 00:01:31,600
I love my dog would be
encoded as 1, 2, 3, 4.

36
00:01:31,600 --> 00:01:34,125
So now, what if I
have the sentence,

37
00:01:34,125 --> 00:01:35,550
I love my cat?

38
00:01:35,550 --> 00:01:37,760
Well, we've already
encoded the words 'I

39
00:01:37,760 --> 00:01:39,770
love my' as 1, 2, 3.

40
00:01:39,770 --> 00:01:41,585
So we can reuse those,

41
00:01:41,585 --> 00:01:44,240
and we can create
a new token for cat,

42
00:01:44,240 --> 00:01:45,710
which we haven't seen before.

43
00:01:45,710 --> 00:01:48,110
So let's make that the number 5.

44
00:01:48,110 --> 00:01:51,530
Now if we just look at
the two sets of encodings,

45
00:01:51,530 --> 00:01:52,550
we can begin to see

46
00:01:52,550 --> 00:01:55,205
some similarity
between the sentences.

47
00:01:55,205 --> 00:01:56,900
I love my dog is 1, 2, 3,

48
00:01:56,900 --> 00:02:00,575
4 and I love my
cat is 1, 2, 3, 5.

49
00:02:00,575 --> 00:02:03,830
So this is at least a
beginning and how we can

50
00:02:03,830 --> 00:02:06,995
start training a neural
network based on words.

51
00:02:06,995 --> 00:02:09,650
Fortunately, TensorFlow
and Keras give us

52
00:02:09,650 --> 00:02:12,410
some APIs that make it
very simple to do this.

53
00:02:12,410 --> 00:02:14,580
We'll look at those next.