We could take character encodings for each character in a set. For example, the ASCII values. But will that help us understand
the meaning of a word? So for example, consider the
word 'LISTEN' as shown here. A common simple character
encoding is ASCII, the American Standard Code for Information Interchange with
the values as shown here. So you might think
you could have a word like LISTEN encoded
using these values. But the problem with
this of course, is that the semantics of the word aren't encoded
in the letters. This could be demonstrated
using the word 'SILENT ' which has a very different
and almost opposite meaning, but with exactly
the same letters. So it seems that training
a neural network with just the letters
could be a daunting task. So how about if we
consider words? What if we could give
words a value and have those values used
in training a network? Now we could be
getting somewhere. So for example,
consider this sentence, I Love my dog. How about giving
a value to each word? What that value is
doesn't matter. It's just that we have
a value per word, and the value is the same for
the same word every time. So a simple encoding for
the sentence would be for example to give
word 'I' the value one. Following on, we could
give the words 'Love', 'my' and 'dog' the values 2, 3, and 4 respectively. So then the sentence, I love my dog would be
encoded as 1, 2, 3, 4. So now, what if I
have the sentence, I love my cat? Well, we've already
encoded the words 'I love my' as 1, 2, 3. So we can reuse those, and we can create
a new token for cat, which we haven't seen before. So let's make that the number 5. Now if we just look at
the two sets of encodings, we can begin to see some similarity
between the sentences. I love my dog is 1, 2, 3, 4 and I love my
cat is 1, 2, 3, 5. So this is at least a
beginning and how we can start training a neural
network based on words. Fortunately, TensorFlow
and Keras give us some APIs that make it
very simple to do this. We'll look at those next.