We could take character encodings
for each character in a set.
For example, the ASCII values.
But will that help us understand the meaning of a word?
So for example, consider the word 'LISTEN' as shown here.
A common simple character encoding is ASCII,
the American Standard Code for
Information Interchange with the values as shown here.
So you might think you could have a word
like LISTEN encoded using these values.
But the problem with this of course,
is that the semantics of
the word aren't encoded in the letters.
This could be demonstrated using the word 'SILENT
' which has a very different and almost opposite meaning,
but with exactly the same letters.
So it seems that training a neural network
with just the letters could be a daunting task.
So how about if we consider words?
What if we could give words a value and
have those values used in training a network?
Now we could be getting somewhere.
So for example, consider this sentence,
I Love my dog.
How about giving a value to each word?
What that value is doesn't matter.
It's just that we have a value per word,
and the value is the same for the same word every time.
So a simple encoding for the sentence would be
for example to give word 'I' the value one.
Following on, we could give the words 'Love',
'my' and 'dog' the values 2,
3, and 4 respectively.
So then the sentence,
I love my dog would be encoded as 1, 2, 3, 4.
So now, what if I have the sentence,
I love my cat?
Well, we've already encoded the words 'I
love my' as 1, 2, 3.
So we can reuse those,
and we can create a new token for cat,
which we haven't seen before.
So let's make that the number 5.
Now if we just look at the two sets of encodings,
we can begin to see
some similarity between the sentences.
I love my dog is 1, 2, 3,
4 and I love my cat is 1, 2, 3, 5.
So this is at least a beginning and how we can
start training a neural network based on words.
Fortunately, TensorFlow and Keras give us
some APIs that make it very simple to do this.
We'll look at those next.